Task · TSO-0124

Alert when an enabled receiver is fail-closed by missing credentials

Description

The health dashboard visualizes tailscale2otel.receiver.misconfigured, but no rule watches it. A non-zero value means an enabled network receiver is rejecting every input with HTTP 403 until the operator supplies its credential or binds it safely; this is low-noise and directly actionable.

Acceptance Criteria

Definition of Done

Implementation Notes

Done in 7841543f. ts2o-receiver-fail-closed: max by (receiver) (tailscale2otel_receiver_misconfigured_ratio) > 0 for 10m, severity advisory, no page label, policy optional so a disabled receiver’s absence stays Ok while a datasource error surfaces as Error. Runbook ingest-receivers, panel ‘Fail-closed receiver misconfiguration’. The 10m window keeps a rolling restart quiet. Fixtures cover a receiver reporting 0 (healthy, not absent, since the point of the signal is that the exporter is healthy while one route accepts nothing) and a sustained 1. Negative-tested on its own: threshold moved to 999, the fixture failed, reverted. Pushed live with gcx –context m7kni; verify-deploy m7kni reports 133 shipped, 133 deployed, 0 missing, 0 orphaned, 0 drifted.

View the source file on GitHub