Task · TSO-0164

OTLP ACK probe recipe: record what a gateway returns on a successful HTTP export

Description

The HTTP OTLP ACK guard (frozen TSO-0148 design section 14) fails closed on a 2xx with a non-empty body that is not protobuf, including a missing Content-Type or a proxy that rewrites the body. All the evidence is from synthetic servers. If the Grafana Cloud gateway can return such a 2xx, every HTTP deployment drops logs and stalls WAL commits. Rob approved on 2026-10-09 a reusable probe that can be rerun before each release. The recipe reads OTLP_ACK_PROBE_ENDPOINT, OTLP_ACK_PROBE_USER and OTLP_ACK_PROBE_TOKEN_FILE from the environment, sends one OTLP/HTTP protobuf log record and one gauge with service.name=tailscale2otel-ackprobe, and prints for each signal only the status code, the Content-Type, the body length and whether the body decodes as the matching Export*ServiceResponse protobuf. It never prints the token, the URL credentials or the body.

Acceptance Criteria

Definition of Done

Implementation Plan

Implement local-only ACK probe with fixture evidence; exact candidate gate and security review before landing.

Implementation Notes

loop20: 2 implementation attempts; Done after acceptance, landed code and green composed gate. Earlier superseded/cancelled CI runs are not passes.

Final Summary

loop20 landed evidence d5f023fadc688a486d8ea9f9d9b33bfd23d92e81. Reusable env/file-only ACK recipe with bounded requests, 401/403 fail-fast, encoded credential redaction, HTTPS outside literal loopback and no redirects. Real local recipe fixtures prove fields, secrecy and request counts. Security original findings corrected and delta PASS. CodeRabbit Go coverage complete; missing-recipe major disproven as shard artifact; skipping tests without just declined. Pure recipe wiring validated; file-as-directory shard failure not claimed pass. Final composed just check on actual shipped SHA129945fb3230f1c5d8a1127ad3e3c7faa2c514f4 passed, unchanged tree. Exact CI e96e74e31e2df4c697435dd4b7a32315e0e02f41 run38003933710 ci-success succeeded; no production diff outside backlog from that tested code. Helm absent-not-required (no landed chart change). Just formatting passed. Generated-input DoD is not applicable or covered by gate; no generator drift or weakened test claimed.

View the source file on GitHub