Description
Found by the SKT-0008 producer validation, 2026-08-27 (recorded there as divergence D12).
The error fraction on traces_spanmetrics_calls_total{status_code="STATUS_CODE_ERROR"} and on the failed service-graph edge is a hard-coded ~1% constant, decoupled from the ledger outcomes that drive the trace lane.
The visible consequence: an active incident moves trace errors but does not move span-metric errors. A user activating a failure scenario sees the error rate climb in Tempo and stay flat on the RED panels built from span metrics — which is the exact opposite of what a real metrics-generator does, since it derives both from the same spans. Incidents and scenarios are a headline synthkit feature, so this undercuts a claim the project makes rather than being an internal detail.
web_service already samples the ledger for this in its metric tick; the app lane does not. The fix is to make the app lane sample the same way, so both derive the fraction from the outcomes actually minted rather than from a constant.
Acceptance Criteria
- #1 The span-metric error fraction is derived from ledger outcomes rather than a constant
- #2 Activating a failure scenario moves the span-metric error rate in step with the trace error rate
- #3 A test covers the incident case, since the flat-line behaviour is only visible under an active scenario
Definition of Done
- #1 make gate (build vet test race rw-proto-check spdx-check forbidden-words)
- #2 make blueprint-schema (only if a blueprint field or construct/workload config struct changed)
- #3 DRY_RUN=true go run ./cmd/synthkit -once -dump — inventory diffed against signals/
Implementation Plan
Write a failing active-scenario regression proving app span-metric errors stay flat, then derive the error fraction from ledger outcomes and prove it moves with trace errors.
Implementation Notes
2026-09-02 validation: the active-scenario regression failed against the prior constant and now passes by comparing emitted span-metric errors directly with ledger trace outcomes. Both app RED metrics and failed service-graph edges derive their error fraction from active ledger requests and calls. Focused tests, integrated just check, and just dump passed; generation was not applicable.
Final Summary
2026-09-02: Replaced the hard-coded app span-metric error fraction with ledger-derived outcomes and proved the active incident moves trace and span-metric errors together.