Stop the capture receiver recording inferred evidence as observed

Description

Two defects in e2e/receiver/, both found while other work tripped over them, and both of the same shape: the receiver records something it INFERRED as though it had OBSERVED it. That is the one thing a ground-truth capture must never do, because every downstream decision then treats a guess as evidence.

1. Histogram.Classic is inferred from the metric name suffix. e2e/receiver/receiver.go:234 sets it from _bucket|_sum|_count alone. So a family carrying only _count and _sum records as classic: true with empty bucket_bounds — which reads as “reality publishes a classic histogram” when the evidence supports only “reality publishes a _count”.

This is not hypothetical harm: it made SKT-0010.07 premise look settled when it is not. That task asserts the corpus proves the real kubelet publishes storage_operation_duration_seconds as a histogram; it proves no such thing, and a lane nearly converted a counter to a histogram — emitting _bucket{le=...} series the capture does not contain — on the strength of it. The lane stopped, correctly, but only because someone read the corpus entry carefully rather than trusting the flag.

Fix: distinguish bucket-OBSERVED from suffix-INFERRED, or record no Histogram at all when no le was ever seen. The second is simpler and loses nothing, since a family with no observed bucket series has no bucket evidence to carry.

2. An OTLP log entry source is derived from service.name. e2e/receiver/receiver.go:740-748. So a captured pod-log entry files under a service name while the corpus curated entries say k8s_pod_logs. Harmless today only because SKT-0010.09 made the comparator classify by SHAPE on both sides — but the recorded source now disagrees with what every other producer records, and the next reader of that field will be misled.

Fix: call inventory.ClassifyLogSource, which SKT-0010.09 made shared exactly so producers stop each deriving their own answer.

Acceptance Criteria

• #1 A family with no observed bucket series does not record as a classic histogram
• #2 Bucket-observed and suffix-inferred are distinguishable, or the inferred case records nothing
• #3 The OTLP log source comes from the shared classifier rather than service.name
• #4 Re-capturing does not change any family whose evidence was genuinely observed
• #5 SKT-0010.07 premise is re-checked against the corrected corpus, since it was built on the inferred flag

Definition of Done

• #1 make gate (build vet test race rw-proto-check spdx-check forbidden-words)
• #2 make blueprint-schema (only if a blueprint field or construct/workload config struct changed)
• #3 DRY_RUN=true go run ./cmd/synthkit -once -dump — inventory diffed against signals/

Implementation Notes

SCOPE SHARPENED 2026-08-27 by lane L21, which deliberately did NOT take this fix and returned the reason.

The receiver-side fix is NOT local. internal/inventory/diff.go compares histogram.representations with BOTH directions certain, so the moment the receiver stops asserting classic from a _count suffix, the reality side records an EMPTY representation set and the synth side legitimate classic claim becomes a NEW contradiction. That trades a false positive for a false negative, which is worse — a false positive is noisy and a false negative is invisible.

So this task needs a PAIRED change: the receiver stops inferring, AND diff.go treats an empty representation set as ABSENT EVIDENCE, mirroring what instrumentEvidenceAbsent already does for the unknown instrument sentinel. That pattern already exists and is the right precedent to follow rather than inventing a second one. Note also that histogramView cannot currently distinguish “no histogram block at all” from “classic: false”, which the fix has to resolve one way or the other.

Direction of travel already helps: SKT-0010.09 added inventory.ClassicHistogramFamily as a shared rule and rewired e2e/inventory to it, so there is now one place the suffix rule lives rather than three.

Do not land the receiver half alone.

DELIVERED 2026-08-27, commit 4fa184e, together with SKT-0013.07 — the fix had to land before the re-capture, so one lab run served both.

What replaced the inference

le is the proof, because Prometheus reserves it and nothing else carries it. A _sum or _count suffix is not even weak evidence: a histogram and a summary expose identically named component series, and CloudWatch’s five-stat expansion emits a genuine <metric>_sum GAUGE.

The fold moved from series time to Snapshot. A component series cannot know whether its family’s bucket series turns up later in the capture window, so proof is only complete once the window is. It uses the SHARED rule — FoldClassicHistogramMetricsWithProof, added beside the existing FoldClassicHistogramMetrics rather than deriving a fourth copy. A producer’s own remote-write metadata declaring HISTOGRAM also proves the family, since that is the producer speaking rather than the name being read; CodeRabbit correctly caught that the RW1 metadata path was missing that half while RW2 had it, which would have folded the same family on v2 and not on v1.

ORDERING BUG I introduced and fixed: the fold must run BEFORE applyDeclaredInstruments, not after. The fold merges a bucket series that proved histogram with the _sum/_count series that proved nothing and contributed the unknown sentinel, and dropping that superseded sentinel is applyDeclaredInstruments’s last act. Folding afterwards left [histogram unknown] on the family. Caught by an existing test.

The paired comparator half, which the scope note was right to insist on

An empty representation set is now ABSENT EVIDENCE, mirroring instrumentEvidenceAbsent. Only the SYNTH direction is gated: if reality observed no representation, synth’s legitimate classic claim must not contradict; if SYNTH has none, reality’s is still a real coverage gap and still reports. That also settles the histogramView question the scope note raised — “no histogram block” and “classic: false” both mean no representation evidence, so they need not be told apart. histogram.native_schemas takes the same gate, and deliberately not a gate on the schema set being empty: an empty reality schema set against an OBSERVED classic representation is real evidence of “not native” and must still contradict.

AC #4 measured, not asserted

Fresh capture of both Alloy permutations, diffed against the committed corpus:

HISTOGRAM-BLOCK changes on shared names: NONE
names removed: 1  — storage_operation_duration_seconds (classic=true, bounds=0)
names added:  51  — of which exactly one is this fix:
                    storage_operation_duration_seconds_count (no histogram)

The other 50 are families a longer capture simply saw (go_, process_, scrape_, up, prober_), unrelated to this fix and scoped to SKT-0010.16. Twelve families carry a histogram block and every one has real captured bounds.

SECOND HARM, now demonstrable: go_gc_duration_seconds arrives as a SUMMARY with quantile, and its _sum/_count now stay separate. Under the old rule they would have folded into it and stamped classic: true — recording a classic histogram for a summary.

AC #5

SKT-0010.07’s premise was already disproved by a prior lane and is confirmed by the corrected corpus: the family is storage_operation_duration_seconds_count with no histogram evidence, and the counter shape is right. The correction also removed the NAME mismatch that had been hiding that task’s real remaining finding — the comparator now pairs the families and reports only-in-reality=[node], the missing label the task notes said was invisible.

Log source

Both log paths now use the shared classifier. The OTLP path keeps the resource identity as a provenance fallback for a shape no rule recognises, so an ordinary application’s OTLP logs do not become unnamed entries.

make gate green. CodeRabbit run pre-push: three findings, all three fixed (RW1 metadata proof, the native_schemas gate, promote usage text).

Final Summary

The capture receiver no longer records inferred evidence as observed. Histogram.Classic came from a metric-name suffix, so a family whose only captured component was _count entered the corpus asserting reality publishes a classic histogram with an empty bucket-bound set; le is now the proof, and folding components into their family happens once the capture window is complete, against the proof it actually accumulated. The paired comparator change the scope note demanded landed with it: an empty representation set is absent evidence rather than a claim, gated on the synth direction only, so this trades no false positive for an invisible false negative. AC #4 was measured rather than asserted — a fresh capture of both Alloy permutations changed the histogram block of no shared family at all, and exactly one name changed, the one this was about. A second harm became demonstrable in the same run: go_gc_duration_seconds is a summary whose _sum and _count the old rule would have folded in and stamped classic. Both log paths now use the shared classifier. SKT-0010.07’s premise is confirmed against the corrected corpus, and correcting it removed a name mismatch that had been hiding that task’s real finding — synth omits the node label — which the comparator now reports. Verified by make gate green and by the measured capture diff; CodeRabbit found three further defects pre-push, all fixed.