Description
The 2026-08-27 report lists 367 extra_metric coverage gaps: metric families a real collector shipped that synthkit does not emit at all. Today they are an undifferentiated list, so nobody can tell a genuine emission gap from a family synthkit deliberately does not model.
Each gap gets one of three verdicts, with evidence:
- should emit — a real deployment of something synthkit already claims to model produces this, so its absence is a hole in a claim the project makes. Carries enough detail for a later implementation wave: the family, its labels as captured, and which construct or workload owns it.
- out of scope — reality produces it, but it belongs to a component synthkit does not claim to model. Records WHY, so the next audit does not re-litigate it.
- unresolved — cannot be decided from the corpus and available documentation. Produces a cantfind.md PENDING rather than a guessed verdict.
Group by owning construct or workload, not by metric name: 367 individual verdicts are unreadable, and a whole exporter arriving or not arriving is the real unit of decision.
The verdict record has to live where someone adding a construct will find it, and it has to survive the next corpus refresh — a verdict that has to be re-derived every time the corpus is recaptured is worthless.
Nothing is implemented here. Emission work the should-emit verdicts justify becomes its own tracked work, sized from the grouped verdicts rather than pre-guessed.
Acceptance Criteria
- #1 Every coverage-gap metric family carries one of the three verdicts
- #2 Verdicts are grouped by owning construct or workload, with a per-group summary a reader can act on
- #3 Each verdict cites its evidence: a corpus entry, vendor documentation, or a live capture, with a date
- #4 Out-of-scope verdicts record why, so the decision is not re-litigated at the next audit
- #5 Unresolved gaps produce cantfind.md PENDING rows rather than a guessed verdict
- #6 The verdict record survives a corpus refresh without needing to be re-derived
- #7 Emission work justified by should-emit verdicts is created as tracked work sized from the groups, not pre-guessed
Definition of Done
- #1 make gate (build vet test race rw-proto-check spdx-check forbidden-words)
- #2 make blueprint-schema (only if a blueprint field or construct/workload config struct changed)
- #3 DRY_RUN=true go run ./cmd/synthkit -once -dump — inventory diffed against signals/
Implementation Notes
DELIVERED 2026-08-27 (lane L4). Record: reality-corpus/verdicts/{coverage-verdicts.json,README.md}.
Headline: 15 of the 367 findings were never coverage gaps
The five kubeproxy_*_duration_seconds families synthkit DOES emit correctly as classic histograms. internal/inventory/gcx_readback.go records raw _bucket/_count/_sum as three separate metric entries, while internal/inventory/synth.go:99 and e2e/inventory/inventory.go:88 both fold them into the family base. Producer asymmetry, not missing emission — and it is the entire k8s/eks extra_metric population. Routed to SKT-0010.08, because the same asymmetry will generate false instrument_mismatch and unexpected_label_key findings for every real histogram family the read-back covers, and those land in the section SKT-0010.05 is about to make fail CI.
Real population: 72 families / 352 findings. Verdicts: 57 should-emit, 1 out of scope, 14 unresolved.
Should-emit, ranked by how much each matters rather than by size
- NAT Gateway, 9 families — highest despite being small. synthkit emits ONE of four directional byte counters and NONE of the four packet counters, so any ingress/egress split or drop-ratio panel is not merely sparse, it is arithmetically WRONG.
- EC2, 11 families — packet counterparts of byte counters already emitted, and three roots completing the cpucredit family. Captured with dimension_InstanceId ONLY for the ebs checks; no ASG rollup observed, so none should be added.
- ALB, 10 families. 4. Firehose, 20 (worth splitting: the four delivery_to_http_endpoint_* roots are the high-value half). 5. EBS, 6. All created as SKT-0010.06.
Out of scope, 1
coredns_hosts_entries. The hosts plugin exports it and the corpus shows it loaded against a k3s-specific Corefile, but corpus authority is k3s only and synthkit models a Corefile with no hosts block — a decision already recorded at coredns.go:430. Not to be re-litigated without EKS-substrate CoreDNS evidence.
Design property that makes the record durable
Keyed on (area, substrate, FOLDED family) using the same fold as internal/inventory/synth.go. So: a refresh adding or dropping stat suffixes or histogram components resolves to the same key; captured_on lives inside the evidence, never the key, so a date bump invalidates nothing; fixing the read-back producer fold renames series but not folded families, so the record is ALREADY correct for the fixed producer; and should_emit is self-retiring — once emitted, the comparator stops reporting it.
The honesty mechanism is the untriaged branch of the proposed consumption: a finding with no verdict entry prints under UNTRIAGED rather than quietly passing, so a refreshed corpus that adds families surfaces them.
Shape defects found while triaging, routed to SKT-0010.07
- storage_operation_duration_seconds: real kubelet publishes a classic histogram; synthkit publishes a standalone counter via st.Add at kubelet.go:308, so no quantile is computable.
- coredns_hosts_reload_timestamp_seconds pinned at 0 for a Corefile with no hosts block. Real CoreDNS registers hosts-plugin metrics only when the plugin loads, so the current shape matches neither state — it invents a third.
cantfind SK-92 through SK-95 filed (the lane proposed SK-84..87; renumbered by the wiring pass to avoid collisions).
Final Summary
Every coverage-gap family carries a recorded verdict in reality-corpus/verdicts/: 57 should-emit, 1 out of scope, 14 unresolved with cantfind PENDINGs (SK-92 to SK-95).
The first finding was that 15 of the 367 reported gaps were never gaps. The five kubeproxy duration families synthkit DOES emit correctly as classic histograms; the gcx read-back records raw _bucket/_count/_sum as three separate entries while both other inventories fold them into the family base. Producer asymmetry, and it is the entire k8s/eks extra_metric population. Routed to SKT-0010.08, because the same asymmetry will generate false instrument and label findings for every real histogram family — findings that land in the section about to fail CI.
Verdicts are grouped by owning construct rather than by metric name, because a whole exporter arriving or not is the real unit of decision. Ranked by consequence rather than size: NAT Gateway is 9 families but ranks first, since synthkit emits one of four directional byte counters and none of the four packet counters, making any ingress/egress or drop-ratio panel arithmetically WRONG rather than merely sparse.
The record is designed to survive a corpus refresh: keyed on folded family name so suffix changes resolve to the same key, with the capture date inside the evidence rather than the key, and should-emit verdicts self-retire once the family is emitted. The proposed gate consumption keeps an UNTRIAGED branch, so a refreshed corpus that adds families surfaces them rather than quietly passing.
Two emitter shape defects surfaced while triaging and were routed to SKT-0010.07 — storage_operation_duration_seconds emitted as a counter where the real kubelet publishes a histogram, so no quantile is computable, and a coredns hosts-plugin metric pinned at zero for a Corefile that does not load the plugin, matching neither real state. Emission work is tracked as SKT-0010.06, sized from the groups. go build clean, JSON parses, hygiene guards clean.