Description
The classification-confidence panels publish an unmatched share that is arithmetically correct and answers the wrong question. Measured live on a 270-stack estate: 299 matched against 89,617 unmatched metric names, and the per-stack Unmatched metric share % has a MEDIAN OF 100. A leadership surface therefore reports that essentially nothing is classified.
Why the number is misleading
The registry is a PRESENCE DETECTOR, not a classifier of every metric name. It carries one sentinel per technology, so the maximum it can ever match on a stack is one name per technology present - a few dozen against thousands of names. The share measures the design, not a gap, and it can never improve materially no matter how many entries are added. Confirmed independently: an estate-wide sweep classified 83 of 53,689 distinct names (0.2%) under registry v1, and v2 with 56 entries moves that to roughly 0.3%.
What to publish instead
Replace the metric-name share with figures whose denominator matches the question:
- Technology presence. Technologies detected on a stack, and the estate distribution of that count. Under registry v2 the live numbers are median 1, mean 2.9, max 17, with 233 of 270 stacks detecting at least one. That is a legible, improvable figure.
- Stacks with no technology detected. The honest coverage gap - 37 of 270 live. This is the number that should drive registry work, and it is small enough to act on.
- Registry reach. For each registry entry, how many measured stacks carry it. Already emitted as
coverage_technology_stacks; surface it as the registry-health panel.
Keep, but reframe
Do not delete the unmatched metric-name COUNT. It stays useful as registry-development input - it is the queue of names nobody has classified - but it belongs in the classification-evidence register table, not in a confidence stat panel, and it must not be expressed as a share of all names.
Metric changes
coverage_metric_names{kind="matched"|"unmatched"} keeps its current meaning but stops feeding a headline. Add a bounded enum for the technology-count distribution rather than a per-stack metric: a kind enum of buckets (0, 1, 2-4, 5+) keeps it to four series. Per-stack technology counts already exist as coverage_stack_technologies.
Every new metric must be declared in collector/emit/budget.py CATALOGUE or tests/test_budget.py fails, and BUDGET.md is regenerated, never hand-edited.
Dashboard changes
The Classification confidence row currently leads with Metric names unmatched by the registry. Rebuild it around technology presence, the no-technology-detected count, and registry reach. The unmatched count moves to the metric-name register table with a column note explaining it is a registry backlog, not a coverage failure.
Acceptance Criteria
- #1 No panel expresses unmatched metric names as a share of all metric names
- #2 Technology-presence distribution is published as a bounded enum
- #3 Stacks detecting zero technologies is published as its own figure
- #4 Registry reach per entry is rendered
- #5 The unmatched metric-name count survives as register-table detail with an explanatory note
- #6 New metrics declared in CATALOGUE; BUDGET.md regenerated by its own command
Definition of Done
- #1 python3 -m pytest tests -q
- #2 tofu fmt -check -recursive terraform; tofu init -backend=false and tofu validate pass for terraform/ and terraform/examples/standalone/
- #3 customer-identifier and shipped-text gates from .github/workflows/ci.yml return clean
Implementation Plan
- Add tests that derive the four technology-count buckets and prove unmatched shares leave every published view.
- Emit a bounded estate technology-presence distribution while preserving the per-stack count and registry reach metrics.
- Replace the Classification confidence dashboard row with technology presence, zero-detection, and registry reach; retain unmatched counts only in the evidence register.
- Regenerate BUDGET.md, run the item gate and CodeRabbit, finalize the task, commit to main, and push.
Implementation Notes
Implemented the four-bucket technology-presence distribution and removed unmatched metric-name shares from the summary and dashboard. The unmatched count remains a non-headline registry-development backlog beside the named evidence table; registry reach remains rendered per entry.
Final verification: 1400 passed, 2 skipped, 6614 subtests; dashboard assembly and metric-render coverage gates passed; OpenTofu formatting and both validations passed; identifier working-tree/history and shipped-text gates passed; CodeRabbit completed with zero findings.
Final Summary
Replaced the misleading unmatched-name share with a four-bucket technology-presence distribution, a direct zero-technology stack figure, and rendered registry reach. Preserved unmatched names only as a clearly described registry-development backlog. Verified with the full offline, dashboard, budget, Terraform, privacy, text, and review gates.