Re-examine every prior capture now that Loki decode works

Description

SKT-0013.04 found that the capture receiver could not decode Alloy real Loki wire format AT ALL, and fixed it. This task deals with the consequence, which is larger than the fix.

handleLoki understood only gzip+JSON — which is what synthkit own sink sends. A real loki.write sends a SNAPPY-COMPRESSED logproto.PushRequest with Content-Type application/x-protobuf. So for the entire life of the k3d lab, the Loki-native lane was SILENTLY INVISIBLE: one permutation reported 21,706 requests decoded and ZERO Loki receipts.

Why that matters beyond one bug: every prior capture recorded no Loki evidence, and nothing distinguished “the collector sent nothing” from “the receiver could not read it”. The corpus therefore asserts an absence it never actually observed, and the fidelity gate has been comparing synthkit log emission against a reality side that was structurally empty. Any conclusion drawn about Loki-native log shapes from a pre-fix capture is unsound — not necessarily WRONG, but unsupported, which is the same problem this project keeps finding in other forms.

So: re-capture, and then re-examine what changes. Specifically worth checking rather than assuming:

• Whether Loki-native log families appear that the corpus has never carried.
• Whether any signals/ claim about the Loki transport was written from a capture that could not have seen it.
• Whether any coverage gap or contradiction involving logs changes verdict once real Loki evidence exists.

The matrix caught this itself, through the partial outcome distinguishing “harness completed but the predicate was unmet” from success — which is exactly what that outcome was built for. Worth noting as evidence the five-outcome design earns its keep.

Acceptance Criteria

• #1 A fresh capture with working Loki decode is taken for every permutation that emits Loki-native
• #2 Loki families the corpus never carried are identified and routed to a verdict rather than silently added
• #3 Any signals/ claim about the Loki transport written from a pre-fix capture is re-checked or marked unsupported
• #4 Log findings whose verdict changes under real evidence are enumerated

Definition of Done

• #1 make gate (build vet test race rw-proto-check spdx-check forbidden-words)
• #2 make blueprint-schema (only if a blueprint field or construct/workload config struct changed)
• #3 DRY_RUN=true go run ./cmd/synthkit -once -dump — inventory diffed against signals/

Implementation Notes

DELIVERED 2026-08-27, commit 4fa184e. Three lab runs: one to re-capture, one to confirm a fix, one diagnostic.

AC #1 — the captures

Both Alloy permutations re-captured with SKT-0013.06’s corrected receiver, 300s common window, both CAPTURED. Loki receipts where every prior run recorded zero: 199 for alloy-default (podLogsViaLoki), 65 for alloy-otlp-podlogs (its cluster-events and manifests lanes are Loki-native regardless of the pod-log transport).

A defect the first working capture immediately exposed

The Loki path FUSED two lanes. alloy-default produced ONE entry under the empty source carrying both container/namespace/stream and action/k8s_kind — the pod-log stream and the manifests stream merged into a union label set matching no real stream. Exactly the harm logfamily.go was written to prevent, in a path that had never been exercised.

Cause, and it is not what I first assumed. I pointed the Loki handler at ClassifyLogSource and the fusion PERSISTED. A diagnostic run keying unclassified streams by job gave the real shapes and the reason:

pod logs:  stream = app_kubernetes_io_name cluster container flags job
                    k8s_cluster_name namespace service_name service_namespace stream
           meta   = pod service_instance_id
manifests: stream = action cluster instance job k8s_cluster_name k8s_kind k8s_namespace_name
           meta   = k8s_daemonset_name k8s_deployment_name k8s_pod_name

A real podLogsViaLoki stream carries pod as STRUCTURED METADATA, not a stream label — a pod name churns on every restart, so keeping it out of the index is deliberate. The identity triple is split across both halves, and a rule reading stream labels only recognises no pod-log stream on that transport at all.

Fixed in BOTH halves, which is the part worth remembering: ClassifyLogStream on the capture side and ShapeLogFamily on the comparator side. I fixed the capture side first and the comparator still reported all three Loki lanes as raw-key-set identities, so nothing paired and every synth pod-log lane read as uncovered. A test now fails if the two diverge again.

AC #2 — the three newly visible Loki families, each with a verdict

• k8s_pod_logs (loki) — REAL, and it CONTRADICTS synthkit. Promoted; see AC #4.
• kubernetes-events (loki) — REAL, synthkit emits it, but the comparator cannot pair it: identifyLog ignores the declared source and promotion elides its value, so the lane name is unrecoverable from a committed document. Reports as an uncovered lane when the true difference is one key (instance). Promoted, verdict recorded, tracked as SKT-0010.15.
• the manifests lane (loki, source-less) — REAL, same pairing problem, same task.

Promoted through a new lab-matrix promote -logs -merge: entries as the capture classified them, cumulative union into the existing untagged k3d document. NOT permutation-tagged, deliberately — synthkit models both pod-log transports behind pod_logs_method, so a mismatch here IS drift, and tagging would have suppressed the contradiction this task exists to surface. Transport is part of log identity, so the Loki and OTLP entries never collide.

AC #3 — the audit result

NO signals/ claim about the Loki transport was written from a pre-fix capture. The one Loki-native block in signals/k8s.md was explicitly labelled “doc-sourced” and was honest about it. Re-checked against real evidence anyway, and it was wrong in four places: pod and service_instance_id are metadata not stream labels, detected_level is absent at egress (Loki derives it destination-side, as the OTLP block already said), and flags is present and documented nowhere. k8s_cluster_name was omitted, and the documented metadata key k8s_pod_name does not exist on this transport. All corrected with provenance.

AC #4 — findings whose verdict changed

ONE new contradiction, and it is a real emission defect invisible for the whole prior life of the lab: synthkit’s Loki pod-log lane emits k8s_namespace_name/k8s_pod_name/k8s_container_name/log_iostream where the collector emits namespace/container/stream; emits detected_level and service_instance_id as stream labels where the collector emits neither; and omits job, flags and all structured metadata. Tracked as SKT-0013.08 with the full captured contract.

Also settled SKT-0013.05 by evidence: the classic spelling is what podLogsViaLoki genuinely puts on the wire, so its six addon lanes are mislabelled as that task predicted. Its AC #3 worry — that a capture would arbitrate by accident — did not materialise, because the capture produced a THIRD shape matching neither candidate.

Contradictions 68 -> 69. Every other finding the Loki evidence added is coverage.

Scoped out, recorded, not dropped: the metric side of the same capture (SKT-0010.16, including two harness families that must never enter the corpus) and the comparator pairing gap (SKT-0010.15).

Final Summary

The Loki-native transport had never been observed once: the capture receiver understood gzip+JSON while a real loki.write sends a snappy-compressed protobuf, so every prior capture recorded no Loki evidence and nothing distinguished the collector sending nothing from the receiver being unable to read it. Both Alloy permutations are now re-captured with working decode, and the first working capture immediately exposed a second defect: the Loki path fused the pod-log stream with the manifests lane into one entry whose union label set matches no real stream. A diagnostic run gave the reason rather than a guess — a real podLogsViaLoki stream carries pod as structured metadata, not a stream label, so a rule reading stream labels only recognises no pod-log stream at all. Both halves of that rule now read both, with a test that fails if the capture and comparator sides diverge again. The audit found no signals/ claim written from a pre-fix capture; the one Loki block was honestly labelled doc-sourced, and re-checking it against evidence corrected it in four places. The payoff is one new contradiction and it is a real emission defect: synthkit’s Loki pod-log lane uses the OTLP transport’s label spellings, emits two keys the collector does not, and omits job, flags and all structured metadata — tracked as SKT-0013.08. Three newly visible Loki families each carry a verdict, and the two that cannot yet pair are tracked as SKT-0010.15 rather than left as noise.