Task · SKT-0013.08

Correct the Loki pod-log emitter to the captured stream shape

Description

Found 2026-08-27 by SKT-0013.07, the first capture in which the lab could decode Alloy’s Loki wire form at all. It is the one NEW contradiction that evidence produced, and it is a real emission defect rather than a permutation difference: synthkit models this exact lane behind pod_logs_method: kubernetes_api|loki.

Captured at collector egress, k8s-monitoring 4.4.0, k3d, 300s window:

stream labels: app_kubernetes_io_name, cluster, container, flags, job,
               k8s_cluster_name, namespace, service_name, service_namespace, stream
structured metadata: pod, service_instance_id

synthkit emits, per the same run’s synth inventory:

stream labels: app_kubernetes_io_name, cluster, detected_level, k8s_cluster_name,
               k8s_container_name, k8s_deployment_name, k8s_namespace_name, k8s_pod_name,
               k8s_statefulset_name, log_iostream, service_instance_id, service_name,
               service_namespace
structured metadata: (none)

The defect is fourfold and each part is independently wrong:

  1. WRONG SPELLINGS. The collector emits the flat Loki convention namespace, container, stream. synthkit emits k8s_namespace_name, k8s_container_name, log_iostream, which is the OTLP transport’s promoted spelling, not this transport’s. A dashboard or alert written against real podLogsViaLoki data matches nothing in synthkit output.
  2. WRONG LEVEL FOR POD IDENTITY. pod is STRUCTURED METADATA on the wire, not a stream label — an index-cardinality choice, since a pod name churns on every restart. synthkit puts k8s_pod_name in the stream labels, which makes every pod restart a new Loki stream. service_instance_id is the same mistake.
  3. EMITS WHAT THE COLLECTOR DOES NOT. detected_level is derived by Loki destination-side and is absent at egress. The OTLP half of signals/k8s.md already said so for that transport; the Loki half asserted it as a stream label because the block was doc-sourced.
  4. OMITS WHAT THE COLLECTOR DOES. No job (<namespace>/<container>), no flags (the CRI partial/full marker, the Loki-path spelling of the OTLP logtag), and no structured metadata at all.

k8s_deployment_name and k8s_statefulset_name are also synth-only. The capture carries k8s_deployment_name as structured metadata on the MANIFESTS lane, not on pod logs, where the workload is named by app_kubernetes_io_name. Reconcile rather than assume.

Do not correct the capture to the synth. signals/k8s.md [slug: k8s-pod-logs] now records the captured contract and the four deltas from the old doc-sourced block; that section is the specification for this task.

Both transports must keep projecting the same CONTENT — switching pod_logs_method may change the observable shape and must never add, drop or alter a log line.

Acceptance Criteria

Definition of Done

Implementation Plan

Lane B owns internal/k8saddon/** plus argocd, certmanager, envoygateway, extdns, karpenter and lbc: first pin the captured stream-label and metadata contract with failing tests, then correct all six Loki-native addon emitters without changing content.

Implementation Notes

Dispatch preflight 2026-08-28: generic EXECUTION route gpt-5.6-luna/max and fork_turns=none confirmed before spawn. Post-spawn route metadata is unexposed by this Codex client; only lane name/status are observable, so absence is recorded as a limitation rather than treated as substitution.

Implemented a shared Loki-native pod-log stream builder and migrated argocd, cert_manager, envoy_gateway, external_dns, karpenter and load_balancer_controller to the captured flat stream-label shape with pod and service_instance_id metadata. Focused package tests pass; review also corrected Karpenter to omit those metadata keys when no real leader pod resolves.

Final evidence: the shared k8saddon Loki pod-log builder drives karpenter, argocd, cert-manager, load-balancer-controller, external-dns and envoy-gateway. Focused tests pin the captured stream-label and structured-metadata sets; the integrated make gate passed.

Final Summary

Reconciled all six addon pod-log emitters to the captured Loki-native wire shape through one shared builder, preserving content while moving pod and service_instance_id to structured metadata.

View the source file on GitHub