gcx live read-back: merge real EKS label names and values into the reality corpus

Description

Layer 2b, the second corpus producer. Closes the fidelity gap the k3d lab structurally cannot: EKS-specific identity and cloud signals. A real EKS cluster already runs the exact chart under audit, so this reads the shapes back off the live stack rather than rebuilding the cluster.

Uses gcx against the operator-selected stack to enumerate, per metric family in scope, the real metric names, label names, and label values, then merges them into the corpus as substrate-tagged entries.

Manual/local only by design: it needs credentials, so it can never be the always-on CI gate. Its job is to keep the corpus honest about the AWS surface between k3d captures.

In scope: EKS node and pod identity (provider_id, instance-type and topology node labels, aws-node and kube-proxy on EKS), plus core CloudWatch families and their stat-suffix expansion. Out of scope: AI/LLM signal areas, Bedrock, AppFlow.

Credential handling follows the repository contract: secrets live only in the ignored .env, never in committed YAML, docs, or task text. The stack to read is operator-selected, never hardcoded.

Acceptance Criteria

• #1 A documented make target reads real metric names, label names and label values for the in-scope families off an operator-selected live stack via gcx
• #2 Read-back output merges into the corpus as entries tagged with the EKS substrate and the capture date
• #3 The target fails with a clear message when no gcx context or credential is configured, and never proceeds against a default or guessed stack
• #4 The scope covers EKS node/pod identity and core CloudWatch families including stat-suffix expansion, and explicitly excludes the AI/LLM areas
• #5 No credential value, stack identifier, or account identifier is written into committed files, corpus entries, or documentation
• #6 The findings report distinguishes drift that only the live read-back can see from drift the k3d lab already covers

Definition of Done

• #1 make gate (build vet test race rw-proto-check spdx-check forbidden-words)
• #2 make blueprint-schema (only if a blueprint field or construct/workload config struct changed)
• #3 DRY_RUN=true go run ./cmd/synthkit -once -dump — inventory diffed against signals/

Implementation Plan

1. Use only the operator-selected gcx context and its explicitly configured Prometheus datasource; run bounded read-only queries for EKS node/pod identity, AWS VPC CNI, kube-proxy, and core non-AI CloudWatch metric families.
2. TDD a root-owned read-back command that converts gcx JSON series into frozen synthkit.telemetry.reality-corpus/v1alpha1 documents, tags them substrate=eks with the capture date, elides deployment-specific values, and cumulative-merges existing documents without treating absence as deletion.
3. Add a documented make target that requires an explicit GCX_CONTEXT, validates context/authentication and datasource configuration with clear failures, invokes only gcx read operations, excludes AI/LLM areas, and never persists stack/account/tenant identifiers.
4. Extend the findings report provenance wording so EKS live-read-only drift is distinguishable from k3d-covered drift, with focused tests.
5. Run the target against the operator-selected stack, inspect the committed corpus for forbidden identifiers, run focused tests and the required structural dump/report checks, then CodeRabbit and a routed final make gate.
6. Commit and push explicit paths only, require exact-SHA ci including hygiene, then finalize SKT-0006.04 atomically and stop the campaign.

Implementation Notes

2026-08-25: The operator selected the live stack that receives telemetry from the real EKS lab running the normal Helm chart. The stack identifier is intentionally omitted from tracker text and will be supplied only through the local GCX_CONTEXT invocation.

2026-08-25 live read-back evidence: an explicit operator-selected context passed gcx configuration and connectivity checks. The bounded 24-hour read returned 588 core CloudWatch metric contracts and 31 EKS Kubernetes contracts. The CloudWatch set contained 116 names for each of the _sum, _average, _maximum, _minimum and sample_count stat suffixes. A separate bounded seven-day read-only availability check returned zero awscni* series, so no unsupported k8s-addons corpus document was created; the target still queries that family explicitly.

Privacy review before acceptance found that CloudWatch resource-tag label names can themselves encode a deployment identity. The candidate was discarded and regenerated after the canonicalizer was tightened to omit every tag_* key and to elide Kubernetes namespace, container and priority values. The regenerated corpus contains no selected context, stack, account, tenant, cluster, resource ID, ARN or identity-bearing tag key. Stable generic job/namespace values, EKS instance types, architecture/OS, and topology region/zone values remain as observed evidence.

The report-only end-to-end target exited 0 with 1,849 findings across four contradiction classes and six coverage-gap classes, including 982 PENDING stubs. Its evidence legend distinguishes gcx_live_readback as live-read-only EKS evidence from k3d_lab evidence.

CodeRabbit review completed with two Major and one Minor issue. Both Major issues were fixed with regression coverage: EKS cluster discovery now requires both an aws:// provider ID and an EKS kubelet-version marker, excluding self-managed AWS Kubernetes; job values are retained only from a strict generic allowlist and otherwise become presence-only. The Minor request to reformat suffix names inside the earlier append-only tracker note was dismissed because replacing prior Backlog notes is prohibited and has no implementation or evidence impact.

Second CodeRabbit review completed with two Major test-semantics suggestions and the same Minor tracker-formatting suggestion. The valid part was applied by asserting the complete sorted-distinct capture-volume count set. The suggestion to retain two differing region values was dismissed because the frozen v1alpha1 cumulative-merge contract requires a cross-run value change to become sticky values_elided rather than a finite union. The suggestion to retain a trusted job value beside a discarded deployment-specific observation was dismissed because that would falsely claim a complete finite value set; the mixed observation correctly makes the entire key presence-only. Test comments now pin both frozen semantics. The repeated Minor note remains non-actionable because Backlog notes are append-only.

2026-08-25 final evidence refresh: make signal-fidelity exited 0 with 1,849 findings: 867 contradictions across four classes and 982 coverage gaps across six classes, with 982 PENDING stubs. The required all-blueprint dry-run dump selected 26 blueprints and emitted a non-empty structural inventory containing 2,644 distinct metric names and 15 distinct profile types; the report comparison consumed the same all-blueprint inventory path. Final corpus privacy checks found zero identity-pattern matches, zero tag_* keys, and no retained values on identity-bearing label keys. The routed Gate lane was preflighted and dispatched at its required route; post-spawn route metadata unexposed by the client. No observable route conflict or substitution occurred.

Final Summary

Implemented an explicit, read-only gcx EKS/CloudWatch read-back and cumulative frozen-corpus merge, with deployment-identity elision, strict EKS discrimination, documented refresh workflow, and report provenance separating live EKS evidence from k3d evidence. Verified against the operator-selected live source (588 CloudWatch and 31 EKS Kubernetes contracts), focused tests/race/vet/docs checks, routed make gate, a 26-blueprint structural dump, privacy scans, and exact-SHA CI run 32875060990 at 501ee647db0f9f04c2186827ab254caada05d908 including hygiene and ci-success.