Description
AI Factories run two Ethernet fabrics as separate telemetry domains: a frontend (north-south access) fabric and a backend (east-west, RoCE) fabric, both NVIDIA Spectrum-X switches on Cumulus Linux. Switches push OTLP to NetQ, NetQ re-exports OTLP to the collector, and direct switch OTLP export is a supplementary path for metrics NetQ does not export, so each metric has exactly one path. Telemetry covers interface state, speed, utilisation, errors, drops, discards, flaps, optics, routing state, switch CPU, memory, temperature, PSU, fan and service state, plus queue, buffer, congestion, PFC, ECN and pause counters on the backend. Switch syslog is shipped separately. synthkit already has a native OTLP metrics path for workloads but no fabric construct.
Acceptance Criteria
- #1 The construct emits native OTLP metrics with names and attributes from the Cumulus Linux OpenTelemetry and NetQ export documentation at a pinned release, documented in a new signals/ fabric area file; the NetQ re-export and direct switch paths are selectable and never both emit the same metric
- #2 Leaf and spine topology per fabric comes from the GPU fixture, with backend ports attached to GPU NICs in a rail-optimised layout and frontend ports to node and storage NICs
- #3 RoCE congestion counters (PFC, ECN, pause, buffer and queue occupancy) are emitted on the backend fabric only
- #4 Switch syslog is emitted through the shared syslog mechanic with real Cumulus message formats for link, routing-neighbour and platform events
- #5 Failure modes cover link flap, optic degradation, a congestion or PFC storm on one rail, switch PSU or fan failure, and a NetQ export outage that produces a telemetry gap while the switches stay healthy
- #6 Every metric, label, attribute and log field is sourced from vendor documentation or a capture at a pinned version and recorded with provenance in the named signals/ area file; anything unconfirmed is a PENDING in cantfind.md, never an invented name
- #7 Rates, counts and cardinality in any shipped blueprint are plausible for a real deployment, with the real-world basis stated in a comment; nothing is raised to force emission
Definition of Done
- #1 just check (fmt-check, lint, gen-check, env-check, docs-check, test, race, hygiene, ui-check, compose-check, helm-test, lab-check, signal-fidelity)
- #2 just gen (only if a blueprint field, construct/workload config struct, or a skill under plugins/synthkit/skills/ changed)
- #3 just dump — inventory diffed against signals/
Implementation Notes
loop47: admitted R08 vendor-contract research; implementation waits for fixture/syslog landing
loop47: two research passes found versioned direct-switch OTLP and NetQ ingestion/TSDB docs, but no complete required NetQ external OTLP re-export/path exclusivity/outage contract or required event formats. Resume with pinned authoritative NetQ/Cumulus output schema and event reference/capture. No direct-only or CLI-certificate-export substitute accepted. Implementation attempts 0; review-repairs 0; infra retries 0; no rejection.