Description
Operators and solution engineers need one loadable blueprint that stands up a complete, generic AI Factory: mixed PCIe and NVL72 compute, an admin and a user Kubernetes cluster, the scheduler, storage, both fabrics, the NVLink and cluster-management planes, power and cooling, the gateway pool and platform automation. Designs for these estates use 60-second collection so every series stays at one data point per minute, and bottom-up sizing lands at roughly 1,000 to 1,200 active series per GPU all-in, with host CPU series (driven by core count), fabric per-interface counters and NVLink per-port telemetry outweighing DCGM itself.
Acceptance Criteria
- #1 blueprints/ai-factory.yaml declares every AI Factory construct with counts and rates whose real-world basis is stated beside them, and carries no customer-specific identifier
- #2 Collection cadence keeps every emitted series at or below one data point per minute
- #3 One consistent label model (site, fabric, rack, node, platform, cluster, GPU model, GPU UUID, storage cluster, scheduler, project, service role) is applied across metrics, logs and traces wherever the real source carries it
- #4 just dump inventory is diffed against the cited signals/ files, and the documented active-series estimate per GPU is checked against the emitted count
- #5 Rates, counts and cardinality in any shipped blueprint are plausible for a real deployment, with the real-world basis stated in a comment; nothing is raised to force emission
Definition of Done
- #1 just check (fmt-check, lint, gen-check, env-check, docs-check, test, race, hygiene, ui-check, compose-check, helm-test, lab-check, signal-fidelity)
- #2 just gen (only if a blueprint field, construct/workload config struct, or a skill under plugins/synthkit/skills/ changed)
- #3 just dump — inventory diffed against signals/
Implementation Notes
loop47 I21 admission under goal dependency semantics: all .04-.14 and .16-.19 now Done or Parked, so parked constructs omitted with explicit reasons in coverage matrix. Generic topology .01 and CPU automation .17 green-landed; no GPU/exporter/syslog/gateway source AC fabricated. Ownedblueprint+coverage transfer toI21. Actual stock/all-in series per GPU remains unproved; report actual emitted count, not1000-1200 target. Defaultscadence <=1DPM/rates unchanged unless justified. No live/credentials/dashboards.
loop47 I21 commission1 stopped before edits, clean c2118af, 0implementation/1evidence-pathsetup retry. Actual public CLI observed12628distinctseries: compute-user10249unadaptedK8s, pilot2369, apps10; vendorGPU0. k8s_monitoring.enabledfalse doesNOTgatebaseemission; rootowned priorcomment/assumption wrong, now corrective cluster.emit admission boundary in isolated root repair. Ownership of blueprint/coverage returnsroot afterterminalnochanges. Nativeagent complex-worker Solmedium verified despite child self-labelledrescue-sol; no phantom worker/rescue attempts charged. SeparateAC4all-inGPUseries validation still unavailable underparkedvendorconsumers; no partialACwaiver/taskDone. Resume after owned root boundarygreenland plus explicitsource-complete sizing or owner-approvedpartialacceptance decision. Evidence codex/scratch/loop47/I21/attempt1-return.md.
loop48 admission cutoff: reserve never admitted, implementation0. DCGM did not land; it was parked in-loop after admitted implementation2 and independent owner/seam diagnosis. Syslog, HGX and gateway landed; driver logs and operators parked on the DCGM dependency. Resume only after DCGM source-correct constraint/clock semantics and applicable green implementation proof, then in-loop landed/parked consumer conditions and GPU adapter proof. compute-user.emit:false remains; no GPU reference-blueprint completion or sizing-AC amendment claimed.