Description
Model-level metrics are usually owned by workload teams rather than the AI Factory platform team, but demos and workload-team stacks need inference serving on the same GPUs: request rate, queue time, time to first token, tokens per second, KV-cache usage and batch size. synthkit’s gen_ai constructs model hosted APIs and gateways, not self-hosted serving engines on GPUs.
Acceptance Criteria
- #1 Real engine metric families (at least vLLM; Triton or NIM where documented) are emitted with provenance, after checking signals/genai.md for overlap
- #2 Serving pods are scheduled through the scheduler construct, so DCGM utilisation and memory follow load on the same GPUs
- #3 Failure modes cover KV-cache exhaustion with rising queue time and a GPU fault taking a replica out
- #4 Every metric, label, attribute and log field is sourced from vendor documentation or a capture at a pinned version and recorded with provenance in the named signals/ area file; anything unconfirmed is a PENDING in cantfind.md, never an invented name
- #5 Rates, counts and cardinality in any shipped blueprint are plausible for a real deployment, with the real-world basis stated in a comment; nothing is raised to force emission
Definition of Done
- #1 just check (fmt-check, lint, gen-check, env-check, docs-check, test, race, hygiene, ui-check, compose-check, helm-test, lab-check, signal-fidelity)
- #2 just gen (only if a blueprint field, construct/workload config struct, or a skill under plugins/synthkit/skills/ changed)
- #3 just dump — inventory diffed against signals/
Implementation Notes
loop47: admitted R19 serving-engine vendor-contract research; implementation waits for DCGM and parked Runai exporter contract
loop47: source research completed at vLLM v0.30.0/Triton v2.73.0. vLLM core roots are model/engine aggregates, KV usage fraction and rates derived from counters; no invented per-GPU engine rate. NIM contract, health/failure event and GPU-fault-to-replica-removal mappings remain PENDING. Implementation not admitted: fixture and Runai/VAST dependencies not landed; no AC ticks.
loop47 dependency park, implementation0: DCGM and Runai consumers .04/.12 unlanded/source-blocked. R19 pinned vLLM0.30.0/Triton2.73.0 names/units retained; no NIM, health/failure-event or GPU-fault-to-replica mapping inferred. Resume after required consumers and eligible serving source profile land.