Description
Dashboards are how the use case is demonstrated: document throughput, first-pass approval rate, review queue states, grounding and readability scores, cost and latency per document type and section, and guard activity. Rules and alerts follow the touchline_* recorded-metric convention under the regulatory prefix. Gate everything on regulatory.
Acceptance Criteria
- #1 A regulatory dashboard in its own folder renders with data for every panel after the load generator has run, checked with a rendered snapshot
- #2 Recording rules and alerts exist under the regulatory prefix, with the alert-expression gotcha for new counter series handled
- #3 Knowledge Graph and Application Observability configuration for the regulatory services is gated on its own selection and does not touch Touchline’s
- #4 A regulatory dashboard renders with data for every panel after the load generator has run (rendered snapshot checked), in content capture on and off modes
- #5 Regulatory recording rules and alerts are added to the existing groups behind a per-rule use_case filter, with the new-counter-series gotcha handled; queries key on the service.namespace value doc-0003 fixes
- #6 Knowledge Graph trace config uses its own priority and no singleton (asserts stack, App Observability global) is duplicated
Definition of Done
- #1 just check
Implementation Plan
Implement the corrected design and frozen regulatory contracts, with disjoint file ownership for parallel work. Prove local runtime/rendered/mock-plan behaviour, integrate into just check, complete CodeRabbit review before landing code, and record live-only acceptance separately. No live deployment, image publication, new credential or CVE exception is authorized by this local implementation pass.
Resume under continued autonomous delivery authorization: add missing bounded document/section cost estimates with explicit unknown pricing, verify emitted telemetry and dashboard queries, then complete live acceptance against the recorded target when deployment prerequisites are established. Separate read-only consumer discovery proceeds in parallel; do not choose a new stack from defaults.
Implementation Notes
Local dashboard/templates/rules/KG complete: 11 panels, seven recording rules and three alerts, separate folder, shared groups filtered per item, priority9001 KG trace config, no duplicate singleton. Actual rendered PromQL/LogQL expressions parse in promtool; new-counter alert red/green proved initial increment detection. SDK cost is shown by agent only; document/section cost allocation remains unavailable and explicitly disclosed. Live data, snapshots in both capture modes and KG priority acceptance remain unverified.
Local templates and query validation reviewed; resume for document/section cost attribution and live populated snapshots in both capture modes after image publication and authorized deployment. Do not claim all panels have live data from local render proof.
Continuation closes local cost gap with app-derived per-generation USD estimates by document_type/section/agent_role/model_name and explicit unpriced-generation counts. Official AWS eu-west-1 price catalog verified exact default EU profile pricing. Model-edge/real-HTTP/OTel tests prove CSR2/ICF3 charged generations, no orchestrator double count, no stub cost, and unknowns remain unpriced. Promtool red/green reproduces new-series undercount1.1vs2 and validates the corrected branch order plus observed zero. Live acceptance is continuing against the configured consumer target.
Final continuation gate passed after preserving an upstream AWS SDK3.1154.0 update from origin/main. Eighteen regulatory runtime/experiment tests and12 mock-provider cases pass. CodeRabbit completed13-file coverage, reporting only two minor new-series query-order defects; both fixed and validated with actual Promtool sample tests, including initial-value plus later-increment and observed-zero cases. Remaining new-panel live acceptance is in progress.
Final Summary
Live dashboard acceptance completed against immutable source 7530badaaa684bc7690a27c090ef7ac102a27f82 and runtime source 80431c41ee435ad77f0aa6303b51437fa0d83aa8. Final snapshots in both capture modes show all 14 panels with a complete legible capture boundary, correct 0–100% ratio axes, visible sparse samples and no errors or No data. All 13 metric queries return finite series. Metadata-only operational request/grounding/readability/latency/cost/guard/SDK metrics were independently verified on the new off-mode pods; stored generation content is blank with usage retained. Online content-score history in the selected range belongs to capture-on and is not claimed as metadata-only content scoring. Retired-pod first-event counter proof passes actual Promtool red/green cases. Full local gate and exact-source CI 39535235286 passed; CodeRabbit covered the runtime/query batch, and the final declarative axis/point change was validated without another review. Teardown remains on the end-to-end task.