Case studyArchitecture, governance, and how to adapt this pattern in a pilot
Business use case
Problem
Pilots ship without run logging, when something goes wrong, nobody can answer which model, which retrieval, or what it cost.
Who benefits
- Platform engineering, SLOs and dashboards per route
- FinOps, token spend visibility before finance asks
- Incident response, trace id ties user report to logs
Success metrics
- 100% of production AI routes emit structured telemetry
- P95 latency tracked per model and per retrieval mode
- Weekly review of empty-retrieval rate on RAG paths
Solution
Wrap generateText (with optional seed RAG) and return a telemetry object alongside the answer, pattern for OpenTelemetry export, log drains, or your observability stack.
Technical implementation
Stack
- AI SDK
generateTextwithusagetokens - searchSeedDocuments for retrieval slice timing
Architecture
Every request returns structured telemetry alongside the answer.
Outcomes and learnings
- Log retrieval empties separately from model errors, different fixes
- Cost estimate is indicative; wire real pricing tables in production
- Same shape works for agents, batch jobs, and workflows with step spans
Delivery playbookDiscovery → pilot → scale
- 1Discovery2–4 wks
Agree mandatory telemetry fields and retention with platform and risk.
- 2Pilot6–8 wks
Dashboard P95 latency, token spend, and empty-retrieval rate for one route.
- 3Scaleongoing
Export to OpenTelemetry; alert on safety flags and cost anomalies per tenant.
Where else this appliesObservability is what turns a demo into an operated service, finance, SRE, and risk all ask different questions from the same trace.
FinOps chargeback
Token and cost estimates per team, model, and feature flag.
Incident debugging
Support ties a bad answer to retrieval hits and model version within minutes.
RAG quality ops
Alert when empty-retrieval rate spikes after index or taxonomy changes.
Vendor routing reviews
Compare latency and spend when gateway routes change between models.
Emit structured JSON from every AI route; forward to your log drain, OpenTelemetry collector, or your observability stack with consistent traceId propagation.