Reference patterns
Examples
Worked patterns that back advisory engagements: business case, implementation, architecture, and demos when you have API keys. Not a product you buy off this site.
Patterns, metrics, and runnable demos for architecture reviews and pilots, from The Ops Toolbox.
25
Patterns
Across 4 stacks
Live
Demos
When keys configured
Pattern picker
Pick a profile close to your initiative. See representative examples and guides.
Policy or compliance answers that must cite approved sources.
Browse by pattern(Agents, RAG, governance, platform, discovery)
Agents & copilots
Streaming tools, escalation, and human oversight.
- Operational Copilot with Streaming Tools
Streaming agent on Vercel AI SDK with tool calling for CRM lookup and ticket creation.
- Multi-Model Routing with Vercel AI Gateway
Compare latency and responses across models through a single gateway endpoint.
- Intent Routing on Amazon Bedrock
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
- Human-in-the-Loop Approval Gates
Agent proposes CRM writes and refunds; supervisors approve or reject before anything executes.
- Policy-as-Code Gates Before Tool Calls
Deterministic rules block or route risky actions before an agent can write to systems of record.
Grounded Q&A
Retrieval, citations, and knowledge-bound answers.
- Enterprise RAG on Azure AI Foundry
Knowledge Q&A using Azure OpenAI and Azure AI Search over seeded policy documents.
- AWS-Native Q&A with Amazon Bedrock
Conversational demo using Bedrock Converse API with optional Knowledge Base retrieval.
- Policy Q&A with Seed RAG on Vercel
Keyword retrieval over in-app policy seeds, then grounded answers via the Vercel AI SDK.
- Production Observability for AI Routes
Structured telemetry on every request: trace id, latency breakdown, tokens, retrieval hits, and cost estimate.
- Ingestion, Chunking, and Deterministic Citations
Turn messy source text into stable chunks and citation ids you can carry into RAG prompts and audits.
Governance & safety
Safety scoring, orchestration, and policy gates.
- Responsible AI Gate with Azure Content Safety
Generate responses with Azure OpenAI while scoring prompts and outputs for harmful content categories.
- Durable Orchestration with Vercel Workflow
A pay-per-invocation durable workflow that orchestrates a multi-step transformation intake pipeline.
- Agent Orchestration on Azure AI Foundry
A governed orchestration pipeline: retrieve context, generate a grounded answer, then safety-score the output.
- Orchestrated Agents on AWS Bedrock
Uses Bedrock Agent Runtime when configured, otherwise runs a multi-step orchestration (context → intent → answer) via Converse.
- Policy Q&A with Seed RAG on Vercel
Keyword retrieval over in-app policy seeds, then grounded answers via the Vercel AI SDK.
- Prompt A/B Evaluation with Structured Rubrics
Compare two system prompts on the same user message, then score with a schema-constrained evaluator.
- Intent Routing on Amazon Bedrock
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
- Azure OpenAI Chat Baseline
Direct Azure OpenAI chat completion without retrieval, a control arm before RAG or orchestration.
- Batch Ticket Triage on Vercel
Classify multiple support snippets into category and priority with one structured model call.
- Human-in-the-Loop Approval Gates
Agent proposes CRM writes and refunds; supervisors approve or reject before anything executes.
- Production Observability for AI Routes
Structured telemetry on every request: trace id, latency breakdown, tokens, retrieval hits, and cost estimate.
- Ingestion, Chunking, and Deterministic Citations
Turn messy source text into stable chunks and citation ids you can carry into RAG prompts and audits.
- Policy-as-Code Gates Before Tool Calls
Deterministic rules block or route risky actions before an agent can write to systems of record.
- PII Redaction Pre-Processor
Redact obvious PII locally before prompts hit external model APIs.
- Conversation Replay + Regression Evals
Store golden conversations and score prompt changes before they ship.
- Incident Assistant: Severity → Runbook → Comms
A multi-step incident helper using seed runbooks: classify severity, pick runbook, draft first comms.
- Vendor Questionnaire Summariser (Security + AI)
Extract what risk teams actually need from vendor docs: retention, regions, subprocessors, red flags, and follow-ups.
- Token Budget Guardrails (Per Session)
Hard caps per session to prevent pilot spend surprises, block after budget is exhausted.
Platform & routing
Gateways, evals, workflows, and intent routing.
- Multi-Model Routing with Vercel AI Gateway
Compare latency and responses across models through a single gateway endpoint.
- Durable Orchestration with Vercel Workflow
A pay-per-invocation durable workflow that orchestrates a multi-step transformation intake pipeline.
- Agent Orchestration on Azure AI Foundry
A governed orchestration pipeline: retrieve context, generate a grounded answer, then safety-score the output.
- Orchestrated Agents on AWS Bedrock
Uses Bedrock Agent Runtime when configured, otherwise runs a multi-step orchestration (context → intent → answer) via Converse.
- Prompt A/B Evaluation with Structured Rubrics
Compare two system prompts on the same user message, then score with a schema-constrained evaluator.
- Intent Routing on Amazon Bedrock
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
- Batch Ticket Triage on Vercel
Classify multiple support snippets into category and priority with one structured model call.
- Production Observability for AI Routes
Structured telemetry on every request: trace id, latency breakdown, tokens, retrieval hits, and cost estimate.
- Conversation Replay + Regression Evals
Store golden conversations and score prompt changes before they ship.
- Incident Assistant: Severity → Runbook → Comms
A multi-step incident helper using seed runbooks: classify severity, pick runbook, draft first comms.
- Batch Clustering + Theme Labels
Group dozens of feedback lines into themes you can route to teams, without building a data platform.
Discovery & structure
Workshop extraction and initiative shaping.
- Transformation Discovery with Claude Structured Output
Extract initiative risks, stakeholders, and systems from unstructured text using schema-constrained generation.
- Meeting Notes to Action Items with Claude
Extract summaries, owners, and follow-ups from meeting notes using schema-constrained generation.
- Vendor Questionnaire Summariser (Security + AI)
Extract what risk teams actually need from vendor docs: retention, regions, subprocessors, red flags, and follow-ups.
Batch & routing
Classify and route many items without a chat UI.
- Intent Routing on Amazon Bedrock
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
- Batch Ticket Triage on Vercel
Classify multiple support snippets into category and priority with one structured model call.
- Batch Clustering + Theme Labels
Group dozens of feedback lines into themes you can route to teams, without building a data platform.
25 examples
Streaming agent on Vercel AI SDK with tool calling for CRM lookup and ticket creation.
Browser streams UI messages with sessionId to the route handler
Compare latency and responses across models through a single gateway endpoint.
Same prompt POSTed once to the compare route
Knowledge Q&A using Azure OpenAI and Azure AI Search over seeded policy documents.
Question sent to searchKnowledge (AI Search or local seed fallback)
Generate responses with Azure OpenAI while scoring prompts and outputs for harmful content categories.
Prompt analysed by Content Safety API (or heuristic fallback)
Conversational demo using Bedrock Converse API with optional Knowledge Base retrieval.
Question enriched with seed context snippets
Extract initiative risks, stakeholders, and systems from unstructured text using schema-constrained generation.
Discovery notes POSTed as plain text
A pay-per-invocation durable workflow that orchestrates a multi-step transformation intake pipeline.
POST starts a workflow run with start()
A governed orchestration pipeline: retrieve context, generate a grounded answer, then safety-score the output.
Retrieve top-k policy chunks (Search or seed fallback)
Uses Bedrock Agent Runtime when configured, otherwise runs a multi-step orchestration (context → intent → answer) via Converse.
If agent env is set: invoke Agent Runtime and stream completion
Keyword retrieval over in-app policy seeds, then grounded answers via the Vercel AI SDK.
Question triggers keyword search over SEED_DOCUMENTS
Compare two system prompts on the same user message, then score with a schema-constrained evaluator.
Same user message run against prompt A and prompt B in parallel
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
Converse classifies message into policy | technical | escalation | general
Direct Azure OpenAI chat completion without retrieval, a control arm before RAG or orchestration.
User message sent to chat.completions with baseline system prompt
Classify multiple support snippets into category and priority with one structured model call.
Client sends array of ticket or comment strings
Extract summaries, owners, and follow-ups from meeting notes using schema-constrained generation.
Raw meeting notes POSTed as text
Agent proposes CRM writes and refunds; supervisors approve or reject before anything executes.
Scenario analysed with generateObject for action proposal
Structured telemetry on every request: trace id, latency breakdown, tokens, retrieval hits, and cost estimate.
Optional seed retrieval timed separately from model call
Turn messy source text into stable chunks and citation ids you can carry into RAG prompts and audits.
Client posts raw text
Deterministic rules block or route risky actions before an agent can write to systems of record.
User proposes an action type and context (tier, amount)
Redact obvious PII locally before prompts hit external model APIs.
Client posts text
Store golden conversations and score prompt changes before they ship.
User supplies a system prompt
A multi-step incident helper using seed runbooks: classify severity, pick runbook, draft first comms.
Incident text posted to server
Extract what risk teams actually need from vendor docs: retention, regions, subprocessors, red flags, and follow-ups.
Vendor pack excerpt posted as text
Group dozens of feedback lines into themes you can route to teams, without building a data platform.
Lines submitted as batch items
Hard caps per session to prevent pilot spend surprises, block after budget is exhausted.
Request includes sessionId and prompt