Reference patterns
Examples
Worked patterns that back advisory engagements: business case, implementation, architecture, and demos when you have API keys. Not a product you buy off this site.
Patterns, metrics, and runnable demos for architecture reviews and pilots, from The Ops Toolbox.
25
Patterns
Across 4 stacks
Live
Demos
When keys configured
Pattern picker
Pick a profile close to your initiative. See representative examples and guides.
Policy or compliance answers that must cite approved sources.
Browse by pattern(Agents, RAG, governance, platform, discovery)
Agents & copilots
Streaming tools, escalation, and human oversight.
- Operational Copilot with Streaming Tools
Streaming agent on Vercel AI SDK with tool calling for CRM lookup and ticket creation.
- Multi-Model Routing with Vercel AI Gateway
Compare latency and responses across models through a single gateway endpoint.
- Intent Routing on Amazon Bedrock
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
- Human-in-the-Loop Approval Gates
Agent proposes CRM writes and refunds; supervisors approve or reject before anything executes.
- Policy-as-Code Gates Before Tool Calls
Deterministic rules block or route risky actions before an agent can write to systems of record.
Grounded Q&A
Retrieval, citations, and knowledge-bound answers.
- Enterprise RAG on Azure AI Foundry
Knowledge Q&A using Azure OpenAI and Azure AI Search over seeded policy documents.
- AWS-Native Q&A with Amazon Bedrock
Conversational demo using Bedrock Converse API with optional Knowledge Base retrieval.
- Policy Q&A with Seed RAG on Vercel
Keyword retrieval over in-app policy seeds, then grounded answers via the Vercel AI SDK.
- Production Observability for AI Routes
Structured telemetry on every request: trace id, latency breakdown, tokens, retrieval hits, and cost estimate.
- Ingestion, Chunking, and Deterministic Citations
Turn messy source text into stable chunks and citation ids you can carry into RAG prompts and audits.
Governance & safety
Safety scoring, orchestration, and policy gates.
- Responsible AI Gate with Azure Content Safety
Generate responses with Azure OpenAI while scoring prompts and outputs for harmful content categories.
- Durable Orchestration with Vercel Workflow
A pay-per-invocation durable workflow that orchestrates a multi-step transformation intake pipeline.
- Agent Orchestration on Azure AI Foundry
A governed orchestration pipeline: retrieve context, generate a grounded answer, then safety-score the output.
- Orchestrated Agents on AWS Bedrock
Uses Bedrock Agent Runtime when configured, otherwise runs a multi-step orchestration (context → intent → answer) via Converse.
- Policy Q&A with Seed RAG on Vercel
Keyword retrieval over in-app policy seeds, then grounded answers via the Vercel AI SDK.
- Prompt A/B Evaluation with Structured Rubrics
Compare two system prompts on the same user message, then score with a schema-constrained evaluator.
- Intent Routing on Amazon Bedrock
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
- Azure OpenAI Chat Baseline
Direct Azure OpenAI chat completion without retrieval, a control arm before RAG or orchestration.
- Batch Ticket Triage on Vercel
Classify multiple support snippets into category and priority with one structured model call.
- Human-in-the-Loop Approval Gates
Agent proposes CRM writes and refunds; supervisors approve or reject before anything executes.
- Production Observability for AI Routes
Structured telemetry on every request: trace id, latency breakdown, tokens, retrieval hits, and cost estimate.
- Ingestion, Chunking, and Deterministic Citations
Turn messy source text into stable chunks and citation ids you can carry into RAG prompts and audits.
- Policy-as-Code Gates Before Tool Calls
Deterministic rules block or route risky actions before an agent can write to systems of record.
- PII Redaction Pre-Processor
Redact obvious PII locally before prompts hit external model APIs.
- Conversation Replay + Regression Evals
Store golden conversations and score prompt changes before they ship.
- Incident Assistant: Severity → Runbook → Comms
A multi-step incident helper using seed runbooks: classify severity, pick runbook, draft first comms.
- Vendor Questionnaire Summariser (Security + AI)
Extract what risk teams actually need from vendor docs: retention, regions, subprocessors, red flags, and follow-ups.
- Token Budget Guardrails (Per Session)
Hard caps per session to prevent pilot spend surprises, block after budget is exhausted.
Platform & routing
Gateways, evals, workflows, and intent routing.
- Multi-Model Routing with Vercel AI Gateway
Compare latency and responses across models through a single gateway endpoint.
- Durable Orchestration with Vercel Workflow
A pay-per-invocation durable workflow that orchestrates a multi-step transformation intake pipeline.
- Agent Orchestration on Azure AI Foundry
A governed orchestration pipeline: retrieve context, generate a grounded answer, then safety-score the output.
- Orchestrated Agents on AWS Bedrock
Uses Bedrock Agent Runtime when configured, otherwise runs a multi-step orchestration (context → intent → answer) via Converse.
- Prompt A/B Evaluation with Structured Rubrics
Compare two system prompts on the same user message, then score with a schema-constrained evaluator.
- Intent Routing on Amazon Bedrock
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
- Batch Ticket Triage on Vercel
Classify multiple support snippets into category and priority with one structured model call.
- Production Observability for AI Routes
Structured telemetry on every request: trace id, latency breakdown, tokens, retrieval hits, and cost estimate.
- Conversation Replay + Regression Evals
Store golden conversations and score prompt changes before they ship.
- Incident Assistant: Severity → Runbook → Comms
A multi-step incident helper using seed runbooks: classify severity, pick runbook, draft first comms.
- Batch Clustering + Theme Labels
Group dozens of feedback lines into themes you can route to teams, without building a data platform.
Discovery & structure
Workshop extraction and initiative shaping.
- Transformation Discovery with Claude Structured Output
Extract initiative risks, stakeholders, and systems from unstructured text using schema-constrained generation.
- Meeting Notes to Action Items with Claude
Extract summaries, owners, and follow-ups from meeting notes using schema-constrained generation.
- Vendor Questionnaire Summariser (Security + AI)
Extract what risk teams actually need from vendor docs: retention, regions, subprocessors, red flags, and follow-ups.
Batch & routing
Classify and route many items without a chat UI.
- Intent Routing on Amazon Bedrock
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
- Batch Ticket Triage on Vercel
Classify multiple support snippets into category and priority with one structured model call.
- Batch Clustering + Theme Labels
Group dozens of feedback lines into themes you can route to teams, without building a data platform.
3 examples
Conversational demo using Bedrock Converse API with optional Knowledge Base retrieval.
Question enriched with seed context snippets
Uses Bedrock Agent Runtime when configured, otherwise runs a multi-step orchestration (context → intent → answer) via Converse.
If agent env is set: invoke Agent Runtime and stream completion
Classify user messages, then answer with intent-specific system prompts on Bedrock Converse.
Converse classifies message into policy | technical | escalation | general