AI LabsbyThe Ops ToolboxOps Toolbox
All examples
Live demo

Enterprise RAG on Azure AI Foundry

Staff ask policy questions and get answers backed by approved documents.

Document Q&AEnterprise
Jump to demo

You're answering an employee policy question — responses should cite approved sources or say when evidence is thin.

Policy help desk

Answers cite approved sources or flag when evidence is thin.

Compliance analystEmployee policy question
Live
Indexed corpus5 documents
  • pol-001Remote Work Policy
  • pol-002Data Classification Standard
  • pol-003AI Acceptable Use
  • pol-004Incident Response SLA
  • pol-005Vendor Security Review
Scenario: External AI tools
Azure AI Search · Indexpol-003Internal
Index
corp-policy-v2
Query mode
Hybrid semantic
Top doc
AI Acceptable Use
Region
Australia East
Employee
E-10492 · Support

Employee question

Case studyArchitecture, governance, and how to adapt this pattern in a pilot

Business use case

Problem

Employees ask HR, security, and policy questions in chat tools that are not grounded in approved documents, creating compliance and liability risk.

Who benefits

  • Employees, fast answers with citations
  • Legal / compliance, retrieval only from indexed, approved corpora
  • IT, Azure AI Foundry as governed control plane for models + search

Success metrics

  • 30% deflection of L1 policy questions without opening a ticket
  • 100% of answers include source title for auditor review
  • Pilot on one corpus (e.g. HR) before multi-domain expansion

Solution

Retrieve-then-generate RAG: Azure AI Search returns top-k chunks; Azure OpenAI answers using only that context and cites sources. Seed script loads sample policies for local and demo environments.

Technical implementation

Stack

  • Azure OpenAI (chat completion) via openai Azure client
  • Azure AI Search, vector/keyword index with id, title, content
  • Fallback, in-memory seed documents when Search env vars are unset

Architecture

Retrieve-then-generate: search happens before the model sees the question, so answers stay tied to approved text.

How it runs
Drawing the flow…

Implementation highlights

  • searchKnowledge() in lib/demos/azure-search.ts unifies Search + fallback
  • System prompt enforces cite-only behaviour and “I don’t know” when context is thin
  • scripts/seed-azure-search.ts uploads lib/demos/seed-documents.ts to your index

Outcomes and learnings

  • Chunking and index schema matter more than model choice for answer quality
  • Always return source metadata alongside text for regulated use cases
  • Foundry links models, search, and monitoring under one governance story
Delivery playbookDiscovery → pilot → scale
  1. 1
    Discovery2–4 wks

    Catalogue approved document sets, classification labels, and retrieval quality benchmarks.

  2. 2
    Pilot6–8 wks

    Index one domain (e.g. HR policies) in AI Search; pilot with compliance reviewers in Foundry.

  3. 3
    Scaleongoing

    Expand indexes by business unit; add citation enforcement, DLP, and Foundry monitoring.

Where else this appliesEnterprise RAG on Foundry is the default when retrieval, models, and monitoring must live under one Microsoft governance story.

Regulated policy assistants

Compliance officers query HR and legal corpora with mandatory citations and audit logs in Azure.

Engineering runbooks

On-call engineers ask procedural questions grounded in internal wiki exports indexed in AI Search.

Healthcare administration

Staff-facing policy Q&A with PHI boundaries enforced outside the model via retrieval filters.

Public sector programs

Citizen-facing explainers tied to official program guides, not generative speculation.

Azure AI Search (or your existing Microsoft search investment) feeds Foundry chat; the same pattern extends to SharePoint-backed corpora and private endpoints.