AI Labs
All examples
Azure AI FoundryOpenAI

Enterprise RAG on Azure AI Foundry

Knowledge Q&A using Azure OpenAI and Azure AI Search over seeded policy documents.

RAGEnterprise

Target outcomes

  • Compliance reviewers expect cited sources on every answer
  • Deflection target: 30% of L1 policy questions without ticket

Initiative playbook

Typical delivery arc for this pattern in enterprise programs.

  1. 1
    Discovery2 to 4 wks

    Catalogue approved document sets, classification labels, and retrieval quality benchmarks.

  2. 2
    Pilot6 to 8 wks

    Index one domain (e.g. HR policies) in AI Search; pilot with compliance reviewers in Foundry.

  3. 3
    Scaleongoing

    Expand indexes by business unit; add citation enforcement, DLP, and Foundry monitoring.

Business use case

Problem

Employees ask HR, security, and policy questions in chat tools that are not grounded in approved documents, creating compliance and liability risk.

Who benefits

  • Employees, fast answers with citations
  • Legal / compliance, retrieval only from indexed, approved corpora
  • IT, Azure AI Foundry as governed control plane for models + search

Success metrics

  • 30% deflection of L1 policy questions without opening a ticket
  • 100% of answers include source title for auditor review
  • Pilot on one corpus (e.g. HR) before multi-domain expansion

Solution

Retrieve-then-generate RAG: Azure AI Search returns top-k chunks; Azure OpenAI answers using only that context and cites sources. Seed script loads sample policies for local and demo environments.

Technical implementation

Stack

  • Azure OpenAI (chat completion) via openai Azure client
  • Azure AI Search, vector/keyword index with id, title, content
  • Fallback, in-memory seed documents when Search env vars are unset

Architecture

Retrieve-then-generate: search happens before the model sees the question, so answers stay tied to approved text.

How it runs
Drawing the flow…

Implementation highlights

  • searchKnowledge() in lib/demos/azure-search.ts unifies Search + fallback
  • System prompt enforces cite-only behaviour and “I don’t know” when context is thin
  • scripts/seed-azure-search.ts uploads lib/demos/seed-documents.ts to your index

Outcomes and learnings

  • Chunking and index schema matter more than model choice for answer quality
  • Always return source metadata alongside text for regulated use cases
  • Foundry links models, search, and monitoring under one governance story

Where else this applies

Enterprise RAG on Foundry is the default when retrieval, models, and monitoring must live under one Microsoft governance story.

Regulated policy assistants

Compliance officers query HR and legal corpora with mandatory citations and audit logs in Azure.

Engineering runbooks

On-call engineers ask procedural questions grounded in internal wiki exports indexed in AI Search.

Healthcare administration

Staff-facing policy Q&A with PHI boundaries enforced outside the model via retrieval filters.

Public sector programs

Citizen-facing explainers tied to official program guides, not generative speculation.

Using this stack elsewhere

Azure AI Search (or your existing Microsoft search investment) feeds Foundry chat; the same pattern extends to SharePoint-backed corpora and private endpoints.

Live demo

The demo is the same code path described above, not a simplified mock UI. Add keys in .env.local when you are ready; the narrative and diagrams stand on their own without them.

Business

Ask something you would not want answered from memory, e.g. external AI tools or remote work. Good answers cite document titles; weak retrieval should say so.

Technical

Azure AI Search when configured, otherwise seeded policies; chat completion with cite-only instructions.

Azure RAG Q&A

Retrieval over policy documents via Azure AI Search (or local seed fallback).

Live