AI security controls & practices

Traditional apps have deterministic logic; AI apps add probabilistic behaviour, untrusted natural language input, and tool calling side effects. Attackers target prompts, retrieved documents, and tool parameters per OWASP LLM Top 10.

Sponsors should expect the same accountability as any production system: named owners, audit trails, and incident response per NIST AI RMF Manage.

Shadow AI (personal accounts, unlogged browser plugins, keys in Slack) is a security programme issue. NIST AI RMF Govern champions should route teams to approved sandboxes.

This guide complements the AWS AI compliance: controls here, artefacts there. Pair with production readiness conversation.

Use this map in NIST AI RMF Govern intake and architecture reviews. Each domain needs an owner and evidence mapped in NIST AI RMF Playbook.

Mature programmes link domains to patterns: HITL for tools, deployment gates for deploys, telemetry for runtime.

Red-team exercises should rotate across domains quarterly per Anthropic jailbreak guidance, not only chat jailbreaks.

InfoSec reviews fail when controls exist in policy but not in the request path.

Bind every session to corporate identity (SSO, Entra ID (conditional access), Okta). Separate roles for end users, approvers, admins, and break-glass operators.

Retrieval indexes must enforce the same document ACLs as source systems. A user who cannot open a file in SharePoint must not see it in RAG (Azure RAG concepts) answers.

Service principals for model APIs per production readiness conversation. Short-lived tokens for tool calls with explicit read vs write scope.

Admin actions such as disable tools require MFA and immutable Bedrock logging audit trails.

API keys in repos are the fastest path to breach. Store secrets in a vault per production readiness conversation; rotate quarterly.

Environment separation: dev keys must not point at production vector indexes. CI needs secret scanning per OWASP LLM.

Never log full API keys. Redact tool arguments before SIEM export per data privacy.

Dual control for production prompt promotion via AI Gateway and eval CI.

Classify corpora before vector indexing. Restricted data excluded from Microsoft 365 Copilot and general apps.

Align retention with Microsoft Copilot data protection and legal holds. Document subprocessors per AWS AI compliance.

Metadata filters on every RAG query: tenant, region, classification, effective dates.

Ingestion pipelines need allow lists per Azure RAG solution guide and malware scanning.

Prompt injection (OpenAI mitigations) is when user or document text overrides system instructions (ignore previous rules, hidden instructions in PDFs). You cannot eliminate it with a single filter; layer controls.

Treat retrieved text as untrusted: delimit context per OpenAI safety best practices; prefer cite-only RAG Q&A.

Use Azure Prompt Shields and Anthropic jailbreak tests; still red-team your domain.

Log jailbreak patterns to SIEM. Spikes often precede incidents cited in MITRE ATLAS.

Tools are the highest-risk surface per OWASP LLM Top 10. Allow lists beat open function calling registries.

Validate tool arguments with schemas via AI SDK tools. Run tools through WAF and API auth like other internal APIs.

Human-in-the-loop before any customer-facing write. Global kill switch: disable agents without taking down read-only Q&A.

Pen-test tools directly. Chat UI tests miss parameter tampering on backend endpoints (agent vs workflow).

Filter inputs and outputs via Azure Content Safety and Bedrock Guardrails. Tune thresholds with legal.

Require citations for policy Q&A per Azure RAG solution guide. Refuse when retrieval is empty (Anthropic hallucinations).

Budget guardrails add latency; document in cost controls architecture diagrams.

Align Microsoft 365 Copilot and custom app rules via Copilot coexistence.

Security teams need visibility like SRE: who asked what, which tools fired, which RAG passages retrieved.

Ship logs to SIEM via Bedrock logging or Foundry monitor. Alert on override rate spikes.

Incident runbook: disable tools, human queue, preserve logs, notify legal, root cause with evals.

Post-incident reviews update golden sets and NIST AI RMF Playbook checklist.

Version prompts, chunking config, and embeddings like application code. PR review for tool schema changes and new data sources.

golden set (OpenAI evals) evals in CI on every prompt or index change. Staging must use production-equivalent safety defaults, not relaxed dev settings.

Model version pinning via AI Gateway. Dependency scanning for AI SDK containers.

Rollback plans for vector index rebuilds that shift RAG behaviour overnight.

Pilots can start lighter if stop rules (NIST AI RMF) and data class are honest. Production requires the full bar unless council approves a time-bound exception.

Document pilot vs production vs enterprise bars so champions align with NIST AI RMF and production readiness conversation.

Private endpoints and DLP are enterprise add-ons per AWS Well-Architected ML lens; plan before procurement.

Security should be a standing NIST AI RMF Govern agenda item, not a one-off gate before production readiness conversation.

NIST AI RMF Govern reviews exceptions (new tool, data class, vendor) with expiry. Champions report shadow AI via Copilot adoption paths.

Platform publishes a control checklist on every intake. Pair champions with security liaison fortnightly.

Celebrate near-miss reports; punishment drives shadow AI underground per Microsoft responsible AI.

When procurement or InfoSec asks for artefacts, use the AWS AI compliance: data-flow diagrams, IAM matrix, sample logs.

Map each control domain to one reference architecture patterns pattern plus production config evidence.

Honest amber gaps beat overclaiming. Reviewers remember credibility at ISO 42001 scale funding time.

Keep evidence per workflow: architecture, eval report, runbook drill, OWASP pen test.