Enterprise Security in the Agentic AI Era

Threat Modeling for AI

8 min read

Threat Modeling for AI

Threat modeling for AI extends classical design-time threat modeling to systems that include models and agents. The classical step asks how an attacker abuses a data flow or trust boundary; the AI extension adds boundaries and assets that traditional STRIDE-style modeling does not name: training data, model weights, the prompt and context window, retrieval corpora, tool and MCP connections, agent memory, and the agent’s autonomy itself. The discipline matters because the standard lists are awareness frameworks, not procedures: naming the risks is not the same as walking a specific architecture and deciding which controls it needs.

As of June 2026 three threat surfaces converge on any deployed agentic system. The LLM-legacy surface is the set of risks inherited from generative models (prompt injection, output handling, disclosure), cataloged in the OWASP LLM Top 10. The agentic-new surface is the set of risks that appear only once a model plans, holds memory, and acts through tools, cataloged in the OWASP ASI Top 10 and its T1–T17 reference model. The supply-chain-persistent surface (tampered models, poisoned packages, rug-pulled tools, hallucinated dependencies) predates AI but acquires AI-specific entry points. A useful threat model accounts for all three on the same architecture.

The taxonomy landscape, and when to use which

Six taxonomies are in active use, plus two structural tests and the wiki’s five-class expansion. They are complementary, not competing: each does a different job. The full cross-walk between them, and onto the control artifacts, is the Threat Taxonomy Reconciliation matrix; this section gives the selection logic.

  • STRIDE-AI is an elicitation method, not a catalog. Use it first, to walk an architecture and surface candidate threats against AI assets. It re-maps the six STRIDE categories onto training data, weights, the prompt/context window, and inference endpoints, and asks the analyst what an attacker does at each.
  • OWASP ASI Top 10 (ASI01–ASI10) is the consensus ranked risk list for agentic systems — the Rosetta Stone the other lists cross-map to. Use it to name and rank what elicitation surfaces.
  • OWASP T1–T17 is the reference threat model the ASI list ranks, and it carries six mitigation playbooks. Use it to move from a named risk to candidate controls.
  • OWASP LLM Top 10 is the non-agentic GenAI base. Use it for the model-layer risks an agentic system inherits regardless of its autonomy.
  • MITRE ATLAS is the adversary technique catalog, the attacker’s-eye view. Use it for detection engineering and red-team planning, where the question is which techniques will be used rather than which risks exist.
  • CSA MAESTRO is a layered decomposition (seven layers, Foundation Models through Agent Ecosystem). Use it when the threat-modeling conversation is architectural: which layer owns a boundary, where cross-layer movement is possible.

Two structural tests sit before enumeration. The Lethal Trifecta (private data access + untrusted content exposure + external communication) is a necessary condition for exfiltration at scale; the Lethal Bifecta (untrusted content + sensitive write capability) is the write-side analogue for damaging action. They are go/no-go checks applied at design time: if the architecture satisfies the condition, the risk is structural and one leg must be removed regardless of which catalog threat is in play.

The wiki’s five threat classes are the expansion beyond the published lists, covering what a peer reviewer raises that the standard taxonomies under-serve: the AI-aware insider, the long-running APT campaign, agent collusion, model-version degradation, and the jurisdictional adversary with regulatory leverage. They are cross-cutting adversary models rather than per-component risks, and three of them collapse to a single control: a continuously-executed, version-pinned eval harness over every artifact.

A method for the agentic case

A repeatable pass runs in five steps, design-time first and revisited at each model or tool change:

  1. Apply the structural tests. Does any agent hold the trifecta or the bifecta? If so, record the structural risk and the leg to remove before going further — it dominates the per-threat analysis.
  2. Elicit with STRIDE-AI. Walk the architecture asset by asset (identities, data stores, memory, prompt, tools, inter-agent channels) and surface candidate threats.
  3. Name and rank with the catalogs. Map each candidate to an ASI category and its T-code; pull the attacker techniques from ATLAS for the ones that need detection coverage.
  4. Check the five classes. Ask the CISO questions the lists miss: own engineers, slow adversaries, colluding agents, the next model version, regulatory cutoff.
  5. Map to controls. Follow each named threat through the reconciliation matrix to its RA plane and CMM domain, then sequence the controls by dependency (identity before egress and observability; policy decision before runtime enforcement).

Worked example — a multi-agent RAG assistant with MCP tools

Consider an enterprise assistant built as three coordinating agents: a planner that decomposes a user request, a retrieval agent that queries a RAG corpus of internal documents, and an executor that calls tools through MCP servers — a ticketing server, a database server, and an email server. Untrusted content enters through retrieved documents and through tool outputs; the corpus mixes documents with different access levels; each agent and each MCP server needs an identity.

Structural tests. The executor reads the private corpus (private data), processes retrieved documents and tool results an attacker can influence (untrusted content), and can send email (external communication): it holds the full trifecta, so exfiltration is structurally possible. The ticketing and database servers give it sensitive writes, so against the same untrusted content it also holds the bifecta. The model records that the email path is the trifecta leg to constrain (route it through a policy decision point or strip it from the executor) before any per-threat control is weighed.

Elicitation and naming. STRIDE-AI over the assets surfaces, and the catalogs name: a poisoned retrieved document that redirects the planner’s objective (ASI01 Goal Hijack / T6, indirect prompt injection); a document that writes a durable false fact into the retrieval agent’s memory (ASI06 Memory & Context Poisoning / T1, memory poisoning); the executor coerced into chaining MCP calls to exfiltrate (ASI02 Tool Misuse / T2, tool-abuse chains); a rug-pulled or typosquatted MCP server (ASI04 Supply Chain / T17, tool poisoning); a forged planner-to-executor instruction (ASI07 Insecure Inter-Agent Communication / T16, MCP protocol abuse); and a fault in one agent amplifying across the mesh (ASI08 Cascading Failures / T5).

The five classes. The RAG curator who silently swaps a high-trust source is the Class 1 insider; a planner and executor that coordinate to stay under a per-agent monitor are Class 3 collusion; a vendor minor-version bump that regresses the executor’s jailbreak resistance is Class 4. Class 5 applies if the underlying model is hosted in a jurisdiction subject to cutoff.

Controls, sequenced by dependency. Per-agent identity for all three agents and the MCP services comes first (CMM D2), because per-agent egress mediation and behavioral baselining are capped by it. A policy decision point outside the model context (D3) authorizes the executor’s actions and downgrades the trifecta on the email path. An agent-aware gateway (D5, RA Egress) brokers every MCP call, fingerprints tools against rug-pull, and enforces per-agent tool scope. On the data plane (D6), answer-time entitlement enforcement keeps the retrieval agent from returning documents above the user’s access level, and cognitive file integrity hashes the system prompts and high-trust sources. Observability (D7) baselines each agent’s tool-call sequence and wires drift to a kill switch. The dependency order — identity, then control, then egress and data, then observability — is the lesson the example teaches: strong guardrails on a weak policy decision point, or per-agent monitoring without per-agent identity, are inert.

Platform-level enforcement, not prompt-level

A threat model is only as good as the layer its controls live at. The recurring architectural error, documented in the standards gap analysis, is placing a control in the prompt — instructing the model not to exfiltrate, not to obey injected text — where the same untrusted input that carries the attack can countermand the instruction. The controls above belong at the framework and runtime layer: a policy decision point the model cannot edit, a gateway the agent cannot bypass, identity the agent cannot forge. Prompt-level measures reduce residual risk; they are never the enforcement boundary. This is the framing principle that decides whether a named threat is actually mitigated or only discouraged.

Connection to the RA and CMM

Every threat this page names lands on a plane of the AAI-S RA and a domain of the CMM through the reconciliation matrix. The RA shows where the control sits in an architecture; the CMM grades how mature the program’s implementation of it is. The threat model is the input to both: it is the enumeration the RA’s Threat-Control Matrix answers and the risk basis the CMM’s evidence requirements are calibrated against.

Where this is still thin

The worked example is one archetype; coding-agent and closed-corpus-chatbot archetypes would exercise different leg combinations. Class 3 (collusion) rests on controlled-lab evidence, not an attributed production incident. STRIDE-AI’s own efficacy rests on a single self-reported case study. These limits are tracked on the threat-classes page and the STRIDE-AI summary.

See also

Sources

Graph View

Backlinks