OWASP AI Exchange

The OWASP AI Exchange is an OWASP Flagship project and open-source living publication on AI security and privacy, edited through a public GitHub repository and published at owaspai.org.¹ The authors group draws on over 170 experts, and the published content runs to more than 300 pages of threats, controls, and guidance.¹ This page summarizes document 0 of the Exchange’s structure, “AI Security Overview” — the entry point that sets the organizing model, the threat and control catalogues, and the risk-analysis method the deeper documents build on. Six deep-dive documents sit behind this overview: general controls, input threats, development-time threats, runtime conventional-security threats, testing, and privacy. Document 4, runtime application security threats, is summarized in Document 4: runtime application security threats below, document 2, threats through use, in Document 2: threats through use, document 3’s development-time material in Document 3: development-time threats (§3.0–§3.2), and document 1’s general governance controls in Document 1: general governance controls (§1.1), general controls for sensitive data limitation in Document 1: general controls for sensitive data limitation (§1.2), and controls to limit unwanted behaviour in Document 1: controls to limit unwanted behaviour (§1.3) below. Coverage of document 2 on this page is its general input controls (§2.0), its evasion material (§2.1), its prompt-injection material (§2.2), its sensitive-data-disclosure and model-exfiltration material (§2.3–§2.4), its resource-exhaustion material (§2.5), and its culture-sensitive-alignment appendix. Coverage of document 1 is §1.1 (general governance controls), §1.2 (general controls for sensitive data limitation), and §1.3 (controls to limit unwanted behaviour). Coverage of document 3 is complete: §3.0 (general development-time controls), §3.1 (broad model poisoning development-time), and §3.2 (sensitive data leak development-time). Coverage of document 5 is its testing-strategies half: the three testing strategies it distinguishes, its threats-to-test-for taxonomy, its general eight-step approach, its agentic-AI testing section, and its two named test procedures for prompt injection and evasion. Document 5’s tool catalogue and the whole of document 6, privacy, are not read.

On this page

• Scope
• Standards liaison
• Organizing model: G.U.A.R.D.
• The AI Security Matrix
• The periodic table of AI security
• Control catalogue
• Risk-analysis method
• Agentic AI treatment
• Document 1: general governance controls (§1.1)
• Document 1: general controls for sensitive data limitation (§1.2)
• Document 1: controls to limit unwanted behaviour (§1.3)
• Document 2: threats through use
• Document 3: development-time threats (§3.0–§3.2)
• Document 4: runtime application security threats
• Document 5: AI security testing (testing strategies)
• Adoption evidence
• Limits

Scope

The Exchange covers all AI: agentic AI, analytical AI, discriminative AI, generative AI, and heuristic (rule-based) systems, plus non-AI data-centric systems — data warehousing, business intelligence, reporting — for the data-poisoning and data-supply-chain material that applies to both.¹ Security carries a narrow definition: preventing unauthorized access, use, disclosure, disruption, modification, or destruction of AI-specific assets, where modification includes manipulating model behavior in unwanted ways.¹

The Exchange sets its own scope boundary: AI security equals threats to AI-specific assets, which the Exchange covers, plus threats to other assets, which other resources cover.² Protection of conventional assets — encrypting a database, hardening a network — sits outside the Exchange’s content. The Exchange labels this category “conventional controls” and defers its detail to resources such as OpenCRE, providing only the AI-specific aspects of those controls (for example, that model-parameter protection can use a Trusted Execution Environment).³

Standards liaison

The Exchange states a formal liaison partnership feeding its content into four external standardization efforts: prEN 18282 (Cybersecurity for the EU AI Act, 70 pages contributed), ISO/IEC 27090 (AI security, 70 pages contributed), ISO/IEC 27091 (AI privacy, contribution stated without a page count), and OpenCRE / OpenCRE-Chat, which links the Exchange’s content to other AI security standards and guidelines through a shared requirements catalogue.¹ The Exchange also states complete alignment with the SANS Critical AI Security Guidelines through a formal, ongoing SANS collaboration; the Exchange’s Essentials section was built jointly with SANS.⁴

The Exchange states that this liaison work makes it represent “the consensus on AI security and privacy.”¹ That is the source’s own positioning claim, recorded here as attributed. This pass has not checked the liaison claim against the prEN 18282 or ISO/IEC 27090 draft texts themselves. Among the taxonomies this wiki tracks, the Exchange’s liaison claim outranks every other: neither the OWASP Top 10 for LLM Applications nor MITRE ATLAS claims a comparable role in drafting external standards.

Document 1’s CHECK COMPLIANCE control names the role behind that liaison claim: Rob van der Veer, the Exchange’s founder, is stated to be liaison officer for the official partnership between the Exchange and CEN/CENELEC/ISO, and co-editor for prEN 18282.⁵ The same passage carries the liaison work’s internal timeline, “at the moment of writing (Oct 2025)”: prEN 18282 in development for almost two years, expected to enter public enquiry in early 2026 and publish in 2026; ISO/IEC 27090 and ISO/IEC 27091 both in development, with the Exchange stating only that 27090 is “in development” at that date.⁵ The wiki’s own CMM standards crosswalk records ISO/IEC 27090 at FDIS in March 2026, a later development stage than the Exchange’s own Oct 2025 self-report — the wiki’s tracked position on 27090 post-dates the source’s, which is unremarkable for a standard in active drafting but worth recording given the source’s living-document claim, above, and the liaison relationship. The Oct 2025 stamp, together with the “as of end of 2023” heading on the same control’s jurisdictional survey (below), are the two internal date markers this page uses to date the source’s most recent substantive update.

Organizing model: G.U.A.R.D.

The Exchange organizes organization-level adoption of AI security into five steps, G.U.A.R.D.: Govern, Understand, Adapt, Reduce, Demonstrate.⁶

1. Govern — general AI governance: an inventory of where AI is applied, defined responsibilities, policy, impact assessment, compliance, and education. The Exchange frames this step as general AI management, not a security-only process.⁶
2. Understand — from the AI inventory, determine which threats apply using the risk-analysis decision tree, then build engineer and security-staff understanding of those threats and their controls. Most of the resulting controls are familiar conventional security countermeasures, except where the organization trains its own model. Controls split into ones the organization implements directly and ones its suppliers own, with the latter routed into supply-chain management.⁶
3. Adapt — extend the security program, threat modeling (cross-team, covering every engineering discipline), testing, supply-chain management, and secure development practice to cover AI-specific assets, threats, and controls. New regulation such as the EU AI Act tends toward outcome-based requirements, in contrast to the control-focused structure of a standard such as ISO/IEC 27001; the Information Security Management System therefore needs assurance processes that demonstrate risks were sufficiently mitigated, beyond showing that controls exist.⁶
4. Reduce — minimize or obfuscate sensitive data and limit the impact of unwanted behavior through privilege management, guardrails, and human oversight. The stated rationale is that AI models can always be wrong or manipulated.⁶
5. Demonstrate — produce evidence of responsible AI security through transparency, testing, documentation, and communication to management, regulators, and clients.⁶

G.U.A.R.D. closes on a position about AI’s security ceiling: securing AI is harder than securing non-AI systems because an LLM exposes an unstable, undocumented interface with an unclear policy set, so security measures applied to AI often cannot reach the assurance level available for other software.⁶ The stated implication is to evaluate whether AI is the right technology for the problem before building it, because removing an unnecessary AI component removes every AI-related risk that component would have carried.⁶

The AI Security Matrix

The Exchange distinguishes three threat types by lifecycle stage — development-time (data obtained and prepared, model trained or obtained), input threats (attackers using the model through inference), and other runtime threats (attacks on the system in operation, not through inference) — and six impacts mapped to three attacker goals: disclose (confidentiality of training/test data, model IP, or input/augmentation data), deceive (integrity of model behavior), and disrupt (availability of the model, or CIA of non-AI-specific assets).² Some attacks chain several of these threats in sequence: hacking a training database, poisoning samples, then presenting crafted inputs once the poisoned model is deployed is one attack expressed as three threat entries.²

The AI Security Matrix organizes this threat catalogue on three axes: asset and impact, attack surface with lifecycle stage, and threat/risk category.⁷ The published table carries eighteen threat/risk-category rows, counted directly from the source table:⁷

Asset & impact	Attack surface (lifecycle)	Threat/risk category
Model behaviour integrity	Runtime — model use	Direct prompt injection
Model behaviour integrity	Runtime — model use	Indirect prompt injection
Model behaviour integrity	Runtime — model use	Evasion (e.g., adversarial examples)
Model behaviour integrity	Runtime — break into deployed model	Model poisoning at runtime (reprogramming)
Model behaviour integrity	Development — engineering environment	Direct development-environment model poisoning
Model behaviour integrity	Development — engineering environment	Data poisoning of train/finetune data
Model behaviour integrity	Development — supply chain	Supply-chain model poisoning
Training data confidentiality	Runtime — model use	Disclosure in output
Training data confidentiality	Runtime — model use	membership inference
Training data confidentiality	Development — engineering environment	Development-time data leak
Model confidentiality	Runtime — model use	Model exfiltration (input-output harvesting)
Model confidentiality	Runtime — break into deployed model	Direct runtime model leak
Model confidentiality	Development — engineering environment	Direct development-time model leak
Model behaviour availability	Model use	AI resource exhaustion
Model input data confidentiality	Runtime — all IT	Input data leak
Any asset, CIA	Runtime — all IT	Output contains conventional injection
Any asset, CIA	Runtime — all IT	Generic runtime security threats
Any asset, CIA	Runtime — all IT	Generic development-environment and supply-chain threats

The periodic table of AI security

The periodic table reuses the same three axes and adds a fourth column: the controls that mitigate each threat/risk category, with deep links into the Exchange’s detailed coverage.⁸ General governance controls apply across every row rather than attaching to a single threat category.⁸ Where the matrix answers which threats apply to a given asset and attack surface, the periodic table answers which controls to check once a threat is judged relevant — the Exchange’s own bridge from threat catalogue to control selection.

Control catalogue

The Exchange groups its controls into four essential categories and eight essential sub-categories, linking to over 50 individual controls detailed in the deep-dive documents.⁴

Category	Sub-categories	Representative controls
Manage	Extend governance/risk/compliance; extend supply-chain management; extend conventional security controls (managing security, development-time, runtime)	AI PROGRAM, SUPPLY CHAIN MANAGE, MODEL ACCESS CONTROL
Have resilient models	Data/model engineering controls	MODEL ALIGNMENT, POISON ROBUST MODEL, EVASION ROBUST MODEL
Watch	Model I/O handling; monitoring	PROMPT INJECTION I/O HANDLING, RATE LIMIT, MONITOR USE
Limit	Minimize/obfuscate sensitive data; limit model behavior	DATA MINIMIZE, LEAST MODEL PRIVILEGE, OVERSIGHT

The full catalogue of named controls beyond the sections summarized on this page lives in the deep-dive documents. Document 3 is summarized in Document 3: development-time threats (§3.0–§3.2) below and document 5’s testing-strategies half in Document 5: AI security testing (testing strategies), which adds no control-catalogue entry because its two procedures carry Category: AI security test; document 5’s tool catalogue and document 6 are out of scope for this page. The Exchange attaches its own caution to the catalogue: many controls are expensive to implement and carry trade-offs against other AI properties that affect model correctness and normal operation, and controls that change the learning process or the training data distribution can have unintended downstream side effects.³ The Exchange advises weighing implementation cost against how effectively a control addresses the threat and the level of the associated risk, and prioritizing low-cost controls against the highest-risk threats first.⁹

Risk-analysis method

The Exchange’s ten-step risk-analysis method builds on ISO 31000 / ISO 23894 risk-framework structure, naming ISO/IEC 27005 as the security risk-management standard behind step evaluation:⁹

1. Identifying risks — walk the decision-tree threat model to determine which threats theoretically apply, how they could realistically happen, and what impact they would have.¹⁰
2. Evaluating risks — combine likelihood and impact; likelihood factors are drawn from named methods (attacker access opportunity from OWASP and FAIR, risk of getting caught from FAIR, capabilities/tools/budget from ISO/IEC 27005, OWASP, and FAIR, susceptibility from ISO/IEC 27005 and FAIR, motive from all three, number of potential attackers from OWASP, and incident data from ISO/IEC 27005).⁹
3. Risk treatment — mitigate, transfer, avoid, or accept.
4. Risk communication and monitoring — maintain a risk register.
5. Arrange responsibility — a shared-responsibility step: by default, the organization that builds and deploys the AI system owns each threat, but hosting, model, extension, and infrastructure providers take shares, arranged through a responsibility matrix. Where a third party will not disclose how it mitigates a risk, the remaining options are to accept the risk, self-mitigate it, or avoid it by not engaging that party.⁹
6. Verify external responsibilities — attain assurance that other organizations are handling the threats assigned to them, for example through audits.
7. Select controls — review the controls tied to each relevant threat and the controls that apply universally, weighing implementation cost against risk reduction.
8. Residual risk acceptance — accept the risk that remains once selected controls are implemented.
9. Further management of selected controls — continuous monitoring, documentation, reporting, and incident response.
10. Continuous risk assessment — repeat the process as threats and feedback evolve.

The Exchange’s sufficiency criterion is that an AI system is sufficiently secure when every identified risk has been treated, meaning transferred, avoided, or accepted.⁹ Controls carry quality parameters — noise levels, anomaly-detection thresholds — that need continuous tuning from simulation results and production feedback rather than a one-time setting.⁹

Agentic AI treatment

The Exchange declines to treat agentic AI as a separate threat landscape: its own text states that the agentic section “highlights agentic attention points only — not a separate threat landscape,” and folds agentic systems into the same AI Security Matrix and periodic table used for every other AI system.¹¹ This is a substantive organizing choice, and it differs from how other taxonomies this wiki tracks structure agentic content: CSA MAESTRO decomposes an agentic system into seven dedicated layers, and the OWASP Top 10 for Agentic Applications publishes ten agentic-specific risk categories as their own taxonomy. The Exchange instead treats agentic behavior as a set of properties that shift which cells of the general matrix apply and how severely.

Agency is presented as a spectrum, with capability, autonomy, and impact increasing across four named points:¹¹

• Mildly agentic — performs actions
• Fairly agentic — also plans
• Quite agentic — also runs in loops
• Very agentic — all of the above without human intervention

Four typical properties determine which security concerns apply:¹¹

• Action — the agent invokes functions rather than only producing text. This raises LEAST MODEL PRIVILEGE to a key control, including an agentic authorization framework of deny-by-default access, infrastructure-level policy enforcement, task-bound tokens, and agent identity.
• Autonomous (optional) — agents trigger each other. That makes OVERSIGHT important and turns working memory into an attack vector, because agent state and plan live there.
• Complex (optional) — agentic behavior is emergent.
• Multi-system (optional) — developers tend to delegate access-control responsibility to the AI through instructions, opening a path for Prompt Injection. Agent-to-service and inter-agent calls need machine-to-machine MODEL ACCESS CONTROL: scoped tokens, session binding, and mutual authentication.

Several security consequences follow from these properties. Tool invocation raises the impact of a successful attack above what text output alone can cause, shifting risk toward goal drift, reward hacking, and runaway escalation, which makes blast-radius control central.¹¹ The Exchange states that security features belong outside the LLM, in the surrounding architecture, because of the model’s own robustness limits — using an agent to implement access control is named as a specific temptation to avoid.¹¹ Agents deployed with their own permission sets open a privilege-escalation path through the confused-deputy pattern.¹¹

Memory and context form three distinct attack surfaces: working memory (in-context), vector stores (subject to persistent poisoning), and cross-session stores (cross-session persistence). A write into persistent memory is framed as a future read attack against other agents; where memory functions as a de facto training-like data source, the Exchange cross-links it to data poisoning.¹¹ Document 4 supplies the control set behind these surfaces.¹²¹³

Jailbreak and agent escape are distinguished as separate failure modes. Jailbreak bypasses safety constraints within the agent’s own authorized boundary; agent escape exceeds that boundary by reaching unauthorized tools or scope. Multi-turn jailbreak calls for session-level OVERSIGHT; escape calls for infrastructure-level LEAST MODEL PRIVILEGE.¹¹ Document 4 states the layer model behind the distinction and warns that conflating the two produces controls only partially effective against each.¹⁴ Agent escape carries its own permalink, control set, and worked example. Deceptive reasoning — a plan that looks well-formed but rests on manipulated intermediate steps — is detected by comparing stated reasoning against actual tool calls, and the Exchange treats it as a consequence of prompt injection or memory manipulation rather than a separate attack class.¹¹ Goal hijacking is classified as an impact — redirected objectives — rather than a distinct threat with its own permalink; the attack vectors that produce it remain prompt injection and related paths.¹¹ The OWASP Top 10 for Agentic Applications ranks the same named thing as ASI01, its highest agentic threat category, and that page carries the contradiction callout recording the split.

Multi-agent systems add a further principle, no transitive trust between agents. Inter-agent security enforcement sits at the message bus or orchestrator layer, through infrastructure controls rather than instructions placed in agent prompts. The orchestrator is named as a high-value target, and monitoring is expected to cover collective agent behavior alongside individual agent actions.¹¹

The Exchange adopts the lethal trifecta, attributed to Simon Willison, as the condition for agentic data exfiltration: Data (attacker control over data that reaches an LLM during a session belonging to a user with the desired access, enabling indirect prompt injection), Access (the LLM or connected agents can reach sensitive data), and Send (the LLM or connected agents can initiate sending that data out).¹¹ The Exchange’s own wording extends the definition to “connected agents,” not only the single model handling the request. Prompt injection, mostly in its indirect form, is named the key threat across most agentic AI systems; after alignment, filtering, and detection are applied, the Exchange states that prompt injection should still be assumed possible, which makes blast-radius control critical rather than optional.¹¹

Four agentic security architecture principles are presented as design constraints rather than a separate control catalogue:¹¹

1. Enforce at infrastructure, not in prompts — access control, policy, and containment belong in the systems surrounding the model.
2. Design for compositional behavior — many tools combined with multi-step chaining produce a workflow space too large to fully pre-specify at design time, so runtime guardrails are required because emergent execution paths cannot all be enumerated in advance; where regulation assumes every workflow is describable before deployment, compositional agentic systems need explicit handling of that gap.
3. Assume cascade across layers — data, reasoning, tools, APIs, and peer agents form one compositional attack surface.
4. Bound blast radius by default — deny-by-default tool access, non-transferable sessions, and no transitive trust between agents limit harm once prompt injection or misalignment occurs.

The Exchange names its own validation gap: tools and methods to evaluate agentic security architecture and deployments are still evolving, which makes design and implementation weaknesses harder to find before production.¹¹

Principle 1 restates the wiki’s Oversight Layer architecture as a design constraint: its PDP/PEP layering keeps enforcement outside the model, and the Exchange’s own inter-agent position — enforce at the message bus or orchestrator rather than in agent prompts — is the same claim in the Exchange’s vocabulary. The LEAST MODEL PRIVILEGE agentic authorization framework named above (deny-by-default, infrastructure policy enforcement, task-bound tokens, agent identity) anchors the deny-by-default rung of the wiki’s CMM D3 Control & Least-Agency ladder, giving that rung a vendor-neutral reference alongside Microsoft ZT4AI. The same requirement states the wiki’s Least Agency Principle in the Exchange’s terms: actions in the prohibited tier are blocked at the platform and runtime layer, and a prompt instruction naming them is not an enforcement point.

Document 1: general governance controls (§1.1)

Document 1 opens on six governance controls, all carrying Category: governance control, grouped under a group-level entry carrying Category: group of controls (permalink /go/governancecontrols/).¹⁵ Like the periodic table’s general governance controls, these six apply across every threat category rather than attaching to one.⁸

AI PROGRAM is the entry-level control: install and execute a program to govern AI, framed as a governance measure that anchors security accountability rather than a security control in itself.¹⁶ Its Quickstart names an eight-step first iteration — board attention, a stakeholder group with assigned responsibilities, identification of applicable laws and regulations, a survey inventorying current AI use and ideas, evaluation of those applications, a risk analysis and first policy, implementation of that policy in tools and procedures, and an AI-literacy program — sitting above a narrower three-step bare-minimum start: inventory current AI use and ideas, run risk analysis to identify threats, controls, and responsible parties, then continue with GUARD step 2.¹⁶ Nine risk-management particularities of AI recur across the control: inductive rather than deductive reasoning, model staleness, data as both opportunity and risk, unfamiliarity producing over- and under-reliance, incomprehensible processing, new technical assets including AI documentation, natural-language interaction, sound and vision recognition, and autonomous action making permissions and oversight dynamic.¹⁶ The control states its own governance/engineering split: ISO/IEC 42001 extends risk management and governance, while ISO/IEC 5338 (behind DEV PROGRAM, below) extends software-lifecycle practice; 42001 does not cover how models are trained, data lineage, continuous validation, model versioning, project planning, or when sensitive data is used in engineering.¹⁶

SEC PROGRAM extends the organization’s information security management system to the whole AI lifecycle and its AI-specific aspects.¹⁷ Its asset list runs to twelve entries, including training, validation, and test data, the model and its hyperparameters, model documentation including experiment records, model input and output, augmentation data, intended model behaviour, and externally sourced training data and models.¹⁷ The control specifies a six-step AI Use Case Privacy and Security Analysis — describe the ecosystem, assess the system of interest, identify concerns, identify risks, identify controls, identify assurance concerns — run against confidentiality, integrity, and availability plus the privacy goals unlinkability, transparency, and intervenability.¹⁷ It also names AI-specific honeypots as a control of particular interest: fake parts of the data, model, or data-science infrastructure exposed on purpose to detect or capture an attacker before the real assets are reached, with eight worked examples including a hardened data service left with an unpatched vulnerability, an exposed data lake, and documentation “accidentally” exposed toward a honeypot.¹⁷ SEC PROGRAM states that serious incidents can trigger parallel regulatory obligations — EU AI Act Art 73 reporting and NIS2 notification timelines among them — coordinated with CHECK COMPLIANCE rather than handled inside monitoring controls.¹⁷

The control’s risk-analysis references cite ISO/IEC 27563:2023 (AI use cases security and privacy) as the source of a set of use-case figures: 132 use cases across 22 application domains, drawn from ISO/IEC TR 24030:2021, of which 11 carry a maximum security-concern rating and 49 a maximum privacy-concern rating.¹⁷ Thirty lines earlier, in the same control’s implementation text, “ISO/IEC TR 27562:2023” is cited for an overlapping discussion of the same security and privacy goals; the two designators differ by a single digit, both are dated 2023, and the source gives no way to resolve which is correct from this control alone. This page attaches the use-case figures to 27563:2023 only, on the source’s own description, and records the pairing as an unverified, unresolved source-side detail rather than as two distinct instruments.

SEC DEV PROGRAM requires secure-development processes so that security is built into the AI system, extending existing secure-development practice rather than standing up an isolated AI-specific framework.¹⁸ Its particularity list covers new engineering roles — data scientists, data engineers, AI engineers — folded into the general development program; new assets, threats, and controls entered into the information security management system and aligned to secure development; an AI supply chain extending to data, models, model hosting, and the capabilities agents interact with dynamically, such as skills and services through MCP; a development environment vulnerable to conventional software risks (open-source package vulnerabilities, CWEs, exposed secrets, sensitive-data leaks) that standard application-security testing tools do not catch without dedicated controls; AI-specific static-analysis and code-quality practice; and an automated-test-coverage figure of 43% industry average against an 80% recommendation, with a stated caution that AI engineering often neglects automated testing because model performance is mistaken for the ground truth of correctness.¹⁸ Its references name OWASP SAMM, NIST SSDF, and the NIST SSDF AI-practices profile.¹⁸

DEV PROGRAM extends general, not security-specific, software-engineering practice to AI development — automated testing, code quality, documentation, versioning — and states plainly not to isolate AI engineering as a separate discipline.¹⁹ ISO/IEC 5338 is the control’s named standard for this practice, covering the complete AI development lifecycle by extending ISO/IEC 12207 with new processes and AI-specific particularities, in contrast to ISO/IEC 42001’s governance-only scope.¹⁹ Related controls named under DEV PROGRAM are SEC DEV PROGRAM, SUPPLY CHAIN MANAGE, CONTINUOUS VALIDATION, and UNWANTED BIAS TESTING.¹⁹

CHECK COMPLIANCE requires AI-relevant laws and regulations to be accounted for in compliance management, with a stated blind-spot objective: legislation’s scope does not necessarily cover every organizational risk, and the EU AI Act specifically does not cover risks to protecting company secrets.⁵ Its nine-row AI regulatory mapping table connects EU AI Act articles (Art 5, 6, 10, 12, 13, 14, 25, 50, 73, and Art 3(23)) and NIS2’s 24-hour/72-hour/1-month notification timeline to an Exchange security hook where one exists, routing provider-conversion and substantial-modification obligations (Art 3(23), Art 25) to the compliance program with no stated Exchange security hook.⁵ Its jurisdictional survey — Canada, the US, Brazil, India, the EU, and China — is headed “as of end of 2023.”⁵ The control’s liaison-officer and prEN 18282 detail is carried under Standards liaison, above.

SEC EDUCATE requires education for AI engineers, development teams, and security professionals, in four steps: determine which threats apply from the AI-use inventory; build engineer and security-staff understanding of those threats and controls, noting that most of the resulting controls are conventional security countermeasures unless the organization trains its own model; use external courses and resources to support that understanding; and distinguish controls the organization must implement from those that are the supplier’s responsibility.²⁰

Document 1’s six governance controls carry twelve explicit Gap: verdicts against named standards, on a two-value scale. Ten grade a standard as covering the control fully — ISO/IEC 42001; the ISO 27000–27005 range; ISO/IEC 27005; ISO/IEC 23894; ISO/IEC 5338, twice; ISO 27002 8.25; ISO 27002 5.36; ISO 27002 6.3; and OpenCRE 616-305. Nine of the ten use the words “covers this control fully”; the tenth, ISO/IEC 5338 under SEC PROGRAM, reads “same as ISO 23894 above” and inherits that verdict. Two read “covers this control minimally” — ISO/IEC 27002 5.37 and OpenCRE 162-655 — and several of the twelve carry a qualifier such as “with said particularity” or “but lacks detail.”^16171819520 The scale grades a named standard’s coverage of one Exchange control; it supplies no organizational maturity criteria and grades no organization or implementation, consistent with the separate point recorded under Limits, below, that the Exchange scores no implementation of its own controls.

Document 1: general controls for sensitive data limitation (§1.2)

Document 1’s second section groups five controls under one thesis: reduce the impact of confidentiality and integrity threats by limiting the data attack surface — the amount and variety of data processed, and the duration it is kept (/go/datalimit/).²¹ Four of the five — DATA MINIMIZE, ALLOWED DATA, SHORT RETAIN, and DISCRETE — carry Category: development-time and runtime control; OBFUSCATE TRAINING DATA is the group’s only development-time AI engineer control.²¹

DATA MINIMIZE removes data fields or records unnecessary for the application, on the stated premise that data absent from the system cannot leak or be manipulated.²² It applies across data collection, preparation, training, evaluation, and runtime logging, and is particularly relevant where a dataset carries personal, confidential, or exposure-restricted information; an exception lets the deployer implement part of the control where better positioned, provided the provider communicates the requirement.²² Four implementation bullets follow: remove fields or records that do not materially affect model performance — correctness, robustness, fairness — based on experimentation or analysis; retain certain identifiers only to support data-removal requests or lifecycle management, excluded from training; update training datasets to reflect removals or corrections made in upstream source data; and preserve original data separately, under access control, for future use.²² Data that is not collected or retained cannot be leaked, reconstructed, or inferred from the system, which also reduces the consequences of dataset theft or unauthorized access.²² Its Particularity states that AI models often tolerate reduced feature sets and incomplete data better than traditional applications, which permits stronger minimization without functional loss.²² Its OpenCRE reference routes to an ENISA entry on respecting differential privacy and to NIST AI 100-2 §2.4.5 on differential privacy; ISO/IEC standards do not yet cover the control.²²

ALLOWED DATA removes data prohibited for the intended purpose, a case the Exchange singles out where consent was not given and the data holds personal information collected for a different purpose.²³ Its stated purpose, beyond compliance, is minimizing the impact of data leakage or manipulation.²³ The entry carries a Description, an Objective, and a References block, and no Implementation section. ISO/IEC 23894 A.8 Privacy is graded as covering the control fully, with a brief section on the idea.²³

SHORT RETAIN removes or anonymizes data once it is no longer needed, or when legally required, for instance under privacy law; exceptions apply where another rule requires keeping a record, such as proof of a transaction.²⁴ The Exchange states that limiting a retention period can be seen as a special form of data minimization.²⁴ Its OpenCRE reference routes to training-data retention management, and ISO/IEC standards do not yet cover the control.²⁴

OBFUSCATE TRAINING DATA attains a degree of obfuscation where exposure-restricted data cannot be removed entirely, applicable where that data is necessary for training, compliance, or risk mitigation, and carrying the same deployer exception as DATA MINIMIZE.²⁵ Five techniques are named. Private Aggregation of Teacher Ensembles (PATE) is a privacy-preserving technique: disjoint teacher models each train on a separate data subset, their predictions are aggregated with added noise, and a student model trains on that noised aggregate rather than on the sensitive data directly. Objective function perturbation is stated to be a differential privacy technique, adding noise calibrated to the objective function’s sensitivity and to a desired privacy level (epsilon) so the trained model does not exactly fit the original data. Masking alters or replaces sensitive features through tokenization, perturbation, generalization, or feature engineering. Encryption has two named models in machine learning: data held encrypted throughout for the data scientists, with a separate group of data engineers preparing and encrypting it beforehand; or data stored and communicated encrypted but used in original form when analyzed, a model the entry states must be combined with access control rather than relied on alone. Tokenization is stated to align with the principles of differential privacy.²⁵ Effectiveness is evaluated through attack testing or by relying on a formal privacy guarantee “such as differential privacy or an equivalent mathematical framework,” and obfuscation reduces the likelihood that training data can be reconstructed or linked back to individuals without eliminating that likelihood.²⁵ Its Particularity states that AI models typically do not require exact or human-readable representations of training data, which permits obfuscation techniques impractical in traditional systems.²⁵ Two limitations are stated: removing or obfuscating personal data is often insufficient, because identity can be induced from other retained data such as locations, times, visited websites, and timestamped activity; and token-based approaches add risk if their mapping tables are compromised.²⁵ The entry names K-anonymity, L-diversity, and T-closeness as statistical properties experts use to assess re-identification risk, states that anonymity is a statistical rather than an absolute concept, and names differential privacy as the framework for analyzing the level of anonymity a dataset retains.²⁵ It closes by naming two forward references that share its purpose of distorting training data for a different reason each: TRAINDATADISTORTION against data poisoning, and EVASIONROBUSTMODEL against evasion.²⁵ ISO/IEC standards do not yet cover the control.

DISCRETE minimizes access to technical details that could help an attacker select or tailor an attack.²⁶ Its implementation carries those details as an asset inside information security management, yielding asset management, data classification, awareness education, policy, and inclusion in risk analysis.²⁶ Three examples follow: weigh this risk before publishing technical articles about the AI system; where a choice exists, prefer a model type or implementation attackers are less familiar with; and minimize technical details in model output.²⁶ ISO 27002 Control 5.9, inventory of information and other associated assets, is graded as covering the control fully, with the particularity that technical data-science details can be sensitive; OpenCRE’s data-classification-and-handling entry carries the same verdict, marked “idem.”²⁶

DISCRETE pulls against AI TRANSPARENCY, described in §1.3 below, and the Exchange states the tension directly rather than resolving it: minimizing technical details that could help an attacker sits against the disclosure AI TRANSPARENCY asks for — five AI system properties the entry states such information can include rather than must carry, and one element it does state as a floor, that users are informed an AI model is involved at all.²⁷²⁶ The Exchange’s own resolution supplies a direction and no threshold: “The key is to minimize information that can help attackers while being transparent.”²⁶ It marks no line for where technical detail crosses from safe disclosure into attacker assistance.

§1.2 adds two Gap: verdicts on document 1’s two-value coverage scale — ISO/IEC 23894 A.8 for ALLOWED DATA and ISO 27002 Control 5.9 for DISCRETE, both “covers this control fully” — plus one verdict marked “idem,” repeating the DISCRETE verdict against the OpenCRE data-classification entry.²³²⁶ DATA MINIMIZE, SHORT RETAIN, and OBFUSCATE TRAINING DATA each state that ISO/IEC standards do not yet cover them, an absence rather than a verdict.²²²⁴²⁵

Document 1: controls to limit unwanted behaviour (§1.3)

The Exchange treats limiting unwanted behaviour as a category distinct from preventing or detecting it, and calls the result blast radius control. §1.3 groups seven controls under that heading: OVERSIGHT, LEAST MODEL PRIVILEGE, MODEL ALIGNMENT, AI TRANSPARENCY, CONTINUOUS VALIDATION, EXPLAINABILITY, and UNWANTED BIAS TESTING.²⁸ The section states that unwanted behaviour arises from causes beyond attack — insufficient or incorrect training data, model staleness and drift, mistakes in model and data engineering, and feedback loops where model output re-enters the training data of later models, which the Exchange names model collapse or recursive pollution — which makes control of that behaviour a shared responsibility rather than a security responsibility alone.²⁸

Mitigating unwanted behaviour creates two threats of its own. Overreliance is the model being trusted too much by users; excessive agency is the model being trusted too much by engineers, who grant it functionality, permissions, or autonomy beyond what the task requires.²⁸ The Exchange routes both to the OWASP Top 10 for LLM Applications — misinformation and over-reliance at LLM07, excessive agency at LLM03 — and states that overreliance applies to every model type rather than to generative models alone.²⁸ A control set that succeeds at bounding blast radius therefore raises the confidence placed in the bounded system, which is the mechanism by which the grant widens again.

OVERSIGHT is the section’s largest control: detection of, and response to, unwanted AI behaviour by automated mechanisms and by humans, filed as a runtime control and positioned as the final checkpoint against a model that can be wrong or manipulated.²⁹ LEAST MODEL PRIVILEGE is the section’s other large control: minimizing what a model can trigger or access, so a manipulated or mistaken model has less to work with, filed as a runtime information-security control.³⁰ LEAST MODEL PRIVILEGE restricts model actions through permissions and is therefore preventative, while OVERSIGHT is reactive or gate-based; the Exchange states both may apply to the same action tier, and nests OVERSIGHT as one clause of the overarching MONITOR USE monitoring control.²⁹ Oversight Layer and Least Agency Principle carry the two controls’ full specification, including the oversight-requirement axis §1.3 adds to the wiki’s existing risk-tier model and the approval-token mechanism carried on HITL. OVERSIGHT’s secure-orchestration guidance — coordination-only orchestrator permissions, a tamper-evident workflow log external to orchestrator memory, and reconciliation against it for phantom steps — is applied concretely to the Control and Egress planes in the Agentic AI Security Reference Architecture.

MODEL ALIGNMENT bakes wanted behaviour into the model itself through training, fine-tuning, and system prompts; the Exchange states it must be combined with deterministic external mechanisms — OVERSIGHT and LEAST MODEL PRIVILEGE among them — for high-risk or regulated use, since alignment alone is a probabilistic, model-internal control.³¹ AI TRANSPARENCY informs users of the AI system’s own properties so they can adjust how far they rely on it, what data they send it, and what further mitigations they apply. The entry names five such properties and states that the information can include them, which makes the list non-exhaustive rather than a required set: the rough working of the model, the training approach, the type of data used and its source, the expected accuracy and robustness of the system’s output, and any residual security risk.²⁷ One element is stated as a floor — the simplest form of transparency is informing users that an AI model is involved, which the entry notes the EU AI Act requires for chatbots — and the Exchange states the control is explicitly not explainability of individual decisions, which is a separate control.²⁷ CONTINUOUS VALIDATION tests model behaviour against a reference set to catch drift, staleness, and permanent manipulation such as data or model poisoning; its stated blind spot is that backdoor poisoning is designed to trigger on input outside normal test sets and is therefore often designed to pass validation tests.³² EXPLAINABILITY and UNWANTED BIAS TESTING close the section as its two stubs — explaining individual model decisions to build trust and catch overreliance, and testing for unwanted bias as a side effect of an attack on model behaviour rather than a security concern in itself — each a few lines with no Implementation or Limitations subsection.³³³⁴

§1.3 carries five Gap: verdicts against named standards, on the same two-value scale document 1 uses for its governance controls. OVERSIGHT names ISO/IEC 42001 B.9.3 for human oversight and autonomy decisions and grades it as covering the control partly — human oversight only, not business logic — then states that no further ISO/IEC standard covers it.²⁹ LEAST MODEL PRIVILEGE names ISO 27002 control 8.2 (privileged access rights) as covering the control fully, with the particularity that privileges assigned to autonomous model decisions are assigned with the risk of unwanted model behaviour in mind, and gives OpenCRE 368-633 the same verdict.³⁰ AI TRANSPARENCY names ISO/IEC 42001 B.7.2 and grades it as covering the control minimally, since it reaches only the data-management part.²⁷ CONTINUOUS VALIDATION names ISO 5338’s continuous-validation content as covering the control fully, and lists ISO/IEC 24029-2:2023, ISO/IEC 24027:2021, ISO/IEC 25059:2023, and CEN/CLC JT021008 without verdicts.³² Three of the seven controls — MODEL ALIGNMENT, EXPLAINABILITY, and UNWANTED BIAS TESTING — carry no standards block at all, which is an absence rather than a nil verdict.³¹³³³⁴

OVERSIGHT routes through two OpenCRE entries: automated AI oversight (205-522), referring to MITRE ATLAS AML.M0020 (Generative AI Guardrails) and NIST AI 100-2 §3.4.5 on an LLM moderator; and human AI oversight (202-604), referring to ATLAS AML.M0030 (Restrict AI Agent Tool Invocation on Untrusted Data), AML.M0029 (Human In-the-Loop for AI Agent Actions), and NIST AI 100-2 §3.4.5 on reinforcement learning from human feedback.²⁹ LEAST MODEL PRIVILEGE routes through OpenCRE 220-442 to ATLAS AML.M0028, AML.M0026, and AML.M0027, and to the ENISA Table 5 entry on applying an RBAC model under least privilege.³⁰ The wiki records ATLAS at v5.6.0 (April 2026); this page cites no other version number.

Document 2: threats through use

Document 2 covers threats an attacker reaches through inference — using the model rather than breaking into the system that runs it. Its general input controls, its evasion material, its prompt-injection material, its sensitive-data-disclosure and model-exfiltration material, its resource-exhaustion material, and its culture-sensitive-alignment appendix are summarized here.

General input controls (§2.0)

Document 2 opens on the input-threat group, which the Exchange also calls threats through use, inference-time attacks, and runtime adversarial attacks. The section designates six controls as generally applicable and defers the rest to each per-attack subsection: MONITOR USE, RATE LIMIT, MODEL ACCESS CONTROL, ANOMALOUS INPUT HANDLING, UNWANTED INPUT SERIES HANDLING, and OBSCURE CONFIDENCE.³⁵ Agent message structure manipulation appears in the group’s own enumeration alongside prompt injection, which places an agentic threat as a first-class member of the general input-threat set rather than an appendix to it.³⁵

The six are a default set adjusted per threat subtype rather than a fixed bundle. Two threat entries state an explicit subtract rule against the generic pointer to the evasion section’s controls: RATE LIMIT, UNWANTED INPUT SERIES HANDLING, and OBSCURE CONFIDENCE are excluded by name for transferability-based evasion and for evasion after data poisoning.³⁶³⁷ All three target a query-based search against the target model, and neither subtype requires querying the target: a transfer attack runs its search against a surrogate, and a poisoning-triggered backdoor needs no search at all. The exclusion is a statement about which controls do not apply to which threats, which the Exchange makes nowhere else in the range.

Four controls state limits that bound what a program may claim from them. MODEL ACCESS CONTROL writes under its own agentic limitations that authentication confirms identity and not intent, so a prompt-injected authenticated agent still passes its auth checks.³⁸ ANOMALOUS INPUT HANDLING detects statistically odd input without knowledge of what malicious input looks like, and states that this is often ineffective against evasion, because adversarial samples are designed to resemble normal input by definition; the same entry states that not all anomalous input is malicious and not all malicious input is anomalous.³⁹ That bound reaches the similarity pre-filter of a tiered detection cascade, whose recall against a crafted adversary is set by the detector’s mechanism rather than by its escalation threshold. OBSCURE CONFIDENCE states that it resembles gradient masking, a fragile defense when used alone, and that an attacker can approximate a withheld confidence indirectly by submitting similar inputs and observing whether outputs change.⁴⁰ UNWANTED INPUT SERIES HANDLING detects patterns across a series of inputs through clustering, pairwise similarity, and frequency analysis over a time window that is not limited to consecutive requests, and states two limitations: legitimate systematic testers and researchers resemble attack patterns, and an attacker can distribute inputs across multiple identities or sources specifically to reduce detectability.⁴¹ The wiki carries that second limitation on behavioral anomaly detection and on the prompt-volume-to-alert ratio, whose central inference it bounds.

MONITOR USE carries the range’s agentic logging set: tool calls, memory writes, reasoning traces, an agent action audit trail, tool chain monitoring, and multi-agent collusion signals, alongside an agentic incident lifecycle covering detection and triage, containment and eradication, and forensic analysis.⁴² Two of its statements reach past logging. Agentic containment must operate at the infrastructure layer and must not depend on the agent cooperating, and a defensive monitoring agent is itself part of the attack surface.⁴² The Exchange also qualifies the signal it recommends collecting: chain-of-thought and reasoning traces are model output rather than guaranteed internal computation, so forensic analysis reconstructs inputs, outputs, and actions rather than hidden intent.⁴² The wiki carries the consequence on Agent Observability and on the CMM D7 ladder.

RATE LIMIT separates its AI purpose from its conventional one. In conventional IT it protects performance; here it primarily mitigates threats that proceed through experimentation, delaying rather than preventing them.⁴³ The Exchange gives a worked figure — an attack needing 10,000 interactions at one per minute takes roughly 167 hours — and a query-volume table spanning zero queries against the target for a transfer attack up to thousands for zero-knowledge evasion and model inversion.⁴³ It names exceptions where the control blocks safety-critical or real-time function, listing emergency dispatch, cybersecurity monitoring, and fraud detection.⁴³ It also admits that low-frequency and single-try attacks including prompt injection remain unaffected, and that parallel access or multiple instances circumvent the limit.⁴³ Its agent fleet subsection sets per-agent and per-session caps on tool invocations, outbound API calls, and orchestration steps, tighter for agents processing untrusted content, with clean termination and an audit record when a hard limit is breached.⁴³ The caps land on Agent Availability Threats as defensive primitives.

MODEL ACCESS CONTROL’s agentic authentication subsection specifies cryptographic identity at channel and message level through mTLS and signed tokens rather than shared static keys, short-lived scoped tokens for agent-to-service calls, multi-agent trust tiers assigned by provenance under an explicit no-transitive-trust rule, inter-agent message signing with replay protection, delegation-chain context propagation, automated credential lifecycle, and session management bound to agent identity and task with no session-state carryover across task boundaries.³⁸ Behavioral trust and reputation scoring with decay and circuit breakers is offered as supplementary only, on the stated grounds that industry scoring methods are not yet standardised, and never as a substitute for identity, policy, or monitoring.³⁸ The session and mutual-authentication elements are graded at CMM D2 L4, and replay protection at D5 L3.

Evasion (§2.1)

Evasion is input crafted so the system performs its task incorrectly: the model behaves correctly on normal data and fails on the crafted sample.⁴⁴ The Exchange separates evasion from prompt injection by what the input manipulates, data in the first case and instructions in the second, and states the impact as the integrity of the model’s own task output.⁴⁴ Four worked examples carry the section: a perturbed traffic sign, a digitally perturbed image, a spam filter defeated by word choice, and an LLM-based content filter bypassed by altering a few words of an offensive message.⁴⁴ The last example places a guardrail classifier inside the threat’s scope, because a classifier is a model performing a task and can be made to perform it incorrectly. The Exchange states that attackers prefer small imperceptible perturbations, which are harder for a human or an automated detector to notice and are typically easier to produce.⁴⁴

Attacker knowledge splits the threat into five subtypes, three of which take the section’s control list unchanged.

Subtype	Attacker position	Control consequence
Zero-knowledge45	No code, training set, parameters, or architecture; query-based search, decision-based or score-based	Section controls unchanged
Perfect-knowledge46	Architecture, parameters, and weights known; gradient-based crafting such as FGSM	Section controls unchanged
Partial-knowledge47	Some system knowledge without gradients; combines surrogate transfer with zero-knowledge technique	Section controls unchanged
Transferability-based36	Adversarial samples built on a surrogate, applied to the target	Subtract the three query-search controls; add the model-theft controls
After data poisoning37	A trigger planted at training time; no search against the target	Subtract the three query-search controls; add the poisoning controls

Most of the taxonomy’s differentiation sits in the threat definitions, and the control assignments differentiate only twice. §2.0 above already records the subtract rule the last two rows carry, and the crosswalk carries its assessor consequence.

The perfect-knowledge entry states that attacks against certified defenses are possible, citing a paper on breaking them.⁴⁶ A published robustness certificate therefore bounds a stated threat model, and it says nothing about the deployed threat surface.

Five controls carry the section. EVASION INPUT HANDLING detects an adversarial sample at the single-sample level through statistical methods, detector networks run as preprocessing or in parallel to the main model, input-distortion comparison, and adversarial-patch detection; single-sample decisioning also catches a transferred attack that series-level analysis misses.⁴⁸ EVASION ROBUST MODEL selects model design, configuration, or training that reduces output sensitivity to small input changes, naming the IBM Adversarial Robustness Toolbox, CleverHans, and Foolbox as the assessment tools.⁴⁹ TRAIN ADVERSARIAL injects correctly labelled adversarial examples into the training set, Madry adversarial training with the PGD attack being the named instance.⁵⁰ INPUT DISTORTION adds noise, smoothing, or JPEG compression to disrupt a precisely crafted sample, and doubles as a detector by running the original and distorted input and treating a large output deviation as an evasion signal.⁵¹ ADVERSARIAL ROBUST DISTILLATION trains a student model on a teacher’s softened outputs to smooth decision boundaries.⁵²

Four of the five state a limit that bounds what a program may claim from them, and the pattern is sharper here than anywhere else in the range. EVASION INPUT HANDLING names its own detector as attackable: an adversarial sample may be crafted to evade both the primary model and a dedicated detector.⁴⁸ INPUT DISTORTION states that zero-knowledge attacks do not rely on gradients and are therefore unaffected by its shattered-gradient mechanism, that applying the model to distorted input reduces accuracy on regular data, and that the model must be retrained with the transformations in place.⁵¹ EVASION ROBUST MODEL recommends gradient masking while citing Athalye, Carlini and Wagner on obfuscated gradients giving a false sense of security, and leaves the tension unresolved in its body text.⁴⁹ TRAIN ADVERSARIAL states significant training overhead, poor scaling with model complexity and input dimension, a risk of overfitting, and weak generalization to new attack methods, and cites Tsipras et al. on robustness trading against accuracy.⁵⁰

ADVERSARIAL ROBUST DISTILLATION carries the pattern in its shortest form. Of the controls in the range, only this one carries no Objective, Applicability, Implementation, Risk-Reduction, or Limitations subsection, and its two primary references are the paper introducing defensive distillation and the paper refuting its robustness, placed one after the other.⁵² The Exchange flags the technique with a caution before any implementation detail, writing that security concerns have arisen about its effectiveness.⁵² A control entry that cites its own refutation supplies a name and a literature pointer without supplying a warrant, which is the distinction Security Controls for AI Stacks carries into its model-layer grade.

ANOMALOUS INPUT HANDLING, summarized under §2.0 above, states that it is often ineffective against evasion because adversarial samples are designed to resemble normal input by definition.³⁹ Read against this section, that admission is the general-control half of the same result. The Exchange files two of its five evasion controls as runtime controls and three as development-time controls, and both runtime controls bound themselves: EVASION INPUT HANDLING names its detector as attackable,⁴⁸ and INPUT DISTORTION exempts zero-knowledge evasion and requires retraining the model with the transformations in place.⁵¹ The general input control that sits alongside them at runtime is ANOMALOUS INPUT HANDLING, which calls itself often ineffective here. The three development-time controls presume a team that trains or fine-tunes the model.

Prompt injection (§2.2)

The Exchange treats prompt injection as a group with two classic members, direct and indirect, and three subtypes that appear once the model runs as an agent: in-context manipulation, stored injection, and multi-agent propagation.⁵³ The structural claim behind the grouping is that user data and system commands share one context plane with no parameterized-query equivalent, so the dominant agentic risk is integrity compromise through the agent’s tools and their side effects rather than a safety violation in generated text.⁵³ The Exchange states plainly that prompt injection is not solvable at the model layer alone, and that model-only defenses evaluated against fixed example attacks provide no guarantee against an adaptive adversary.⁵³

Direct prompt injection is described as social engineering of a generative AI, separated from evasion by what it manipulates — instructions rather than data — and enumerated across ten attack forms from role-play conditioning to training-data extraction.⁵⁴ Jailbreak is named as the subset aimed at defeating alignment, through competing objectives or through input that is out-of-distribution for alignment training and in-distribution for the base model.⁵⁴ In multi-turn agents the Exchange makes jailbreak a session-level property: constraints that hold on turn one degrade under incremental reframing, and an agent that refuses at turn one and complies at turn ten has been jailbroken across the session rather than at any single request.⁵⁴ Multimodal input is treated as an instruction channel in its own right, on the stated basis that models fuse visual and textual, and sometimes audio, embeddings into a shared latent space.⁵⁴ The section routes system prompt leakage to the OWASP Top 10 for LLM Applications, where the wiki’s verified category is LLM07:2025, System Prompt Leakage.⁵⁴⁵⁵

Indirect prompt injection is likened to remote code execution and named the dominant threat class for systems that retrieve external content, invoke tools, or share memory across sessions.⁵⁶ Five structural mitigations are named as a bundle: capping concurrent risk factors and otherwise requiring human oversight; privilege-based data flow control, which the Exchange identifies with CaMeL; instruction and data separation; paired input and output firewalls at the agent-to-tool boundary; and detection operating at text, model, and execution levels, of which execution-level detection is called often the most reliable.⁵³

The seven layers of protection are the Exchange’s defense-in-depth ordering for this threat, carried with their stated per-layer weaknesses on Prompt Injection Containment.⁵⁷ Two controls sit behind them: PROMPT INJECTION I/O HANDLING, which specifies Unicode normalization, locale-independent case-folding, confusable collapsing under Unicode UTS #39, and instruction-token escaping ahead of detection;⁵⁸ and INPUT SEGREGATION, which delimits untrusted data with hard-to-spoof markers and is stated to be a partial mitigation, because models may still attend to marked text and may not follow instructions to ignore it.⁵⁹ The wiki carries the second technique as sentinel tokens.

A commercial position runs against the Exchange’s detection-layer ranking. Cyera’s agent-security release, fetched 2026-08-31, argues that tooling watching “prompts, outputs, or individual tool calls” misses what data an agent can reach and whose permissions it inherits, and sells discovery and classification as the prior control (Cyera Agent Guardian Release). The Exchange ranks execution-level detection above the text and model layers for reliability, and the AAI-S reference architecture carries that ranking and records the disagreement as open. Cyera publishes no measurement on either side of it.

Agent message structure manipulation carries its own permalink and threat entry: forging, replaying, or altering the structured messages passing between agents, tools, and orchestration layers, so that task parameters, tool arguments, routing metadata, conversation state, or schema fields drive a downstream component into an unintended action.⁶⁰ The Exchange separates it from indirect prompt injection by target — the message fabric and its protocol fields rather than natural-language content — and states that it reaches single agentic flows as well as multi-agent systems.⁶⁰ The threat page carries the control set.

Two wiki positions sharpen against the subtypes above. Orchestration Hijacking models inter-agent compromise as an agent’s planner being subverted by adversarial content; the Exchange’s multi-agent propagation adds a privilege-mediated path in which the receiving planner stays intact and complies correctly, and the escalation comes from the privilege difference alone.⁵³ Multi-Agent Runtime Security enumerates the threat shapes that have no single-agent analogue, and message structure manipulation is the exception the Exchange states reaches single-agent tool loops.⁶⁰

Sensitive data disclosure and model exfiltration (§2.3–§2.4)

Sensitive data disclosure through use groups two threats against training-data confidentiality.⁶¹ The first is disclosure in model output: output may carry personal data, copyrighted text, or other sensitive material from the training set or from input including augmentation data, reached through ordinary use or through attacker provocation.⁶² The Exchange states the structural form of the problem — once training data is embedded in a model, the access-right variations that governed the original data cannot be controlled any more.⁶² An access model survives in a document store and does not survive absorption into weights, which sets the boundary where answer-time entitlement enforcement stops.

SENSITIVE OUTPUT HANDLING is the Exchange’s runtime control for it: detect exposure-restricted data — personal data, confidential identifiers, passwords, tokens — in output and filter, mask, stop, or log it before the output reaches a user or a downstream system, as a final safeguard behind access control and prompt-level instruction.⁶³ It specifies recitation detection against an indexed training-data set alongside pattern matching, and GenAI-based semantic detection to complement rules.⁶³ Its Limitations block states four: pattern matching misses sensitive data that matches no known pattern, false positives can cause serious system malfunction or block legitimate output, subtle or context-dependent disclosures are difficult to detect automatically, and an attacker can obfuscate output to defeat detection by base64-encoding a token.⁶³ A fifth limit sits in the entry’s risk-reduction guidance rather than that block: recitation checks reach only indexed data and may miss shorter or paraphrased disclosures.⁶³ The entry also states that instructing a model not to disclose data through a system prompt is insufficient on its own, because such instructions are bypassable through direct prompt injection.⁶³ That statement bounds a different control rather than this one, and it is the Exchange’s stated reason for placing the safeguard at output time.

The second threat is model inversion and membership inference. Inversion reconstructs part of the training set by optimizing input to maximize confidence indications in output; membership inference presents input identifying an entity and reads output confidence to infer that entity’s presence in the training set.⁶⁴ Overfitting raises the feasibility of both, and model capacity, model type, and regularization set how much a model overfits.⁶⁴ The Exchange cites MITRE ATLAS AML.T0024.001 and AML.T0024.000, OWASP Top 10 for ML ML03:2023 and ML04:2023, and NIST AI 100-2 §2.4.2 for the pair.⁶⁴ Its only threat-specific control is SMALL MODEL: keep the model small enough that it lacks the capacity to store detail at the level of individual training samples.⁶⁵ The entry also routes to four general input controls — MONITOR USE, RATE LIMIT, MODEL ACCESS CONTROL, and OBSCURE CONFIDENCE — and to sensitive data limitation, and it names the two model-leak threats as prerequisites worth controlling, on the stated grounds that both attacks are much more efficient with full access to model attributes.⁶⁴ The threat entry adds a model-family preference, recommending linear models or Naive Bayes classifiers over neural networks and decision trees where the risk applies.⁶⁴ SMALL MODEL runs a single sentence, the shortest control block in the range, with no Objective, Applicability, Implementation, or Limitations subsection, and states that ISO/IEC standards do not yet cover it.⁶⁵ The wiki’s differential privacy page carries the quantitative mitigation for the same threat, which this entry does not name.

Model exfiltration is the query-based route to a replica: an attacker collects input-output pairs through harvested logs, intercepted traffic, or large-scale input variation, and trains a new model that reproduces the original.⁶⁶ The Exchange names it model stealing, model extraction, model distillation, and model theft by use, and scopes the risk to models held as intellectual property or exposed to evasion risk, excluding models that are already public.⁶⁶ Its impact statement runs parallel to the one document 4 makes for the break-in route at §4.3: a replica lets an attacker search for input attacks offline, free of the rate limiting, access control, and detection the production system applies.⁶⁶⁶⁷ The two routes yield replicas of different fidelity, and the Exchange bounds the difference. It scopes the query-based replica to mainly evasion attacks, states that other input attacks need a much more detailed copy of the model than this form of theft typically achieves, and names direct development-time model leak and direct runtime model leak as the routes that can produce an exact copy.⁶⁶ An assessor reading the two entries together treats the break-in routes as the higher-fidelity theft and query-based exfiltration as the evasion-rehearsal route. The Exchange states that where an attacker can reach the model and the model allows intensive use, model exfiltration is typically hard to protect against, and that detection always requires further analysis because the same usage pattern can be benign.⁶⁶

A third impact claim in the same entry is attributed and unverified. The Exchange states that Anthropic claimed in February 2026 that exfiltration attacks by competitors could produce models able to generate harmful content against the intent of the original model makers.⁶⁶ The Exchange cites no publication for it. It is recorded here as the Exchange’s report of a vendor claim and carries an unverified mark.⁶⁸

MODEL WATERMARKING embeds a hidden marker in a trained model so that an owner can verify derivation if a suspected copy appears, for legal claims, contractual enforcement, and regulatory investigation.⁶⁹ The entry states its own objective as post-theft ownership verification rather than theft prevention, and requires that a technique survive fine-tuning and pruning without creating ownership ambiguity between claimants.⁶⁹ It then states that watermarking is effective evidence for direct model theft and limited for model exfiltration, because typical watermark markers sit in data that is not in-distribution for the queries an exfiltration attack sends; entangled watermarking is named as the technique that closes the gap.⁶⁹ The control is therefore filed under the threat it is weakest against, while the threats it is strongest against, direct runtime model leak and direct development-time model leak, hold their own permalinks in documents 3 and 4.

AI resource exhaustion (§2.5)

Specific input drives the model to consume funds or capacity beyond its budget. The Exchange names two attacker goals under one threat: depletion of funds, and unavailability of the AI system affecting dependent processes, organizations, and individuals.⁷⁰ It cites MITRE ATLAS AML.T0029 Denial of ML service, and names the sponge attack, also called an energy-latency attack, as input designed to increase model computation time.⁷⁰ The Exchange calls that form a denial-of-wallet attack and states it can also cause denial of service.⁷⁰ A financial-depletion goal filed under an availability impact means a program reading the matrix row as service degradation under-reads it. The same input costs money while the service continues to answer correctly.

Two controls are specific to the row. DOS INPUT VALIDATION rejects or corrects input that would trigger exhaustion, and extends the same discipline to LLM-generated tool arguments: size limits, rejection of path traversal and shell metacharacters, and per-tool rate limits against injection-driven tool-execution loops.⁷¹ LIMIT RESOURCES caps resource use for a single model input, and its agent subsection sets hard platform-level caps per agent or session across CPU time, memory, disk I/O, network egress, tool invocations, and wall-clock execution time.⁷² Containers, API gateways, or orchestration must enforce those caps, and the agent must not, which places the agent outside the trust boundary for its own resource governance.⁷² On breach the entry requires clean termination and an audit event; it also requires tighter tiers for low-trust and untrusted-content workloads, and fleet-wide consumption monitoring for correlated spikes and slow exhaustion attacks.⁷²

The section states the bound on what the second control buys. Resource limits bound cost and availability impact and do not prevent all harm within the allocated budget.⁷² A quota caps the bill and the outage, and the actions an agent takes below the ceiling are ungraded by it. Validation acts before the cost is incurred and the quota bounds cost already being incurred, which is why this wiki anchors the two in different domains: DOS INPUT VALIDATION at CMM D4 and LIMIT RESOURCES at D5. Both entries state that ISO/IEC standards do not yet cover them, and both record an ISO 27002 gap: DOS INPUT VALIDATION without qualification, LIMIT RESOURCES except for monitoring.⁷¹⁷² The caps land on Agent Availability Threats as defensive primitives.

Culture-sensitive alignment (appendix)

Document 2 closes on an appendix arguing that alignment carries jurisdiction-dependent nuance: topics acceptable for public discussion in one country are taboo in another, so the same output can be judged a successful prompt injection under one jurisdiction’s alignment scheme and compliant under another’s.⁷³ A comparison table scores thirteen countries and regions on whether their AI-governance documents carry explicit cultural or ideological sensitivity requirements, marking two as explicit, two as implied, and nine as carrying no cultural or political content constraint.⁷³ The appendix also treats semantic drift, the shift of word meanings over time, as an alignment maintenance problem: lengthening the system prompt is the named compensation, and the appendix states that emerging news and events cannot be addressed in real time by that mechanism, requiring continuously updated red-teaming instead.⁷³

Attachment surface for the alignment-scheme dependence

The appendix is a policy comparison rather than a control specification, and it is filed here without synthesis, on the same basis as the copyright section recorded under Limits. One sentence in it is security-load-bearing and has no page in this wiki: a guardrail verdict is relative to an alignment scheme, so a measured bypass rate for one region does not carry to another. The place that would carry it is CMM D4’s recorded data-class scope for the output classifier, which today records data classes and carries no jurisdiction dimension.

Document 3: development-time threats (§3.0–§3.2)

The development-time surface (§3.0)

Document 3 covers the engineering environment and the supply chain as attack surfaces, across data engineering, model engineering, and the tooling both run on. The section states seven particularities that separate an AI development environment from a conventional one: the data in it is real and typically sensitive, because a model must be trained on real data where a conventional development environment can hold fakes; its data, code, configuration and parameters are targets for behaviour manipulation, which the Exchange names poisoning; source code, configuration and parameters are critical intellectual property; the supply chain gains four new elements — data, model, AI components, model hosting, and abilities; external software components run inside the environment and can reach assets there, training data among them; AI software components run development-time as well as in production, which widens that reach to model parameters; and model development can cross trust boundaries through federated learning, merged PEFT modules, and model conversion services, which spreads training data and extends the attack surface at the same time.⁷⁴ The Exchange records ISO/IEC 42001 B.7.2 as briefly mentioning development-time data security risks.⁷⁴

Five controls answer the surface. DEV SECURITY extends the security management system over the AI-specific assets — training data, test data, model parameters, technical documentation — through encryption at rest, technical and centralized access control, operational security, and logging and monitoring for manipulation outside office hours, and it carries an integrity-checking programme across build, deploy and supply-chain stages.⁷⁵ SEGREGATE DATA partitions the environment into five areas — external, application development, data engineering, training, operational — so the training environment can carry controls against access from the less-protected areas around it.⁷⁶ CONF COMPUTE hides training data and model parameters from model engineers while in use.⁷⁷ FEDERATED LEARNING distributes a training set across organizations.⁷⁸ SUPPLY CHAIN MANAGE governs where data and models come from.⁷⁹

Model signing and dataset integrity (§3.0)

Two integrity problems in DEV SECURITY reach past the environment. Cryptographic model signing is named a significant step toward provable provenance and is stated to differ from SSL or Authenticode signing in a way that matters: a model is a set of associated artifacts in varying formats — tokenizers, vocab files, configs, inference code — all required for it to run, so any change to any of them can introduce malicious behaviour or degrade performance, and comprehensive verification must cover all of them. No standard exists for this at the time of writing; the Exchange names the OpenSSF Model Signing SIG as the body defining one, with possible interplay with ML-BOM and AI-BOM codified into the certificate.⁷⁵ The second problem is a dataset composed of pointers rather than content. The Exchange’s example is LAION-400M, whose entries are image URLs, and it states that data behind a URL is neither permanent nor beyond manipulation, citing Carlini et al., Poisoning Web-Scale Datasets is Practical; hashing dataset entries is the stated answer.⁷⁵

Supply chain management (§3.0)

SUPPLY CHAIN MANAGE states four ways an AI supply chain differs from a conventional one: four new supplied assets and services (data, models, model hosting, abilities, with models deliverable as fine-tuning artifacts such as LoRA modules); the supply chain including the organization’s own departments, which brings data provenance inside the control; new AI-specific development tooling; and some of that tooling executing development-time, which exposes training data and model parameters if the tooling is corrupted.⁷⁹ Supplied abilities are named as their own class — skills, MCP servers, plugins, external agents, and custom tool adapters — with the observation that third-party runtime services are not new to software and that agentic AI uses them at scale and for critical functions.⁷⁹ The control’s implementation runs to provenance records (origin and versioning of models and datasets including pre-trained lineage, checksums, training-data sources and augmentation steps, dependencies and environment requirements, ownership), lifecycle-aware update points, integrity verification, a per-agent bill of materials signed at origin and verified in the deployment pipeline with deployments rejected on mismatch, dependency-vulnerability management with severity-based remediation SLAs and named compensating controls where patching is infeasible, supplier evaluation across seven dimensions, pre-execution assessment of models from less trusted sources, and interface security for supplied services.⁷⁹ Its stated limits are that the control relies on the accuracy and completeness of records and attestations, that complex multi-party chains resist full traceability, and that trust decisions stay probabilistic rather than absolute.⁷⁹

Broad model poisoning (§3.1)

§3.1 groups three threats under manipulation of development elements to alter model behaviour: data poisoning, development-environment model poisoning, and supply-chain model poisoning, with the stated impact being integrity of model behaviour and its consequences failed fraud detection, safety-relevant decisions, reputational damage, and liability.⁸⁰ The section’s threat model places poisoning at five points: supplied data, a supplied model, the data-preparation domain, the training environment, and training data collected at runtime; and it states that a security-separated training environment makes controls against upstream poisoning implementable.⁸⁰ MODEL ENSEMBLE is the control specific to the group, deploying a model as several models over a randomly split training set so a deviating output identifies possible manipulation, at an effectiveness that falls as the poisoned share of the dataset rises.⁸¹

Data poisoning (§3.1.1) is defined by five routes — change in storage, change in transit, change at the supplier before the data is obtained, a model trained on poisoned data at the supplier, and manipulation of data entry in operation feeding into training — each also executable by an insider, and the Exchange states that the manipulated data can be training data or the in-context-learning data used to augment input.⁸² It splits the threat into targeted or backdoor poisoning, which fires on specific inputs, and sabotage, which degrades results for regular inputs, and states the detectability asymmetry between them: sabotage surfaces on ordinary traffic, while a backdoor evades review because a model holds no reviewable code, its parameters are unreadable to a human, and testing runs on normal cases the trigger avoids by design.⁸² Five controls answer it: MORE TRAIN DATA to outnumber poisoned samples, DATA QUALITY CONTROL to detect them, TRAIN DATA DISTORTION to corrupt inserted triggers, POISON ROBUST MODEL to reduce recall of poisoned data, and TRAIN ADVERSARIAL with adversarial robust distillation to improve generalization.⁸²

Direct development-time model poisoning (§3.1.2) manipulates the model itself rather than its training data — storage of model parameters, replacement of the model, command or code injection through custom or lambda layers, weight and architecture modification, and embedded deserialization attacks executing during unpacking or execution — with CONTINUOUS VALIDATION named as the performance-deviation detection control.⁸³ Supply-chain model poisoning (§3.1.3) covers a manipulated third-party pre-trained model obtained and further used or fine-tuned, which the Exchange names a transfer learning attack where the supplied model is trained on further.⁸⁴ Its control split is the part that carries: protection of model parameters at the moment of manipulation is outside the hands of the party that obtained the model, so that party holds the data-poisoning controls, POISON ROBUST MODEL, adversarial robust distillation, and supply chain management, and the supplier owes the rest.⁸⁴ That split is why the wiki’s CMM dependency-rules registry records §3.1.3 as an argument for treating a weak-D8-caps-D6 dependency as a soft cap rather than a hard one: a receiver with a compromised supplier still has a bounded, stated set of its own controls to raise integrity with, which is degradation rather than a ceiling.

Development-time leaks (§3.2)

§3.2 groups three threats whose impact is confidentiality rather than integrity, against the same engineering environment §3.1 attacks for behaviour (/go/devleak/). Development-time data leak (§3.2.1) is unauthorized access to train or test data through a leak of the development environment, and the Exchange states three routes: engineers working with real data, because a model must be trained on it; training data collected at runtime, which makes a live system an attack surface for a development-time threat; and cloud-hosted or externally managed GenAI models, to which company documents must travel to be trained or fine-tuned on.⁸⁵ Direct development-time model leak (§3.2.2) is unauthorized access to model attributes — parameters, weights, architecture — with insider access, compromised repositories, and weak storage controls named as routes.⁸⁶ Source code/configuration leak (§3.2.3) covers the code and configuration that preprocess the data and train the model, with model intellectual property as the breached asset.⁸⁷

The model-leak entry states the impact twice, and the second half reaches other sections. Beyond intellectual-property theft, a copied model lets an attacker run input attacks against it free of the rate limiting, access control, and detection the production system applies, which covers preparation for evasion and prompt injection as well as extraction.⁸⁶ That is the argument §2.4 makes for model exfiltration and §4.3 for direct runtime model leak, and §3.2.2 names both as the alternative ways of model theft, the fidelity difference §2.4 states above holding between them.⁸⁶ The section’s risk-identification test scopes the threat to a model held as a trade secret or exposed to any input-attack risk, and excludes a model that is already public because there is then no need to steal it — the same exclusion §2.4 states for exfiltration.⁸⁶

The control sets are near-identical across the three and add nothing to the catalogue: general controls with sensitive data limitation, then DEV SECURITY, SEGREGATE DATA for all three, CONF COMPUTE for the first two, and SUPPLY CHAIN MANAGE plus MODEL WATERMARKING for the model-leak entry alone.⁸⁵⁸⁶⁸⁷ §3.2 is therefore a threat-side section: it supplies the confidentiality half of the §3.0 group minus FEDERATED LEARNING, plus MODEL WATERMARKING, which this page already carries from §2.4 — so it adds no control the catalogue did not already carry, without being the same group §3.0 introduced.

Agentic development-time threats (§3.0)

§3.0 carries four agentic attention points and routes their mechanics back to §3.1 rather than duplicating them.⁷⁴ Malicious agent training — agent backdoors and biased fine-tuning — follows the data-poisoning, development-environment-poisoning and supply-chain paths, answered by reproducible pipelines, chain of custody, adversarial pre-deployment testing, monitoring during training for statistical anomalies in the training-data distribution, unexpected loss landscapes and behavioural triggers active only under specific input patterns, and approved weight checksums captured after training and compared before deploy and on update. Agent behaviour manipulation covers the post-training shaping surface — system prompts, RLHF datasets, reward functions, agent configuration — under version control, access restriction, runtime integrity checks against approved versions, separated author, review and deploy roles, and adversarial stage testing. Tool integration vulnerabilities cover MCP servers, skills, plugins and bespoke connectors, audited for hardcoded credentials, insecure deserialization, missing validation and leaky error handling, with credentials held in a secret manager rather than agent-accessible configuration. Planning system poisoning treats plan libraries, templates and heuristics as integrity-protected artefacts, with generated plans validated against policy before execution and planning decisions logged.⁷⁴

Document 4: runtime application security threats

Document 4 covers conventional security attacks that carry AI-specific consequences, distinct from the input threats of document 2 and the development-time threats of document 3. Its premise is additive: conventional attacks acquire AI-specific consequences — model IP theft, side-channel inference of model structure, exfiltration through generated output, vector-store exposure — that need AI-specific control aspects layered on the conventional baseline.⁸⁸ The document states that some of its controls are conventional security controls applied to AI-specific threats, and that ISO 27002 covers the technical and operational baseline only partly and at a high abstraction level.⁸⁸ It routes detail to OpenCRE, Common Criteria protection profiles (ISO/IEC 15408, evaluated per ISO 18045), and OWASP ASVS rather than duplicating general application-security material.⁸⁸

Model parameters and I/O at runtime

Direct runtime model poisoning alters model parameters, or the model’s input and output logic, inside the live system.⁸⁹ Direct runtime model leak steals parameters by breaking into the deployed system, and includes side-channel attacks that infer model structure, the type of data processed, or specific parameter values from timing, power consumption, or electromagnetic emissions during inference, without taking the whole model.⁶⁷ The Exchange names DeepSniffer as the side-channel example, and masking (random delays and noise during inference) and shielding (hardware electromagnetic and acoustic shielding) as countermeasures, with a Trusted Execution Environment isolating model operations inside a secure enclave.⁹⁰ MODEL OBFUSCATION stores the model with minimal technical information to slow an attacker who already has runtime storage access, which makes it a post-breach mitigation rather than a preventive control.⁹¹

The impact argument is the part that reaches beyond model IP. A stolen model lets an attacker rehearse input attacks — evasion, prompt injection — against their own copy, free of the rate limiting, access control, and detection the production system applies.⁶⁷ Model confidentiality therefore protects the effectiveness of unrelated input-attack defenses elsewhere in the framework. The Exchange carves out the case where the model is already public, since there is nothing to steal.⁶⁷

Data confidentiality at runtime

Input data leak is a confidentiality breach of prompts and other model input through a conventional attack on data at rest or in transit.⁹² The Exchange names five complicating factors: metadata linking a sensitive conversation to a user; cloud inference, where input is unencrypted at inference time and third parties may log it; stored input reachable by court order or subpoena; augmentation data that travels with the prompt and may surface in output; and external services invoked by model actions, which spread the input further.⁹² It states that opting out of third-party cloud AI monitoring can be necessary for security reasons, because that monitoring can itself expose sensitive data.⁸⁸

Direct augmentation data leak covers the retrieved documents and system prompts inserted into a prompt.⁹³ Two claims in this section are load-bearing for retrieval deployments. Augmentation data typically sits in a vector database, outside the source archive’s regular storage and protection, so it needs protection of its own. The vectors representing that data are themselves vulnerable to information extraction, which puts the embeddings inside the protection scope rather than only the documents they came from.⁹³ The Exchange states it is best to assume augmentation data can leak to output, and derives from that assumption the requirement that access rights on the data align with the rights of the users who can see the output.⁹³

Output as an injection and exfiltration surface

Model output may carry conventional injection attacks such as cross-site scripting, which become vulnerabilities when the output is rendered or executed downstream.⁹⁴ A manipulated model can emit JavaScript that runs and sends data to a third party. The Exchange also puts a definitional edge case in scope deliberately: packing sensitive data into URLs the user clicks, or into images a browser retrieves, is not injection under classic definitions, and it is included because the mechanism matches — data is executed by a web request.⁹⁴ The paired control is output encoding.⁹⁴

Augmentation data integrity and agent memory

Augmentation data manipulation is an integrity breach of vector stores, system-prompt storage, or agent working memory, producing manipulated behaviour without touching model weights.¹² The Exchange separates three surfaces requiring separate controls: in-context manipulation during an active session, usually an indirect prompt injection problem unless the session writes the content onward; persistent memory poisoning of vector stores, knowledge bases, and long-term memory retrieved across sessions, where content written by one agent or session may be retrieved by another, making a compromised write a future read attack; and cross-session state persistence, where adversarial content survives into the next session through shared memory or state channels even when the original injection arrived in-context.¹² The worked example is a multi-agent support system on a shared vector store, where a fabricated return policy submitted in a support request is summarized into the store and served to other customers until the entry is found and removed.¹²

AUGMENTATION DATA INTEGRITY specifies the control set: treat vector-store and shared agent-memory content as an untrusted external input surface carrying the same sanitisation and segregation obligations as user messages; record provenance per write (source, writer identity, timestamp, partition); verify integrity before an entry enters an agent’s active context and quarantine failures; restrict which agents and sessions may read from and write to specific memory partitions; sanitise session boundaries by reviewing and resetting agent context between tasks; log memory state changes append-only and immutably for incident replay; and hold planning artefacts — plan libraries, templates, heuristics — under integrity verification and policy validation before execution.¹³ The Exchange describes cross-agent memory access without explicit authorisation as lateral movement through shared state, and concedes that distinguishing legitimate memory updates from adversarial poisoning at scale remains difficult, naming structural access control and write authorization as the compensating approach.¹³ These controls land on the wiki’s memory poisoning and agent memory isolation pages and on CMM D6.

Agent escape and sandboxing

Document 4 gives agent escape its own threat entry: an autonomous agent operating outside its defined security boundary by invoking unauthorised tools, reaching systems outside its scope, or acting beyond its assigned task.¹⁴ The distinction from jailbreak sharpens the one document 0 draws. Jailbreak overrides safety constraints while staying inside the operational boundary and is a reasoning-layer problem; escape is a policy-enforcement problem at the infrastructure layer; and the Exchange states that conflating the two produces controls only partially effective against each.¹⁴ Infrastructure enforcement can block escape even where jailbreak succeeds at the reasoning layer.

The controls are capability-based access control at the backend restricting tool sets, data sources, and action space independently of LLM reasoning, and role and scope boundary enforcement at every tool invocation — where a valid call to an individually authorised tool made during an out-of-scope task is itself an escape event.¹⁴ OVERSIGHT covers session-level jailbreak drift, and red teaming must include multi-turn and multi-session paths, because single-turn jailbreak testing underestimates production agentic risk.¹⁴ Two limitations are stated: an agent with very broad authorised scope can cause harm through jailbreak without technically escaping, and capability enforcement may be difficult to retrofit where tool access was historically managed in prompts alone.¹⁴

Agent sandboxing is specified as a runtime operational control rather than pre-deployment testing: each live agent runs in a bounded environment across compute, memory, storage, network, and IPC, so compromise, malfunction, or prompt injection cannot spread past its boundary.⁹⁵ The implementation names a dedicated container, microVM, or OS-enforced sandbox with separate PID, network, mount, and UID namespaces; a read-only root with ephemeral writable layers discarded on termination; mandatory access control through seccomp, AppArmor, or SELinux; default-deny egress through a monitored proxy or service mesh; blocking of direct agent-to-agent network paths in favour of an authenticated orchestration layer or message bus; DNS restriction; segmentation of agents processing untrusted content away from sensitive internal services; tool credentials held outside the sandbox in a controlled credential store; destruction of transient state and in-sandbox credentials on termination; and per-agent resource quotas enforced by the platform rather than by agent self-management.⁹⁵

The limitations are stated as plainly as the controls. Container or hypervisor escape undermines containment. Shared inference, credential, and policy services create implicit cross-agent channels. Network segmentation cannot stop exfiltration through legitimately permitted APIs. Host-OS variance weakens the same configuration on a different host. Sandbox overhead scales with concurrent agents.⁹⁵ The wiki’s Agent Sandboxing practice and isolation landscape carry these against the shipping products.

A numbering gap in the source

Document 4’s sandboxing implementation list numbers its sub-areas “(4.9.1)” and “(4.9.3)” with no “(4.9.2)” appearing anywhere in the section. Recorded here rather than silently normalized, since a reader following the source’s own numbering will find the same gap.⁹⁵

Document 5: AI security testing (testing strategies)

Document 5 distinguishes three strategies for testing an AI system’s security: conventional security testing (pentesting); model performance validation, which tests whether the model behaves according to its specified acceptance criteria and, for security, detects permanent behavioral alteration through data or model poisoning; and AI security testing, the document’s own subject, defined as the part of AI red teaming that tests whether the model withstands specific attacks by simulating them.⁹⁶ The document names two reference projects rather than duplicating their content: the CSA Agentic AI Red Teaming Guide, which it describes as a collaboration between the Cloud Security Alliance and the AI Exchange, and the separate OWASP AI Testing Guide project.⁹⁶

Threats to test for

Document 5 partitions threats to test for by AI paradigm rather than by lifecycle stage. Predictive AI systems carry evasion, model exfiltration, and model poisoning as the threats beyond conventional security testing.⁹⁶ Generative AI systems carry prompt injection, sensitive-data output as a prompt-injection outcome, and insecure output handling from unfiltered generated content.⁹⁶ Agentic systems add non-deterministic multi-step execution, dynamic tool use, inter-agent communication, and persistent state, which the document states raise goal hijacking, unauthorized tool invocation, multi-agent propagation, persistent memory poisoning, agent escape, and delegation-chain abuse as threats beyond single-turn generative testing.⁹⁶

General approach and agentic testing

The document’s general approach runs eight steps: define objectives and scope aligned with organizational, compliance, and risk-management requirements; understand the AI system’s model, use cases, and deployment scenarios; identify potential threats through threat modeling and attack-surface exploration; develop attack scenarios and edge cases; execute tests, manual or automated; assess and document the resulting vulnerabilities and risks; prioritize remediation, implement mitigations, and calculate residual risk; and validate fixes by retesting the system post-remediation.⁹⁶

Agentic testing extends this approach over a wider attack surface — tools, orchestration, inter-agent channels, and session-persistent state rather than model input and output alone.⁹⁶ Its coverage-driven methodology:⁹⁶

• Threat-models the agentic system before testing: agents, orchestrators, tools, data sources, trust boundaries, and every external input surface.
• Confirms designed controls under normal conditions before adversarial load, since an untested baseline cannot be distinguished from a control that fails under attack.
• Runs single-turn and multi-turn prompt injection tests separately, since single-turn resistance does not predict session-level degradation across incremental reframing — a pattern the document names Crescendo without defining it further. The wiki’s definition sits on PyRIT’s orchestrator page, and a vendor efficacy figure for it on Promptfoo’s; this document corroborates neither.
• Tests tool-call validation independently of the LLM, by sending crafted invocations directly to the access-control or API gateway layer, since a restriction that exists only in a system prompt is not enforced against an injected instruction.
• Exercises failure modes: context-window saturation, tool errors, partial task completion, unexpected orchestrator routing.
• Defines minimum coverage criteria up front — which layers were tested, to what depth, with what corpus size — and reports untested threat categories explicitly; a coverage gap is a finding rather than an omission.
• Combines AI red teaming with conventional application-security testing, since an MCP server may be reachable for SSRF, SQL injection, or cross-site scripting.

The document directs readers to the CSA Agentic AI Red Teaming Guide as the primary agentic methodology and names paths it states are not always captured by single-turn LLM tests: goal-level red teaming pursuing an adversarial objective across sessions and attack paths; multi-turn testing exercising the same session-level reframing named above; cross-agent paths, including a compromised sub-agent influencing the orchestrator or privilege escalation through delegation chains; human oversight tested as a social surface, where urgency framing, confusion injection, or approval fatigue may bypass OVERSIGHT gates that work under normal review; supply-chain scenarios, substituted model variants or tampered tool implementations that bypass output filtering — the adversarial counterpart D8 grades no rung for, alongside the signature verification it does grade; and protocol testing of MCP, A2A, and other inter-agent protocol implementations for implementation weaknesses rather than prompt-layer attacks alone.⁹⁶ Findings should include reproduction steps and observed reproduction rates rather than a single pass or fail, since LLM behavior is probabilistic.⁹⁶

The document scopes an agentic penetration test across four layers: the LLM reasoning layer (prompt injection, goal hijacking, deceptive-reasoning induction); the tool execution layer (validation bypass, unauthorized invocation, parameter tampering); the infrastructure layer (API gateway controls, credential exposure, key management, and MONITOR USE log integrity, verifying that the agent cannot suppress or alter logs under adversarial conditions); and the inter-agent communication layer (message tampering, identity spoofing, trust-boundary exploitation).⁹⁶ Findings are prioritized against an agentic-aware severity model: autonomous execution scope, persistence across sessions, multi-agent propagation potential, and irreversibility of impact.⁹⁶

Prompt-injection and evasion test procedures

Document 5 publishes two named test procedures, each carrying Category: AI security test rather than a control-catalogue entry.

Testing against prompt injection presents a crafted set of inputs against direct and indirect prompt injection and sensitive-data output, and evaluates the resulting risk.⁹⁷ The procedure assembles and tailors an attack corpus, pairs each input with a detection method, presents attack inputs through the production system API rather than directly to the model so the deployed filtering and detection mechanisms are in the path, routes attack inputs through the same insertion mechanisms untrusted data uses when indirect injection is relevant, adds variation algorithms — synonym substitution, encoding, reformatting — against the detection mechanism, runs the test multiple times against production configuration to account for non-determinism, and evaluates severity of harm against likelihood of presentation. The test fails where the severity of harm cannot be accepted given how likely the attack input is to be presented.⁹⁷

Testing against evasion looks for feasible inputs that produce unintended output of unacceptable severity, and checks whether the system can limit or stop the search for such inputs and whether a surrogate model could be built to prepare the attack.⁹⁸ The procedure establishes feasibility criteria — a perturbation limit expressed as a norm-based bound, a perceptual similarity measure such as SSIM or PSNR, or a domain-specific rule — then searches in order: perfect-knowledge, zero-knowledge, transfer with a surrogate, and heuristic, each skippable once an earlier search finds a feasible input of unacceptable impact. The test fails where a feasible evasion input of unacceptable impact severity is found.⁹⁸

Adoption evidence

The AI Exchange project began in October 2022 as the “AI security and privacy guide” and was rebranded “AI Exchange” a year later to emphasize global collaboration; it was awarded OWASP Flagship project status in March 2025, alongside the OWASP GenAI Security Project.¹ Rob van der Veer, who founded the project and is credited by the source with 34 years of experience in AI and security, is the provenance behind the standards-liaison position above; the source names him as the individual working in ISO/IEC 27090, ISO/IEC 27091, and the EU AI Act at CEN/CENELEC.¹ The Exchange positions itself as the comprehensive core resource next to the GenAI Security Project’s narrower, awareness-focused deliverables such as the LLM Top 10.⁹⁹ It publishes attributed endorsements from named individuals at three organizations: Dutch Railways (Director Cybersecurity), Peloton Interactive (Chief Security & Trust Officer), and Lenovo (Staff AI Security Engineer).¹ The formal SANS collaboration and the prEN 18282 / ISO/IEC 27090 / ISO/IEC 27091 liaison work, both described above, are themselves adoption signals distinct from user-facing testimonials.¹ The project’s roadmap names active 2026 workstreams, including an agentic-AI content extension and a harmonization effort mapping the Exchange to NIST, MITRE ATLAS, and ETSI, targeted for delivery before summer 2026.¹⁰⁰

Limits

The validation gap the Exchange names for agentic security architecture applies to document 0’s own guidance: nothing in it scores whether an implementation of the Exchange’s controls achieves its stated effect.¹¹ Document 5 narrows that gap for two threats rather than closing it. Its prompt-injection procedure presents attack inputs through the production system API so the deployed filtering and detection mechanisms are in the path, and issues a pass or fail on whether the severity of harm can be accepted; its evasion procedure fails a system where a feasible evasion input produces unacceptable impact.⁹⁷⁹⁸ Both score one threat’s controls under attack. Neither scores a control set as a whole, and no other threat entry read so far in documents 1 through 5’s testing-strategies half carries a comparable procedure; document 5’s tool catalogue and document 6 remain unread. The AI Security Matrix and periodic table function as a threat catalogue; the Exchange’s own ten-step risk-analysis method is the separate mechanism that turns the catalogue into a ranked, context-specific set of risks, and a matrix row’s presence alone does not indicate the underlying threat applies to a given system.⁷⁹ The consensus claim in the Standards liaison section is attributed to the source and has not been checked against the prEN 18282 or ISO/IEC 27090 draft texts in this pass. The overview also carries a substantial section on AI and copyright — infringement risk in training data, source-code copyright, vendor indemnification commitments — framed by the source itself as a legal and policy topic; it exists in the document and is noted here without synthesis, since it sits outside this wiki’s security scope axes.¹⁰¹

The eighteen-row matrix is the Exchange’s sorted threat view rather than its complete threat set. Document 4 gives direct augmentation data leak, augmentation data manipulation, and agent escape their own permalinks and control sets without a corresponding matrix row.⁹³¹²¹⁴ A threat’s absence from the matrix is therefore evidence about the matrix’s axes, and a reader using the matrix as an enumeration will under-count.

Document 1’s twelve Gap: verdicts, recorded above, grade named standards’ coverage of a governance control; they do not close the gap this section states, since a coverage judgment about a standard is not a scoring of an implementation. The same document’s jurisdictional survey under CHECK COMPLIANCE is headed “as of end of 2023,” a three-year-old jurisdictional list inside a document this pass retrieved in 2026-08 and describes as continuously updated, above.⁵

Document 1 §1.1 cites no OWASP sibling project by identifier — no LLM## category, no ASI## category, no numeric claim about a sibling OWASP deliverable — so the identifier discrepancies already on record elsewhere on this page (the LLM07:2025/LLM08 note, above) gain no further instance from this tranche. Three GenAI Security Project deliverables are named by title rather than fetched in this pass — the LLM AI Cybersecurity & Governance Checklist, the LLM and GenAI Security Center of Excellence Guide, and the AI security solutions landscape¹⁶¹⁷¹⁸ — and are recorded here as unchecked rather than verified. §1.2 cites MITRE ATLAS identifiers under DISCRETE — AML.M0000, AML.M0001, and AML.T0002 — and, like §1.1, cites no OWASP sibling project by identifier.²⁶

§3.0 and §3.1 cite two OWASP sibling projects, both for data poisoning. The data-poisoning entry names OWASP for LLM 05: Data and Model Poisoning inline, linked to the 2026 edition of the GenAI LLM Top 10 — the category number is present in that anchor text, the edition year is not — and separately cites the same 2026-edition category in its References block as OWASP Top10 for LLM: sec. LLM05:2026: Data and Model Poisoning, which carries both, alongside OWASP Top10 for ML: sec. ML02:2023: Data Poisoning Attack. The supply-chain-model-poisoning entry cites only the inline form, OWASP for LLM 04: Supply Chain — again the category number without the edition year — and no companion reference-block citation in this tranche. The 2026-edition citations repeat the edition-year pattern already recorded above for LLM07/LLM08; this pass has not reconciled LLM05:2026 or the unnumbered LLM 04 reference against the wiki’s verified 2025 category set. Six of the section’s controls state that ISO/IEC standards do not cover them yet — CONF COMPUTE, FEDERATED LEARNING, MORE TRAIN DATA, TRAIN DATA DISTORTION, POISON ROBUST MODEL, and MODEL ENSEMBLE — which is an absence rather than a coverage verdict, on the same basis as §1.2’s three. Three controls carry a graded ISO/IEC verdict instead of an absence one: SEGREGATE DATA (ISO 27002 8.31, partial coverage),⁷⁶ SUPPLY CHAIN MANAGE (full coverage, with a data-provenance gap a separate provenance standard fills),⁷⁹ and DATA QUALITY CONTROL (minimal coverage under the ISO/IEC 5259 series).¹⁰² TRAIN ADVERSARIAL states neither verdict.

Notes

Sources

• OWASP AI Exchange

Footnotes

1. OWASP AI Exchange — About the AI Exchange, retrieved 2026-08-17. 170+ experts, 300+ pages, prEN 18282 (70 pages) and ISO/IEC 27090 (70 pages) contributions, ISO/IEC 27091 contribution, OpenCRE/OpenCRE-Chat link, “represents the consensus” claim, founding history (October 2022, rebranded a year later, OWASP Flagship status March 2025), named endorsements. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰

2. OWASP AI Exchange — Threats overview, retrieved 2026-08-17. Three threat types by lifecycle, six impacts mapped to three attacker goals, scope-boundary statement. ↩ ↩² ↩³

3. OWASP AI Exchange — Controls overview, retrieved 2026-08-17. Scope-of-controls statement and the cost/trade-off caution. ↩ ↩²

4. OWASP AI Exchange — Essentials, retrieved 2026-08-17. Four categories, eight sub-categories, 50+ controls; SANS alignment statement. ↩ ↩²

5. OWASP AI Exchange — CHECK COMPLIANCE, retrieved 2026-08-19. The blind-spot objective, the nine-row AI regulatory mapping table, the end-of-2023 jurisdictional survey, and the Oct-2025-dated ISO/IEC 27090/27091 and prEN 18282 liaison-officer and co-editor detail. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

6. OWASP AI Exchange — How to organize AI Security, retrieved 2026-08-17. The G.U.A.R.D. steps and their closing position. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸

7. OWASP AI Exchange — AI Security Matrix, retrieved 2026-08-17. Eighteen threat/risk-category rows, counted from the source’s HTML table. ↩ ↩² ↩³

8. OWASP AI Exchange — Periodic table of AI security, retrieved 2026-08-17. ↩ ↩² ↩³

9. OWASP AI Exchange — Risk analysis, retrieved 2026-08-17. Ten-step method, likelihood-factor attributions, sufficiency criterion, control-tuning note. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

10. OWASP AI Exchange — Identifying risks (threat model), retrieved 2026-08-17. ↩

11. OWASP AI Exchange — Agentic AI overview, retrieved 2026-08-17. Agency spectrum, four properties, memory surfaces, jailbreak/escape distinction, goal hijacking, no-transitive-trust, lethal trifecta, four architecture principles, validation gap. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵ ↩¹⁶

12. OWASP AI Exchange — Augmentation data manipulation, retrieved 2026-08-18. ↩ ↩² ↩³ ↩⁴ ↩⁵

13. OWASP AI Exchange — AUGMENTATION DATA INTEGRITY, retrieved 2026-08-18. ↩ ↩² ↩³

14. OWASP AI Exchange — Agent escape, retrieved 2026-08-18. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

15. OWASP AI Exchange — General governance controls, retrieved 2026-08-19. The group-level category and permalink for §1.1. ↩

16. OWASP AI Exchange — AI PROGRAM, retrieved 2026-08-19. The Quickstart eight-step iteration, the bare-minimum three-step start, the nine risk-management particularities, the ISO/IEC 42001-vs-5338 governance/engineering split, and the 42001 “covers this control fully” verdict. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

17. OWASP AI Exchange — SEC PROGRAM, retrieved 2026-08-19. The twelve-item AI asset list, the six-step AI Use Case Privacy and Security Analysis, the AI-specific honeypot control and its eight worked examples, the regulatory-incident coordination statement, the ISO/IEC 27563:2023 use-case figures, and the adjacent “ISO/IEC TR 27562:2023” citation recorded here as unresolved. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸

18. OWASP AI Exchange — SEC DEV PROGRAM, retrieved 2026-08-19. The anti-isolation statement, the particularity list including the AI development environment’s exposure to conventional software risks, the 43%/80% automated-test-coverage figures, and the OWASP SAMM / NIST SSDF / NIST SSDF AI-practices references. ↩ ↩² ↩³ ↩⁴ ↩⁵

19. OWASP AI Exchange — DEV PROGRAM, retrieved 2026-08-19. The general-software-engineering framing, the do-not-isolate-AI-engineering statement, the ISO/IEC 5338-extends-ISO/IEC-12207 claim, and the related-controls list. ↩ ↩² ↩³ ↩⁴

20. OWASP AI Exchange — SEC EDUCATE, retrieved 2026-08-19. The four-step education process and the conventional-controls-unless-training-your-own-model statement. ↩ ↩²

21. OWASP AI Exchange — General controls for sensitive data limitation, retrieved 2026-08-20. The group thesis and the category line of each of the five controls: DATA MINIMIZE, ALLOWED DATA, SHORT RETAIN, and DISCRETE as development-time and runtime controls, OBFUSCATE TRAINING DATA as a development-time AI engineer control. ↩ ↩²

22. OWASP AI Exchange — DATA MINIMIZE, retrieved 2026-08-20. The applicability statement, the deployer exception, the four implementation bullets, the risk-reduction statement, the Particularity on model tolerance for reduced data, the OpenCRE routing to ENISA and NIST AI 100-2 differential-privacy entries, and the ISO/IEC coverage gap. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

23. OWASP AI Exchange — ALLOWED DATA, retrieved 2026-08-20. The purpose-limitation requirement and its consent case, the absence of an Implementation section, and the ISO/IEC 23894 A.8 Privacy Gap: verdict of full coverage with a brief section on the idea. ↩ ↩² ↩³ ↩⁴

24. OWASP AI Exchange — SHORT RETAIN, retrieved 2026-08-20. The statement that limiting the retention period of data can be seen as a special form of data minimization, the exception case, the OpenCRE routing to training-data retention management, and the ISO/IEC coverage gap. ↩ ↩² ↩³ ↩⁴

25. OWASP AI Exchange — OBFUSCATE TRAINING DATA, retrieved 2026-08-20. The applicability statement and deployer exception, the five named obfuscation techniques (PATE, objective function perturbation, masking, encryption, tokenization), the two encryption models, the risk-reduction and effectiveness-evaluation statement, the Particularity on obfuscation techniques impractical in traditional systems, the two stated limitations, the K-anonymity / L-diversity / T-closeness assessment statement, the differential-privacy definition paragraph, the TRAINDATADISTORTION and EVASIONROBUSTMODEL cross-references, and the ISO/IEC coverage gap. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸

26. OWASP AI Exchange — DISCRETE, retrieved 2026-08-20. The implementation statement placing technical details as an asset in information security management, the three stated examples, the AI TRANSPARENCY weighing note and its stated resolution, and the ISO 27002 Control 5.9 and OpenCRE data-classification Gap: verdicts. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸

27. OWASP AI Exchange — AI TRANSPARENCY, retrieved 2026-08-19. The five properties the entry states the information can include — the rough working of the model, the training approach, the type of data used and its source, expected accuracy and robustness of the output, and any residual security risk — phrased as a non-exhaustive list rather than a required set; the stated floor that the simplest form of transparency is informing users that an AI model is involved, which the entry notes the EU AI Act requires for chatbots; the explainability distinction; and the ISO/IEC 42001 B.7.2 gap verdict. ↩ ↩² ↩³ ↩⁴

28. OWASP AI Exchange — Controls to limit the effects of unwanted behaviour, retrieved 2026-08-19. The blast-radius framing, the causes of unwanted behaviour beyond attack, and the overreliance/excessive-agency threats the control set creates. ↩ ↩² ↩³ ↩⁴

29. OWASP AI Exchange — OVERSIGHT, retrieved 2026-08-19. The automated/human oversight split, the OpenCRE and MITRE ATLAS references, and the ISO/IEC 42001 B.9.3 gap verdict. ↩ ↩² ↩³ ↩⁴

30. OWASP AI Exchange — LEAST MODEL PRIVILEGE, retrieved 2026-08-19. The permissions/attack-surface framing, the OpenCRE and MITRE ATLAS references, and the ISO 27002 8.2 / OpenCRE 368-633 gap verdicts. ↩ ↩² ↩³

31. OWASP AI Exchange — MODEL ALIGNMENT, retrieved 2026-08-19. The probabilistic-model-internal-control statement and the requirement to combine it with deterministic external mechanisms for high-risk use. ↩ ↩²

32. OWASP AI Exchange — CONTINUOUS VALIDATION, retrieved 2026-08-19. The backdoor-poisoning blind spot and the ISO 5338 gap verdict. ↩ ↩²

33. OWASP AI Exchange — EXPLAINABILITY, retrieved 2026-08-19. ↩ ↩²

34. OWASP AI Exchange — UNWANTED BIAS TESTING, retrieved 2026-08-19. ↩ ↩²

35. OWASP AI Exchange — Threats through use (input threats), retrieved 2026-08-18. The group’s alternative names, the five enumerated threat types, and the six generally applicable controls. ↩ ↩²

36. OWASP AI Exchange — Transferability-based evasion, retrieved 2026-08-18 and 2026-08-19. The subtype definition, the six enumerated surrogate sources, and the named exclusion of RATE LIMIT, UNWANTED INPUT SERIES HANDLING, and OBSCURE CONFIDENCE. ↩ ↩²

37. OWASP AI Exchange — Evasion after data poisoning, retrieved 2026-08-18 and 2026-08-19. The subtype definition, the trigger planted at training time, and the named exclusion of the same three controls. ↩ ↩²

38. OWASP AI Exchange — MODEL ACCESS CONTROL, retrieved 2026-08-18. The agentic authentication subsection, the no-transitive-trust rule, the reputation-scoring caveat, and the identity-not-intent limitation. ↩ ↩² ↩³

39. OWASP AI Exchange — ANOMALOUS INPUT HANDLING, retrieved 2026-08-18. The detection framings, the similarity measures by modality, and the stated ineffectiveness against evasion. ↩ ↩²

40. OWASP AI Exchange — OBSCURE CONFIDENCE, retrieved 2026-08-18. The gradient-masking resemblance and the indirect-approximation admission. ↩

41. OWASP AI Exchange — UNWANTED INPUT SERIES HANDLING, retrieved 2026-08-18. Series-level detection across a non-consecutive time window, and the identity-distribution and legitimate-tester limitations. ↩

42. OWASP AI Exchange — MONITOR USE, retrieved 2026-08-18. The AI-specific logging set, the agentic incident lifecycle, the infrastructure-layer containment requirement, the monitoring-agent attack-surface statement, and the reasoning-trace qualification. ↩ ↩² ↩³

43. OWASP AI Exchange — RATE LIMIT, retrieved 2026-08-18. The experimentation-mitigation scope, the 10,000-interaction worked example, the query-volume table, the safety-critical exceptions, the stated limitations, and the agent fleet rate limits. ↩ ↩² ↩³ ↩⁴ ↩⁵

44. OWASP AI Exchange — Evasion, retrieved 2026-08-19. The definition, the separation from prompt injection by manipulated object, the four worked examples, and the stated attacker preference for imperceptible perturbations. ↩ ↩² ↩³ ↩⁴

45. OWASP AI Exchange — Zero-knowledge evasion, retrieved 2026-08-19. The black-box definition and the decision-based / score-based split. ↩

46. OWASP AI Exchange — Perfect-knowledge evasion, retrieved 2026-08-19. The white-box definition, FGSM, and the statement that attacks against certified defenses are possible. ↩ ↩²

47. OWASP AI Exchange — Partial-knowledge evasion, retrieved 2026-08-19. The gray-box definition and its combination of surrogate transfer with zero-knowledge technique. ↩

48. OWASP AI Exchange — EVASION INPUT HANDLING, retrieved 2026-08-19. Single-sample detection methods, detector networks, and the stated circumvention of both model and detector. ↩ ↩² ↩³

49. OWASP AI Exchange — EVASION ROBUST MODEL, retrieved 2026-08-19. Named mechanisms, the Adversarial Robustness Toolbox / CleverHans / Foolbox tool set, and the obfuscated-gradients reference. ↩ ↩²

50. OWASP AI Exchange — TRAIN ADVERSARIAL, retrieved 2026-08-19. Madry adversarial training with PGD, and the stated overhead, scaling, overfitting, and generalization costs. ↩ ↩²

51. OWASP AI Exchange — INPUT DISTORTION, retrieved 2026-08-19. Random Transformations, the comparison-based detection use, and the stated zero-knowledge exemption, accuracy loss, and retraining requirement. ↩ ↩² ↩³

52. OWASP AI Exchange — ADVERSARIAL ROBUST DISTILLATION, retrieved 2026-08-19. The caution preceding any implementation detail, the absent subsections, and the Papernot 2016 / Carlini and Wagner 2016 reference pair. ↩ ↩² ↩³

53. OWASP AI Exchange — Prompt injection, retrieved 2026-08-18. The threat group, the shared context plane, the three agentic subtypes, the model-layer statement, the five structural mitigations, and the detection-layer precedence. ↩ ↩² ↩³ ↩⁴ ↩⁵

54. OWASP AI Exchange — Direct prompt injection, retrieved 2026-08-18. Ten attack forms, the jailbreak definition and its two strategies, multi-turn session-level jailbreak, multimodal instruction channels, and the system-prompt-leakage cross-reference. ↩ ↩² ↩³ ↩⁴ ↩⁵

55. Identifier discrepancy, unreconciled. The Exchange cites the category as “LLM08: Hidden Context Exposure” in a 2026 edition of the OWASP Top 10 for LLM Applications. The edition this wiki verified against genai.owasp.org is 2025, where System Prompt Leakage is LLM07:2025 and LLM08:2025 is Vector and Embedding Weaknesses. The source’s number therefore resolves to a different live category, so this page writes LLM07:2025 and records the Exchange’s identifier as attributed and unreconciled. See OWASP Top 10 for LLM Applications. ↩

56. OWASP AI Exchange — Indirect prompt injection, retrieved 2026-08-18. The remote-code-execution comparison and the dominant-threat-class statement. ↩

57. OWASP AI Exchange — Seven layers of prompt injection protection, retrieved 2026-08-18. ↩

58. OWASP AI Exchange — PROMPT INJECTION I/O HANDLING, retrieved 2026-08-18. Unicode NFKC normalization, locale-independent case-folding, confusable collapsing under Unicode UTS #39, instruction-token escaping. ↩

59. OWASP AI Exchange — INPUT SEGREGATION, retrieved 2026-08-18. Hard-to-spoof markers, the partial-mitigation classification, and the stated limitations. ↩

60. OWASP AI Exchange — Agent message structure manipulation, retrieved 2026-08-18. The threat definition, the distinction from indirect prompt injection by target, and the statement that it reaches single agentic flows. ↩ ↩² ↩³

61. OWASP AI Exchange — Sensitive data disclosure through use, retrieved 2026-08-19. ↩

62. OWASP AI Exchange — Disclosure of sensitive data in model output, retrieved 2026-08-19. The disclosure mechanism, the augmentation-data path, and the statement that original access-right variations cannot be controlled once training data is embedded in the model. ↩ ↩²

63. OWASP AI Exchange — SENSITIVE OUTPUT HANDLING, retrieved 2026-08-19. The exposure-restricted data classes, the enforcement point at output time, recitation detection against an indexed training set, and the statement that system-prompt instruction is insufficient on its own. The four limits quoted here are the entry’s Limitations block; the recitation-reach limit is stated separately under its risk-reduction guidance. ↩ ↩² ↩³ ↩⁴ ↩⁵

64. OWASP AI Exchange — Model inversion and membership inference, retrieved 2026-08-19. The two attack definitions, the overfitting relationship, the model-family preference, the ATLAS / OWASP ML / NIST AI 100-2 identifiers, and the entry’s control list: sensitive data limitation, MONITOR USE, RATE LIMIT, MODEL ACCESS CONTROL, OBSCURE CONFIDENCE, SMALL MODEL, and the two model-leak threats. ↩ ↩² ↩³ ↩⁴ ↩⁵

65. OWASP AI Exchange — SMALL MODEL, retrieved 2026-08-19. The capacity argument and the ISO/IEC coverage gap. ↩ ↩²

66. OWASP AI Exchange — Model exfiltration, retrieved 2026-08-19. The harvesting routes, the alternative names, the public-model exception, the rehearsal impact, the intensive-use difficulty statement, the benign-use detection caveat, the bound scoping the replica’s input attacks to mainly evasion, and the February 2026 Anthropic claim. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

67. OWASP AI Exchange — Direct runtime model leak, retrieved 2026-08-18. ↩ ↩² ↩³ ↩⁴

68. Attributed and unverified. The Exchange states the claim as Anthropic’s and cites no publication for it (OWASP AI Exchange — Model exfiltration, retrieved 2026-08-19). The wiki records it as the Exchange’s report of a vendor claim, on the same basis it marks the Exchange’s own prEN 18282 and ISO/IEC 27090 liaison figures unverified. ↩

69. OWASP AI Exchange — MODEL WATERMARKING, retrieved 2026-08-19. The attribution objective, the survivability and ambiguity requirements, the stated limitation against model exfiltration, and the entangled-watermarking reference. ↩ ↩² ↩³

70. OWASP AI Exchange — AI resource exhaustion, retrieved 2026-08-19. The two attacker goals, the MITRE ATLAS AML.T0029 citation, the sponge / energy-latency attack, and the denial-of-wallet framing. ↩ ↩² ↩³

71. OWASP AI Exchange — DOS INPUT VALIDATION, retrieved 2026-08-19. The validation scope, the agent tool-parameter subsection, and the two stated standards gaps (ISO 27002 and ISO/IEC). ↩ ↩²

72. OWASP AI Exchange — LIMIT RESOURCES, retrieved 2026-08-19. The per-input cap, the six per-agent resource dimensions, the infrastructure-enforcement rule, clean termination with an audit event, the tighter tiers for untrusted-content workloads, fleet-wide consumption monitoring, the ISO 27002 gap stated except for monitoring alongside the ISO/IEC coverage gap, and the stated bound that resource limits bound cost and availability impact without preventing all harm within the budget. ↩ ↩² ↩³ ↩⁴ ↩⁵

73. OWASP AI Exchange — Culture-sensitive alignment, retrieved 2026-08-19. The jurisdiction-dependence argument, the thirteen-region comparison table and its three verdict classes, and the semantic-drift discussion. ↩ ↩² ↩³

74. OWASP AI Exchange — Development-time threats, retrieved 2026-08-20. The seven stated particularities of the AI development environment, the ISO/IEC 42001 B.7.2 note, and the four agentic development-time attention points routing back to §3.1. ↩ ↩² ↩³ ↩⁴

75. OWASP AI Exchange — DEV SECURITY, retrieved 2026-08-20. The AI-specific asset list (training data, test data, model parameters, technical documentation); the build-stage, deploy-stage and supply-chain integrity-check sets; the statement that a model comprises associated artifacts of varying formats — tokenizers, vocab files, configs, inference code — so signing must cover all of them, with no standard yet existing and the OpenSSF Model Signing SIG working on a specification; and the dataset-by-reference integrity problem, where a dataset holding URL pointers such as LAION-400M is exposed to manipulation or removal of the referenced content, answered by hashing dataset entries. ↩ ↩² ↩³

76. OWASP AI Exchange — SEGREGATE DATA, retrieved 2026-08-20. Storage of sensitive development data — training or test data, model parameters, technical documentation — in separated areas with restricted access; the five example areas (external, application development, data engineering, training, operational); the statement that the training environment can carry controls against access from the less-protected development areas, mitigating data poisoning; and the ISO 27002 8.31 partial-coverage verdict. ↩ ↩²

77. OWASP AI Exchange — CONF COMPUTE, retrieved 2026-08-20. Use of data-science execution-environment features to hide training data and model parameters from model engineers while in use, and the statement that ISO/IEC standards do not cover it yet. ↩

78. OWASP AI Exchange — FEDERATED LEARNING, retrieved 2026-08-20. The decentralized architecture; the stated benefits of regulatory compliance, confidentiality, scalability and data diversity; and the challenges — remaining disclosure risk through the model, insufficient protection against an active and dishonest central party extracting user data from received gradients, added poisoning attack surface across a client network, and the statement that ISO/IEC standards do not cover it yet. ↩

79. OWASP AI Exchange — SUPPLY CHAIN MANAGE, retrieved 2026-08-20. The four new supplied assets (data, models, model hosting, abilities) including fine-tuning artifacts such as LoRA modules; the statement that the supply chain may include the own organization, making data provenance part of the control; supplied abilities as their own class (skills, MCP servers, plugins, external agents, custom tool adapters); the provenance record set, lifecycle update points, per-agent bill of materials, dependency-vulnerability management with compensating controls, the seven-dimension supplier evaluation, and the pre-execution model-assessment set; and the Limitations block on incomplete provenance claims and probabilistic trust decisions. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

80. OWASP AI Exchange — Broad model poisoning development-time, retrieved 2026-08-20. The definition of development-time model poisoning as manipulation of development elements to alter model behaviour, its three subtypes (data poisoning, development-environment model poisoning, supply-chain model poisoning), the stated impact on model-behaviour integrity, and the four-stage poisoning threat model covering supplied data, supplied models, the data-preparation and training environments, and runtime-collected training data. ↩ ↩²

81. OWASP AI Exchange — MODEL ENSEMBLE, retrieved 2026-08-20. The category line “development-time AI engineer control - including specific runtime implementation”; deployment as an ensemble over a randomly split training set so a deviating output signals possible manipulation; and the stated effectiveness bound that the approach weakens as the share of poisoned samples rises. ↩

82. OWASP AI Exchange — Data poisoning, retrieved 2026-08-20. The five manipulation routes (storage, transit, change at the supplier before or after training, and manipulation of operational data entry feeding into training), each also executable by an insider; the statement that manipulated data can be training data or in-context-learning data used to augment input; the two categories, targeted or backdoor poisoning and sabotage; the detectability asymmetry; and the five named controls (MORE TRAIN DATA, DATA QUALITY CONTROL, TRAIN DATA DISTORTION, POISON ROBUST MODEL, TRAIN ADVERSARIAL with adversarial robust distillation). ↩ ↩² ↩³

83. OWASP AI Exchange — Direct development-time model poisoning, retrieved 2026-08-20. Manipulation of model parameters, storage of parameters, replacement of the model, command or code injection through custom or lambda layers, weight and architecture modification, and embedded deserialization attacks executing during model unpacking or execution; and CONTINUOUS VALIDATION as the named performance-deviation detection control. ↩

84. OWASP AI Exchange — Supply-chain model poisoning, retrieved 2026-08-20. The definition covering a manipulated third-party pre-trained model obtained and further used or fine-tuned; the transfer learning attack naming; the statement that manipulation may be by data poisoning or by direct parameter change and that parameter protection at the moment of manipulation is outside the obtaining party’s control; and the control split between what the receiver holds (data-poisoning controls, POISON ROBUST MODEL, adversarial robust distillation, SUPPLY CHAIN MANAGE) and what the supplier owes. ↩ ↩²

85. OWASP AI Exchange — Development-time data leak, retrieved 2026-08-25. Unauthorized access to train or test data through a data leak of the development environment, with the stated impact a confidentiality breach of sensitive train/test data; the statement that training or test data can be confidential because it is sensitive data such as personal data, or intellectual property; and the three routes named — the development environment where engineers work with real data, training data collected at runtime making a live system an attack surface, and cloud-hosted or externally managed GenAI models to which training data must travel. ↩ ↩²

86. OWASP AI Exchange — Direct development-time model leak, retrieved 2026-08-25. Unauthorized access to model attributes — parameters, weights, architecture — through stealing data from the development environment including the supply chain, with insider access, compromised repositories, and weak storage controls named as routes; the two-part impact of intellectual-property theft and a copy on which an attacker can experiment to find working inputs for extraction, evasion or prompt injection without the production system’s rate limiting, access control and detection; model exfiltration and direct runtime model leak named as the alternative ways of model theft; the risk-identification scope to a model held as a trade secret or exposed to input-attack risk, excluding an already-public model; and the control list naming SUPPLY CHAIN MANAGE as specifically protecting model attributes. ↩ ↩² ↩³ ↩⁴ ↩⁵

87. OWASP AI Exchange — Source code/configuration leak, retrieved 2026-08-25. Unauthorized access to the source code and configuration that preprocess data and train the model, with model intellectual property as the breached asset. ↩ ↩²

88. OWASP AI Exchange — Generic security threats, retrieved 2026-08-18. ↩ ↩² ↩³ ↩⁴

89. OWASP AI Exchange — Direct runtime model poisoning, retrieved 2026-08-18. ↩

90. OWASP AI Exchange — RUNTIME MODEL CONFIDENTIALITY, retrieved 2026-08-18. ↩

91. OWASP AI Exchange — MODEL OBFUSCATION, retrieved 2026-08-18. ↩

92. OWASP AI Exchange — Input data leak, retrieved 2026-08-18. ↩ ↩²

93. OWASP AI Exchange — Direct augmentation data leak, retrieved 2026-08-18. ↩ ↩² ↩³ ↩⁴

94. OWASP AI Exchange — Output contains conventional injection, retrieved 2026-08-18. ↩ ↩² ↩³

95. OWASP AI Exchange — Agent sandboxing and isolation, retrieved 2026-08-18. ↩ ↩² ↩³ ↩⁴

96. OWASP AI Exchange — AI security testing, retrieved 2026-08-19. The three-strategy distinction, the CSA Agentic AI Red Teaming Guide collaboration and OWASP AI Testing Guide references, the paradigm-split threats-to-test-for taxonomy, the general eight-step approach, the agentic coverage-driven methodology, the red-teaming exercise paths, the four-layer penetration-test model, and the agentic-aware severity model. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹²

97. OWASP AI Exchange — Testing against Prompt injection, retrieved 2026-08-19. The test description, the nine-step procedure, the production-API and augmentation-path presentation requirements, the variation-algorithm step and its stated limited impact on estimated risk, and the pass/fail evaluation step. ↩ ↩² ↩³

98. OWASP AI Exchange — Testing against Evasion, retrieved 2026-08-19. The three-aspect test description, the feasibility-criteria step, the ordered perfect-knowledge/zero-knowledge/transfer/heuristic search procedure, and the pass/fail evaluation step. ↩ ↩² ↩³

99. OWASP AI Exchange — Relevant OWASP AI initiatives, retrieved 2026-08-17. Positioning against the OWASP GenAI Security Project and the LLM Top 10. ↩

100. OWASP AI Exchange — AI Exchange roadmap, retrieved 2026-08-17. 2026 workstreams, including agentic-AI extension and NIST/ATLAS/ETSI harmonization. ↩

101. OWASP AI Exchange — How about copyright?, retrieved 2026-08-17. ↩

102. OWASP AI Exchange — DATA QUALITY CONTROL, retrieved 2026-08-20. The five named detection methods (statistical deviation and outlier detection, spectral signatures over covariance of learned feature representations, activation clustering, Reject on Negative Impact, gradient fingerprinting); the filter-versus-alert threshold split; and the ISO/IEC 5259 series minimal-coverage verdict, on the particularity that the standard does not address detecting malicious changes such as statistical deviations. ↩