CMM D9: Operations and Human Factors

Agentic AI Security CMM — D9 Operations & Human Factors (Deep Dive)

Companion deep-dive to the CMM’s D9 domain, written under the recalibration method. D9 is the most process- and labor-heavy domain in the model, and the least product-driven. About half of it has no product answer on any platform and is pure operating-model work: HITL-fatigue and oversight-quality measurement, IR-runbook authoring, decommission drills, bus-factor continuity. Where products exist, they are platform-native primitives repurposed from adjacent domains (D2 identity lifecycle, D7 observability), not D9-specific tools.

Single-source grounding

Levels and cost model synthesize the recalibration method against the regulated-FI stress test plus vendor documentation. Tooling status is a May 2026 snapshot.

Threat coverage

D9 is cross-cutting: it appears in every one of the five threat classes — decommission and rollback (Class 4), sustained threat hunting (Class 2), incident response and HITL-fatigue measurement against ASI09, and the vendor-cutoff playbook (Class 5) — and its bus-factor and continuity prerequisites gate every other domain’s L5 claim. The full mapping is in the Threat Taxonomy Reconciliation matrix.

Control landscape (dated)

Capability	What ships today	Status (May 2026)	Platform-native (MS / AWS / GCP)
AI-specific IR framework / playbooks	CoSAI AI Incident Response Framework (CACAO playbooks for prompt injection, memory injection, RAG poisoning)	published 2025-10-301	vendor-neutral; runs on any SOAR
Post-deployment monitoring categories	NIST AI 800-4 (descriptive, not prescriptive)	published Mar 20262	—
System-prompt-leakage test cases	OWASP LLM07:2025 System Prompt Leakage (canary-string method; verified by the LLM Top 10 review)	live	—
IR orchestration / SOAR for agents	Sentinel + Security Copilot; CACAO on Step Functions	MS Security Analyst Agent + NL playbook generator shipped 20263	MS strongest; AWS via AgentCore; GCP thin
Agent decommission / orphan reaping	NHI lifecycle (Okta NHI, Oasis); SCIM deprovisioning	GA across NHI vendors	MS: Entra Agent ID disable/delete with cascade child cleanup; sponsor-based Lifecycle Workflows preview4
Canary-token / prompt-leak trip-wire	OSS pattern; embedded in red-team suites	stable pattern	none native — build-it-yourself hook
HITL-fatigue / oversight-quality measurement	no product — risk-tiered oversight is the documented pattern; rubber-stamp / queue-age metrics are hand-rolled	—	none on any platform — pure process5
Bus-factor / continuity test	no product — deputy + runbook-continuity drill is an org practice	—	none — pure process

The load-bearing center of D9 has no product on any stack: HITL-fatigue measurement, bus-factor continuity, IR-runbook authoring and drilling, decommission cadence. That is a market gap, not a Microsoft gap, so no procurement decision closes it.

What the Microsoft ZT4AI Governance pillar does supply for D9 is the lifecycle-accountability layer — Entra ID Governance sponsors with manager-transfer, and the Entra Agent ID disable/delete-with-cascade reaper — crosswalked to D9 in the 2026-Q2 ZT4AI review, which confirms the human-factors center has no product on any stack. The RAI Standard’s A5 (Human oversight and control) goal sets the human-oversight outcome at the goal level, and Agent 365’s lifecycle management (create / review / decommission via the registry plus the Entra ID Governance sponsor model) supplies the management-plane mechanism behind it — the goal-and-mechanism pairing is set out in Agent 365 review. Neither instrument supplies the human-factors instrumentation (HITL-fatigue measurement, continuity testing) at the center of this domain.

Capability-decoupled levels

Stated as capabilities per rule 1; a control counts when it operates in production per rule 2.

• L1 — Initial. No guardrail SLAs, no decommission procedure, no HITL-queue monitoring, no system-prompt-confidentiality control, no IR runbook.
• L2 — Developing. A runbook documents guardrail fail behavior, decommission and credential rotation on owner departure, HITL-queue monitoring, and basic system-prompt protection. Decommission may be manual.
• L3 — Defined. Guardrail latency/cost is measured per agent with a tested fail-mode (fail-closed for high-risk tier); an orphan reaper runs on a scheduled SLA (met platform-native by Entra Agent ID disable/delete-with-cascade, or NHI/SCIM deprovisioning); HITL approval-rate and queue-age are tracked; a system-prompt trip-wire is deployed (canary tokens + LLM07:2025 cases); a model-deprecation policy is published; the org participates in at least one disclosure community; a named AI-security role exists with bus-factor ≥2. The IR runbook is adapted from CoSAI v1.0 CACAO playbooks and exercised at least once.
• L4 — Managed. Quantitative HITL-fatigue indicators are tracked (rubber-stamp rate, queue p95) — pure process, no tool; benign drift is separated from adversarial; decommission drills run quarterly; model versions are pinned in production; an AI-VEX-equivalent is published for own components; the org joins coordinated-disclosure exercises; a per-credential dependency map is maintained.
• L5 — Optimizing. A closed loop runs: every guardrail / decommission / HITL incident yields a measurable controls update within a published SLA; attested zero orphaned credentials, zero prompt leaks, and zero undeprecated models for at least two quarters; a quarterly deputy-and-runbook continuity test; HITL-fatigue indicators stay within published thresholds.
• L5+ — Leading Edge. Org-level AI risk-observability metrics published externally; named contribution of drift-detection patterns or bypass classes back to standards bodies; cross-org coordinated-disclosure leadership.

Nothing in D9 L1–L5 depends on just-GA’d tooling. The dependencies are CoSAI v1.0 (out six months), stable OTel/canary patterns, and the GA Entra Agent ID lifecycle path. D9 is therefore cadence-safe for a regulated buyer, unlike D2/D3/D5 where per-task tokens sit on early-stage OSS.

Right-sizing by deployment shape

D9 is where right-sizing bites hardest: a single low-risk bot held to mesh-grade IR and continuity is the canonical over-scoping error.

Deployment shape	Realistic D9 target	Why
Member-facing RAG bot (no tools) — the persona	L2 → L3, narrow	Only system-prompt confidentiality (canary + LLM07:2025 cases) and a basic owner-departure decommission runbook are load-bearing. No HITL queue to fatigue (read-only), no multi-agent IR, no quarterly drill. Holding this bot to L4 KPIs is the over-scoping the recalibration prevents
Coding / copilot	L3 → L4	Decommission cadence and prompt-leakage matter; real HITL volume on writes makes rubber-stamp measurement load-bearing
MCP / skill provider serving others	L4 + selective L5	Third-party blast radius justifies federated CVE disclosure and a skill/MCP deprecation policy
High-autonomy multi-agent mesh	L4 minimum, L5 where resourced	HITL-fatigue-at-scale is the dominant operational risk; closed-loop IR and continuity testing earn their cost only here

The lethal-trifecta test lowers D9 the same way it lowers D3/D5: a contained, low-autonomy bot legitimately scores lower, and the score records that as an intentional trade-off.

Cost model

Licensing is near-zero for an E5/Azure or AWS incumbent; the spend is people and run-rate.

Level	Licensing	Operational labor (the bottleneck)	Run-rate
L2	~0	~0.1–0.25 FTE: write the first runbook	—
L3	~0 (Entra / Sentinel / OTel entitled)	the dominant cost: adapt CoSAI CACAO playbooks to your agents; stand up the canary hook + LLM07:2025 cases; staff a named deputy and run one continuity test; participate in a disclosure community	latency/cost-span logging into the SIEM (couples to D7 ingest)
L4	~0 incremental (MS) / small (AWS)	recurring: quarterly decommission drills; HITL-rotation staffing plus hand-built rubber-stamp / queue dashboards (no product); benign-vs-adversarial drift triage; disclosure-exercise participation	drift + HITL telemetry ingest
L5	~0	highest: a closed-loop SLA needs an on-call rotation owning AI-IR; quarterly attestation evidence; a standing continuity test	continuous attestation + incident-replay logging

The costs the recalibration asked to surface: an on-call rotation is D9-L5’s largest hidden cost; IR-runbook authoring runs multi-week per agent class even though CoSAI ships the shape; decommission drills are quarterly facilitation labor; and HITL-rotation staffing covers both reviewers and the analyst-time to hand-build the fatigue dashboards, because no product measures oversight quality.

Customer critiques folded in

• “D9 is L1–L2: no formal decommission or HITL-fatigue tracking.” The recalibrated L3 is the realistic near-term target for the persona’s bot and is narrow: canary + LLM07:2025 cases, an owner-departure decommission runbook via Entra Agent ID (already owned), and a named deputy. There is no HITL queue for a read-only RAG bot, so the recalibration right-sizes the fatigue-tracking critique away rather than meeting it with tooling.
• “L5 assumes a cadence regulated FIs can’t follow.” D9’s L1–L5 dependencies are all stable or standards-based, so D9 is cadence-safe; the aspirational items sit at L5+.
• “Cost under-tells the dominant spend.” Licensing is near-zero; the real spend is on-call, drill facilitation, and hand-built HITL dashboards, all labor and run-rate.
• “Vendor-neutral catalog is an integration tax.” The platform-native column names Entra Agent ID and Sentinel for the all-Microsoft buyer; the off-stack residual (HITL-fatigue measurement, continuity practice) has no product on any stack, so it is a market gap, not a Microsoft gap.

Open questions

• HITL-fatigue measurement has no product and no metric standard; thresholds (when is approval-without-comment too high?) are unresolved.
• No “AI-VEX” product exists yet; generic VEX is GA, but AI-component exploitability disclosure is emerging only.
• NIST AI 800-4 is descriptive, so the benign-vs-adversarial drift requirement leans on its taxonomy without an implementation spec.
• Entra Agent ID orphan governance is split GA/preview — disable/delete-with-cascade is usable, but sponsor-based Lifecycle Workflows are preview.
• No FFIEC/GLBA/NCUA IR-notification crosswalk yet (including the GLBA breach-notification timeline); deferred to the crosswalk.

Cross-cutting note and the L4→L5 gate

D9 is cross-cutting, not a per-plane domain; its capabilities interact indirectly (HITL fatigue degrades D3’s approval gates; decommission lag orphans D2 identities; drift remediation feeds on D7 telemetry). Per the dependency rules, D9 operational lag no longer drags upstream domains down: D9 weakness is honest signal in its own row, not a punitive cap.

The L4→L5 “campaign, not a step” gate sits substantially inside D9. Two of its four conditions are D9 controls: bus-factor ≥2 with a documented continuity test, and the two-quarter stable-L4 history that D9’s closed-loop attestations evidence. So D9 is not merely one domain climbing to L5. Its continuity-test and clean-state attestations are the org-wide gate evidence every other domain’s L5 claim depends on. A program cannot reach L5 anywhere without clearing D9’s continuity bar.

Notes

Footnotes

1. CoSAI — AI Incident Response Framework, dated 2025-10-30 on the CoSAI resources listing per CoSAI review. NIST-IR lifecycle adapted to AI; CACAO playbooks. ↩

2. NIST — Challenges to the Monitoring of Deployed AI Systems (AI 800-4), 2026. Six monitoring categories; descriptive, not prescriptive. ↩

3. Microsoft — What’s new in Microsoft Sentinel, March 2026, 2026. Security Analyst Agent; natural-language SOAR playbook generator. ↩

4. Microsoft Learn — How agent identity deletion works, 2026. Disable/delete with cascade child cleanup; sponsor-based Lifecycle Workflows in preview. ↩

5. Ou — Confirmation fatigue and the protocol gap in agentic AI oversight, 2026. No dedicated HITL spec or oversight-quality measurement product as of early 2026. ↩