Agent Memory Isolation
Definition
Agent Memory Isolation is the architectural practice of assigning each agent a distinct, non-sharable memory namespace so that one agent cannot read, write to, or be contaminated by another agent’s stored memories. The identity that keys the namespace must be determined at the infrastructure level — not by the agent itself — to prevent impersonation.
The failure mode it prevents
When multiple agents share a single memory store (e.g., a Postgres table with no namespace partitioning), their stored facts, goals, and preferences bleed across agent boundaries:
- Agent A stores a goal (“optimize for $1,000/month revenue”)
- Agent B, unrelated to that goal, retrieves it because both agents share the same memory space
- Agent B’s outputs are contaminated by Agent A’s objectives
This is a form of Memory Poisoning originating not from adversarial injection but from architectural non-isolation — a cross-agent state leakage at design time.
Implementation pattern (McMillin, 2026)
Brooks McMillin describes the minimal viable pattern in his [un]prompted 2026 talk:
- Each agent is a subclass of a base
Agentclass in Python - The subclass class name (e.g.,
TaskManagerAgent) serves as the namespace key - When the agent initializes its local MCP server, the class name is passed to the MCP server
- The MCP server maintains the namespace mapping and enforces isolation — the LLM cannot influence its own namespace key
The critical property: the namespace key is determined by code structure (class hierarchy), not by anything the LLM could assert in a prompt. This structurally prevents a prompt-injected agent from claiming a different identity and reading another agent’s memories.
Memory access model
Within a namespace, two access modes are available:
- Auto-injection: important memories (importance score above a threshold) are automatically injected into the agent’s context at startup
- On-demand search: agents can invoke a
search_memoryMCP tool to retrieve specific memories by query
Selective auto-injection rather than full injection prevents context bloat — a common failure mode when all memories are injected regardless of relevance.
Design invariants
| Property | Requirement |
|---|---|
| Identity anchor | Outside the LLM (code structure, infrastructure registration) |
| Namespace key | Not settable by the agent at runtime |
| Cross-namespace reads | Blocked at the memory store or MCP server layer |
| Write isolation | Agent A cannot write into Agent B’s namespace |
Relationship to identity primitives
This pattern is an instance of Ambient vs Derived Authority reasoning applied to memory: in a shared-namespace system, every agent has ambient access to all memories. The isolation pattern derives each agent’s memory access from a specific, narrow identity anchor.
For stronger guarantees — especially in multi-tenant or adversarially hardened environments — the class-name anchor should be replaced with a cryptographic identity primitive. See Capability-Based Authorization and Tenuo Warrants for the production-grade counterpart.
Relationship to memory poisoning
Memory Poisoning is usually framed as external adversarial injection. Agent memory isolation prevents an orthogonal failure: internal cross-contamination between benign agents. Both result in an agent’s behavior being influenced by content it should not have access to; the difference is attacker vs. architectural design.
Primary sources
- Building Secure Agentic Systems — Brooks McMillin — the primary practitioner source for this pattern; includes failure modes, the fix, and a live demo
- Memory Poisoning (Agentic AI) — the adversarial counterpart concept