LlamaFirewall

Sources: LlamaFirewall paper, arXiv:2505.03574 (see the summary page) · Purple Llama repo

Open-source AI guardrail framework published by Meta AI (arXiv:2505.03574, submitted April 2025) and distributed in the Purple Llama project. Designed for building secure AI agents; provides three specialized guardrail components that operate at different points in the agent execution pipeline. The framework is a final runtime defense layer supporting system-level, use-case-specific safety policies rather than a single universal filter.

Architecture: Three Components

PromptGuard 2

Input-side DeBERTa classifier for jailbreak and prompt injection detection, shipped in two sizes — 86M (98% AUC, 97.5% recall at 1% FPR, 92.4 ms) and 22M (88.7% recall at 1% FPR, 19.3 ms) — trading recall for latency. Operates before the LLM processes the input.¹

AlignmentCheck

Inspects the agent’s chain-of-thought reasoning before tool execution for signs of goal hijacking, using a guardrail LLM (Llama 3.3 70B / Llama 4 Maverick). This is a prospective control — it fires after the model has reasoned but before it acts, catching indirect injections that pass input-layer detection but manifest as abnormal reasoning. Addresses OWASP ASI01 (Agent Goal Hijack). Still experimental per the paper; alone it cuts attack success ~83% on AgentDojo. Combined with PromptGuard 2, the framework drops AgentDojo attack success from 17.6% to 1.75% at roughly 5% utility cost.¹

CodeShield

Static analysis (Semgrep + regex, eight languages) for LLM-generated code before execution. Catches dangerous patterns (shell injection, file deletion, credential access) in code the agent writes and is about to run: 96% precision, 79% recall, 60–70 ms latency, validated against CyberSecEval3-labeled completions.¹

Positioning

LlamaFirewall operates at the input and reasoning layers (model layer in the Security Controls for AI Stacks taxonomy). For containment, it is combined with platform-level controls. The key architectural note: LlamaFirewall guardrails should be deployed at the framework/runtime layer, not as prompt instructions, to achieve their effectiveness guarantees.

Relationship to Traditional Security

LlamaFirewall maps to IPS/WAF at the model layer — pattern-matching and behavioral analysis on inputs and reasoning rather than network packets and HTTP requests. AlignmentCheck is novel: no traditional equivalent exists for prospective chain-of-thought auditing.

As an external baseline

Uber’s ADR paper uses LlamaFirewall (Llama Guard 3-8B plus heuristic rules, official thresholds) as one of three detection baselines. The results split by benchmark and expose LlamaFirewall’s operating point. On the enterprise ADR-Bench (260 benign / 42 malicious), it fires 40 false positives — precision 0.167, F1 0.178, the lowest of the four detectors and 19x ADR’s cost per task — the failure mode that makes a high-false-positive guardrail unusable for production alerting under class imbalance.² On AgentDojo (prompt injection), the picture inverts: LlamaFirewall reaches recall 0.974, near-best at catching attacks, but 21 false alarms drag precision to 0.638.² The takeaway is not that one detector is better but that input/reasoning-layer guardrails optimize for recall on injection and pay for it in precision on benign enterprise traffic.

See Also

• Prompt Injection Containment for Agentic Systems — the practice page covering the two-layer detection + containment model
• Security Controls for AI Stacks — model layer where LlamaFirewall operates
• Meta — publisher

Footnotes

1. All component figures from the LlamaFirewall paper, arXiv:2505.03574 (Meta, 2025). ↩ ↩² ↩³

2. §5 Evaluation, Table 2, arXiv:2605.17380: LlamaFirewall scores precision 0.167 / recall 0.190 / F1 0.178 with 40 FPs on ADR-Bench, and precision 0.638 / recall 0.974 / F1 0.771 with 21 FPs on AgentDojo. ↩ ↩²