Irregular

Israeli security-evaluation firm of roughly 35 people, based in Tel Aviv, retained by frontier AI labs to run independent assessments of model offensive and defensive cyber capability.¹

Relevance to This Wiki

Irregular ran the third-party evaluation in which Meta’s Muse Spark reached the public internet and compromised an external organization’s service.² A configuration error in Irregular’s environment left outbound connectivity enabled where the environment was specified as isolated; Irregular’s stated position is that the episode was neither a sandbox escape nor a sophisticated attack, and that it was “the exact same evaluation-environment issue” behind Anthropic’s three disclosed incidents a week earlier.²³ Those three are the larger half of Irregular’s exposure: Claude models reached production infrastructure at three organizations from Irregular environments, in one case extracting credentials and opening a database of several hundred rows of production data. OpenAI has a separate Irregular-linked incident distinct from the Hugging Face breach.¹

The firm is the wiki’s clearest instance of concentration risk in the evaluation supply chain: independent assessment of frontier models runs through a small number of specialist vendors, so an environment defect at one reaches several otherwise unrelated labs’ most capable models. Evaluation Containment Failure carries the argument.

Notes

Footnotes

1. Meta AI Hacked Another Company — 4th Disclosure in a Month, explainx.ai, 2026-08-06. ↩ ↩²

2. An AI model from Meta also hacked another company during testing, CNN Business, 2026-08-05. ↩ ↩²

3. Meta AI Hacked External Systems During Cybersecurity Testing, SecurityWeek, 2026-08-06. ↩