Paper
Agents of Chaos
Authors
Natalie Shapira, Chris Wendler, Avery Yen, Gabriele Sarti, Koyena Pal, Olivia Floody, Adam Belfki, Alex Loftus, Aditya Ratan Jannali, Nikhil Prakash, Jasmine Cui, Giordano Rogers, Jannik Brinkmann, Can Rager, Amir Zur, Michael Ripa, Aruna Sankaranarayanan, David Atkinson, Rohit Gandikota, Jaden Fiotto-Kaufman, EunJeong Hwang, Hadas Orgad, P Sam Sahil, Negev Taglicht, Tomer Shabtay, Atai Ambus, Nitay Alon, Shiri Oron, Ayelet Gordon-Tapiero, Yotam Kaplan, Vered Shwartz, Tamar Rott Shaham, Christoph Riedl, Reuth Mirsky, Maarten Sap, David Manheim, Tomer Ullman, David Bau
Abstract
We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. In several cases, agents reported task completion while the underlying system state contradicted those reports. We also report on some of the failed attempts. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings. These behaviors raise unresolved questions regarding accountability, delegated authority, and responsibility for downstream harms, and warrant urgent attention from legal scholars, policymakers, and researchers across disciplines. This report serves as an initial empirical contribution to that broader conversation.
Metadata
Related papers
Vibe Coding XR: Accelerating AI + XR Prototyping with XR Blocks and Gemini
Ruofei Du, Benjamin Hersh, David Li, Nels Numan, Xun Qian, Yanhe Chen, Zhongy... • 2026-03-25
Comparing Developer and LLM Biases in Code Evaluation
Aditya Mittal, Ryan Shar, Zichu Wu, Shyam Agarwal, Tongshuang Wu, Chris Donah... • 2026-03-25
The Stochastic Gap: A Markovian Framework for Pre-Deployment Reliability and Oversight-Cost Auditing in Agentic Artificial Intelligence
Biplab Pal, Santanu Bhattacharya • 2026-03-25
Retrieval Improvements Do Not Guarantee Better Answers: A Study of RAG for AI Policy QA
Saahil Mathur, Ryan David Rittner, Vedant Ajit Thakur, Daniel Stuart Schiff, ... • 2026-03-25
MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination
Zhuo Li, Yupeng Zhang, Pengyu Cheng, Jiajun Song, Mengyu Zhou, Hao Li, Shujie... • 2026-03-25
Raw Data (Debug)
{
"raw_xml": "<entry>\n <id>http://arxiv.org/abs/2602.20021v1</id>\n <title>Agents of Chaos</title>\n <updated>2026-02-23T16:28:48Z</updated>\n <link href='https://arxiv.org/abs/2602.20021v1' rel='alternate' type='text/html'/>\n <link href='https://arxiv.org/pdf/2602.20021v1' rel='related' title='pdf' type='application/pdf'/>\n <summary>We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. In several cases, agents reported task completion while the underlying system state contradicted those reports. We also report on some of the failed attempts. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings. These behaviors raise unresolved questions regarding accountability, delegated authority, and responsibility for downstream harms, and warrant urgent attention from legal scholars, policymakers, and researchers across disciplines. This report serves as an initial empirical contribution to that broader conversation.</summary>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.AI'/>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.CY'/>\n <published>2026-02-23T16:28:48Z</published>\n <arxiv:primary_category term='cs.AI'/>\n <author>\n <name>Natalie Shapira</name>\n </author>\n <author>\n <name>Chris Wendler</name>\n </author>\n <author>\n <name>Avery Yen</name>\n </author>\n <author>\n <name>Gabriele Sarti</name>\n </author>\n <author>\n <name>Koyena Pal</name>\n </author>\n <author>\n <name>Olivia Floody</name>\n </author>\n <author>\n <name>Adam Belfki</name>\n </author>\n <author>\n <name>Alex Loftus</name>\n </author>\n <author>\n <name>Aditya Ratan Jannali</name>\n </author>\n <author>\n <name>Nikhil Prakash</name>\n </author>\n <author>\n <name>Jasmine Cui</name>\n </author>\n <author>\n <name>Giordano Rogers</name>\n </author>\n <author>\n <name>Jannik Brinkmann</name>\n </author>\n <author>\n <name>Can Rager</name>\n </author>\n <author>\n <name>Amir Zur</name>\n </author>\n <author>\n <name>Michael Ripa</name>\n </author>\n <author>\n <name>Aruna Sankaranarayanan</name>\n </author>\n <author>\n <name>David Atkinson</name>\n </author>\n <author>\n <name>Rohit Gandikota</name>\n </author>\n <author>\n <name>Jaden Fiotto-Kaufman</name>\n </author>\n <author>\n <name>EunJeong Hwang</name>\n </author>\n <author>\n <name>Hadas Orgad</name>\n </author>\n <author>\n <name>P Sam Sahil</name>\n </author>\n <author>\n <name>Negev Taglicht</name>\n </author>\n <author>\n <name>Tomer Shabtay</name>\n </author>\n <author>\n <name>Atai Ambus</name>\n </author>\n <author>\n <name>Nitay Alon</name>\n </author>\n <author>\n <name>Shiri Oron</name>\n </author>\n <author>\n <name>Ayelet Gordon-Tapiero</name>\n </author>\n <author>\n <name>Yotam Kaplan</name>\n </author>\n <author>\n <name>Vered Shwartz</name>\n </author>\n <author>\n <name>Tamar Rott Shaham</name>\n </author>\n <author>\n <name>Christoph Riedl</name>\n </author>\n <author>\n <name>Reuth Mirsky</name>\n </author>\n <author>\n <name>Maarten Sap</name>\n </author>\n <author>\n <name>David Manheim</name>\n </author>\n <author>\n <name>Tomer Ullman</name>\n </author>\n <author>\n <name>David Bau</name>\n </author>\n </entry>"
}