AgentLeak: A Full-Stack Benchmark for Privacy Leakage in Multi-Agent LLM Systems
arXiv:2602.11510v2 Announce Type: replace Abstract: Multi-agent Large Language Model (LLM) systems create privacy risks that current benchmarks cannot measure. When agents coordinate on tasks, sensitive data passes through inter-agent messages, shared memory, and tool arguments, all pathways that output-only audits never inspect. We introduce AgentLeak, to the best of our knowledge the first full-stack benchmark for privacy leakage covering internal channels. It spans 1,000 scenarios across healthcare, finance, legal, and corporate domains, paired with a 32-class attack taxonomy and a three-ti — Faouzi El Yagoubi, Godwin Badu-Marfo, Ranwa Al Mallah
View PDF HTML (experimental)
Abstract:Multi-agent Large Language Model (LLM) systems create privacy risks that current benchmarks cannot measure. When agents coordinate on tasks, sensitive data passes through inter-agent messages, shared memory, and tool arguments, all pathways that output-only audits never inspect. We introduce AgentLeak, to the best of our knowledge the first full-stack benchmark for privacy leakage covering internal channels. It spans 1,000 scenarios across healthcare, finance, legal, and corporate domains, paired with a 32-class attack taxonomy and a three-tier detection pipeline. A factorial evaluation crossing five production LLMs (GPT-4o, GPT-4o-mini, Claude 3.5 Sonnet, Mistral Large, and Llama 3.3 70B) with all 1,000 scenarios, yielding 4,979 validated execution traces, reveals that multi-agent configurations reduce per-channel output leakage (C1: 27.2% vs 43.2% in single-agent) but introduce unmonitored internal channels that raise total system exposure to 68.9% (aggregated across C1, C2, C5). Internal channels account for most of this gap: inter-agent messages (C2) leak at 68.8%, compared to 27.2% on C1 (output channel). This means that output-only audits miss 41.7% of violations. Safety-aligned models achieve lower leakage on both external and internal channels, yet no model eliminates it. Across all five models and four domains, the pattern C2 $\geq$ C1 holds consistently, confirming that inter-agent communication is the primary vulnerability. These results establish that output-only auditing is fundamentally insufficient for multi-agent systems and that privacy controls must be extended to inter-agent communication channels.
Comments: 17 pages, 10 figures, 13 tables. Code and dataset available at this https URL
Subjects:
Artificial Intelligence (cs.AI)
MSC classes: 68T01
ACM classes: K.4.1; I.2.11; I.2.7
Cite as: arXiv:2602.11510 [cs.AI]
(or arXiv:2602.11510v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2602.11510
arXiv-issued DOI via DataCite
Submission history
From: Faouzi El Yagoubi [view email] [v1] Thu, 12 Feb 2026 03:10:44 UTC (1,721 KB) [v2] Fri, 27 Mar 2026 23:13:47 UTC (621 KB)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
researchpaperarxiv
Fair decisions, clear reasons: Creating Fuzzy AI with fairness built in from the start - Asia Research News |
<a href="https://news.google.com/rss/articles/CBMirAFBVV95cUxOS2ZFSlhpUDZueldBUXpTQ1MtZ3QwX0l1bjhsQzAxRjF1ZnhQOHNDcTA4VzM0a3FKQ0pSRUY3Q2JCa3lVTnBxU29jLU5Gd2Rnb2stQmxnSl95MDNVZGlCSHVJZVFfQWp1azR3UnNOU3pkZjlXTW1ESTU4V0lMdy1RbFRLUC10anpoUUpwZ0dKc2E2VVdvUDBCb2tQbHRLSTNrcW8zMHJiMGdmNHJC?oc=5" target="_blank">Fair decisions, clear reasons: Creating Fuzzy AI with fairness built in from the start</a> <font color="#6f6f6f">Asia Research News |</font>
ANU partners with AI safety company Anthropic to strengthen its AI research and teaching - anu.edu.au
<a href="https://news.google.com/rss/articles/CBMiwwFBVV95cUxQd1FoSHltem1yYjdFNzJGRFZtY0lOLXpnOVRiMzZENEdXZVRDYV9JQVRFbnNwdVB3QU5tYm4tWDF5V3YzYkIwYTdNYThVUjdRcjNoTmRQZWs4SWM0d1BZMi1VdEdHTG1xVWhrTkx4YzRrckVGZDFtRVdDdDVPSGFiQnBPR0tycHZrR0VSUndzODlFRi1STFM2R0NlN2RsbUo5Y3l1NzhlZmxvWXA0NHpNVDBYeGxwT0lJWFpvaUxUNDJma0U?oc=5" target="_blank">ANU partners with AI safety company Anthropic to strengthen its AI research and teaching</a> <font color="#6f6f6f">anu.edu.au</font>
Exclusive | Caltech Researchers Claim Radical Compression of High-Fidelity AI Models - WSJ
<a href="https://news.google.com/rss/articles/CBMiuANBVV95cUxQcVJ1dXhZMV9JTFB6RDhvYnpjc0hmQmFyOXMwaHpGRGx5VGo1MXlrQTQwb1c1VkRrMVNkbFdBcFNXcXZydjNyQzA2aEpIMnhPUXRvWkR4dlVnXzIwTC00cVlRbXY2WHZ5NTlWeDUzMDF6TjNVb2w1MGJCUGNPWWdWcFE3dmdmb1pIUXlQLU85SG4tVlNpWWxqRmUyU0dyYnUxZzJMN0dVUEVWTXR6Ni1WS3Ffa3A5aU43WWlibjhLYWZTaUhqTmFuLTdTbk5EMnBwQ2NuSEstZUpQQnVnQ3Fwa1oweW1rLXZscHlvVnhLdTA0WlMtdE82YW83cjhwRUpmQW9TbFM5VlN2SGpJU1RFUVpkYmItMEt6VElZUHYzbHpQLWtiTnNvMFdJZUNEXzJVR1l2bWQ1NlpYbjhrZFVfVlNpMGhXZzFFdWZIV1Bmbjc3WXYxQXFFWU1ROFlwZ1hZZUp3MHVjbHpWVHB4SF9JSEd0VTJudjB2UGxVNmxiWGNZOG9sUlhzdnVFS0tXRUdHdGZZZ0dDZlJBb01SWUV3RDJTR1hVbldJemFwYWl5QlAzeVJWVGc1Mg?oc=5" target="_blank">Exclusive | Caltech Researchers Claim Radical Compression of High-Fidelity AI Models</a> <font color="#6f6f6f">WSJ</font>
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
More in Research Papers
Beyond Symbolic Solving: Multi Chain-of-Thought Voting for Geometric Reasoning in Large Language Models
arXiv:2604.00890v1 Announce Type: new Abstract: Geometric Problem Solving (GPS) remains at the heart of enhancing mathematical reasoning in large language models because it requires the combination of diagrammatic understanding, symbolic manipulation and logical inference. In existing literature, researchers have chiefly focused on synchronising the diagram descriptions with text literals and solving the problem. In this vein, they have either taken a neural, symbolic or neuro-symbolic approach. But this solves only the first two of the requirements, namely diagrammatic understanding and symbolic manipulation, while leaving logical inference underdeveloped. The logical inference is often limited to one chain-of-thought (CoT). To address this weakness in hitherto existing models, this paper
Google research suggests encryption technique used by Bitcoin will be cracked by quantum computers around 2029 — search giant says quantum attacks need to be prepared for now
Google research suggests encryption technique used by Bitcoin will be cracked by quantum computers around 2029 — search giant says quantum attacks need to be prepared for now
ARGS: Auto-Regressive Gaussian Splatting via Parallel Progressive Next-Scale Prediction
arXiv:2604.00494v1 Announce Type: new Abstract: Auto-regressive frameworks for next-scale prediction of 2D images have demonstrated strong potential for producing diverse and sophisticated content by progressively refining a coarse input. However, extending this paradigm to 3D object generation remains largely unexplored. In this paper, we introduce auto-regressive Gaussian splatting (ARGS), a framework for making next-scale predictions in parallel for generation according to levels of detail. We propose a Gaussian simplification strategy and reverse the simplification to guide next-scale generation. Benefiting from the use of hierarchical trees, the generation process requires only \(\mathcal{O}(\log n)\) steps, where \(n\) is the number of points. Furthermore, we propose a tree-based tra
Benchmarking Filtered Approximate Nearest Neighbor Search Algorithms on Transformer-based Embedding Vectors
arXiv:2507.21989v3 Announce Type: replace-cross Abstract: Advances in embedding models for text, image, audio, and video drive progress across multiple domains, including retrieval-augmented generation, recommendation systems, and others. Many of these applications require an efficient method to retrieve items that are close to a given query in the embedding space while satisfying a filter condition based on the item's attributes, a problem known as filtered approximate nearest neighbor search (FANNS). By performing an in-depth literature analysis on FANNS, we identify a key gap in the research landscape: publicly available datasets with embedding vectors from state-of-the-art transformer-based text embedding models that contain abundant real-world attributes covering a broad spectrum of a
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!