Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation

View PDF HTML (experimental)

Abstract:Retrieval-Augmented Generation (RAG) has emerged as a powerful approach to boost the capabilities of large language models (LLMs) by incorporating external, up-to-date knowledge sources. However, this introduces a potential vulnerability to knowledge poisoning attacks, where attackers can compromise the knowledge source to mislead the generation model. One such attack is the PoisonedRAG in which the injected adversarial texts steer the model to generate an attacker-chosen response to a target question. In this work, we propose novel defense methods, FilterRAG and ML-FilterRAG, to mitigate the PoisonedRAG attack. First, we propose a new property to uncover distinct properties to differentiate between adversarial and clean texts in the knowledge data source. Next, we employ this property to filter out adversarial texts from clean ones in the design of our proposed approaches. Evaluation of these methods using benchmark datasets demonstrate their effectiveness, with performances close to those of the original RAG systems.

Comments: Preprint for Submission

Subjects:

Machine Learning (cs.LG); Information Retrieval (cs.IR)

Cite as: arXiv:2508.02835 [cs.LG]

(or arXiv:2508.02835v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2508.02835

arXiv-issued DOI via DataCite

Submission history

From: Kennedy Edemacu [view email] [v1] Mon, 4 Aug 2025 19:03:52 UTC (346 KB) [v2] Fri, 27 Mar 2026 16:32:20 UTC (356 KB)