OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answering

View PDF HTML (experimental)

Abstract:Long-horizon omnimodal question answering answers questions by reasoning over text, images, audio, and video. Despite recent progress on OmniLLMs, low-resource long audio-video QA still suffers from costly dense encoding, weak fine-grained retrieval, limited proactive planning, and no clear end-to-end optimization. To address these issues, we propose OmniRAG-Agent, an agentic omnimodal QA method for budgeted long audio-video reasoning. It builds an image-audio retrieval-augmented generation module that lets an OmniLLM fetch short, relevant frames and audio snippets from external banks. Moreover, it uses an agent loop that plans, calls tools across turns, and merges retrieved evidence to answer complex queries. Furthermore, we apply group relative policy optimization to jointly improve tool use and answer quality over time. Experiments on OmniVideoBench, WorldSense, and Daily-Omni show that OmniRAG-Agent consistently outperforms prior methods under low-resource settings and achieves strong results, with ablations validating each component.

Subjects:

Computation and Language (cs.CL)

Cite as: arXiv:2602.03707 [cs.CL]

(or arXiv:2602.03707v4 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2602.03707

arXiv-issued DOI via DataCite

Submission history

From: Xinyu Mu [view email] [v1] Tue, 3 Feb 2026 16:28:24 UTC (15,724 KB) [v2] Wed, 4 Feb 2026 03:33:14 UTC (15,724 KB) [v3] Sun, 22 Feb 2026 15:44:32 UTC (15,724 KB) [v4] Mon, 30 Mar 2026 11:14:57 UTC (15,726 KB)