V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models

View PDF HTML (experimental)

Abstract:Video large language models (VideoLLMs) show strong capability in video understanding, yet long-context inference is still dominated by massive redundant visual tokens in the prefill stage. We revisit token compression for VideoLLMs under a tight budget and identify a key bottleneck, namely insufficient spatio-temporal information coverage. Existing methods often introduce discontinuous coverage through coarse per-frame allocation or scene segmentation, and token merging can further misalign spatio-temporal coordinates under MRoPE-style discrete (t,h,w) bindings. To address these issues, we propose V-CAST (Video Curvature-Aware Spatio-Temporal Pruning), a training-free, plug-and-play pruning policy for long-context video inference. V-CAST casts token compression as a trajectory approximation problem and introduces a curvature-guided temporal allocation module that routes per-frame token budgets to semantic turns and event boundaries. It further adopts a dual-anchor spatial selection mechanism that preserves high-entropy visual evidence without attention intervention, while keeping retained tokens at their original coordinates to maintain positional alignment. Extensive experiments across multiple VideoLLMs of different architectures and scales demonstrate that V-CAST achieves 98.6% of the original performance, outperforms the second-best method by +1.1% on average, and reduces peak memory and total latency to 86.7% and 86.4% of vanilla Qwen3-VL-8B-Instruct.

Comments: Code: \url{this https URL}

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2603.27650 [cs.CV]

(or arXiv:2603.27650v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2603.27650

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Xuyang Liu [view email] [v1] Sun, 29 Mar 2026 11:53:32 UTC (33,362 KB)