vGamba: Attentive State Space Bottleneck for efficient Long-range Dependencies in Visual Recognition

View PDF HTML (experimental)

Abstract:Capturing long-range dependencies (LRD) efficiently is a core challenge in visual recognition, and state-space models (SSMs) have recently emerged as a promising alternative to self-attention for addressing it. However, adapting SSMs into CNN-based bottlenecks remains challenging, as existing approaches require complex pre-processing and multiple SSM replicas per block, limiting their practicality. We propose vGamba, a hybrid vision backbone that replaces the standard bottleneck convolution with a single lightweight SSM block, the Gamba cell, which incorporates 2D positional awareness and an attentive spatial context (ASC) module for efficient LRD modeling. Results on diverse downstream vision tasks demonstrate competitive accuracy against SSM-based models such as VMamba and ViM, while achieving significantly improved computation and memory efficiency over Bottleneck Transformer (BotNet). For example, at $2048 \times 2048$ resolution, vGamba is $2.07 \times$ faster than BotNet and reduces peak GPU memory by 93.8% (1.03GB vs. 16.78GB), scaling near-linearly with resolution comparable to ResNet-50. These results demonstrate that Gamba Bottleneck effectively overcomes the memory and compute constraints of BotNet global modeling, establishing it as a practical and scalable backbone for high-resolution vision tasks.

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2503.21262 [cs.CV]

(or arXiv:2503.21262v3 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2503.21262

arXiv-issued DOI via DataCite

Submission history

From: Yunusa Haruna [view email] [v1] Thu, 27 Mar 2025 08:39:58 UTC (14,993 KB) [v2] Mon, 17 Nov 2025 13:12:41 UTC (12,902 KB) [v3] Mon, 30 Mar 2026 05:07:32 UTC (17,132 KB)