iiANET: Inception Inspired Attention Hybrid Network for efficient Long-Range Dependency

View PDF HTML (experimental)

Abstract:The recent emergence of hybrid models has introduced a transformative approach to computer vision, gradually moving beyond conventional convolutional neural networks and vision transformers. However, efficiently combining these two approaches to better capture long-range dependencies in complex images remains a challenge. In this paper, we present iiANET (Inception Inspired Attention Network), an efficient hybrid visual backbone designed to improve the modeling of long-range dependencies in complex visual recognition tasks. The core innovation of iiANET is the iiABlock, a unified building block that integrates a modified global r-MHSA (Multi-Head Self-Attention) and convolutional layers in parallel. This design enables iiABlock to simultaneously capture global context and local details, making it effective for extracting rich and diverse features. By efficiently fusing these complementary representations, iiABlock allows iiANET to achieve strong feature interaction while maintaining computational efficiency. Extensive qualitative and quantitative evaluations on some SOTA benchmarks demonstrate improved performance.

Comments: 17 pages, 7 figures. Published in Transactions on Machine Learning Research (TMLR). Available at this https URL

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2407.07603 [cs.CV]

(or arXiv:2407.07603v3 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2407.07603

arXiv-issued DOI via DataCite

Journal reference: Transactions on Machine Learning Research (12/2025)

Submission history

From: Yunusa Haruna [view email] [v1] Wed, 10 Jul 2024 12:39:02 UTC (11,616 KB) [v2] Sat, 12 Apr 2025 11:32:38 UTC (1,954 KB) [v3] Sun, 29 Mar 2026 17:57:18 UTC (30,560 KB)