Unleashing the Potential of Mamba: Boosting a LiDAR 3D Sparse Detector by Using Cross-Model Knowledge Distillation

View PDF HTML (experimental)

Abstract:The LiDAR 3D object detector that strikes a balance between accuracy and speed is crucial for achieving real-time perception in autonomous driving. However, many existing LiDAR detection models depend on complex feature transformations, leading to poor real-time performance and high resource consumption, which limits their practical effectiveness. In this work, we propose a faster LiDAR 3D object detector, a framework that adaptively aligns sparse voxels to enable efficient heterogeneous knowledge distillation, called FASD. We aim to distill the Transformer sequence modeling capability into Mamba models, significantly boosting accuracy through knowledge transfer. Specifically, we first design the architecture for cross-model knowledge distillation to impart the global contextual understanding capabilities of the Transformer to Mamba. Transformer-based teacher model employ a scale-adaptive attention mechanism to enhance multiscale fusion. In contrast, Mamba-based student model leverages feature alignment through spatial-based adapters, supervised with latent space feature and span-head distillation losses, leading to improved performance and efficiency. We evaluated the FASD on the Waymo and nuScenes datasets, achieving a 4x reduction in resource consumption and a 1-2% performance improvement over the baseline, while also delivering significant gains in accuracy and efficiency in real deployment.

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2409.11018 [cs.CV]

(or arXiv:2409.11018v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2409.11018

arXiv-issued DOI via DataCite

Submission history

From: Rui Yu [view email] [v1] Tue, 17 Sep 2024 09:30:43 UTC (10,767 KB) [v2] Mon, 30 Mar 2026 17:02:43 UTC (8,035 KB)