Can Vision Foundation Models Navigate? Zero-Shot Real-World Evaluation and Lessons Learned

View PDF HTML (experimental)

Abstract:Visual Navigation Models (VNMs) promise generalizable, robot navigation by learning from large-scale visual demonstrations. Despite growing real-world deployment, existing evaluations rely almost exclusively on success rate, whether the robot reaches its goal, which conceals trajectory quality, collision behavior, and robustness to environmental change. We present a real-world evaluation of five state-of-the-art VNMs (GNM, ViNT, NoMaD, NaviBridger, and CrossFormer) across two robot platforms and five environments spanning indoor and outdoor settings. Beyond success rate, we combine path-based metrics with vision-based goal-recognition scores and assess robustness through controlled image perturbations (motion blur, sunflare). Our analysis uncovers three systematic limitations: (a) even architecturally sophisticated diffusion and transformer-based models exhibit frequent collisions, indicating limited geometric understanding; (b) models fail to discriminate between different locations that are perceptually similar, however some semantics differences are present, causing goal prediction errors in repetitive environments; and (c) performance degrades under distribution shift. We will publicly release our evaluation codebase and dataset to facilitate reproducible benchmarking of VNMs.

Subjects:

Robotics (cs.RO); Machine Learning (cs.LG)

Cite as: arXiv:2603.25937 [cs.RO]

(or arXiv:2603.25937v1 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2603.25937

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Maeva Guerrier [view email] [v1] Thu, 26 Mar 2026 22:04:49 UTC (34,206 KB)