ELViS: Efficient Visual Similarity from Local Descriptors that Generalizes Across Domains

View PDF HTML (experimental)

Abstract:Large-scale instance-level training data is scarce, so models are typically trained on domain-specific datasets. Yet in real-world retrieval, they must handle diverse domains, making generalization to unseen data critical. We introduce ELViS, an image-to-image similarity model that generalizes effectively to unseen domains. Unlike conventional approaches, our model operates in similarity space rather than representation space, promoting cross-domain transfer. It leverages local descriptor correspondences, refines their similarities through an optimal transport step with data-dependent gains that suppress uninformative descriptors, and aggregates strong correspondences via a voting process into an image-level similarity. This design injects strong inductive biases, yielding a simple, efficient, and interpretable model. To assess generalization, we compile a benchmark of eight datasets spanning landmarks, artworks, products, and multi-domain collections, and evaluate ELViS as a re-ranking method. Our experiments show that ELViS outperforms competing methods by a large margin in out-of-domain scenarios and on average, while requiring only a fraction of their computational cost. Code available at: this https URL

Comments: ICLR 2026

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2603.28603 [cs.CV]

(or arXiv:2603.28603v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2603.28603

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Pavel Suma [view email] [v1] Mon, 30 Mar 2026 15:53:42 UTC (24,430 KB)