WikiCLIP: An Efficient Contrastive Baseline for Open-domain Visual Entity Recognition

View PDF HTML (experimental)

Abstract:Open-domain visual entity recognition (VER) seeks to associate images with entities in encyclopedic knowledge bases such as Wikipedia. Recent generative methods tailored for VER demonstrate strong performance but incur high computational costs, limiting their scalability and practical deployment. In this work, we revisit the contrastive paradigm for VER and introduce WikiCLIP, a simple yet effective framework that establishes a strong and efficient baseline for open-domain VER. WikiCLIP leverages large language model embeddings as knowledge-rich entity representations and enhances them with a Vision-Guided Knowledge Adaptor (VGKA) that aligns textual semantics with visual cues at the patch level. To further encourage fine-grained discrimination, a Hard Negative Synthesis Mechanism generates visually similar but semantically distinct negatives during training. Experimental results on popular open-domain VER benchmarks, such as OVEN, demonstrate that WikiCLIP significantly outperforms strong baselines. Specifically, WikiCLIP achieves a 16% improvement on the challenging OVEN unseen set, while reducing inference latency by nearly 100 times compared with the leading generative model, AutoVER. The project page is available at this https URL

Comments: Accepted by CVPR26, codes and weights are publicly available

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2603.09921 [cs.CV]

(or arXiv:2603.09921v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2603.09921

arXiv-issued DOI via DataCite

Submission history

From: Longtian Qiu [view email] [v1] Tue, 10 Mar 2026 17:18:53 UTC (4,975 KB) [v2] Sun, 29 Mar 2026 15:41:56 UTC (4,969 KB)