MLLM-based Textual Explanations for Face Comparison

View PDF HTML (experimental)

Abstract:Multimodal Large Language Models (MLLMs) have recently been proposed as a means to generate natural-language explanations for face recognition decisions. While such explanations facilitate human interpretability, their reliability on unconstrained face images remains underexplored. In this work, we systematically analyze MLLM-generated explanations for the unconstrained face verification task on the challenging IJB-S dataset, with a particular focus on extreme pose variation and surveillance imagery. Our results show that even when MLLMs produce correct verification decisions, the accompanying explanations frequently rely on non-verifiable or hallucinated facial attributes that are not supported by visual evidence. We further study the effect of incorporating information from traditional face recognition systems, viz., scores and decisions, alongside the input images. Although such information improves categorical verification performance, it does not consistently lead to faithful explanations. To evaluate the explanations beyond decision accuracy, we introduce a likelihood-ratio-based framework that measures the evidential strength of textual explanations. Our findings highlight fundamental limitations of current MLLMs for explainable face recognition and underscore the need for a principled evaluation of reliable and trustworthy explanations in biometric applications. Code is available at this https URL.

Comments: Accepted at 14th International Workshop on Biometrics and Forensics (IWBF)

Subjects:

Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Cite as: arXiv:2603.16629 [cs.CV]

(or arXiv:2603.16629v3 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2603.16629

arXiv-issued DOI via DataCite

Submission history

From: Redwan Sony [view email] [v1] Tue, 17 Mar 2026 15:01:00 UTC (7,932 KB) [v2] Wed, 18 Mar 2026 19:45:53 UTC (7,926 KB) [v3] Thu, 26 Mar 2026 19:05:31 UTC (7,926 KB)