When to Think and When to Look: Uncertainty-Guided Lookback
arXiv:2511.15613v3 Announce Type: replace-cross Abstract: Test-time thinking (that is, generating explicit intermediate reasoning chains) is known to boost performance in large language models and has recently shown strong gains for large vision language models (LVLMs). However, despite these promising results, there is still no systematic analysis of how thinking actually affects visual reasoning. We provide the first such analysis with a large scale, controlled comparison of thinking for LVLMs, evaluating ten variants from the InternVL3.5 and Qwen3-VL families on MMMU-val under generous toke — Jing Bi, Filippos Bellos, Junjia Guo, Yayuan Li, Chao Huang, Yolo Y. Tang, Luchuan Song, Susan Liang, Zhongfei Mark Zhang, Jason J. Corso, Chenliang Xu
Authors:Jing Bi, Filippos Bellos, Junjia Guo, Yayuan Li, Chao Huang, Yolo Y. Tang, Luchuan Song, Susan Liang, Zhongfei Mark Zhang, Jason J. Corso, Chenliang Xu
View PDF HTML (experimental)
Abstract:Test-time thinking (that is, generating explicit intermediate reasoning chains) is known to boost performance in large language models and has recently shown strong gains for large vision language models (LVLMs). However, despite these promising results, there is still no systematic analysis of how thinking actually affects visual reasoning. We provide the first such analysis with a large scale, controlled comparison of thinking for LVLMs, evaluating ten variants from the InternVL3.5 and Qwen3-VL families on MMMU-val under generous token budgets and multi pass decoding. We show that more thinking is not always better; long chains often yield long wrong trajectories that ignore the image and underperform the same models run in standard instruct mode. A deeper analysis reveals that certain short lookback phrases, which explicitly refer back to the image, are strongly enriched in successful trajectories and correlate with better visual grounding. Building on this insight, we propose uncertainty guided lookback, a training free decoding strategy that combines an uncertainty signal with adaptive lookback prompts and breadth search. Our method improves overall MMMU performance, delivers the largest gains in categories where standard thinking is weak, and outperforms several strong decoding baselines, setting a new state of the art under fixed model families and token budgets. We further show that this decoding strategy generalizes, yielding consistent improvements on five additional benchmarks, including two broad multimodal suites and math focused visual reasoning datasets.
Comments: Accepted to CVPR 2026
Subjects:
Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
Cite as: arXiv:2511.15613 [cs.CV]
(or arXiv:2511.15613v3 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2511.15613
arXiv-issued DOI via DataCite
Submission history
From: Jing Bi [view email] [v1] Wed, 19 Nov 2025 17:01:02 UTC (8,681 KB) [v2] Tue, 25 Nov 2025 16:38:28 UTC (8,681 KB) [v3] Fri, 27 Mar 2026 17:10:24 UTC (8,681 KB)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
researchpaperarxiv
Enhancing REST API Fuzzing with Access Policy Violation Checks and Injection Attacks
arXiv:2604.00702v1 Announce Type: new Abstract: Due to their widespread use in industry, several techniques have been proposed in the literature to fuzz REST APIs. Existing fuzzers for REST APIs have been focusing on detecting crashes (e.g., 500 HTTP server error status code). However, security vulnerabilities can have major drastic consequences on existing cloud infrastructures. In this paper, we propose a series of novel automated oracles aimed at detecting violations of access policies in REST APIs, as well as executing traditional attacks such as SQL Injection and XSS. These novel automated oracles can be integrated into existing fuzzers, in which, once the fuzzing session is completed, a ``security testing'' phase is executed to verify these oracles. When a security fault is detected,
LibScan: Smart Contract Library Misuse Detection with Iterative Feedback and Static Verification
arXiv:2604.00657v1 Announce Type: new Abstract: Smart contracts are self-executing programs that manage financial transactions on blockchain networks. Developers commonly rely on third-party code libraries to improve both efficiency and security. However, improper use of these libraries can introduce hidden vulnerabilities that are difficult to detect, leading to significant financial losses. Existing automated tools struggle to identify such misuse because it often requires understanding the developer's intent rather than simply scanning for known code patterns. This paper presents LibScan, an automated detection framework that combines large language model (LLM)-based semantic reasoning with rule-based code analysis, identifying eight distinct categories of library misuse in smart contra
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
More in Research Papers
An Empirical Study on How Architectural Topology Affects Microservice Performance and Energy Usage
arXiv:2604.00080v1 Announce Type: new Abstract: Microservice architectures form the backbone of modern software systems for their scalability, resilience, and maintainability, but their rise in cloud-native environments raises energy efficiency concerns. While prior research addresses microservice decomposition and placement, the impact of topology, the structural arrangement and interaction pattern among services, on energy efficiency remains largely underexplored. This study quantifies the impact of topologies on energy efficiency and performance across six canonical ones (Sequential Fan-Out, Parallel Fan-Out, Chain, Hierarchical, Probabilistic, Mesh), each instantiated at 5-, 10-, and 20-service scales using the $\mu\text{Bench}$ framework. We measure throughput, response time, energy u
Extreme Conformal Prediction: Reliable Intervals for High-Impact Events
arXiv:2505.08578v3 Announce Type: replace-cross Abstract: Conformal prediction is a popular method to construct prediction intervals with marginal coverage guarantees from black-box machine learning models. In applications with potentially high-impact events, such as flooding or financial crises, regulators often require very high confidence for such intervals. However, if the desired level of confidence is too large relative to the amount of data used for calibration, then classical conformal methods provide infinitely wide, thus, uninformative prediction intervals. In this paper, we propose — Olivier C. Pasche, Henry Lam, Sebastian Engelke
Gradient-flow SDEs have unique transient population dynamics
arXiv:2505.21770v3 Announce Type: replace-cross Abstract: Identifying the drift and diffusion of an SDE from its population dynamics is a notoriously challenging task. Researchers in machine learning and single-cell biology have only been able to prove a partial identifiability result: for potential-driven SDEs, the gradient-flow drift can be identified from temporal marginals if the Brownian diffusivity is already known. Existing methods therefore assume that the diffusivity is known a priori, despite it being unknown in practice. We dispel the need for this assumption by providing a complete — Vincent Guan, Joseph Janssen, Nicolas Lanzetti, Antonio Terpin, Geoffrey Schiebinger, Elina Robeva
Uncertainty Quantification With Multiple Sources
arXiv:2405.06479v4 Announce Type: replace-cross Abstract: Weighted conformal prediction (WCP) has been commonly used to quantify prediction uncertainty under covariate shift. However, the effectiveness of WCP relies heavily on the degree of overlap between the training and test covariate distributions. This challenge is exacerbated in multi-source settings with varying covariate distributions, where direct application of WCP can be impractical. In this paper, we address the multi-source setup by leveraging WCP under the assumption of a shared conditional distribution. We investigate two extens — Mufang Ying, Wenge Guo, Koulik Khamaru, Ying Hung
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!