Poppy: Polarization-based Plug-and-Play Guidance for Enhancing Monocular Normal Estimation

View PDF HTML (experimental)

Abstract:Monocular surface normal estimators trained on large-scale RGB-normal data often perform poorly in the edge cases of reflective, textureless, and dark surfaces. Polarization encodes surface orientation independently of texture and albedo, offering a physics-based complement for these cases. Existing polarization methods, however, require multi-view capture or specialized training data, limiting generalization. We introduce Poppy, a training-free framework that refines normals from any frozen RGB backbone using single-shot polarization measurements at test time. Keeping backbone weights frozen, Poppy optimizes per-pixel offsets to the input RGB and output normal along with a learned reflectance decomposition. A differentiable rendering layer converts the refined normals into polarization predictions and penalizes mismatches with the observed signal. Across seven benchmarks and three backbone architectures (diffusion, flow, and feed-forward), Poppy reduces mean angular error by 23-26% on synthetic data and 6-16% on real data. These results show that guiding learned RGB-based normal estimators with polarization cues at test time refines normals on challenging surfaces without retraining.

Comments: project page: this https URL

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2603.27891 [cs.CV]

(or arXiv:2603.27891v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2603.27891

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Irene Kim [view email] [v1] Sun, 29 Mar 2026 22:05:09 UTC (62,845 KB)