Efficient Domain Adaptation for Text Line Recognition via Decoupled Language Models

View PDF HTML (experimental)

Abstract:Optical character recognition remains critical infrastructure for document digitization, yet state-of-the-art performance is often restricted to well-resourced institutions by prohibitive computational barriers. End-to-end transformer architectures achieve strong accuracy but demand hundreds of GPU hours for domain adaptation, limiting accessibility for practitioners and digital humanities scholars. We present a modular detection-and-correction framework that achieves near-SOTA accuracy with single-GPU training. Our approach decouples lightweight visual character detection (domain-agnostic) from domain-specific linguistic correction using pretrained sequence models including T5, ByT5, and BART. By training the correctors entirely on synthetic noise, we enable annotation-free domain adaptation without requiring labeled target images. Evaluating across modern clean handwriting, cursive script, and historical documents, we identify a critical "Pareto frontier" in architecture selection: T5-Base excels on modern text with standard vocabulary, whereas ByT5-Base dominates on historical documents by reconstructing archaic spellings at the byte level. Our results demonstrate that this decoupled paradigm matches end-to-end transformer accuracy while reducing compute by approximately 95%, establishing a viable, resource-efficient alternative to monolithic OCR architectures.

Comments: Accepted to the International Conference on Machine Intelligence Theory and Applications (MiTA 2026)

Subjects:

Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

Cite as: arXiv:2603.28028 [cs.CV]

(or arXiv:2603.28028v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2603.28028

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Arundhathi Dev [view email] [v1] Mon, 30 Mar 2026 04:39:26 UTC (125 KB)