Live
Black Hat USAAI BusinessBlack Hat AsiaAI BusinessThe AI-Powered Agency: A Developer Playbook for Selling AI Services in 2026Dev.to AIYour AI Chatbot Isn't Stupid. It Just Has No Memory. Here's How We Fixed That.Dev.to AIInternational RegLab Project reports on AI use in nuclear power plant operations - Nuclear Energy Agency (NEA)Google News: AIAI Agent Tools for Small Business Owners: A Practical GuideDev.to AIPRH Germany sues OpenAI for ‘copyright infringement’ of children’s series - The BooksellerGoogle News: OpenAIEmail obfuscation: What works in 2026?!DEV CommunityReply Signs Strategic Collaboration Agreement with AWS to Accelerate AI-Driven Cloud Transformation - Press Release HubGoogle News: Generative AIDeepSource vs Qodana: Code Quality Platforms Compared (2026)DEV CommunityThe Senior Angular Take‑Home That Made Me Rethink Tech InterviewsDEV CommunityClaude Code Leak: 16 Lessons on Building Production-Ready AI SystemsAnalytics VidhyaImage Optimisation Strategies for Better LCP ScoresDEV CommunityStop Building AI Into Your Product. Start Building Products With AI.DEV CommunityBlack Hat USAAI BusinessBlack Hat AsiaAI BusinessThe AI-Powered Agency: A Developer Playbook for Selling AI Services in 2026Dev.to AIYour AI Chatbot Isn't Stupid. It Just Has No Memory. Here's How We Fixed That.Dev.to AIInternational RegLab Project reports on AI use in nuclear power plant operations - Nuclear Energy Agency (NEA)Google News: AIAI Agent Tools for Small Business Owners: A Practical GuideDev.to AIPRH Germany sues OpenAI for ‘copyright infringement’ of children’s series - The BooksellerGoogle News: OpenAIEmail obfuscation: What works in 2026?!DEV CommunityReply Signs Strategic Collaboration Agreement with AWS to Accelerate AI-Driven Cloud Transformation - Press Release HubGoogle News: Generative AIDeepSource vs Qodana: Code Quality Platforms Compared (2026)DEV CommunityThe Senior Angular Take‑Home That Made Me Rethink Tech InterviewsDEV CommunityClaude Code Leak: 16 Lessons on Building Production-Ready AI SystemsAnalytics VidhyaImage Optimisation Strategies for Better LCP ScoresDEV CommunityStop Building AI Into Your Product. Start Building Products With AI.DEV Community
Eigenvector logo
AI NEWS HUBbyEIGENVECTOR

JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding

arXivMarch 31, 202610 min read0 views
Source Quiz

arXiv:2603.27942v1 Announce Type: cross Abstract: Japanese scene text poses challenges that multilingual benchmarks often fail to capture, including mixed scripts, frequent vertical writing, and a character inventory far larger than the Latin alphabet. Although Japanese is included in several multilingual benchmarks, these resources do not adequately capture the language-specific complexities. Meanwhile, existing Japanese visual text datasets have primarily focused on scanned documents, leaving in-the-wild scene text underexplored. To fill this gap, we introduce JaWildText, a diagnostic benchm — Koki Maeda (Institute of Science Tokyo, Tokyo, Japan, Research and Development Center for Large Language Models, National Institute of Informatics, Tokyo, Japan), Naoaki Okazaki (Institute of Science Tokyo, Tokyo, Japan, Research and Development Center for Large Language Models, National Institute of Informatics, Tokyo, Japan)

View PDF HTML (experimental)

Abstract:Japanese scene text poses challenges that multilingual benchmarks often fail to capture, including mixed scripts, frequent vertical writing, and a character inventory far larger than the Latin alphabet. Although Japanese is included in several multilingual benchmarks, these resources do not adequately capture the language-specific complexities. Meanwhile, existing Japanese visual text datasets have primarily focused on scanned documents, leaving in-the-wild scene text underexplored. To fill this gap, we introduce JaWildText, a diagnostic benchmark for evaluating vision-language models (VLMs) on Japanese scene text understanding. JaWildText contains 3,241 instances from 2,961 images newly captured in Japan, with 1.12 million annotated characters spanning 3,643 unique character types. It comprises three complementary tasks that vary in visual organization, output format, and writing style: (i) Dense Scene Text Visual Question Answering (STVQA), which requires reasoning over multiple pieces of visual text evidence; (ii) Receipt Key Information Extraction (KIE), which tests layout-aware structured extraction from mobile-captured receipts; and (iii) Handwriting OCR, which evaluates page-level transcription across various media and writing directions. We evaluate 14 open-weight VLMs and find that the best model achieves an average score of 0.64 across the three tasks. Error analyses show recognition remains the dominant bottleneck, especially for kanji. JaWildText enables fine-grained, script-aware diagnosis of Japanese scene text capabilities, and will be released with evaluation code.

Comments: 18 pages

Subjects:

Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Cite as: arXiv:2603.27942 [cs.CV]

(or arXiv:2603.27942v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2603.27942

arXiv-issued DOI via DataCite

Submission history

From: Koki Maeda [view email] [v1] Mon, 30 Mar 2026 01:36:16 UTC (5,114 KB) [v2] Tue, 31 Mar 2026 07:50:48 UTC (5,114 KB)

Was this article helpful?

Sign in to highlight and annotate this article

AI
Ask AI about this article
Powered by Eigenvector · full article context loaded
Ready

Conversation starters

Ask anything about this article…

Daily AI Digest

Get the top 5 AI stories delivered to your inbox every morning.

Knowledge Map

Knowledge Map
TopicsEntitiesSource
JaWildText:…researchpaperarxivaiartificial-…arXiv

Connected Articles — Knowledge Graph

This article is connected to other articles through shared AI topics and tags.

Knowledge Graph100 articles · 228 connections
Scroll to zoom · drag to pan · click to open

Discussion

Sign in to join the discussion

No comments yet — be the first to share your thoughts!

More in Research Papers