Papers arxiv:2603.20466 Copy markdown

Diffutron: A Masked Diffusion Language Model for Turkish Language

Published on Mar 20 · Submitted by Talha Rüzgar Akkuş on Mar 30 · Diffutron Upvote 6

• Authors: Şuayp Talha Kocabay , Talha Rüzgar Akkuş

Abstract

Masked diffusion language models offer a resource-efficient, non-autoregressive approach to text generation for Turkish through progressive instruction tuning and LoRA-based pre-training.

AI-generated summary

Masked Diffusion Language Models (MDLMs) have emerged as a compelling non-autoregressive alternative to standard large language models; however, their application to morphologically rich languages remains limited. In this paper, we introduce Diffutron, a masked diffusion language model specifically designed for Turkish. Our approach leverages a resource-efficient training pipeline, starting with LoRA-based continual pre-training of a multilingual encoder on a large-scale corpus. To enable generative capabilities, we employ a progressive instruction-tuning strategy, sequentially adapting the model on general and task-specific instruction sets. Experimental results across comprehensive benchmarks demonstrate that, despite its compact size, our model achieves competitive performance compared to existing multi-billion-parameter baselines. These findings validate the effectiveness of masked diffusion modeling combined with multi-stage tuning for non-autoregressive text generation in Turkish.

View arXiv page View PDF Add to collection

Community

Q-bert Paper author Paper submitter 3 days ago

Reply librarian-bot 2 days ago

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

• EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Training (2026)

• Raising Bars, Not Parameters: LilMoo Compact Language Model for Hindi (2026)

• AraModernBERT: Transtokenized Initialization and Long-Context Encoder Modeling for Arabic (2026)

• ViCLSR: A Supervised Contrastive Learning Framework with Natural Language Inference for Natural Language Understanding Tasks (2026)

• MzansiText and MzansiLM: An Open Corpus and Decoder-Only Language Model for South African Languages (2026)

• MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation (2026)

• Dicta-LM 3.0: Advancing The Frontier of Hebrew Sovereign LLMs (2026)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Reply EditPreview Upload images, audio, and videos by dragging in the text input, pasting, or clicking here. Tap or paste here to upload images Comment

· Sign up or log in to comment

Upvote 6

Get this paper in your agent:

hf papers read 2603.20466 Don't have the latest CLI? curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 3

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2603.20466 in a Space README.md to link it from this page.

Collections including this paper 1