TinyLoRA – Learning to Reason in 13 Parameters

View PDF HTML (experimental)

Abstract:Recent research has shown that language models can learn to \textit{reason}, often via reinforcement learning. Some work even trains low-rank parameterizations for reasoning, but conventional LoRA cannot scale below the model dimension. We question whether even rank=1 LoRA is necessary for learning to reason and propose TinyLoRA, a method for scaling low-rank adapters to sizes as small as one parameter. Within our new parameterization, we are able to train the 8B parameter size of Qwen2.5 to 91% accuracy on GSM8K with only 13 trained parameters in bf16 (26 total bytes). We find this trend holds in general: we are able to recover 90% of performance improvements while training $1000x$ fewer parameters across a suite of more difficult learning-to-reason benchmarks such as AIME, AMC, and MATH500. Notably, we are only able to achieve such strong performance with RL: models trained using SFT require $100-1000x$ larger updates to reach the same performance.

Subjects:

Machine Learning (cs.LG)

Cite as: arXiv:2602.04118 [cs.LG]

(or arXiv:2602.04118v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2602.04118

arXiv-issued DOI via DataCite

Submission history

From: John Morris [view email] [v1] Wed, 4 Feb 2026 01:20:04 UTC (1,595 KB)