Models transformer neural network training announce product application

On the Mirage of Long-Range Dependency, with an Application to Integer Multiplication

arXiv cs.LGby Zichao WeiApril 1, 20261 min read0 views

arXiv:2603.29069v1 Announce Type: new Abstract: Integer multiplication has long been considered a hard problem for neural networks, with the difficulty widely attributed to the O(n) long-range dependency induced by carry chains. We argue that this diagnosis is wrong: long-range dependency is not an intrinsic property of multiplication, but a mirage produced by the choice of computational spacetime. We formalize the notion of mirage and provide a constructive proof: when two n-bit binary integers are laid out as a 2D outer-product grid, every step of long multiplication collapses into a $3 \times 3$ local neighborhood operation. Under this representation, a neural cellular automaton with only 321 learnable parameters achieves perfect length generalization up to $683\times$ the training rang

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Connected Papers Toggle

Litmaps Toggle

scite.ai Toggle

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

Links to Code Toggle

DagsHub Toggle

GotitPub Toggle

Huggingface Toggle

Links to Code Toggle

ScienceCast Toggle

Demos

Replicate Toggle

Spaces Toggle

Recommenders and Search Tools

Link to Influence Flower

Core recommender toggle

IArxiv recommender toggle

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Original source

arXiv cs.LG

https://arxiv.org/abs/2603.29069

Was this article helpful?