On the limited utility of parallel data for learning shared multilingual representations
arXiv:2603.29026v1 Announce Type: new Abstract: Shared multilingual representations are essential for cross-lingual tasks and knowledge transfer across languages. This study looks at the impact of parallel data, i.e. translated sentences, in pretraining as a signal to trigger representations that are aligned across languages. We train reference models with different proportions of parallel data and show that parallel data seem to have only a minimal effect on the cross-lingual alignment. Based on multiple evaluation methods, we find that the effect is limited to potentially accelerating the representation sharing in the early phases of pretraining, and to decreasing the amount of language-specific neurons in the model. Cross-lingual alignment seems to emerge on similar levels even without
View PDF HTML (experimental)
Abstract:Shared multilingual representations are essential for cross-lingual tasks and knowledge transfer across languages. This study looks at the impact of parallel data, i.e. translated sentences, in pretraining as a signal to trigger representations that are aligned across languages. We train reference models with different proportions of parallel data and show that parallel data seem to have only a minimal effect on the cross-lingual alignment. Based on multiple evaluation methods, we find that the effect is limited to potentially accelerating the representation sharing in the early phases of pretraining, and to decreasing the amount of language-specific neurons in the model. Cross-lingual alignment seems to emerge on similar levels even without the explicit signal from parallel data.
Subjects:
Computation and Language (cs.CL)
Cite as: arXiv:2603.29026 [cs.CL]
(or arXiv:2603.29026v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.29026
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Julius Leino [view email] [v1] Mon, 30 Mar 2026 21:37:34 UTC (1,252 KB)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
modeltrainingannounce
The end of 'shadow AI' at enterprises? Kilo launches KiloClaw for Organizations to enable secure AI agents at scale
As generative AI matures from a novelty into a workplace staple, a new friction point has emerged: the "shadow AI" or "Bring Your Own AI (BYOAI)" crisis. Much like the unsanctioned use of personal devices in years past, developers and knowledge workers are increasingly deploying autonomous agents on personal infrastructure to manage their professional workflows. "Our journey with Kilo Claw has been to make it easier and easier and more accessible to folks," says Kilo co-founder Scott Breitenother. Today, the company dedicated to providing a portable, multi-model, cloud-based AI coding environment is moving to formalize this "shadow AI" layer: it's launching KiloClaw for Organizations and KiloClaw Chat, a suite of tools designed to provide enterprise-grade governance over personal AI a
From Linux Admin to DevOps & AI: My Journey Begins
<h2> Why I'm Sharing This Journey </h2> <p>Hi! I'm Nazmur, a Junior Linux Administrator who's decided to level up. Like many in operations, I've realized that the future belongs to those who can bridge <strong>traditional sysadmin work</strong> with modern <strong>DevOps practices</strong> and emerging <strong>AI technologies</strong>.</p> <p>This is the first post in my journey from Linux admin → DevOps Engineer → AI/ML Engineer.</p> <h2> Where I Started </h2> <p>As a Linux admin, my daily work involves:</p> <ul> <li>Managing and monitoring Linux servers</li> <li>Writing bash scripts to automate repetitive tasks</li> <li>Troubleshooting system issues</li> <li>Ensuring uptime and reliability</li> </ul> <p>It's solid work, but I kept asking myself: <em>"How can I do this faster? More effici
Multi-Cloud Strategy: When and How to Go Multi-Cloud
<h2> Introduction </h2> <p>Every few months, another major cloud outage makes headlines. AWS us-east-1 goes down, taking half the internet with it. A misconfigured Azure deployment affects thousands of customers. These incidents fuel the multi-cloud narrative: "Don't put all your eggs in one basket."</p> <p>But multi-cloud comes with significant costs—complexity, operational overhead, and often higher expenses. While some organizations genuinely benefit from multi-cloud, many adopt it for the wrong reasons and regret the decision.</p> <p>In this comprehensive guide, we'll explore when multi-cloud makes sense, when it doesn't, and how to implement it successfully if you truly need it.</p> <h2> What is Multi-Cloud? </h2> <h3> Definition </h3> <p>Multi-cloud means using services from multiple
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
More in Models
I Let an AI Agent Run My Freelance Life. It Almost Burned It Down.
<p>For the past few days I kept seeing OpenClaw everywhere. YouTube, Instagram, that one tech Discord I lurk in but never actually talk in. Everyone losing their minds over it.</p> <p><em>"It negotiated $4,200 off a car price." "It runs my entire inbox." "It's the future of computing."</em></p> <p>I had a rough idea what it was, some kind of AI agent. And the intern brain immediately went: if this is basically an automation tool, I can fix my entire chaotic freelance workflow with it.</p> <p>Classic. Give a sleep-deprived software intern a new shiny tool and watch what happens.</p> <p>I'm juggling a software internship by day and freelance client work on the side. My problems aren't glamorous. Client meetings clashing with job interview slots. Cold emails to recruiters I keep meaning to se
From Coin Toss to LLM — Understanding Random Variables
<h1> From Coin Toss to LLM — Understanding Random Variables </h1> <p>A beginner friendly guide to probability and random variables — no math background needed.</p> <h2> 1. What is Probability? </h2> <p>Probability is a number that measures how likely something is to happen.</p> <p>This number is always between 0 and 1:</p> <div class="table-wrapper-paragraph"><table> <thead> <tr> <th>Value</th> <th>Meaning</th> </tr> </thead> <tbody> <tr> <td>0</td> <td>Impossible — will never happen</td> </tr> <tr> <td>1</td> <td>Certain — will always happen</td> </tr> <tr> <td>0.5</td> <td>Equal chance — may or may not happen</td> </tr> </tbody> </table></div> <h3> Example </h3> <p>Flip a fair coin. Two outcomes are possible — heads or tails. Neither is more likely than the other.</p> <p>So the probabili
5 Things I Believed About SEO That AI Search Proved Completely Wrong
<p>`<br> I have been writing technical content for about three years. In that time I developed a fairly confident set of beliefs about what makes content rank well. Keyword research, internal linking, page speed, backlink acquisition, content length. I followed the playbook. My traffic grew. I assumed I understood what I was doing.</p> <p>Then, about six weeks ago, I started seriously looking at whether my content was being cited by ChatGPT, Perplexity AI, and Google Gemini. Not just vaguely appearing somewhere, but actually being used as a source when someone asked a question my posts directly answer.</p> <p>The results were uncomfortable. And almost everything I had been confident about turned out to be either wrong or only partially true when applied to how AI search actually works.</p>
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!