Polysemanticity or Polysemy? Lexical Identity Confounds Superposition Metrics
arXiv:2604.00443v1 Announce Type: new Abstract: If the same neuron activates for both "lender" and "riverside," standard metrics attribute the overlap to superposition--the neuron must be compressing two unrelated concepts. This work explores how much of the overlap is due a lexical confound: neurons fire for a shared word form (such as "bank") rather than for two compressed concepts. A 2x2 factorial decomposition reveals that the lexical-only condition (same word, different meaning) consistently exceeds the semantic-only condition (different word, same meaning) across models spanning 110M-70B parameters. The confound carries into sparse autoencoders (18-36% of features blend senses), sits in <=1% of activation dimensions, and hurts downstream tasks: filtering it out improves word sense di
View PDF HTML (experimental)
Abstract:If the same neuron activates for both "lender" and "riverside," standard metrics attribute the overlap to superposition--the neuron must be compressing two unrelated concepts. This work explores how much of the overlap is due a lexical confound: neurons fire for a shared word form (such as "bank") rather than for two compressed concepts. A 2x2 factorial decomposition reveals that the lexical-only condition (same word, different meaning) consistently exceeds the semantic-only condition (different word, same meaning) across models spanning 110M-70B parameters. The confound carries into sparse autoencoders (18-36% of features blend senses), sits in <=1% of activation dimensions, and hurts downstream tasks: filtering it out improves word sense disambiguation and makes knowledge edits more selective (p = 0.002).
Comments: 21 pages
Subjects:
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2604.00443 [cs.CL]
(or arXiv:2604.00443v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2604.00443
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Iyad Ait Hou [view email] [v1] Wed, 1 Apr 2026 03:39:47 UTC (2,671 KB)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
modelannouncefeature
The AI Gaslight
Why Tech Billionaires Are Selling a Utopia to Build an Empire From “vibe coding” tech debt to digital sweatshops — how the AI industry is sacrificing the working class to summon a machine we cannot control. A few weeks ago, I made a very public, very painful admission about building my startup, Nexa. Caught up in the deafening hype of the AI bubble, I stopped writing deep architectural code and started relying entirely on Large Language Models (LLMs) to “vibe code” my MVP. The AI acted like a sycophant. It flattered me. It told me my ideas were brilliant. It made me feel like a 10x engineer. But when real users touched the product, the system choked. Beneath the beautiful UI was a terrifying ocean of unscalable spaghetti code and suppressed errors. I realized the hard way that AI doesn’t m
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
More in Models
AI That Improves AI: What Happens When Agents Start Rewriting Themselves?
From Darwin Gödel Machine to HyperAgents-understanding how AI is evolving from static models to systems that continuously improve themselves What happens when an AI system is no longer just solving problems but also begins to change itself in order to solve them better? Not retrained. Not fine-tuned.But actively rewriting it’s own code, it’s own workflow and eventually improving the way it improves itself ! At first, it sounds like science fiction. However, the idea of a machine that can modify itself has been discussed for decades, earlier framed as a theoretical construct-something powerful yet out of reach . One of the earliest formulations imagined a system that could rewrite its own code but only after proving that the modification would lead to better performance. Agreed, it was a be
Bounded Autonomy: Controlling LLM Characters in Live Multiplayer Games
arXiv:2604.04703v1 Announce Type: new Abstract: Large language models (LLMs) are bringing richer dialogue and social behavior into games, but they also expose a control problem that existing game interfaces do not directly address: how should LLM characters participate in live multiplayer interaction while remaining executable in the shared game world, socially coherent with other active characters, and steerable by players when needed? We frame this problem as bounded autonomy, a control architecture for live multiplayer games that organizes LLM character control around three interfaces: agent-agent interaction, agent-world action execution, and player-agent steering. We instantiate bounded autonomy with probabilistic reply-chain decay, an embedding-based action grounding pipeline with fa
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!