CircuitProbe: Predicting Reasoning Circuits in Transformers via Stability Zone Detection
arXiv:2604.00716v1 Announce Type: new Abstract: Transformer language models contain localized reasoning circuits, contiguous layer blocks that improve reasoning when duplicated at inference time. Finding these circuits currently requires brute-force sweeps costing 25 GPU hours per model. We propose CircuitProbe, which predicts circuit locations from activation statistics in under 5 minutes on CPU, providing a speedup of three to four orders of magnitude. We find that reasoning circuits come in two types: stability circuits in early layers, detected through the derivative of representation change, and magnitude circuits in late layers, detected through anomaly scoring. We validate across 9 models spanning 6 architectures, including 2025 models, confirming that CircuitProbe top predictions m
View PDF HTML (experimental)
Abstract:Transformer language models contain localized reasoning circuits, contiguous layer blocks that improve reasoning when duplicated at inference time. Finding these circuits currently requires brute-force sweeps costing 25 GPU hours per model. We propose CircuitProbe, which predicts circuit locations from activation statistics in under 5 minutes on CPU, providing a speedup of three to four orders of magnitude. We find that reasoning circuits come in two types: stability circuits in early layers, detected through the derivative of representation change, and magnitude circuits in late layers, detected through anomaly scoring. We validate across 9 models spanning 6 architectures, including 2025 models, confirming that CircuitProbe top predictions match or are within 2 layers of the optimal circuit in all validated cases. A scaling experiment across the Qwen 2.5 family reveals that layer duplication consistently benefits models under 3B parameters but degrades performance in 7B+ models, making this a practical scaling technique for small language models. CircuitProbe requires as few as 10 calibration examples and its predictions are stable across English, Hindi, Chinese, and French.
Comments: 11 pages, 1 figure, 3 tables. Code available at this https URL
Subjects:
Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
ACM classes: I.2.6; I.2.7
Cite as: arXiv:2604.00716 [cs.AI]
(or arXiv:2604.00716v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2604.00716
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Rajkiran Panuganti [view email] [v1] Wed, 1 Apr 2026 10:26:12 UTC (104 KB)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
modellanguage modeltransformer
AI Agent Tools for Small Business Owners: A Practical Guide
The AI landscape is overwhelming. Hundreds of tools, new launches every week, and most of them are designed for enterprise teams with dedicated engineering staff. If you're a small business owner — running a service company, an e-commerce shop, or a solo consulting practice — you need tools that actually work without a full-time developer to maintain them. Here's a practical breakdown of the AI agent tools that matter most for small business operations in 2026, focused on what's real and useful today. What AI Agents Actually Do for Small Businesses Forget the hype about artificial general intelligence. For small businesses, AI agents solve three specific problems: They monitor things you can't watch 24/7 — revenue, inventory, customer messages, system health They handle repetitive tasks on
Your AI Chatbot Isn't Stupid. It Just Has No Memory. Here's How We Fixed That.
I had a moment in a session a few weeks ago that I haven't stopped thinking about. Someone asked an AI chatbot what their company's refund policy was. The bot answered confidently, fluently, with zero hesitation. It was also completely wrong. It had invented a policy — 14 days, original packaging, contact support@ — from thin air, because it had never actually seen the company's documentation. It wasn't broken. It was doing exactly what it was designed to do: predict the most plausible-sounding next word. And "most plausible" and "accurate" are not the same thing. That's the dirty secret of LLMs fresh out of training. They're brilliant at sounding right. They're not inherently good at being right — especially about things that aren't in their training data. The fix has a name: RAG. Retriev
The AI-Powered Agency: A Developer Playbook for Selling AI Services in 2026
A freelance brand designer I follow on X shared her numbers last month. In 2024, she was serving three to four clients at a time, billing around $150K per year. In 2025, she added AI to her workflow, not as a gimmick but as actual production infrastructure. She now serves fifteen to twenty concurrent clients, her annual revenue hit $720K, and she works fewer hours than before. She did not build a SaaS product. She did not raise money. She did not hire a team. She just got very good at using AI tools to deliver the same quality of work in a fraction of the time, and charged based on the value of the output rather than the hours it took. This is the model Y Combinator highlighted in their Spring 2026 Request for Startups. Their advice was blunt: instead of selling access to an AI tool for $5
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!