What an Autonomous Agent Discovers About Molecular Transformer Design: Does It Transfer?

View PDF HTML (experimental)

Abstract:Deep learning models for drug-like molecules and proteins overwhelmingly reuse transformer architectures designed for natural language, yet whether molecular sequences benefit from different designs has not been systematically tested. We deploy autonomous architecture search via an agent across three sequence types (SMILES, protein, and English text as control), running 3,106 experiments on a single GPU. For SMILES, architecture search is counterproductive: tuning learning rates and schedules alone outperforms the full search (p = 0.001). For natural language, architecture changes drive 81% of improvement (p = 0.009). Proteins fall between the two. Surprisingly, although the agent discovers distinct architectures per domain (p = 0.004), every innovation transfers across all three domains with <1% degradation, indicating that the differences reflect search-path dependence rather than fundamental biological requirements. We release a decision framework and open-source toolkit for molecular modeling teams to choose between autonomous architecture search and simple hyperparameter tuning.

Comments: 18 pages, 3 figures, 8 tables; code and data at this https URL

Subjects:

Artificial Intelligence (cs.AI)

Cite as: arXiv:2603.28015 [cs.AI]

(or arXiv:2603.28015v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2603.28015

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Edward Wijaya [view email] [v1] Mon, 30 Mar 2026 04:08:37 UTC (142 KB)