Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Driving

View PDF HTML (experimental)

Abstract:Reinforcement Learning (RL) has shown excellent performance in solving decision-making and control problems of autonomous driving, which is increasingly applied in diverse driving scenarios. However, driving is a multi-attribute problem, leading to challenges in achieving multi-objective compatibility for current RL methods, especially in both policy updating and policy execution. On the one hand, a single value evaluation network limits the policy updating in complex scenarios with coupled driving objectives. On the other hand, the common single-type action space structure limits driving flexibility or results in large behavior fluctuations during policy execution. To this end, we propose a Multi-objective Ensemble-Critic reinforcement learning method with Hybrid Parametrized Action for multi-objective compatible autonomous driving. Specifically, an advanced MORL architecture is constructed, in which the ensemble-critic focuses on different objectives through independent reward functions. The architecture integrates a hybrid parameterized action space structure, and the generated driving actions contain both abstract guidance that matches the hybrid road modality and concrete control commands. Additionally, an uncertainty-based exploration mechanism that supports hybrid actions is developed to learn multi-objective compatible policies more quickly. Experimental results demonstrate that, in both simulator-based and HighD dataset-based multi-lane highway scenarios, our method efficiently learns multi-objective compatible autonomous driving with respect to efficiency, action consistency, and safety.

Comments: 14 pages, accepted for publication in IEEE Transactions on Neural Networks and Learning Systems (T-NNLS)

Subjects:

Robotics (cs.RO); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Machine Learning (cs.LG)

Cite as: arXiv:2501.08096 [cs.RO]

(or arXiv:2501.08096v4 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2501.08096

arXiv-issued DOI via DataCite

Related DOI:

https://doi.org/10.1109/TNNLS.2026.3674573

DOI(s) linking to related resources

Submission history

From: Guizhe Jin [view email] [v1] Tue, 14 Jan 2025 13:10:13 UTC (3,325 KB) [v2] Fri, 28 Mar 2025 14:49:25 UTC (2,811 KB) [v3] Tue, 19 Aug 2025 16:41:15 UTC (2,590 KB) [v4] Sat, 28 Mar 2026 08:07:30 UTC (2,592 KB)