Secure Reinforcement Learning: On Model-Free Detection of Man in the Middle Attacks

View PDF HTML (experimental)

Abstract:We consider the problem of learning-based man-in-the-middle (MITM) attacks in cyber-physical systems (CPS), and extend our previously proposed Bellman Deviation Detection (BDD) framework for model-free reinforcement learning (RL). We refine the standard MDP attack model by allowing the reward function to depend on both the current and subsequent states, thereby capturing reward variations induced by errors in the adversary's transition estimate. We also derive an optimal system-identification strategy for the adversary that minimizes detectable value deviations. Further, we prove that the agent's asymptotic learning time required to secure the system scales linearly with the adversary's learning time, and that this matches the optimal lower bound. Hence, the proposed detection scheme is order-optimal in detection efficiency. Finally, we extend the framework to asynchronous and intermittent attack scenarios, where reliable detection is preserved.

Subjects:

Systems and Control (eess.SY); Machine Learning (cs.LG)

Cite as: arXiv:2603.27592 [eess.SY]

(or arXiv:2603.27592v1 [eess.SY] for this version)

https://doi.org/10.48550/arXiv.2603.27592

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Rishi Rani [view email] [v1] Sun, 29 Mar 2026 09:18:00 UTC (344 KB)