Architecting software monitors for control-flow anomaly detection through large language models and conformance checking

View PDF HTML (experimental)

Abstract:Context: Ensuring high levels of dependability in modern computer-based systems has become increasingly challenging due to their complexity. Although systems are validated at design time, their behavior can be different at runtime, possibly showing control-flow anomalies due to ``unknown unknowns''. Objective: We aim to detect control-flow anomalies through software monitoring, which verifies runtime behavior by logging software execution and detecting deviations from expected control flow. Methods: We propose a methodology to develop software monitors for control-flow anomaly detection through Large Language Models (LLMs) and conformance checking. The methodology builds on existing software development practices to maintain traditional V&V while providing an additional level of robustness and trustworthiness. It leverages LLMs to link design-time models and implementation code, automating source-code instrumentation. The resulting event logs are analyzed via conformance checking, an explainable and effective technique for control-flow anomaly detection. Results: We test the methodology on a case-study scenario from the European Railway Traffic Management System / European Train Control System (ERTMS/ETCS), which is a railway standard for modern interoperable railways. The results obtained from the ERTMS/ETCS case study demonstrate that LLM-based source-code instrumentation can achieve up to 82.849% control-flow coverage of the reference design-time process model, while the subsequent conformance checking-based anomaly detection reaches a peak performance of 95.957% F1-score and 93.669% AUC. Conclusion: Incorporating domain-specific knowledge to guide LLMs in source-code instrumentation significantly allowed obtaining reliable and quality software logs and enabled effective control-flow anomaly detection through conformance checking.

Subjects:

Software Engineering (cs.SE); Machine Learning (cs.LG)

Cite as: arXiv:2511.10876 [cs.SE]

(or arXiv:2511.10876v2 [cs.SE] for this version)

https://doi.org/10.48550/arXiv.2511.10876

arXiv-issued DOI via DataCite

Related DOI:

https://doi.org/10.1016/j.infsof.2026.108133

DOI(s) linking to related resources

Submission history

From: Francesco Vitale Dr. [view email] [v1] Fri, 14 Nov 2025 01:11:26 UTC (761 KB) [v2] Fri, 27 Mar 2026 15:25:41 UTC (750 KB)