Synergizing Large Language Models and Task-specific Models for Time Series Anomaly Detection

View PDF HTML (experimental)

Abstract:In anomaly detection, methods based on large language models (LLMs) can incorporate expert knowledge by reading professional document, while task-specific small models excel at extracting normal data patterns and detecting value fluctuations from training data of target applications. Inspired by the human nervous system, where the brain stores expert knowledge and the peripheral nervous system and spinal cord handle specific tasks like withdrawal and knee-jerk reflexes, we propose CoLLaTe, a framework designed to facilitate collaboration between LLMs and task-specific models, leveraging the strengths of both models for anomaly detection. In particular, we first formulate the collaboration process and identify two key challenges in the collaboration: (1) the misalignment between the expression domains of the LLMs and task-specific small models, and (2) error accumulation arising from the predictions of both models. To address these challenges, we then introduce two key components in CoLLaTe: a model alignment module and a collaborative loss function. Through theoretical analysis and experimental validation, we demonstrate that these components effectively mitigate the identified challenges and achieve better performance than both LLM-based and task-specific models.

Comments: This work has been submitted to the IEEE for possible publication

Subjects:

Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Cite as: arXiv:2501.05675 [cs.AI]

(or arXiv:2501.05675v5 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2501.05675

arXiv-issued DOI via DataCite

Submission history

From: Feiyi Chen [view email] [v1] Fri, 10 Jan 2025 02:57:08 UTC (1,558 KB) [v2] Sat, 18 Jan 2025 05:30:01 UTC (1,833 KB) [v3] Thu, 23 Jan 2025 04:23:49 UTC (1,833 KB) [v4] Tue, 6 May 2025 07:57:08 UTC (1,754 KB) [v5] Mon, 30 Mar 2026 10:50:11 UTC (943 KB)