MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization

View PDF HTML (experimental)

Abstract:Recent advancements in Large Language Models (LLMs) have expanded context windows to million-token scales, yet benchmarks for evaluating memory remain limited to short-session synthetic dialogues. We introduce \textsc{MemoryCD}, the first large-scale, user-centric, cross-domain memory benchmark derived from lifelong real-world behaviors in the Amazon Review dataset. Unlike existing memory datasets that rely on scripted personas to generate synthetic user data, \textsc{MemoryCD} tracks authentic user interactions across years and multiple domains. We construct a multi-faceted long-context memory evaluation pipeline of 14 state-of-the-art LLM base models with 6 memory method baselines on 4 distinct personalization tasks over 12 diverse domains to evaluate an agent's ability to simulate real user behaviors in both single and cross-domain settings. Our analysis reveals that existing memory methods are far from user satisfaction in various domains, offering the first testbed for cross-domain life-long personalization evaluation.

Comments: Published as a workshop paper in Lifelong Agent @ ICLR 2026

Subjects:

Computation and Language (cs.CL)

Cite as: arXiv:2603.25973 [cs.CL]

(or arXiv:2603.25973v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2603.25973

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Weizhi Zhang [view email] [v1] Thu, 26 Mar 2026 23:28:47 UTC (8,365 KB)