SERSEM: Selective Entropy-Weighted Scoring for Membership Inference in Code Language Models

View PDF HTML (experimental)

Abstract:As Large Language Models (LLMs) for code increasingly utilize massive, often non-permissively licensed datasets, evaluating data contamination through Membership Inference Attacks (MIAs) has become critical. We propose SERSEM (Selective Entropy-Weighted Scoring for Membership Inference), a novel white-box attack framework that suppresses uninformative syntactical boilerplate to amplify specific memorization signals. SERSEM utilizes a dual-signal methodology: first, a continuous character-level weight mask is derived through static Abstract Syntax Tree (AST) analysis, spellchecking-based multilingual logic detection, and offline linting. Second, these heuristic weights are used to pool internal transformer activations and calibrate token-level Z-scores from the output logits. Evaluated on a 25,000-sample balanced dataset, SERSEM achieves a global AUC-ROC of 0.7913 on the StarCoder2-3B model and 0.7867 on the StarCoder2-7B model, consistently outperforming the implemented probability-based baselines Loss, Min-K% Prob, and PAC. Our findings demonstrate that focusing on human-centric coding anomalies provides a significantly more robust indicator of verbatim memorization than sequence-level probability averages.

Comments: Accepted to the FSE 2026 Poisoned Chalice Competition

Subjects:

Software Engineering (cs.SE); Cryptography and Security (cs.CR)

Cite as: arXiv:2604.01147 [cs.SE]

(or arXiv:2604.01147v1 [cs.SE] for this version)

https://doi.org/10.48550/arXiv.2604.01147

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Semih Çağlar [view email] [v1] Wed, 1 Apr 2026 17:03:58 UTC (915 KB)