Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning
Abstract
When a language model is distilled from a teacher that shares its base parameters, the student can inherit behavioural traits of the teacher---including misalignment---even from data that is semantically unrelated to the trait (Cloud et al., 2026). All existing defences require knowing the trait. We remove this requirement and prove that a subliminally transferred trait can be recovered from the parameter updates alone, without supervision, by exploiting a single asymmetry: the trait is expressed consistently across diverse datasets generated by the teacher, whereas task-specific and shared capabilities vary. The resulting estimator returns a weight-space direction that can be amplified to discover and interpret the trait or projected out to suppress it. Applied to the subliminal-learning benchmarks of (Cloud et al., 2026), our method recovers the transmitted animal-preference traits without any trait labels and suppresses their expression with almost no impact to task performance.
Related
- Where Is the Truth When AI Agents Disagree? Dependence-Aware Belief Propagation for Unsupervised Truth Discovery (Extended Version)
- Self-Organized Conformal Prediction: Reducing Regional Coverage Gaps with Unsupervised Group Discovery
- Pattern Intelligence: Unsupervised, LLM-Interpreted Discovery of Systematic Underpayment and Denial Patterns in Healthcare Revenue Cycle Management
- Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation
- Iterative Calibration and Learning: Mitigating Teacher Bias in LLM Distillation without Target-Domain Reward Feedback
BibTeX
@misc{egan2026defence,
title = {{Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning}},
author = {Joseph Egan and Ciarán M. Gilligan-Lee and Michael O'Riordan and Yuchen Zhu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.13536},
eprint = {2610.13536},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.13536},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning
AU - Joseph Egan
AU - Ciarán M. Gilligan-Lee
AU - Michael O'Riordan
AU - Yuchen Zhu
PY - 2026
DA - 2026/10/09
PB - arXiv
DO - 10.48550/arxiv.2610.13536
UR - https://arxiv.org/abs/2610.13536
AB - When a language model is distilled from a teacher that shares its base parameters, the student can inherit behavioural traits of the teacher---including misalignment---even from data that is semantically unrelated to the trait (Cloud et al., 2026). All existing defences require knowing the trait. We remove this requirement and prove that a subliminally transferred trait can be recovered from the parameter updates alone, without supervision, by exploiting a single asymmetry: the trait is expressed consistently across diverse datasets generated by the teacher, whereas task-specific and shared capabilities vary. The resulting estimator returns a weight-space direction that can be amplified to discover and interpret the trait or projected out to suppress it. Applied to the subliminal-learning benchmarks of (Cloud et al., 2026), our method recovers the transmitted animal-preference traits without any trait labels and suppresses their expression with almost no impact to task performance.
ER -