Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning
Abstract
Fine-tuning a pretrained language model on a curated dataset can produce spurious correlations between the fine-tuning task and unintended latent factors---such as misaligned personas or political slant---that the curation procedure has entangled with the task. The model can latch onto these spurious correlations, leading to bias and reduced out-of-distribution generalisation. We prove that under reasonable assumptions on task complexity and the spurious correlation, such latent factors can be identified, without supervision, from the weights of a naive LoRA fine-tune. Existing approaches to removing bias, such as activation steering, remove identified factors from residual-stream activations, either at inference or during training. We argue, however, that the goal should be to remove the spurious correlation, not the latent factor itself, as the pretrained model may rely on it for genuine task signal. To enable this, we propose GRASP, GRadient projection of Associated Spurious Patterns, which prevents the model from acquiring new reliance on the identified latent factor while preserving any pretrained content along it. We validate on three fine-tuning tasks. The first two involve emergent misalignment, where fine-tuning on a narrow task---in our case, writing insecure code and giving bad medical advice---leads to misaligned responses on unrelated topics. Here our method completely removes misalignment in the insecure code case and reduces it by ~5x in the bad medical advice case, beating all baselines in the trade-off between misalignment-reduction and task-preservation. The last is a novel political-bias experiment, where fine-tuning on right-skewed Reddit financial-advice data causes political-lean drift on unrelated topics. Here our method reduces drift by more than half, while improving financial task performance, beating all baselines.
Related
- Spurious quantum correlations
- Common Signals, Spurious Cross-Excitation, and Identification in Multivariate Hawkes Processes
- Spurious effects in random-intercept cross-lagged panel models: Results from simulations and reanalyses of data on self-esteem and problematic eating behaviors used by Beckers et al. (2023)
- No Spurious Local Minima in Full-Order Linear Quadratic Gaussian Control
- Robust Estimation under Spurious Correlation via Statistical Counterfactual Augmentation
BibTeX
@misc{gilliganlee2026unsupervised,
title = {{Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning}},
author = {Ciarán M. Gilligan-Lee and Joseph Egan and Yuchen Zhu and Michael O'Riordan},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.27676},
eprint = {2605.27676},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.27676},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning
AU - Ciarán M. Gilligan-Lee
AU - Joseph Egan
AU - Yuchen Zhu
AU - Michael O'Riordan
PY - 2026
DA - 2026/10/09
PB - arXiv
DO - 10.48550/arxiv.2605.27676
UR - https://arxiv.org/abs/2605.27676
AB - Fine-tuning a pretrained language model on a curated dataset can produce spurious correlations between the fine-tuning task and unintended latent factors---such as misaligned personas or political slant---that the curation procedure has entangled with the task. The model can latch onto these spurious correlations, leading to bias and reduced out-of-distribution generalisation. We prove that under reasonable assumptions on task complexity and the spurious correlation, such latent factors can be identified, without supervision, from the weights of a naive LoRA fine-tune. Existing approaches to removing bias, such as activation steering, remove identified factors from residual-stream activations, either at inference or during training. We argue, however, that the goal should be to remove the spurious correlation, not the latent factor itself, as the pretrained model may rely on it for genuine task signal. To enable this, we propose GRASP, GRadient projection of Associated Spurious Patterns, which prevents the model from acquiring new reliance on the identified latent factor while preserving any pretrained content along it. We validate on three fine-tuning tasks. The first two involve emergent misalignment, where fine-tuning on a narrow task---in our case, writing insecure code and giving bad medical advice---leads to misaligned responses on unrelated topics. Here our method completely removes misalignment in the insecure code case and reduces it by ~5x in the bad medical advice case, beating all baselines in the trade-off between misalignment-reduction and task-preservation. The last is a novel political-bias experiment, where fine-tuning on right-skewed Reddit financial-advice data causes political-lean drift on unrelated topics. Here our method reduces drift by more than half, while improving financial task performance, beating all baselines.
ER -