On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance
Abstract
Large Language Models (LLMs) are increasingly used for zero-shot annotation and LLM-as-a-judge tasks, yet their reliability hinges on how model-internalized priors interact with user-provided instructions. We investigate three dimensions of this interaction: (1) how an LLM's familiarity with data and task definitions relates to performance, (2) whether additional information in prompts can correct zero-shot errors ("decision stickiness"), and (3) model susceptibility to misaligned task definitions. We introduce Definition-Specific Familiarity (DSF), which measures alignment between a model's elicited concept and the target definition. Across nine LLMs and six toxicity datasets (five primary datasets plus an additional robustness dataset), DSF predicts annotation performance after controlling for dataset identity (partial r=+0.41). This association remains positive across all prompting conditions tested. In contrast, three common text-memorization metrics show no positive association. We show that prompting has limited corrective power: only 34.8% of zero-shot errors are corrected by additional instructions or examples, with high-confidence errors especially persistent. Misaligned definitions systematically shift predictions without reducing reported confidence, making confidence unreliable for detecting definition-policy mismatch. Together, these findings establish definition alignment as a practical model-selection criterion and show that better prompting alone cannot substitute for validating model-policy fit.
Related
- Adaptive control of pool-type research nuclear reactors: theory limits, pragmatic actuation constraints, and realized performance and adaptation
- Achievability of CSS codes performance limits over Pauli channels under separate ML decoding with bounded graphical complexity
- Comparative Performance Analysis of Optimization Solvers for Linear Support Vector Machines: Impact of Preconditioning and Dimensionality An Empirical Study Using the Benchopt Framework
- The impact of big data capabilities and business model innovation on new venture performance
- A Comparative Evaluation of Missing Data Imputation Techniques and Their Impact on Classification Performance in Healthcare
BibTeX
@misc{casanova2026limits,
title = {{On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance}},
author = {Etienne Casanova and Rafal Kocielnik and R. Michael Alvarez},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2606.00467},
eprint = {2606.00467},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.00467},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance
AU - Etienne Casanova
AU - Rafal Kocielnik
AU - R. Michael Alvarez
PY - 2026
DA - 2026/10/02
PB - arXiv
DO - 10.48550/arxiv.2606.00467
UR - https://arxiv.org/abs/2606.00467
AB - Large Language Models (LLMs) are increasingly used for zero-shot annotation and LLM-as-a-judge tasks, yet their reliability hinges on how model-internalized priors interact with user-provided instructions. We investigate three dimensions of this interaction: (1) how an LLM's familiarity with data and task definitions relates to performance, (2) whether additional information in prompts can correct zero-shot errors ("decision stickiness"), and (3) model susceptibility to misaligned task definitions. We introduce Definition-Specific Familiarity (DSF), which measures alignment between a model's elicited concept and the target definition. Across nine LLMs and six toxicity datasets (five primary datasets plus an additional robustness dataset), DSF predicts annotation performance after controlling for dataset identity (partial r=+0.41). This association remains positive across all prompting conditions tested. In contrast, three common text-memorization metrics show no positive association. We show that prompting has limited corrective power: only 34.8% of zero-shot errors are corrected by additional instructions or examples, with high-confidence errors especially persistent. Misaligned definitions systematically shift predictions without reducing reported confidence, making confidence unreliable for detecting definition-policy mismatch. Together, these findings establish definition alignment as a practical model-selection criterion and show that better prompting alone cannot substitute for validating model-policy fit.
ER -