Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding
Abstract
In open-ended generation, LLMs frequently fall into the "likelihood trap", characterized by repetitive degeneration and vocabulary dullness, resulting in a discrepancy between machine-generated and human-written text. While post-hoc tail truncation (e.g., Top-p, Min-p) avoids sampling from the unreliable tail, it can misalign generation with human lexical preferences by over-sampling from the uncalibrated head; fixed scalar repetition penalties, in turn, ignore how the scale of the logit distribution varies across inference steps, which can disrupt semantic coherence. To address both shortcomings, we propose Variance-Calibrated Modulation (VCM), a training-free pre-decoding intervention. VCM directly reshapes the probability distribution prior to truncation via two dynamic mechanisms: (1) Contextual Searchlight via PMI, which naturally suppresses global stopwords and elevates context-evoked tokens, and (2) Adaptive Self-Debiasing, which utilizes real-time logit standard deviation to provide scale-invariant penalization. In experiments across open-ended generation, factual QA, and mathematical reasoning, we show that VCM consistently mitigates the likelihood trap. With negligible computational overhead, VCM integrates with existing decoding strategies, improving diversity and coherence and, particularly at higher decoding temperatures, reasoning accuracy. Our code is publicly available on GitHub: https://github.com/AetherDing/VCM
Related
- R2-GP: Variance-Calibrated Priors for Additive Gaussian Processes
- Breaking the Shot-Noise Barrier: Cached Recycled Variance-Reduced Gradients for Quantum Optimization
- Can Large Language Model-Generated Responses Support Assessment Development? A Human-Calibrated Rasch Benchmark
- Predictive Likelihood Ratios for Language Model Watermark Detection
- llmClassificR: Text Classification with Large Language Models, Network Psychometrics, and Calibrated Uncertainty in R
BibTeX
@misc{ding2026breaking,
title = {{Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding}},
author = {Yuanhao Ding and Meimingwei Li and Esteban Garces Arias and Matthias Aßenmacher and Christian Heumann and Chongsheng Zhang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.22511},
eprint = {2606.22511},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.22511},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding
AU - Yuanhao Ding
AU - Meimingwei Li
AU - Esteban Garces Arias
AU - Matthias Aßenmacher
AU - Christian Heumann
AU - Chongsheng Zhang
PY - 2026
DA - 2026/09/02
PB - arXiv
DO - 10.48550/arxiv.2606.22511
UR - https://arxiv.org/abs/2606.22511
AB - In open-ended generation, LLMs frequently fall into the "likelihood trap", characterized by repetitive degeneration and vocabulary dullness, resulting in a discrepancy between machine-generated and human-written text. While post-hoc tail truncation (e.g., Top-p, Min-p) avoids sampling from the unreliable tail, it can misalign generation with human lexical preferences by over-sampling from the uncalibrated head; fixed scalar repetition penalties, in turn, ignore how the scale of the logit distribution varies across inference steps, which can disrupt semantic coherence. To address both shortcomings, we propose Variance-Calibrated Modulation (VCM), a training-free pre-decoding intervention. VCM directly reshapes the probability distribution prior to truncation via two dynamic mechanisms: (1) Contextual Searchlight via PMI, which naturally suppresses global stopwords and elevates context-evoked tokens, and (2) Adaptive Self-Debiasing, which utilizes real-time logit standard deviation to provide scale-invariant penalization. In experiments across open-ended generation, factual QA, and mathematical reasoning, we show that VCM consistently mitigates the likelihood trap. With negligible computational overhead, VCM integrates with existing decoding strategies, improving diversity and coherence and, particularly at higher decoding temperatures, reasoning accuracy. Our code is publicly available on GitHub: https://github.com/AetherDing/VCM
ER -