Skip to content
DraconiticResearch
Search Sources API
186,541 studies31 sourcesUpdated 40 min ago
PreprintarXiv licencearXiv

How Do Uncertainty Estimators Align with Hallucination Occurrence in LLMs?

  • Yedidia Agnimo
  • Anna Korba
  • Annabelle Blangero
  • Nicolas Chesneau
  • Karteek Alahari
Open at sourcePDFDOICite

Abstract

Large language models (LLMs) can hallucinate, but do model uncertainty estimators reliably reveal when they do? Although uncertainty estimators are increasingly used to identify unreliable generations, their relationship with hallucination remains poorly understood and is often taken for granted. We systematically evaluate 46 uncertainty estimators across complementary hallucination settings, covering both intrinsic hallucinations, where outputs are unsupported by the provided context, and extrinsic hallucinations, where claims are unsupported by expected model knowledge. Across four benchmarks and multiple LLMs, we find that the association between uncertainty and hallucination varies widely and is often weak, depending on the hallucination setting and the model under evaluation. These results challenge the use of uncertainty as a direct signal of hallucination and clarify when it provides useful information.

Related

  1. ABC-Align: Prediction-Powered Alignment with Adaptive Bias Control
    Eric Frankel, Banghua Zhu, Sewoong Oh +1
    arXivPreprintarXiv licenceFull textStatisticsarXiv
  2. Learning Rules, Memorizing Facts: An Information-Theoretic View of Hallucination
    Gengze Xu, Anxin Guo, Yifei Hu +3
    arXivPreprintCC BY 4.0Full textGeneral mathematicsarXiv
  3. A Note on the Failure of L^2 Boundedness in Self-Similar Navier-Stokes Constructions: Algorithmic Hallucination vs. Physical Reality
    Aziz Arbai
    HALPreprintCC BY 4.0Full textAnalysis & PDEsFluid dynamicsHAL
  4. Fix What's Wrong, Keep What's Right: Hallucination Repair for LLM-Based Optimization Modeling
    Jiageng Wu, Chao Shen, Zihan Guo +5
    arXivPreprintarXiv licenceFull textOptimisation & controlarXiv
  5. Eigenvector distributions and optimal shrinkage estimators for large covariance and precision matrices
    Xiucai Ding, Yun Li, Fan Yang
    Applied and Computational Harmonic AnalysisPeer-reviewedLicence not statedFull text5 citedStatisticsOpenAlex

Cite

BibTeXRIS
BibTeX
@misc{agnimo2026uncertainty,
  title = {{How Do Uncertainty Estimators Align with Hallucination Occurrence in LLMs?}},
  author = {Yedidia Agnimo and Anna Korba and Annabelle Blangero and Nicolas Chesneau and Karteek Alahari},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2605.27016},
  eprint = {2605.27016},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.27016},
}
RIS
TY  - UNPB
TI  - How Do Uncertainty Estimators Align with Hallucination Occurrence in LLMs?
AU  - Yedidia Agnimo
AU  - Anna Korba
AU  - Annabelle Blangero
AU  - Nicolas Chesneau
AU  - Karteek Alahari
PY  - 2026
DA  - 2026/10/12
PB  - arXiv
DO  - 10.48550/arxiv.2605.27016
UR  - https://arxiv.org/abs/2605.27016
AB  - Large language models (LLMs) can hallucinate, but do model uncertainty estimators reliably reveal when they do? Although uncertainty estimators are increasingly used to identify unreliable generations, their relationship with hallucination remains poorly understood and is often taken for granted. We systematically evaluate 46 uncertainty estimators across complementary hallucination settings, covering both intrinsic hallucinations, where outputs are unsupported by the provided context, and extrinsic hallucinations, where claims are unsupported by expected model knowledge. Across four benchmarks and multiple LLMs, we find that the association between uncertainty and hallucination varies widely and is often weak, depending on the hallucination setting and the model under evaluation. These results challenge the use of uncertainty as a direct signal of hallucination and clarify when it provides useful information.
ER  -