C4DF: confidence calibrated contextual control for diverse and faithful image captioning.
- 1Department of Computer Engineering, MIT Academy of Engineering, Affiliated to Savitribai Phule Pune University, Pune, India.
Abstract
Image captioning models generate coherent descriptions that are linguistically fluent. Because of similarity-based training objectives, generated captions are repetitive. In most techniques that aim to generate diverse, human-like captions, strong language priors lead to visually unsupported entities. This paper proposes Confidence-Calibrated Contextual Control for Diverse and Faithful Image Captions (C4DF), a decoding-time framework that reduces hallucinated objects while maintaining diversity and fluency. Image-specific pseudo-context adaptively suppresses tokens guided by confidence calibration. Prefix token conditioning controls diversity, while confidence-calibrated token suppression during decoding improves grounding. Experiments on the benchmark dataset demonstrate an improvement in CIDEr from 46.96 to 51.69 while reducing the noun non-context rate by 9.48%. Qualitative analysis confirms this balance between diversity and faithfulness without compromising on linguistic fluency.
Related
- Trustworthy Stress Detection from Wearable Sensor Signals for Preventive Anxiety Monitoring: Subject-Independent Accuracy and Calibrated Confidence with the Autonomic-Branch Quantum-Graph Network
- Confidence-Ordering Reversal under Contextual Priors in Neural Decoding
- Calibrated Reliance on Large Language Models for Personalized Surgical Care: A Critical Narrative Review of Confidence, Clinical Reasoning and Decision Support
- HistoMRIFuseNet: retrospective confidence-calibrated decision fusion of independent histopathology and DCE-MRI classifiers
- HybridSwingNet for explainable swing trading using multi-encoder deep learning and confidence-calibrated signal execution
BibTeX
@article{kharat2026c4df,
title = {{C4DF: confidence calibrated contextual control for diverse and faithful image captioning.}},
author = {Sharmila Kharat and Sunita Barve},
journal = {MethodsX},
year = {2026},
month = oct,
doi = {10.1016/j.mex.2026.104199},
url = {https://europepmc.org/article/MED/42926827},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - C4DF: confidence calibrated contextual control for diverse and faithful image captioning.
AU - Sharmila Kharat
AU - Sunita Barve
PY - 2026
DA - 2026/10/10
JO - MethodsX
DO - 10.1016/j.mex.2026.104199
UR - https://europepmc.org/article/MED/42926827
AB - Image captioning models generate coherent descriptions that are linguistically fluent. Because of similarity-based training objectives, generated captions are repetitive. In most techniques that aim to generate diverse, human-like captions, strong language priors lead to visually unsupported entities. This paper proposes Confidence-Calibrated Contextual Control for Diverse and Faithful Image Captions (C4DF), a decoding-time framework that reduces hallucinated objects while maintaining diversity and fluency. Image-specific pseudo-context adaptively suppresses tokens guided by confidence calibration. Prefix token conditioning controls diversity, while confidence-calibrated token suppression during decoding improves grounding. Experiments on the benchmark dataset demonstrate an improvement in CIDEr from 46.96 to 51.69 while reducing the noun non-context rate by 9.48%. Qualitative analysis confirms this balance between diversity and faithfulness without compromising on linguistic fluency.
SN - 2215-0161
ER -