Conformal Factuality Control for Multi-Hop Retrieval-Augmented Generation
Abstract
Retrieval-augmented generation (RAG) can ground large language models in external evidence, but retrieved context does not guarantee that generated claims are factually supported. This problem is especially relevant in multi-hop RAG, where retrieval and reasoning proceed through multiple dependent stages. We study whether claim-level conformal factuality control, previously developed for RAG, remains effective in this setting. We apply split-conformal claim filtering to multi-hop RAG and evaluate it on HotpotQA, Natural Questions, and TriviaQA using Llama 3.1 8B and GPT-4o-mini, together with a single-hop reference experiment. Across all six multi-hop model-dataset configurations, increasingly stringent conformal targets consistently increase the fraction of responses whose retained claims are fully supported. At the 95% target, this rate ranges from 95.80% to 97.20%, compared with 55.60%-76.03% without filtering. However, the improvement is strongly selective: only 4.41%-31.09% of generated claims are retained and 9.70%-51.40% of responses remain non-empty at the 95% target. These results show that conformal factuality extends to multi-hop RAG, while demonstrating that nominal reliability must be interpreted jointly with claim retention and abstention.
Related
- EntroPrefill: Renyi-Guided Context Pruning with Conditional Stability Guarantees for Retrieval-Augmented Generation
- Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment
- GNA: Granular Neighbor Assembly for Retrieval-Augmented Multivariate Time-Series Forecasting
- Optimal Control Under Signal Temporal Logic Constraints: A Robust Reformulation via Augmented Dynamics
- Conformal Risk-Averse Decision Making with Optimized Certainty Equivalent Risk Control
BibTeX
@misc{rehman2026conformal,
title = {{Conformal Factuality Control for Multi-Hop Retrieval-Augmented Generation}},
author = {Muhammad Aimal Rehman and Chi-Kuang Yeh},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38222},
eprint = {2609.38222},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38222},
}
RIS
TY - UNPB
TI - Conformal Factuality Control for Multi-Hop Retrieval-Augmented Generation
AU - Muhammad Aimal Rehman
AU - Chi-Kuang Yeh
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2609.38222
UR - https://arxiv.org/abs/2609.38222
AB - Retrieval-augmented generation (RAG) can ground large language models in external evidence, but retrieved context does not guarantee that generated claims are factually supported. This problem is especially relevant in multi-hop RAG, where retrieval and reasoning proceed through multiple dependent stages. We study whether claim-level conformal factuality control, previously developed for RAG, remains effective in this setting. We apply split-conformal claim filtering to multi-hop RAG and evaluate it on HotpotQA, Natural Questions, and TriviaQA using Llama 3.1 8B and GPT-4o-mini, together with a single-hop reference experiment. Across all six multi-hop model-dataset configurations, increasingly stringent conformal targets consistently increase the fraction of responses whose retained claims are fully supported. At the 95% target, this rate ranges from 95.80% to 97.20%, compared with 55.60%-76.03% without filtering. However, the improvement is strongly selective: only 4.41%-31.09% of generated claims are retained and 9.70%-51.40% of responses remain non-empty at the 95% target. These results show that conformal factuality extends to multi-hop RAG, while demonstrating that nominal reliability must be interpreted jointly with claim retention and abstention.
ER -