A Dual-Layer Framework for LLM Hallucination Mitigation: Design and Preliminary Evaluation
- 1Khwopa College of Engineering
Abstract
Large language models (LLMs) produce fluent but factually incorrect statements, known as hallucinations, which are risky in medicine, law and finance. Retrieval-augmented generation (RAG) conditions the input but does not check the output, whereas post-generation verifiers check the output but do not shape generation. We describe a two-layer framework that requires no fine-tuning of the base LLM. A Pre-Generation Control Layer (PGCL) classifies each query by domain and risk, retrieves evidence for high-risk queries, builds a constrained prompt and sets a risk-dependent decoding temperature. A Post-Generation Verification Layer (PGVL) splits the answer into atomic claims, scores each against a hierarchical Tree-RAG index with an LLM judge and a retrieval-similarity signal, and compares the resulting TruthScore with a threshold conditioned on the semantic entropy of sampled answers. Failing claims are handled by a Surgical Self-Correction Loop. We report a small pilot (50 HotpotQA and 25 TruthfulQA questions, Llama-3.3-70B) and, mainly, what it reveals. For 86% of verified claims no evidence was retrieved, so the judge largely assessed its own output. On identical claims, the entropy-conditioned threshold changed 1.6-9.9% of accept decisions, all toward rejection. Of 120 rejected claims, 85 ended as "cannot verify" and only 7 of 30 genuine rewrites passed re-verification. The gold HotpotQA answer appeared in 8 of 50 answers before verification and 9 of 50 after. We therefore find no measurable improvement in answer correctness, and we document confounds (a shared fact memory, unstable risk classification, run-to-run variance) that future evaluations must control.
Related
- A unified multi-domain framework for hallucination detection and reliability evaluation in large language models
- Benzothiazole–Triazole–Indazole Conjugates: Design, Characterization, and Preliminary In Vitro MTT-Based Antiproliferative Evaluation
- Hallucination in Time-series Large Language Models: An empirical lnvestigation and analysis of mitigation strategies
- HalluciGuard-eye: knowledge graph-enhanced retrieval-augmented generation for hallucination detection and mitigation in ophthalmic multimodal large language models
- Preliminary evaluation of the BeH20 co-parenting program using a multiple baseline experimental design.
BibTeX
@misc{baral2026duallayer,
title = {{A Dual-Layer Framework for LLM Hallucination Mitigation: Design and Preliminary Evaluation}},
author = {Anish Baral and Bijay Kadariya and Sushil Sharma},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23616052},
url = {https://zenodo.org/records/23616052},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - A Dual-Layer Framework for LLM Hallucination Mitigation: Design and Preliminary Evaluation
AU - Anish Baral
AU - Bijay Kadariya
AU - Sushil Sharma
PY - 2026
DA - 2026/10/24
PB - Zenodo
DO - 10.5281/zenodo.23616052
UR - https://zenodo.org/records/23616052
AB - Large language models (LLMs) produce fluent but factually incorrect statements, known as hallucinations, which are risky in medicine, law and finance. Retrieval-augmented generation (RAG) conditions the input but does not check the output, whereas post-generation verifiers check the output but do not shape generation. We describe a two-layer framework that requires no fine-tuning of the base LLM. A Pre-Generation Control Layer (PGCL) classifies each query by domain and risk, retrieves evidence for high-risk queries, builds a constrained prompt and sets a risk-dependent decoding temperature. A Post-Generation Verification Layer (PGVL) splits the answer into atomic claims, scores each against a hierarchical Tree-RAG index with an LLM judge and a retrieval-similarity signal, and compares the resulting TruthScore with a threshold conditioned on the semantic entropy of sampled answers. Failing claims are handled by a Surgical Self-Correction Loop. We report a small pilot (50 HotpotQA and 25 TruthfulQA questions, Llama-3.3-70B) and, mainly, what it reveals. For 86% of verified claims no evidence was retrieved, so the judge largely assessed its own output. On identical claims, the entropy-conditioned threshold changed 1.6-9.9% of accept decisions, all toward rejection. Of 120 rejected claims, 85 ended as "cannot verify" and only 7 of 30 genuine rewrites passed re-verification. The gold HotpotQA answer appeared in 8 of 50 answers before verification and 9 of 50 after. We therefore find no measurable improvement in answer correctness, and we document confounds (a shared fact memory, unstable risk classification, run-to-run variance) that future evaluations must control.
ER -