A retrieval-augmented multi-stage prompting framework for legal analysis in the Indian judicial system
- 1School of Computer Science and Engineering (SCOPE), Vellore Institute of Technology Chennai , ,
Abstract
The aim of this endeavor was to create an Artificial Intelligence (AI) based legal researcher that utilizes a multi-stage Retrieval-Augmented Generation (RAG) methodology to solve the challenges of finding and comprehending pertinent legal information in the documents. The traditional methods for finding legal documents will usually yield very few results that are useful to someone without a law degree. This new system uses semantic search, keyword retrieval and Large Language Model (LLM) reasoning methods to improve the legal research process. The submitted query will be transformed into a vector embedding while simultaneously being classified into the appropriate category for retrieval. Once the query is classified, the system will use a hybrid retrieval mechanism to find the most relevant legal document based on both BM25 (Best Matching 25) and FAISS (Facebook AI Similarity Search) search algorithms and will combine these results using Reciprocal Rank Fusion (RRF) in order to determine the best results. Later, it will be processed through LLM models to produce formatted (structured) answers to the query in accordance with the relevant legal citation. The overall performance of the proposed system demonstrates improved answer relevancy, reliability and contextual accuracy, thereby making it an effective tool for legal research and assistance. LLaMA-3.3-70B achieved the highest scores in Answer Relevancy (0.91) and Faithfulness (0.92), proving its ability to generate accurate, intent aligned, and evidence-grounded legal responses with minimal hallucination. Gemini-2.0-Flash showed higher Context Recall (0.78), indicating strong retrieval capability, while Mistral-Large excelled in BLEU (0.769) and semantic similarity, highlighting fluent text generation but weaker evidence grounding.
Related
- GEO-BERT: A Retrieval augmented LLM Framework for Intelligent Dataset Retrieval from the Gene Expression Omnibus
- Efficient training-free query routing framework for federated retrieval-augmented generation
- Real-world evaluation of a multilingual agentic retrieval-augmented clinical AI framework for evidence-grounded medical decision support
- MacroRisk-agent: a retrieval-augmented multi-agent framework with deterministic validation gates for auditable stress testing
- Automated risk scoring for venous thromboembolism using large language models with expert knowledge-augmented prompting: a multicenter validation study
BibTeX
@article{kanishkavardhan2026retrievalaugmented,
title = {{A retrieval-augmented multi-stage prompting framework for legal analysis in the Indian judicial system}},
author = {Kanishkavardhan P. and Gowtham S. and Maheswari N.},
journal = {Frontiers in big data},
year = {2026},
month = oct,
doi = {10.3389/fdata.2026.1933830},
url = {https://europepmc.org/article/PMC/PMC13724461},
note = {CC BY},
}
RIS
TY - JOUR
TI - A retrieval-augmented multi-stage prompting framework for legal analysis in the Indian judicial system
AU - Kanishkavardhan P.
AU - Gowtham S.
AU - Maheswari N.
PY - 2026
DA - 2026/10/19
JO - Frontiers in big data
DO - 10.3389/fdata.2026.1933830
UR - https://europepmc.org/article/PMC/PMC13724461
AB - The aim of this endeavor was to create an Artificial Intelligence (AI) based legal researcher that utilizes a multi-stage Retrieval-Augmented Generation (RAG) methodology to solve the challenges of finding and comprehending pertinent legal information in the documents. The traditional methods for finding legal documents will usually yield very few results that are useful to someone without a law degree. This new system uses semantic search, keyword retrieval and Large Language Model (LLM) reasoning methods to improve the legal research process. The submitted query will be transformed into a vector embedding while simultaneously being classified into the appropriate category for retrieval. Once the query is classified, the system will use a hybrid retrieval mechanism to find the most relevant legal document based on both BM25 (Best Matching 25) and FAISS (Facebook AI Similarity Search) search algorithms and will combine these results using Reciprocal Rank Fusion (RRF) in order to determine the best results. Later, it will be processed through LLM models to produce formatted (structured) answers to the query in accordance with the relevant legal citation. The overall performance of the proposed system demonstrates improved answer relevancy, reliability and contextual accuracy, thereby making it an effective tool for legal research and assistance. LLaMA-3.3-70B achieved the highest scores in Answer Relevancy (0.91) and Faithfulness (0.92), proving its ability to generate accurate, intent aligned, and evidence-grounded legal responses with minimal hallucination. Gemini-2.0-Flash showed higher Context Recall (0.78), indicating strong retrieval capability, while Mistral-Large excelled in BLEU (0.769) and semantic similarity, highlighting fluent text generation but weaker evidence grounding.
SN - 2624-909X
ER -