A Reproducible Benchmark and Evidence-Retrieval Software Framework for Silicon Detector R&D Literature
Abstract
Silicon pixel detector R&D depends on a large and rapidly growing technical literature, including beam-test and irradiation studies, performance measurements, simulation, and design reports. Locating the supporting evidence passage for a measurement, operating condition, or design decision is therefore a computing and data-science challenge for detector-development workflows. General-purpose language models are insufficient unless grounded in traceable primary sources, particularly in a domain with specialised terminology, configuration-dependent measurements, and rapidly evolving experimental results. We address this with a reproducible, general-purpose framework for evidence-grounded retrieval over technical literature, using silicon pixel detector R&D as a demanding validation domain. The framework combines sparse lexical retrieval, dense semantic retrieval, and hybrid reciprocal-rank fusion, with an optional graph-guided exploration layer and grounded, abstention-aware response generation. The accompanying benchmark provides manually curated chunk-level evidence annotations, source-level diagnostics, semantic relevance checks, and negative-query abstention tests over two detector query sets. We evaluate six retrieval configurations across 378 source documents and 8,442 indexed chunks. Hybrid sparse-dense retrieval gives the strongest strict evidence recovery, achieving Hit@5 of 0.917 on the core benchmark and 0.951 on the curated extension benchmark, while graph-based methods are more effective for literature exploration and source discovery. Graph expansion is therefore best employed as a discovery layer over the hybrid retrieval backbone. The framework provides reusable software for traceable, 1 evidence-grounded knowledge access in silicon detector R&D and high-energy physics instrumentation.
Related
- Qlippy: A Retrieval-Augmented GenAI Assistant for Reproducible Quantum Workflows and Experiment Tracking
- FlowMLLab: An open-source framework for reproducible computational-fluid-dynamics and scientific-machine-learning experiments
- CELEBI: Design and Realization of a Reproducible Physics Analysis Framework
- iPINN for Broadband CARS Phase Retrieval: A Framework for Function Approximation and Inverse Modeling Problems in Nonlinear Spectroscopy
- A literature-guided descriptor-based framework for filtering composition search spaces
BibTeX
@misc{gao2026reproducible,
title = {{A Reproducible Benchmark and Evidence-Retrieval Software Framework for Silicon Detector R&D Literature}},
author = {Tianqi Gao and Ruobing Jiang and Dawei Fu and Qiang Li and Matthew Kenzie},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2606.24725},
eprint = {2606.24725},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.24725},
}
RIS
TY - UNPB
TI - A Reproducible Benchmark and Evidence-Retrieval Software Framework for Silicon Detector R&D Literature
AU - Tianqi Gao
AU - Ruobing Jiang
AU - Dawei Fu
AU - Qiang Li
AU - Matthew Kenzie
PY - 2026
DA - 2026/10/21
PB - arXiv
DO - 10.48550/arxiv.2606.24725
UR - https://arxiv.org/abs/2606.24725
AB - Silicon pixel detector R&D depends on a large and rapidly growing technical literature, including beam-test and irradiation studies, performance measurements, simulation, and design reports. Locating the supporting evidence passage for a measurement, operating condition, or design decision is therefore a computing and data-science challenge for detector-development workflows. General-purpose language models are insufficient unless grounded in traceable primary sources, particularly in a domain with specialised terminology, configuration-dependent measurements, and rapidly evolving experimental results. We address this with a reproducible, general-purpose framework for evidence-grounded retrieval over technical literature, using silicon pixel detector R&D as a demanding validation domain. The framework combines sparse lexical retrieval, dense semantic retrieval, and hybrid reciprocal-rank fusion, with an optional graph-guided exploration layer and grounded, abstention-aware response generation. The accompanying benchmark provides manually curated chunk-level evidence annotations, source-level diagnostics, semantic relevance checks, and negative-query abstention tests over two detector query sets. We evaluate six retrieval configurations across 378 source documents and 8,442 indexed chunks. Hybrid sparse-dense retrieval gives the strongest strict evidence recovery, achieving Hit@5 of 0.917 on the core benchmark and 0.951 on the curated extension benchmark, while graph-based methods are more effective for literature exploration and source discovery. Graph expansion is therefore best employed as a discovery layer over the hybrid retrieval backbone. The framework provides reusable software for traceable, 1 evidence-grounded knowledge access in silicon detector R&D and high-energy physics instrumentation.
ER -