Reliable LLM-Generated Programs for High-Energy Physics Experiments through Graph-Grounded Software Knowledge
Abstract
Extracting physics information from modern particle-physics experiments requires multistage analyses implemented on top of large and highly interconnected software ecosystems. General-purpose large language models (LLMs) often produce unreliable programs for such tasks because a user request alone rarely specifies the required APIs, dependencies, and usage conventions. We organize these software relations before generation and retrieve task-relevant knowledge at inference time. Using the open-source ROOT framework as a representative and reproducible testbed, we evaluate a complete grounding system that combines hybrid retrieval over a heterogeneous software knowledge graph, skill-selected workflow examples, and execution-guided repair. On a benchmark of 275 ROOT tasks, grounding improves first-attempt execution from 58.5% to 76.0% under Claude Code orchestration and from 51.3% to 64.0% under standalone orchestration. Final success increases from 90.5% to 96.0% and from 78.9% to 90.9%, respectively, while the average generation cost per successful task increases by only 1.3% and 3.2%. The gains persist under a strong coding agent, indicating that explicit software knowledge remains valuable even when agentic scaffolding is already in place. Because the method captures software relations common to large codebases rather than facts specific to ROOT or a particular model, it should transfer to other experiment frameworks and proprietary software, especially where documentation is sparse or internal dependencies are complex.
Related
- Hierarchical Multi-Agent Reasoning Systems for Knowledge-Grounded Material Substitution
- Analog Quantum Experiments Meet Automatic Differentiation - from Programs to Atoms
- Do Better Scores Mean Better Physics? Physics-Grounded Explanations for Sim2Real Neural Operators
- Physics-Grounded Simulation Enables Zero-Shot Molecular and Reaction Analysis in Scanning Tunneling Microscopy
- Claims of emergence in AI should be grounded in information decomposition
BibTeX
@misc{sun2026reliable,
title = {{Reliable LLM-Generated Programs for High-Energy Physics Experiments through Graph-Grounded Software Knowledge}},
author = {Yue Sun and Tong Liu and Yipu Liao and Jingde Chen and Ke Li},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.01095},
eprint = {2609.01095},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.01095},
}
RIS
TY - UNPB
TI - Reliable LLM-Generated Programs for High-Energy Physics Experiments through Graph-Grounded Software Knowledge
AU - Yue Sun
AU - Tong Liu
AU - Yipu Liao
AU - Jingde Chen
AU - Ke Li
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2609.01095
UR - https://arxiv.org/abs/2609.01095
AB - Extracting physics information from modern particle-physics experiments requires multistage analyses implemented on top of large and highly interconnected software ecosystems. General-purpose large language models (LLMs) often produce unreliable programs for such tasks because a user request alone rarely specifies the required APIs, dependencies, and usage conventions. We organize these software relations before generation and retrieve task-relevant knowledge at inference time. Using the open-source ROOT framework as a representative and reproducible testbed, we evaluate a complete grounding system that combines hybrid retrieval over a heterogeneous software knowledge graph, skill-selected workflow examples, and execution-guided repair. On a benchmark of 275 ROOT tasks, grounding improves first-attempt execution from 58.5% to 76.0% under Claude Code orchestration and from 51.3% to 64.0% under standalone orchestration. Final success increases from 90.5% to 96.0% and from 78.9% to 90.9%, respectively, while the average generation cost per successful task increases by only 1.3% and 3.2%. The gains persist under a strong coding agent, indicating that explicit software knowledge remains valuable even when agentic scaffolding is already in place. Because the method captures software relations common to large codebases rather than facts specific to ROOT or a particular model, it should transfer to other experiment frameworks and proprietary software, especially where documentation is sparse or internal dependencies are complex.
ER -