QBugLM: An Agentic Benchmarking Framework for LLM-based Quantum Software Debugging
Abstract
Quantum software bugs often yield silent, incorrect outputs rather than explicit errors, making them particularly difficult to detect and repair with conventional techniques. Although large language models (LLMs) have shown strong performance on classical software engineering tasks, their ability to debug quantum code remains largely unexplored. To bridge this gap, we propose QBugLM, a multi-agent framework that automates the quantum software debugging pipeline, from taxonomy-driven bug injection to LLM-based detection and repair, and finally to simulation-based validation, for framework-agnostic OpenQASM 3.0 programs. We further conduct a comprehensive case study using QBugLM to benchmark two LLMs, Claude 4.6 Sonnet and Qwen3 Coder Next, across different prompting strategies, bug categories, and quantum programs. Our results show that iterative feedback is critical, as a single retry raises Pass@1 from below 25% to above 80%. Moreover, simpler structured prompting can even outperform Chain-of-Thought and ReAct for reasoning-capable models under fixed-resource constraints. Our work takes initial steps toward benchmarking LLM capabilities for debugging quantum programs and offers practical insights to support future efforts in automated quantum software repair.
Versions
Related
- Aitomia: An Agentic Framework for AI-Driven Atomistic and Quantum Chemical Simulations
- Agentic programs: an emerging form of scientific software in computational materials science
- Shot Allocation Strategies in Debugging Quantum Circuits
- SQD-Agent: LLM-driven agentic framework for Quantum Chemistry workflows
- An Open-Source Hardware and Software Toolkit to Enable Agentic RHEED-Guided Thin-Film Synthesis
BibTeX
@misc{pham2026qbuglm,
title = {{QBugLM: An Agentic Benchmarking Framework for LLM-based Quantum Software Debugging}},
author = {An B. B. Pham and Hoa T. Nguyen and Muhammad Usman},
howpublished = {arXiv},
year = {2026},
month = jun,
doi = {10.48550/arxiv.2606.07314},
eprint = {2606.07314},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.07314},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - QBugLM: An Agentic Benchmarking Framework for LLM-based Quantum Software Debugging
AU - An B. B. Pham
AU - Hoa T. Nguyen
AU - Muhammad Usman
PY - 2026
DA - 2026/06/05
PB - arXiv
DO - 10.48550/arxiv.2606.07314
UR - https://arxiv.org/abs/2606.07314
AB - Quantum software bugs often yield silent, incorrect outputs rather than explicit errors, making them particularly difficult to detect and repair with conventional techniques. Although large language models (LLMs) have shown strong performance on classical software engineering tasks, their ability to debug quantum code remains largely unexplored. To bridge this gap, we propose QBugLM, a multi-agent framework that automates the quantum software debugging pipeline, from taxonomy-driven bug injection to LLM-based detection and repair, and finally to simulation-based validation, for framework-agnostic OpenQASM 3.0 programs. We further conduct a comprehensive case study using QBugLM to benchmark two LLMs, Claude 4.6 Sonnet and Qwen3 Coder Next, across different prompting strategies, bug categories, and quantum programs. Our results show that iterative feedback is critical, as a single retry raises Pass@1 from below 25% to above 80%. Moreover, simpler structured prompting can even outperform Chain-of-Thought and ReAct for reasoning-capable models under fixed-resource constraints. Our work takes initial steps toward benchmarking LLM capabilities for debugging quantum programs and offers practical insights to support future efforts in automated quantum software repair.
ER -