GRPO-QPS: Target-Preserving Reinforcement Learning for Quantum Posterior Sampling
Abstract
Bayesian quantum tomography requires efficient inference while preserving a posterior fixed by the prior and Born likelihood. Learned transport provides fast amortized samples, but reward tuning can reshape the generated distribution rather than improve exploration of this fixed target. We introduce GRPO-QPS, a target-preserving framework in which GRPO learns proposal behavior and an exact Metropolis correction preserves the posterior after training. Across the evaluated reconstruction benchmarks, GRPO-QPS improves over BuresTomFlow and Flow-GRPO on thermal, cat, Dicke, and cluster families, and it closely matches an exact two-qubit reference posterior. Tuned conventional MCMC is slightly stronger on several original continuous benchmarks where the available fixed proposals already match the posterior geometry well. To test whether this reflects a fundamental limitation of learned exploration, we evaluate a more challenging multimodal thermal posterior. At six qubits and 800 shots, the learned proposal achieves a minimum effective sample size of 102 per 1,000 likelihood calls, compared with 28 for prior independence, 27 for a tuned fixed mixture, and 20 for Haario adaptive Metropolis. A record-conditioned policy also transfers to unseen 3,000-shot records, matching or exceeding the strongest conventional baseline in all nine held-out seed-record comparisons. These results show that GRPO-QPS combines target-preserving Bayesian inference with broad gains over learned transport baselines and a sampling advantage when efficient exploration requires proposal geometry beyond the evaluated conventional kernels.
Related
- Reinforcement learning for robust calibration of multiqudit quantum gates
- Reinforcement learning to disentangle multiqubit quantum states from partial observations
- Quantum Hierarchical Reinforcement Learning via Variational Quantum Circuits
- Quantum Enhanced Deep Reinforcement Learning for Distribution Network Reconfiguration
- Quantum control of solid state qubits via Reinforcement Learning
BibTeX
@misc{wang2026grpoqps,
title = {{GRPO-QPS: Target-Preserving Reinforcement Learning for Quantum Posterior Sampling}},
author = {Yufeng Wang and Parivesh Priye and Lu Wei and Haibin Ling},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.14711},
eprint = {2609.14711},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.14711},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - GRPO-QPS: Target-Preserving Reinforcement Learning for Quantum Posterior Sampling
AU - Yufeng Wang
AU - Parivesh Priye
AU - Lu Wei
AU - Haibin Ling
PY - 2026
DA - 2026/09/21
PB - arXiv
DO - 10.48550/arxiv.2609.14711
UR - https://arxiv.org/abs/2609.14711
AB - Bayesian quantum tomography requires efficient inference while preserving a posterior fixed by the prior and Born likelihood. Learned transport provides fast amortized samples, but reward tuning can reshape the generated distribution rather than improve exploration of this fixed target. We introduce GRPO-QPS, a target-preserving framework in which GRPO learns proposal behavior and an exact Metropolis correction preserves the posterior after training. Across the evaluated reconstruction benchmarks, GRPO-QPS improves over BuresTomFlow and Flow-GRPO on thermal, cat, Dicke, and cluster families, and it closely matches an exact two-qubit reference posterior. Tuned conventional MCMC is slightly stronger on several original continuous benchmarks where the available fixed proposals already match the posterior geometry well. To test whether this reflects a fundamental limitation of learned exploration, we evaluate a more challenging multimodal thermal posterior. At six qubits and 800 shots, the learned proposal achieves a minimum effective sample size of 102 per 1,000 likelihood calls, compared with 28 for prior independence, 27 for a tuned fixed mixture, and 20 for Haario adaptive Metropolis. A record-conditioned policy also transfers to unseen 3,000-shot records, matching or exceeding the strongest conventional baseline in all nine held-out seed-record comparisons. These results show that GRPO-QPS combines target-preserving Bayesian inference with broad gains over learned transport baselines and a sampling advantage when efficient exploration requires proposal geometry beyond the evaluated conventional kernels.
ER -