Modeling quantum neural network gradient with reinforcement learning
Abstract
Training quantum neural networks (QNNs) on near-term hardware remains hampered by two compounding difficulties: the exponential vanishing of gradient variance known as the barren plateau, and the 𝒪(L · 2n) time and memory cost of differentiating through an n-qubit, L-layer circuit. We propose RLQ-Grad, a reinforcement-learning-based optimizer in which a classical policy πϕ (a spectrally-normalized PPO agent) learns to propose parameter updates directly, conditioned on the QNN's current parameters, loss, accuracy, and previous update. Because the surrogate gradient is emitted by a classical network rather than obtained by differentiating through the unitary U(θ), its variance is not constrained by the barren plateau concentration bound, and its cost scales with the number of trainable parameters rather than the Hilbert-space dimension. We prove these properties formally and verify them on a hardware-efficient ansatz across four supervised benchmarks with up to n=20 qubits. RLQ-Grad preserves a near-flat gradient-variance curve where backpropagation, parameter-shift, and adjoint differentiation decay by 1 to 2 orders of magnitude. Accounting for the full training pipeline (PPO rollouts, actor-critic updates, and optimizer states), RLQ-Grad needs under 2 MB of memory and runs 2490×, 7876×, and 673× faster per iteration than these three methods at n=20. It improves top-1 accuracy by up to +10% over gradient-based baselines on circuits of up to 12 qubits, and matches dedicated barren plateau mitigation methods on CIFAR-10 at 14 to 20 qubits, where evolutionary and gradient-free optimizers collapse to chance.
Related
- Quantum Enhanced Deep Reinforcement Learning for Distribution Network Reconfiguration
- Discovering network resilience formulas with symbolized reinforcement learning
- Reinforcement learning for robust calibration of multiqudit quantum gates
- Reinforcement learning to disentangle multiqubit quantum states from partial observations
- Quantum Hierarchical Reinforcement Learning via Variational Quantum Circuits
BibTeX
@misc{luu2026modeling,
title = {{Modeling quantum neural network gradient with reinforcement learning}},
author = {Nhan Trong Luu and Duong Trung Luu and Nam Ngoc Pham and Thang Cong Truong},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.31066},
eprint = {2609.31066},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.31066},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Modeling quantum neural network gradient with reinforcement learning
AU - Nhan Trong Luu
AU - Duong Trung Luu
AU - Nam Ngoc Pham
AU - Thang Cong Truong
PY - 2026
DA - 2026/09/25
PB - arXiv
DO - 10.48550/arxiv.2609.31066
UR - https://arxiv.org/abs/2609.31066
AB - Training quantum neural networks (QNNs) on near-term hardware remains hampered by two compounding difficulties: the exponential vanishing of gradient variance known as the barren plateau, and the 𝒪(L · 2^n) time and memory cost of differentiating through an n-qubit, L-layer circuit. We propose RLQ-Grad, a reinforcement-learning-based optimizer in which a classical policy π_ϕ (a spectrally-normalized PPO agent) learns to propose parameter updates directly, conditioned on the QNN's current parameters, loss, accuracy, and previous update. Because the surrogate gradient is emitted by a classical network rather than obtained by differentiating through the unitary U(θ), its variance is not constrained by the barren plateau concentration bound, and its cost scales with the number of trainable parameters rather than the Hilbert-space dimension. We prove these properties formally and verify them on a hardware-efficient ansatz across four supervised benchmarks with up to n=20 qubits. RLQ-Grad preserves a near-flat gradient-variance curve where backpropagation, parameter-shift, and adjoint differentiation decay by 1 to 2 orders of magnitude. Accounting for the full training pipeline (PPO rollouts, actor-critic updates, and optimizer states), RLQ-Grad needs under 2 MB of memory and runs 2490×, 7876×, and 673× faster per iteration than these three methods at n=20. It improves top-1 accuracy by up to +10% over gradient-based baselines on circuits of up to 12 qubits, and matches dedicated barren plateau mitigation methods on CIFAR-10 at 14 to 20 qubits, where evolutionary and gradient-free optimizers collapse to chance.
ER -