A Sim-to-Real Study of Surface-Code Decoder Benchmarking
Abstract
Quantum error-correction decoders are typically benchmarked against synthetic circuit-level noise, under the assumption that a decoder's ranking under such noise transfers to hardware and improves as the noise model becomes more realistic. The Willow processor, the first to operate below the surface-code threshold, allows us to test this assumption. We rank a panel of six decoders using a four-rung ladder of noise models with increasing fidelity, evaluated against real data across three code distances, two bases, and fifteen round counts. Rank agreement with hardware appears once the noise model gives each operation type its own error rate. Calibrating the model to the device improves absolute error rates but not rank agreement. We additionally provide the first independent evaluation of NVIDIA's Ising pre-decoder on hardware, at code distances below its training receptive field and via a mapping onto the lattice on which it was trained. Under these conditions, it holds no accuracy-latency advantage: another panel decoder matches or improves on it in both per-cycle error rate and decode latency in 278 of the 280 evaluations. We release the full pipeline and the per-shot outcome of every evaluation, so future decoders and devices can be compared.
Related
- Modular Neural Decoder for Surface Code Memory and Logic on Superconducting Processors
- A Code-Agnostic Graph Neural Network Decoder from the Detection Error Model
- Auditing Syndrome-to-Decoder Mappings for Repetition, Surface, CSS, and Digitized-GKP Interfaces
- Real-time decoder for a MegaQuOp quantum computer using a single CPU
- Benchmarking the Quantum Leap: Auditing LLM Evaluation for Quantum Code Generation
BibTeX
@misc{manor2026simtoreal,
title = {{A Sim-to-Real Study of Surface-Code Decoder Benchmarking}},
author = {Shay J. Manor and Leila S. Erhili and Yassine Jebbouri},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.04557},
eprint = {2609.04557},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.04557},
note = {CC BY-SA 4.0},
}
RIS
TY - UNPB
TI - A Sim-to-Real Study of Surface-Code Decoder Benchmarking
AU - Shay J. Manor
AU - Leila S. Erhili
AU - Yassine Jebbouri
PY - 2026
DA - 2026/09/03
PB - arXiv
DO - 10.48550/arxiv.2609.04557
UR - https://arxiv.org/abs/2609.04557
AB - Quantum error-correction decoders are typically benchmarked against synthetic circuit-level noise, under the assumption that a decoder's ranking under such noise transfers to hardware and improves as the noise model becomes more realistic. The Willow processor, the first to operate below the surface-code threshold, allows us to test this assumption. We rank a panel of six decoders using a four-rung ladder of noise models with increasing fidelity, evaluated against real data across three code distances, two bases, and fifteen round counts. Rank agreement with hardware appears once the noise model gives each operation type its own error rate. Calibrating the model to the device improves absolute error rates but not rank agreement. We additionally provide the first independent evaluation of NVIDIA's Ising pre-decoder on hardware, at code distances below its training receptive field and via a mapping onto the lattice on which it was trained. Under these conditions, it holds no accuracy-latency advantage: another panel decoder matches or improves on it in both per-cycle error rate and decode latency in 278 of the 280 evaluations. We release the full pipeline and the per-shot outcome of every evaluation, so future decoders and devices can be compared.
ER -