STORM: RDMA-based Monte Carlo Transport Scheme for Distributed-Memory Particle Simulations
Abstract
Monte Carlo particle transport enables high-fidelity astrophysical radiation and neutrino simulations - from core-collapse supernovae and neutron-star mergers to accretion flows - by handling multidimensional geometries, frequency dependence, and moving media without angular discretization. However, inter-rank communication limits scalability on unstructured meshes: standard two-sided MPI requires receivers to post receives and poll completions, creating per-iteration progress overhead that grows with the number of communication partners. Such problems have not demonstrated high scaling efficiency at O(104) cores. We present STORM (Scalable Transport via One-sided Remote Memory), an open-source library for Monte Carlo transport on general meshes, physics, and boundary conditions. STORM provides a lock-free, mesh-independent communication layer that replaces MPI's matched-send/receive semantics with Remote Direct Memory Access (RDMA) - one-sided operations that write directly into a remote rank's memory without involving its CPU. Each rank pair shares a single-producer, single-consumer ring buffer; RDMA writes transfer particles while receivers remain passive. A two-sided MPI backend provides a portable fallback. In an adversarial uniform-emission benchmark, the RDMA backend sustains >97% weak-scaling and >88% strong-scaling efficiency up to 13,440~cores (112~cores per network adapter), with $1.14−1.27\times$ speedups over the two-sided alternative. In a Hohlraum IMC benchmark at 4480 ranks, it is 1.41× faster because MPI progress overhead is reduced by 6.1×. By decoupling communication from physics models and mesh representations, STORM removes a barrier to scaling Monte Carlo transport in astrophysical multiphysics codes, enabling coupled radiation-hydrodynamics with energy- and angle-resolved photon or neutrino transport on dynamically evolving meshes at scale.
Versions
Related
- Identifying heating processes in simulations with an entropy-based scheme: A single jet episode in a galaxy cluster
- Resolving the thermal component of stellar feedback with the "anchor-and-refine" scheme in galactic simulations
- Analysis of wave processes using beam-driven Langmuir/Z-mode waveforms generated in particle-in-cell simulations
- A comparison of particle and fluid approaches for modeling the streaming instability in global patch simulations
- An Implicit Conservative Finite Volume Scheme with Poisson Brackets for the Steady-State Transport of Galactic Cosmic Rays
BibTeX
@misc{mizrachi2026storm,
title = {{STORM: RDMA-based Monte Carlo Transport Scheme for Distributed-Memory Particle Simulations}},
author = {Maor Mizrachi and Barak Raveh and Elad Steinberg},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2607.20639},
eprint = {2607.20639},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.20639},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - STORM: RDMA-based Monte Carlo Transport Scheme for Distributed-Memory Particle Simulations
AU - Maor Mizrachi
AU - Barak Raveh
AU - Elad Steinberg
PY - 2026
DA - 2026/10/18
PB - arXiv
DO - 10.48550/arxiv.2607.20639
UR - https://arxiv.org/abs/2607.20639
AB - Monte Carlo particle transport enables high-fidelity astrophysical radiation and neutrino simulations - from core-collapse supernovae and neutron-star mergers to accretion flows - by handling multidimensional geometries, frequency dependence, and moving media without angular discretization. However, inter-rank communication limits scalability on unstructured meshes: standard two-sided MPI requires receivers to post receives and poll completions, creating per-iteration progress overhead that grows with the number of communication partners. Such problems have not demonstrated high scaling efficiency at O(10^4) cores. We present STORM (Scalable Transport via One-sided Remote Memory), an open-source library for Monte Carlo transport on general meshes, physics, and boundary conditions. STORM provides a lock-free, mesh-independent communication layer that replaces MPI's matched-send/receive semantics with Remote Direct Memory Access (RDMA) - one-sided operations that write directly into a remote rank's memory without involving its CPU. Each rank pair shares a single-producer, single-consumer ring buffer; RDMA writes transfer particles while receivers remain passive. A two-sided MPI backend provides a portable fallback. In an adversarial uniform-emission benchmark, the RDMA backend sustains >97% weak-scaling and >88% strong-scaling efficiency up to 13,440~cores (112~cores per network adapter), with $1.14−1.27\times$ speedups over the two-sided alternative. In a Hohlraum IMC benchmark at 4480 ranks, it is 1.41× faster because MPI progress overhead is reduced by 6.1×. By decoupling communication from physics models and mesh representations, STORM removes a barrier to scaling Monte Carlo transport in astrophysical multiphysics codes, enabling coupled radiation-hydrodynamics with energy- and angle-resolved photon or neutrino transport on dynamically evolving meshes at scale.
ER -