DeepWaveRL: Self-Supervised Full Waveform Inversion via Reinforcement Learning
Abstract
Full Waveform Inversion (FWI) is a fundamental inverse problem in geophysics to estimate subsurface properties, such as velocity, from seismic measurements. Recently, self-supervised learning methods remove the need for ground-truth velocity maps during training by leveraging a forward operator to simulate seismic data from predictions and backpropagate the gradients of reconstruction loss through it. In this work, we consider a more challenging setting without accessing such gradients. To this end, we incorporate reinforcement learning into self-supervised FWI and propose a framework named DeepWaveRL. Specifically, we reformulate FWI as a policy learning problem, where the policy network predicts velocity maps as actions, and a forward operator is used solely for simulation. A global scalar reward is computed based on data misfit for each prediction and used to weight policy gradients. Thus, learning proceeds without accessing the gradients from the forward operator. Given the indirect and coarse supervision signals, we further identify key strategies to stabilize learning and mitigate the spatial credit assignment bottleneck. Experimental results show that, despite relying entirely on a single scalar reward, DeepWaveRL achieves competitive performance compared to supervised and self-supervised baselines. These findings suggest that our framework offers a flexible and complementary learning paradigm for the FWI research community.
Related
- DispSAC: Surface-wave dispersion inversion using soft actor-critic reinforcement learning
- Seismic imaging of the crust and upper mantle beneath Taiwan using full-waveform inversion of teleseismic body waves
- Uncertainty Quantification in Full Waveform Inversion with Gradient Flows in Probability Space
- Effect of the interfacial continuity on the elastic full-waveform inversion
- Full Waveform Inversion with a Learned Preconditioner
BibTeX
@misc{jin2026deepwaverl,
title = {{DeepWaveRL: Self-Supervised Full Waveform Inversion via Reinforcement Learning}},
author = {Peng Jin and Yinan Feng and Yinpeng Chen and Youzuo Lin and Sharon Huang},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.03264},
eprint = {2611.03264},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.03264},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - DeepWaveRL: Self-Supervised Full Waveform Inversion via Reinforcement Learning
AU - Peng Jin
AU - Yinan Feng
AU - Yinpeng Chen
AU - Youzuo Lin
AU - Sharon Huang
PY - 2026
DA - 2026/11/02
PB - arXiv
DO - 10.48550/arxiv.2611.03264
UR - https://arxiv.org/abs/2611.03264
AB - Full Waveform Inversion (FWI) is a fundamental inverse problem in geophysics to estimate subsurface properties, such as velocity, from seismic measurements. Recently, self-supervised learning methods remove the need for ground-truth velocity maps during training by leveraging a forward operator to simulate seismic data from predictions and backpropagate the gradients of reconstruction loss through it. In this work, we consider a more challenging setting without accessing such gradients. To this end, we incorporate reinforcement learning into self-supervised FWI and propose a framework named DeepWaveRL. Specifically, we reformulate FWI as a policy learning problem, where the policy network predicts velocity maps as actions, and a forward operator is used solely for simulation. A global scalar reward is computed based on data misfit for each prediction and used to weight policy gradients. Thus, learning proceeds without accessing the gradients from the forward operator. Given the indirect and coarse supervision signals, we further identify key strategies to stabilize learning and mitigate the spatial credit assignment bottleneck. Experimental results show that, despite relying entirely on a single scalar reward, DeepWaveRL achieves competitive performance compared to supervised and self-supervised baselines. These findings suggest that our framework offers a flexible and complementary learning paradigm for the FWI research community.
ER -