Keeping Score: Adaptive, Tuning-Free Loss Weighting for Score-Augmented Neural Ratio Estimation
Abstract
Neural likelihood surrogates (e.g., Neural Ratio Estimation) for stochastic process models are commonly trained via probabilistic classification on simulated data, which forces a tradeoff between surrogate quality and training costs. For structured models where the exact score ∇θlog p(x | θ) is available, this information can be incorporated into training by augmenting the cross-entropy loss with a score-matching term. However, the optimal weighting of the two losses is not known a priori, and selecting it by hand requires expensive tuning that undercuts the computational savings. We propose an adaptive, tuning-free algorithm that sets the score loss weights during training based on loss gradients, adding minimal overhead to standard classifier training. We evaluate our approach on case studies involving network dynamics and spatial processes, demonstrating that it improves surrogate quality at a drastically lower computational cost than generating more training data. Notably, in some cases, our approach achieves downstream inference performance equivalent to a 10x increase in training data with less than a 1.1x increase in training time.
Related
- On relations between the Cochran--Mantel--Haenszel test and Augmented Inverse Probability Weighting
- A Nonconvex Framework for Joint Sparse Sample Weighting and Parameter Estimation
- Optimal Weighting of Training Data in Adaptive Coding
- Weighting Scheme and Construction in IPCW Survival Estimation
- Adaptive spline-based weighting functions for blended survival curve extrapolation
BibTeX
@misc{shen2026keeping,
title = {{Keeping Score: Adaptive, Tuning-Free Loss Weighting for Score-Augmented Neural Ratio Estimation}},
author = {Alexander Shen and Mikael Kuusela},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.12118},
eprint = {2605.12118},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.12118},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Keeping Score: Adaptive, Tuning-Free Loss Weighting for Score-Augmented Neural Ratio Estimation
AU - Alexander Shen
AU - Mikael Kuusela
PY - 2026
DA - 2026/10/08
PB - arXiv
DO - 10.48550/arxiv.2605.12118
UR - https://arxiv.org/abs/2605.12118
AB - Neural likelihood surrogates (e.g., Neural Ratio Estimation) for stochastic process models are commonly trained via probabilistic classification on simulated data, which forces a tradeoff between surrogate quality and training costs. For structured models where the exact score ∇_θlog p(x | θ) is available, this information can be incorporated into training by augmenting the cross-entropy loss with a score-matching term. However, the optimal weighting of the two losses is not known a priori, and selecting it by hand requires expensive tuning that undercuts the computational savings. We propose an adaptive, tuning-free algorithm that sets the score loss weights during training based on loss gradients, adding minimal overhead to standard classifier training. We evaluate our approach on case studies involving network dynamics and spatial processes, demonstrating that it improves surrogate quality at a drastically lower computational cost than generating more training data. Notably, in some cases, our approach achieves downstream inference performance equivalent to a 10x increase in training data with less than a 1.1x increase in training time.
ER -