Neural Scaling Laws for Jet Generation
Abstract
Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.
Related
- Crossing over universal scaling laws in two-dimensional driven dissipative condensates
- Thermal scaling laws for open-water swimming
- Dropout Universality: Scaling Laws and Optimal Scheduling at the Edge-of-Chaos
- Detectability limits of scaling laws
- Edge energy fluctuation: A unified theory of friction scaling laws in twisted layered material interfaces
BibTeX
@misc{amram2026neural,
title = {{Neural Scaling Laws for Jet Generation}},
author = {Oz Amram and Darius A. Faroughy and Tjarko Gerdes and Anna Hallin and Gregor Kasieczka and Michael Krämer and Humberto Reyes-Gonzalez and David Shih},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.28940},
eprint = {2605.28940},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.28940},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Neural Scaling Laws for Jet Generation
AU - Oz Amram
AU - Darius A. Faroughy
AU - Tjarko Gerdes
AU - Anna Hallin
AU - Gregor Kasieczka
AU - Michael Krämer
AU - Humberto Reyes-Gonzalez
AU - David Shih
PY - 2026
DA - 2026/10/06
PB - arXiv
DO - 10.48550/arxiv.2605.28940
UR - https://arxiv.org/abs/2605.28940
AB - Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.
ER -