Precision Tokenization for Multi-Modal Data in High Energy Physics
Abstract
Foundation models have recently gained significant attention in fields such as high energy physics (HEP). The tokenizers that convert detector data into discrete inputs for such models, however, are comparatively unexplored. This paper addresses this gap by learning discrete and compressed tokenized representations of HEP datasets. We demonstrate the approach on four modalities covering two different simulated detectors that vary in complexity. Treating tokenization as a rate-distortion problem, a systematic hyperparameter scan yields four design rules for tokenizing HEP data that hold across all set-based modalities studied. The cost of tokenization is evaluated on tasks including jet pT regression, particle flow reconstruction, and flavor tagging. We find that tokenization preserves much of the information needed for these tasks at compression factors of approximately 4-11× while keeping the jet pT resolution below 1%. We also present heptokens, an open-source library for tokenizing HEP data.
Related
- Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks
- Photodiode based multi-modal diagnostic for low-energy neutral beam injection in the LTX-β spherical tokamak
- Physics-guided deep neural networks for Laguerre–Gaussian modal decomposition via PCA-ranked Zernike descriptors
- Reconstruction and Synthesis of Arbitrary Optical Intensity Patterns in Extremely Multimode Waveguide Cavities via Physics-Informed Optimized Modal Superposition
- A Multi-Modal Assessment to Evaluate Changes Following Pediatric Robot-Assisted Gait Training: A Case Study
BibTeX
@misc{krupa2026precision,
title = {{Precision Tokenization for Multi-Modal Data in High Energy Physics}},
author = {Jeffrey Krupa and Samuel Klein and Nilotpal Kakati and Daniel Murnane and Baran Hashemi and Eilam Gross and Lukas Heinrich and Michael Kagan},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.13876},
eprint = {2610.13876},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.13876},
}
RIS
TY - UNPB
TI - Precision Tokenization for Multi-Modal Data in High Energy Physics
AU - Jeffrey Krupa
AU - Samuel Klein
AU - Nilotpal Kakati
AU - Daniel Murnane
AU - Baran Hashemi
AU - Eilam Gross
AU - Lukas Heinrich
AU - Michael Kagan
PY - 2026
DA - 2026/10/09
PB - arXiv
DO - 10.48550/arxiv.2610.13876
UR - https://arxiv.org/abs/2610.13876
AB - Foundation models have recently gained significant attention in fields such as high energy physics (HEP). The tokenizers that convert detector data into discrete inputs for such models, however, are comparatively unexplored. This paper addresses this gap by learning discrete and compressed tokenized representations of HEP datasets. We demonstrate the approach on four modalities covering two different simulated detectors that vary in complexity. Treating tokenization as a rate-distortion problem, a systematic hyperparameter scan yields four design rules for tokenizing HEP data that hold across all set-based modalities studied. The cost of tokenization is evaluated on tasks including jet p_T regression, particle flow reconstruction, and flavor tagging. We find that tokenization preserves much of the information needed for these tasks at compression factors of approximately 4-11× while keeping the jet p_T resolution below 1%. We also present heptokens, an open-source library for tokenizing HEP data.
ER -