All against the machine: the Solo score for rating skill in variable environments
Abstract
We propose a distribution-free metric to rate individual skill in ``player-versus-environment'' settings, where participants face heterogeneous tasks without direct opponents. Such settings are common in digital platforms, games, education, finance, and the benchmark evaluation of AI agents. They combine high randomness, tasks of widely varying difficulty, and unknown heterogeneity across individuals. Our metric maps each task outcome to a bounded performance score with zero population mean and variance bounded by 1/3, regardless of the outcome distribution, so that scores are directly comparable across tasks. Aggregating these scores over tasks yields an interpretable skill score for each individual (the ``Solo'' rating), and a reshuffling null model tests whether that score exceeds what chance alone would produce. We validate the approach on the progression of 2×105 players in the mobile games Candy Crush Saga and Bubble Witch 3 Saga. The metric identifies high- and low-skill players with high statistical confidence, their classification persists over hundreds of subsequent levels, and a windowed version of the score tracks changes in performance along progression.
Related
- Machine Learning Assisted Design of Complex and High Entropy Alloys by Hybrid HiPIMS/Pulsed-DC PVD Process for Low Carbon Energy Applications in Extreme Environments
- Testing machine-learned distributions against Monte Carlo data for the QCD chiral phase transition
- Robust optimal reinsurance and ESG investment under rating-innovation ambiguity and ESG rating uncertainty
- Intensity–phase–modal degradation of Bessel–Gaussian vortex beams in sea-fog scattering environments
- Exponential advantage in continuous-variable quantum state learning
BibTeX
@misc{reguera2026against,
title = {{All against the machine: the Solo score for rating skill in variable environments}},
author = {David Reguera and Xavier R. Hoffmann and Irene Pérez and Pol Colomer-de-Simón and Miquel Masoliver and Xavier Guardiola and Jan Wedekind and Marián Boguñá},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.04523},
eprint = {2610.04523},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.04523},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - All against the machine: the Solo score for rating skill in variable environments
AU - David Reguera
AU - Xavier R. Hoffmann
AU - Irene Pérez
AU - Pol Colomer-de-Simón
AU - Miquel Masoliver
AU - Xavier Guardiola
AU - Jan Wedekind
AU - Marián Boguñá
PY - 2026
DA - 2026/10/03
PB - arXiv
DO - 10.48550/arxiv.2610.04523
UR - https://arxiv.org/abs/2610.04523
AB - We propose a distribution-free metric to rate individual skill in ``player-versus-environment'' settings, where participants face heterogeneous tasks without direct opponents. Such settings are common in digital platforms, games, education, finance, and the benchmark evaluation of AI agents. They combine high randomness, tasks of widely varying difficulty, and unknown heterogeneity across individuals. Our metric maps each task outcome to a bounded performance score with zero population mean and variance bounded by 1/3, regardless of the outcome distribution, so that scores are directly comparable across tasks. Aggregating these scores over tasks yields an interpretable skill score for each individual (the ``Solo'' rating), and a reshuffling null model tests whether that score exceeds what chance alone would produce. We validate the approach on the progression of 2×10^5 players in the mobile games Candy Crush Saga and Bubble Witch 3 Saga. The metric identifies high- and low-skill players with high statistical confidence, their classification persists over hundreds of subsequent levels, and a windowed version of the score tracks changes in performance along progression.
ER -