Interpreting Public-Dataset SOH Benchmarks: A Systematic Review of Evaluation Protocols and Reproducibility Evidence
- 1School of Intelligent Manufacturing, Nanjing Polytechnic Institute, Nanjing 210048, China
Abstract
Studies that use the same public lithium-ion battery dataset do not necessarily evaluate the same state-of-health (SOH) estimation task; reported performance depends on how the experiment is defined. We mapped 408 studies and then examined a nested set of 168 with SOH-REF. This subset was not prespecified as representative, so its percentages are descriptive only. All four Core interpretability elements could be identified in 95/168 studies (56.5%). Same-cell and unseen-cell tests addressed different generalization settings. Preprocessing fitting scope was assessable in 40/168 studies (23.8%), whereas leakage risk was assessable in 64/168 (38.1%). Uncertainty evidence was uncommon, and explicit article-reported public implementation evidence was rare. These classifications reflect what the papers reported; NOT_ASSESSABLE does not indicate poor methodology or establish whether leakage was present. An accuracy value is therefore most informative when accompanied by the health-target definition, physical train-test relationship, preprocessing fitting scope, metric context, and aggregation procedure. More explicit reporting of these elements would make public-dataset SOH benchmarks easier to interpret and reuse without imposing a single evaluation design.
Related
- Reproducibility, Benchmarks, and Mechanistic Diversity: Building Trust in Computational Electrocatalysis for the AI Era
- Identifying and Addressing Systematic Data Leakage in Protein-Ligand Affinity Benchmarks
- Systematic Evaluation of Halogen Substitution in ortho-Benzoxy Phenylboronic Acid Catalysts for Amidation Reactions
- Systematic Evaluation of Mixture-Design for Multi-Oxide Magnesium Aluminosilicate Glass Development: Model Comparison, Modifier Interactions, and Predictive Limitations
- A Pathway-Economical Synthesis Enabling Systematic Evaluation: Enantiodivergent Access to Aspidosperma Alkaloids Unlocks Divergent Antitumor Mechanisms.
BibTeX
@article{li2026interpreting,
title = {{Interpreting Public-Dataset SOH Benchmarks: A Systematic Review of Evaluation Protocols and Reproducibility Evidence}},
author = {Qingde Li and Renqun Wang},
journal = {Batteries},
year = {2026},
month = nov,
doi = {10.3390/batteries12110462},
url = {https://www.mdpi.com/2313-0105/12/11/462},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Interpreting Public-Dataset SOH Benchmarks: A Systematic Review of Evaluation Protocols and Reproducibility Evidence
AU - Qingde Li
AU - Renqun Wang
PY - 2026
DA - 2026/11/04
JO - Batteries
DO - 10.3390/batteries12110462
UR - https://www.mdpi.com/2313-0105/12/11/462
AB - Studies that use the same public lithium-ion battery dataset do not necessarily evaluate the same state-of-health (SOH) estimation task; reported performance depends on how the experiment is defined. We mapped 408 studies and then examined a nested set of 168 with SOH-REF. This subset was not prespecified as representative, so its percentages are descriptive only. All four Core interpretability elements could be identified in 95/168 studies (56.5%). Same-cell and unseen-cell tests addressed different generalization settings. Preprocessing fitting scope was assessable in 40/168 studies (23.8%), whereas leakage risk was assessable in 64/168 (38.1%). Uncertainty evidence was uncommon, and explicit article-reported public implementation evidence was rare. These classifications reflect what the papers reported; NOT_ASSESSABLE does not indicate poor methodology or establish whether leakage was present. An accuracy value is therefore most informative when accompanied by the health-target definition, physical train-test relationship, preprocessing fitting scope, metric context, and aggregation procedure. More explicit reporting of these elements would make public-dataset SOH benchmarks easier to interpret and reuse without imposing a single evaluation design.
SN - 2313-0105
ER -