When Do Models Win? A Learning Curve Benchmark for Molecular Property Prediction in Low-Data Regimes
- 1¹Department of Information Display
- 2Kyung Hee University
Abstract
In molecular property prediction, labeled data are often limited to tens to thousands of samples, yet it remains unclear which model classes are most effective in these low-data regimes and how their ranking evolves with data size. We present a learning-curve benchmark spanning traditional machine learning, graph neural networks, transformer-based molecular language models, rotationally equivariant and SE(3)-invariant 3D networks, descriptor-based hybrids, multi-task learning variants, and graph–sequence fusion baselines. Training size is varied from 50 to 3,000 samples, with dataset-dependent maxima, on QM9, ESOL, Lipophilicity, and BACE under scaffold splits with repeated trials. Four patterns emerge. (1) Model optimality is regime-dependent: deep learning leads continuous physicochemical properties (ESOL, Lipophilicity) while Gaussian Process Regression leads bioactivity (BACE) across most of the range, with clear traditional-versus-deep-learning crossovers. (2) The benefit of 3D inductive bias is target-dependent: on QM9 an equivariant 3D network trained on DFT geometries is strong on LUMO and the HOMO–LUMO gap but weak on HOMO, although all three targets use the same DFT-optimized geometries; single-conformer ETKDG inputs instead give near-flat curves on Lipophilicity and BACE. (3) Pretraining chiefly raises the low-data floor rather than the mean. Its advantage concentrates where no single model dominates, typically at intermediate sizes, and its clearest effect is on worst-case accuracy: in a matched Uni-Mol pair, pretraining lifts the minimum QM9 R² at N≤200 by approximately 0.6 on every target. Neither parameter count nor pretraining-corpus size predicts the ranking — the compact ChemBERTa-2 (3.4M parameters) outperforms the far larger MoLFormer (44.5M) and SELFormer (86.8M) on ESOL, yet the ordering reverses on the other datasets. (4) Under data scarcity, model selection beats averaging: even oracle ensembles barely improve on the best single model at the smallest sizes (at most +1.2% in RMSE at N=50, and negative on ESOL). Together these results give data-scale-aware guidelines for model selection and establish learning-curve analysis as a principled tool for characterizing model behavior across the property–scale plane. Scientific Contribution: This study establishes a comprehensive learning-curve benchmark for molecular property prediction in low-data regimes, comparing a broad suite of model families across four datasets spanning physicochemical, quantum-mechanical, and bioactivity properties. It shows that the best-performing model family is jointly determined by data scale and target type — including reproducible crossovers between traditional machine learning and deep learning — and converts these findings into actionable, data-scale-aware model-selection guidelines.
Related
- Topology-Guided Multi-Level Graph Learning: A Unified Framework for Molecular Property Prediction
- Polymer Property Prediction via an Automated Molecular Dynamics Pipeline and Transfer Learning
- MatUQ: a benchmark for uncertainty-aware out-of-distribution materials property prediction with graph neural networks
- PolyBench: A Reproducible Open Benchmark Platform for Repeat-Unit-Based Polymer Property Prediction
- Weighted Views Representation with Multi-Head Coulomb Attention for Molecular Property Prediction
BibTeX
@misc{lee¹2026when,
title = {{When Do Models Win? A Learning Curve Benchmark for Molecular Property Prediction in Low-Data Regimes}},
author = {Junho Lee¹},
howpublished = {ChemRxiv},
year = {2026},
month = sep,
doi = {10.26434/chemrxiv.15001253/v6},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15001253/v6},
note = {CC BY},
}
RIS
TY - UNPB
TI - When Do Models Win? A Learning Curve Benchmark for Molecular Property Prediction in Low-Data Regimes
AU - Junho Lee¹
PY - 2026
DA - 2026/09/01
PB - ChemRxiv
DO - 10.26434/chemrxiv.15001253/v6
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15001253/v6
AB - In molecular property prediction, labeled data are often limited to tens to thousands of samples, yet it remains unclear which model classes are most effective in these low-data regimes and how their ranking evolves with data size. We present a learning-curve benchmark spanning traditional machine learning, graph neural networks, transformer-based molecular language models, rotationally equivariant and SE(3)-invariant 3D networks, descriptor-based hybrids, multi-task learning variants, and graph–sequence fusion baselines. Training size is varied from 50 to 3,000 samples, with dataset-dependent maxima, on QM9, ESOL, Lipophilicity, and BACE under scaffold splits with repeated trials. Four patterns emerge. (1) Model optimality is regime-dependent: deep learning leads continuous physicochemical properties (ESOL, Lipophilicity) while Gaussian Process Regression leads bioactivity (BACE) across most of the range, with clear traditional-versus-deep-learning crossovers. (2) The benefit of 3D inductive bias is target-dependent: on QM9 an equivariant 3D network trained on DFT geometries is strong on LUMO and the HOMO–LUMO gap but weak on HOMO, although all three targets use the same DFT-optimized geometries; single-conformer ETKDG inputs instead give near-flat curves on Lipophilicity and BACE. (3) Pretraining chiefly raises the low-data floor rather than the mean. Its advantage concentrates where no single model dominates, typically at intermediate sizes, and its clearest effect is on worst-case accuracy: in a matched Uni-Mol pair, pretraining lifts the minimum QM9 R² at N≤200 by approximately 0.6 on every target. Neither parameter count nor pretraining-corpus size predicts the ranking — the compact ChemBERTa-2 (3.4M parameters) outperforms the far larger MoLFormer (44.5M) and SELFormer (86.8M) on ESOL, yet the ordering reverses on the other datasets. (4) Under data scarcity, model selection beats averaging: even oracle ensembles barely improve on the best single model at the smallest sizes (at most +1.2% in RMSE at N=50, and negative on ESOL). Together these results give data-scale-aware guidelines for model selection and establish learning-curve analysis as a principled tool for characterizing model behavior across the property–scale plane. Scientific Contribution: This study establishes a comprehensive learning-curve benchmark for molecular property prediction in low-data regimes, comparing a broad suite of model families across four datasets spanning physicochemical, quantum-mechanical, and bioactivity properties. It shows that the best-performing model family is jointly determined by data scale and target type — including reproducible crossovers between traditional machine learning and deep learning — and converts these findings into actionable, data-scale-aware model-selection guidelines.
ER -