Language-model-derived training data for polymer glass transition prediction: a controlled benchmark
- 1Burgas State University “Prof. Dr. Assen Zlatarov
Abstract
Experimental polymer property data are scarce and costly, whereas large language models (LLMs) have absorbed much of the chemical literature and can be queried for a fraction of a cent per polymer. We ask whether LLM-produced data can replace or supplement experimental data for predicting the glass transition temperature (Tg). With Claude Sonnet 5 as the teacher we built two synthetic datasets for USD 10.28: a generated set of 7,912 polymers whose structures and Tg values were both written by the LLM, and a labeled set of 9,925 hypothetical PI1M polymers whose Tg the LLM predicted. Both were cleaned with RDKit, deduplicated with a cut-invariant polymer key, and purged of every polymer of a held-out test set (1,471 polymers) and validation set (589) drawn from a curated experimental collection. A fine-tuned MoLFormer student was trained along a learning curve of 250 to 5,297 real polymers, with and without a preceding stage on synthetic data, and compared with a fingerprint random forest, two controls (self-training pseudo-labels and shuffled labels) and the teacher’s own zero-shot predictions, over 5 seeds. Trained on synthetic data alone, the student reached a mean absolute error (MAE) of 36.8 ◦C (generated) and 38.5 ◦C (labeled), within about 1 ◦C of the teacher (37.3 ◦C) and far from a student trained on all real data (25.2 ◦C). Synthetic data helped only when real data were scarce: with 250 or 500 real polymers, a synthetic first stage reduced the MAE by 3 ◦C to 5 ◦C on average and on every seed, a gain that shuffled labels abolished and that free self-training pseudo-labels reproduced only in part; against a baseline trained with more patient early stopping it was 3 ◦C to 4 ◦C at 250 real polymers and about 2 ◦C at 500; with 5,297 real polymers the gain was below 1 ◦C and not distinguishable from zero. Of the unique generated polymers, 8.3 % were copies of real polymers, 56 % of the valid generated pairs were repeats, and the teacher’s labels were rounded to a few values and shifted to lower Tg. LLM-produced data are thus a cheap and, below a few hundred measurements, useful supplement, not a substitute for measurements.
Related
- Surfactant-Toolkit and SurfBench-200: an open computation toolkit and a benchmark of tool-augmented language-model reasoning in surfactant science
- Can Universal Machine-Learning Interatomic Potentials Capture Small Polarons? A Geometry-Based Benchmark in Transition-Metal Oxides
- Hybrid-CatBoost: an empirical comparison of molecular representations for aqueous-solubility prediction on the TDC benchmark
- CCSBase2: A Unified Benchmark Dataset and Machine Learning Framework for Robust CCS Prediction.
- MatUQ: a benchmark for uncertainty-aware out-of-distribution materials property prediction with graph neural networks
BibTeX
@misc{rusev2026languagemodelderived,
title = {{Language-model-derived training data for polymer glass transition prediction: a controlled benchmark}},
author = {Rostislav Rusev},
howpublished = {ChemRxiv},
year = {2026},
month = sep,
doi = {10.26434/chemrxiv.15009353/v1},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15009353/v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Language-model-derived training data for polymer glass transition prediction: a controlled benchmark
AU - Rostislav Rusev
PY - 2026
DA - 2026/09/23
PB - ChemRxiv
DO - 10.26434/chemrxiv.15009353/v1
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15009353/v1
AB - Experimental polymer property data are scarce and costly, whereas large language models (LLMs) have absorbed much of the chemical literature and can be queried for a fraction of a cent per polymer. We ask whether LLM-produced data can replace or supplement experimental data for predicting the glass transition temperature (Tg). With Claude Sonnet 5 as the teacher we built two synthetic datasets for USD 10.28: a generated set of 7,912 polymers whose structures and Tg values were both written by the LLM, and a labeled set of 9,925 hypothetical PI1M polymers whose Tg the LLM predicted. Both were cleaned with RDKit, deduplicated with a cut-invariant polymer key, and purged of every polymer of a held-out test set (1,471 polymers) and validation set (589) drawn from a curated experimental collection. A fine-tuned MoLFormer student was trained along a learning curve of 250 to 5,297 real polymers, with and without a preceding stage on synthetic data, and compared with a fingerprint random forest, two controls (self-training pseudo-labels and shuffled labels) and the teacher’s own zero-shot predictions, over 5 seeds. Trained on synthetic data alone, the student reached a mean absolute error (MAE) of 36.8 ◦C (generated) and 38.5 ◦C (labeled), within about 1 ◦C of the teacher (37.3 ◦C) and far from a student trained on all real data (25.2 ◦C). Synthetic data helped only when real data were scarce: with 250 or 500 real polymers, a synthetic first stage reduced the MAE by 3 ◦C to 5 ◦C on average and on every seed, a gain that shuffled labels abolished and that free self-training pseudo-labels reproduced only in part; against a baseline trained with more patient early stopping it was 3 ◦C to 4 ◦C at 250 real polymers and about 2 ◦C at 500; with 5,297 real polymers the gain was below 1 ◦C and not distinguishable from zero. Of the unique generated polymers, 8.3 % were copies of real polymers, 56 % of the valid generated pairs were repeats, and the teacher’s labels were rounded to a few values and shifted to lower Tg. LLM-produced data are thus a cheap and, below a few hundred measurements, useful supplement, not a substitute for measurements.
ER -