PairIT: Autoregressive Transformers for Low-Data Molecule Optimization
Abstract
Generating novel molecules with desired properties is a central challenge in drug discovery. Although foundational work focused on de novo generation, practical use cases require a more complex setting: conditioning on an initial seed molecule, optimizing multiple properties, and operating in data-sparse regimes. We introduce PairIT, a flexible training framework for autoregressive transformers over SMILES sequences that enables seeded generation by learning from pairs of structurally similar molecules. By leveraging matched pairs, the model learns to generate local, chemically meaningful edits that improve a property of the seed molecule. Prompt tokens extend this framework by specifying the target property, desired direction, and approximate magnitude of change. Multiple prompt tokens can be concatenated to enable multi-property optimization at inference time. For properties where pair supervision alone is insufficient, reinforcement learning post-training can further improve optimization, providing a tunable trade-off between proximity to the seed and the optimization objective. PairIT naturally supports pretraining on a large dataset and fine-tuning on a smaller dataset, which is critical for performance in data-sparse regimes. Surprisingly, we observe diminishing returns to the scale of the pretraining dataset. We demonstrate empirically on MoleculeNet and Therapeutics Data Commons datasets that, with appropriate paired-data curation, a standard transformer model of modest size (68M parameters) can achieve robust seeded molecular optimization in low-data regimes.
Related
- Quantifying Multi-Objective Optimization in Generative Chemistry for Molecule Design
- LinkerGen: Guided Autoregressive Generation of Organic Linkers for Reticular Chemistry
- Long-horizon Forecasting of Sodium-ion Battery Charge-profile Evolution and Degradation using Autoregressive Deep Sequence Models
- Drugon: Phase-Encoded Morgan Fingerprints Match Pretrained Chemical Transformers in Low-Data Affinity Prediction
- Recovering Small Molecule Ligand—RNA Binding Pocket Binding Modes and Exploring Potential Binding Pathways Using Metadynamics
BibTeX
@misc{lawrence2026pairit,
title = {{PairIT: Autoregressive Transformers for Low-Data Molecule Optimization}},
author = {Hannah Lawrence and Bodhi P. Vani and Ji Won Park and Natasa Tagasovska and Samuel Don Stanton and Andrew Martin Watkins and Michael Maser and Ewa M. Nowara},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.19123},
eprint = {2610.19123},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.19123},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - PairIT: Autoregressive Transformers for Low-Data Molecule Optimization
AU - Hannah Lawrence
AU - Bodhi P. Vani
AU - Ji Won Park
AU - Natasa Tagasovska
AU - Samuel Don Stanton
AU - Andrew Martin Watkins
AU - Michael Maser
AU - Ewa M. Nowara
PY - 2026
DA - 2026/10/13
PB - arXiv
DO - 10.48550/arxiv.2610.19123
UR - https://arxiv.org/abs/2610.19123
AB - Generating novel molecules with desired properties is a central challenge in drug discovery. Although foundational work focused on de novo generation, practical use cases require a more complex setting: conditioning on an initial seed molecule, optimizing multiple properties, and operating in data-sparse regimes. We introduce PairIT, a flexible training framework for autoregressive transformers over SMILES sequences that enables seeded generation by learning from pairs of structurally similar molecules. By leveraging matched pairs, the model learns to generate local, chemically meaningful edits that improve a property of the seed molecule. Prompt tokens extend this framework by specifying the target property, desired direction, and approximate magnitude of change. Multiple prompt tokens can be concatenated to enable multi-property optimization at inference time. For properties where pair supervision alone is insufficient, reinforcement learning post-training can further improve optimization, providing a tunable trade-off between proximity to the seed and the optimization objective. PairIT naturally supports pretraining on a large dataset and fine-tuning on a smaller dataset, which is critical for performance in data-sparse regimes. Surprisingly, we observe diminishing returns to the scale of the pretraining dataset. We demonstrate empirically on MoleculeNet and Therapeutics Data Commons datasets that, with appropriate paired-data curation, a standard transformer model of modest size (68M parameters) can achieve robust seeded molecular optimization in low-data regimes.
ER -