Skip to content
DraconiticResearch
Search Sources API
197,693 studies31 sourcesUpdated 14 min ago
Peer-reviewedCC BYJournal of Cheminformatics

Transformers for molecular property prediction: domain adaptation efficiently improves performance

  • Afnan Sultan1
  • Max Rausch-Dupont2
  • Shahrukh Rafi Khan3,2
  • Olga Kalinina3,4
  • Dietrich Klakow5
  • Andrea Volkamer6,7
7 affiliations
  1. 1Data Driven Drug Design, Center for Bioinformatics, PharmaScienceHub (PSH), Saarland University, Universität Campus, 66123, Saarbrücken, Saarland, Germany
  2. 2Spoken Language Systems, PharmaScienceHub (PSH), Saarland Informatics Campus, Saarland University, Universität Campus, 66123, Saarbrücken, Saarland, Germany
  3. 3Drug Bioinformatics, Center for Bioinformatics, Medical Faculty, PharmaScienceHub (PSH), Saarland University, Saarbrücken, 66123, Saarland, Germany
  4. 4Helmholtz Institute for Pharmaceutical Research Saarland (HIPS), Helmholtz Centre for Infection Research (HZI), PharmaScienceHub (PSH), Saarbrücken, 66123, Saarland, Germany
  5. 5Spoken Language Systems, PharmaScienceHub (PSH), Saarland Informatics Campus, Saarland University, Universität Campus, 66123, Saarbrücken, Saarland, Germany. [email protected]
  6. 6Data Driven Drug Design, Center for Bioinformatics, PharmaScienceHub (PSH), Saarland University, Universität Campus, 66123, Saarbrücken, Saarland, Germany. [email protected]
  7. 7Helmholtz Institute for Pharmaceutical Research Saarland (HIPS), Helmholtz Centre for Infection Research (HZI), PharmaScienceHub (PSH), Saarbrücken, 66123, Saarland, Germany. [email protected]
Open at sourceDOICite

Abstract

Over the past six years, molecular transformer models have become a part of the computational toolbox for drug discovery. Most existing models are pre-trained on millions to billions of molecules from large-scale unlabeled datasets such as ZINC or ChEMBL. However, the extent to which such large-scale pre-training improves molecular property prediction remains unclear. This study investigates the potential of transformer models for molecular property prediction while addressing their current limitations. We explore strategies to enhance performance, including the influence of pre-training dataset size and the benefits of domain adaptation through chemically informed objectives. Our results show that increasing the pre-training dataset beyond approximately 400–800K molecules does not improve performance across seven datasets covering five ADME endpoints: lipophilicity, permeability, solubility (two datasets), microsomal stability (two datasets), and plasma protein binding. In contrast, applying domain adaptation on a small number of domain-relevant molecules ( ≤ 4K ) using multi-task regression of physicochemical properties significantly improves model performance across all datasets ( P -value < 0.001). Furthermore, we find that a model pre-trained on ∼ 400K molecules and adapted on a small domain-specific dataset outperforms larger-scale transformer models like MolFormer and performs comparably to MolBERT. Benchmarking these models alongside baseline representations using RDKit descriptors and Morgan fingerprints reveals that incorporating chemically, physically, and topologically informed features consistently leads to superior performance, regardless of whether used with traditional or transformer-based architectures. While traditional practices such as a random forest model with RDKit descriptors remain strong baselines, this study identifies concrete practices that significantly enhance the performance of transformer models. In particular, aligning pre-training and adaptation with chemically meaningful tasks and domain-relevant data offers a promising path forward for future advancements in molecular property prediction. Our models are available on HuggingFace to allow for easy use and adaptation at https://huggingface.co/collections/UdS-LSV/domain-adaptation-molecular-transformers-6821e7189ada6b7d0a5b62d4 . Scientific contribution We introduce the first systematic evaluation of domain adaptation strategies for molecular transformer models in predicting key ADME properties, such as lipophilicity, solubility, clearance, etc. Our results demonstrate that combining pre-training with domain adaptation significantly improves predictive performance and generalization across diverse chemical datasets (P-value < 1e−9 ). This contribution advances cheminformatics by offering practical guidelines and open resources for developing more accurate and robust molecular transformer models for property prediction.

Related

  1. Developing a Quantum Physics-Inspired Machine Learning Model for Partition Coefficient Prediction and Comparing Its Performance with Empirical Descriptors and Molecular Graph-Based Models
    Abhishek Tripathi, Riya Rajput, Mohammad Seraj +2
    ACS OmegaPeer-reviewedCC BY-NC-ND 4.0Full textGeneral chemistryCrossref
  2. Drugon: Phase-Encoded Morgan Fingerprints Match Pretrained Chemical Transformers in Low-Data Affinity Prediction
    Masoud Azizi
    ChemRxivPreprintCC BYFull textGeneral chemistryChemRxiv
  3. Few-Shot Early-Release Adaptation Enables Generalizable Prediction of PLGA Drug Release
    Tengxiao Liu, Juchen Zhang, Alireza Mansouri +1
    ChemRxivPreprintCC BYFull textMaterials & polymersChemRxiv
  4. Machine Learning of Methacrylated Dextran Phase Separation for Prediction, Transferability, and Adaptation
    Jordan Carden, Erfan Moaseri, Elizabeth Roberge +2
    ChemRxivPreprintCC BYFull textMaterials & polymersChemRxiv
  5. Machine-learning prediction of optical band gaps and photocatalytic performance in rare-earth-doped spinel ferrites
    Gaurav Katoch, Gazal Sharma, A. Johnson Santhosh
    Scientific ReportsPeer-reviewedCC BY-NC-ND 4.0Full textMaterials & polymersCrossref

Cite

BibTeXRIS
BibTeX
@article{sultan2026transformers,
  title = {{Transformers for molecular property prediction: domain adaptation efficiently improves performance}},
  author = {Afnan Sultan and Max Rausch-Dupont and Shahrukh Rafi Khan and Olga Kalinina and Dietrich Klakow and Andrea Volkamer},
  journal = {Journal of Cheminformatics},
  year = {2026},
  month = jul,
  doi = {10.1186/s13321-026-01252-z},
  url = {https://doi.org/10.1186/s13321-026-01252-z},
  note = {CC BY},
}
RIS
TY  - JOUR
TI  - Transformers for molecular property prediction: domain adaptation efficiently improves performance
AU  - Afnan Sultan
AU  - Max Rausch-Dupont
AU  - Shahrukh Rafi Khan
AU  - Olga Kalinina
AU  - Dietrich Klakow
AU  - Andrea Volkamer
PY  - 2026
DA  - 2026/07/29
JO  - Journal of Cheminformatics
DO  - 10.1186/s13321-026-01252-z
UR  - https://doi.org/10.1186/s13321-026-01252-z
AB  - Over the past six years, molecular transformer models have become a part of the computational toolbox for drug discovery. Most existing models are pre-trained on millions to billions of molecules from large-scale unlabeled datasets such as ZINC or ChEMBL. However, the extent to which such large-scale pre-training improves molecular property prediction remains unclear. This study investigates the potential of transformer models for molecular property prediction while addressing their current limitations. We explore strategies to enhance performance, including the influence of pre-training dataset size and the benefits of domain adaptation through chemically informed objectives. Our results show that increasing the pre-training dataset beyond approximately 400–800K molecules does not improve performance across seven datasets covering five ADME endpoints: lipophilicity, permeability, solubility (two datasets), microsomal stability (two datasets), and plasma protein binding. In contrast, applying domain adaptation on a small number of domain-relevant molecules ( ≤ 4K ) using multi-task regression of physicochemical properties significantly improves model performance across all datasets ( P -value < 0.001). Furthermore, we find that a model pre-trained on ∼ 400K molecules and adapted on a small domain-specific dataset outperforms larger-scale transformer models like MolFormer and performs comparably to MolBERT. Benchmarking these models alongside baseline representations using RDKit descriptors and Morgan fingerprints reveals that incorporating chemically, physically, and topologically informed features consistently leads to superior performance, regardless of whether used with traditional or transformer-based architectures. While traditional practices such as a random forest model with RDKit descriptors remain strong baselines, this study identifies concrete practices that significantly enhance the performance of transformer models. In particular, aligning pre-training and adaptation with chemically meaningful tasks and domain-relevant data offers a promising path forward for future advancements in molecular property prediction. Our models are available on HuggingFace to allow for easy use and adaptation at https://huggingface.co/collections/UdS-LSV/domain-adaptation-molecular-transformers-6821e7189ada6b7d0a5b62d4 . Scientific contribution We introduce the first systematic evaluation of domain adaptation strategies for molecular transformer models in predicting key ADME properties, such as lipophilicity, solubility, clearance, etc. Our results demonstrate that combining pre-training with domain adaptation significantly improves predictive performance and generalization across diverse chemical datasets (P-value < 1e^−9 ). This contribution advances cheminformatics by offering practical guidelines and open resources for developing more accurate and robust molecular transformer models for property prediction.
SN  - 1758-2946
ER  -