TAME: Element-wise Mixture-of-Experts Fusion for Data-Efficient and Interpretable Molecular Property Prediction
9 affiliations
- 1Theoretical Chemistry
- 2University of Munich (LMU
- 3Bielefeld University of Applied Sciences (HSBI
- 4Machine Learning Group
- 5TU Berlin
- 6BIFOLD–Berlin Institute for the Foundations of Learning and Data
- 7Korea University
- 8MPI for Informatics
- 9MPI for Solid State Research
Abstract
Property prediction for molecules and materials is bottlenecked by label scarcity, and in this regime the operative failure mode is not a low mean but an unpredictable tail: individual training runs collapse. The three dominant molecular representations fail in complementary ways—physicochemical descriptors are exact but fixed, graph neural networks learn topology but degenerate under sparse supervision, and language models supply semantic context but cannot compute exact quantities—yet combining them is itself the learning problem, because modality-level scalar gating commits the whole model to one trust weight per source. We introduce TAME, a tri-modal encoder whose element-wise Mixture-of-Experts router assigns an independent modality mixture to every hidden coordinate, stabilized by a balance–entropy regularizer, a closed-form gate initialization (β = log τ) that removes router burn-in, and two-stage self-supervised graph pretraining. On scaffold-split BACE (≈ 1.5k molecules, 100 seeds), the fusion stage improves level and spread together: median ROC-AUC rises from 0.752 to 0.817 and macro-F1 from 0.634 to 0.703, while seed-to-seed variance falls five-fold and a collapse tail reaching ROC-AUC ≈ 0.45 disappears. Because backbone, checkpoint, split, seeds and optimizer recipe are held fixed, the effect is attributable to the fusion stage alone. The router allocates a distinct mixture to each coordinate, which no scalar gate can represent, and the entropy weight moves that allocation continuously between graded mixing and a near-binary regime in which each coordinate is claimed outright by one modality. Gate mass additionally tracks encoder quality without supervision, shifting from text to topology once pretraining lifts the graph embedding out of rank-1 collapse. The same three experts map onto crystal graphs, composition–structure descriptors and literature-mined text, making TAME a transferable template for multimodal representation learning in materials discovery.
Related
- Embedded Morgan Fingerprints for more efficient molecular property predictions with machine learning
- Closed-loop Auto Research for Molecular Property Prediction: Discovering and Certifying Generalizable Improvements
- Topology-Guided Multi-Level Graph Learning: A Unified Framework for Molecular Property Prediction
- Geometry-Aware Line Graph Transformer Pretraining for Molecular Property Prediction
- Embedded Morgan Fingerprints for More Efficient Molecular Property Predictions With Machine Learning
BibTeX
@misc{schiller2026tame,
title = {{TAME: Element-wise Mixture-of-Experts Fusion for Data-Efficient and Interpretable Molecular Property Prediction}},
author = {Robert P. Schiller and Christoph Weisser and Klaus-Robert Müller and Christian Ochsenfeld and Parastoo Semnani},
howpublished = {ChemRxiv},
year = {2026},
month = sep,
doi = {10.26434/chemrxiv.15008270/v1},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15008270/v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - TAME: Element-wise Mixture-of-Experts Fusion for Data-Efficient and Interpretable Molecular Property Prediction
AU - Robert P. Schiller
AU - Christoph Weisser
AU - Klaus-Robert Müller
AU - Christian Ochsenfeld
AU - Parastoo Semnani
PY - 2026
DA - 2026/09/03
PB - ChemRxiv
DO - 10.26434/chemrxiv.15008270/v1
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15008270/v1
AB - Property prediction for molecules and materials is bottlenecked by label scarcity, and in this regime the operative failure mode is not a low mean but an unpredictable tail: individual training runs collapse. The three dominant molecular representations fail in complementary ways—physicochemical descriptors are exact but fixed, graph neural networks learn topology but degenerate under sparse supervision, and language models supply semantic context but cannot compute exact quantities—yet combining them is itself the learning problem, because modality-level scalar gating commits the whole model to one trust weight per source. We introduce TAME, a tri-modal encoder whose element-wise Mixture-of-Experts router assigns an independent modality mixture to every hidden coordinate, stabilized by a balance–entropy regularizer, a closed-form gate initialization (β = log τ) that removes router burn-in, and two-stage self-supervised graph pretraining. On scaffold-split BACE (≈ 1.5k molecules, 100 seeds), the fusion stage improves level and spread together: median ROC-AUC rises from 0.752 to 0.817 and macro-F1 from 0.634 to 0.703, while seed-to-seed variance falls five-fold and a collapse tail reaching ROC-AUC ≈ 0.45 disappears. Because backbone, checkpoint, split, seeds and optimizer recipe are held fixed, the effect is attributable to the fusion stage alone. The router allocates a distinct mixture to each coordinate, which no scalar gate can represent, and the entropy weight moves that allocation continuously between graded mixing and a near-binary regime in which each coordinate is claimed outright by one modality. Gate mass additionally tracks encoder quality without supervision, shifting from text to topology once pretraining lifts the graph embedding out of rank-1 collapse. The same three experts map onto crystal graphs, composition–structure descriptors and literature-mined text, making TAME a transferable template for multimodal representation learning in materials discovery.
ER -