Skip to content
DraconiticResearch
Search Sources API
201,683 studies31 sourcesUpdated 49 min ago
Peer-reviewedCC BY-NC-NDNature Communications

Scaling multimodal materials representation learning with synthetic narratives

  • Yang Jeong Park1,2,3
  • Mayank Kumaran4
  • Chia-Wei Hsu1
  • Elsa Olivetti5,6
  • Ju Li7,8,9
9 affiliations
  1. 1Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
  2. 2Department of Materials Science and Engineering, Ulsan National Institute of Science and Technology, Ulju-gun, Ulsan, Republic of Korea
  3. 3Department of Nuclear Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
  4. 4Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, IL, USA
  5. 5Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected]
  6. 6MIT-IBM Watson AI Lab, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected]
  7. 7Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected]
  8. 8Department of Nuclear Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected]
  9. 9MIT-IBM Watson AI Lab, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected]
Open at sourcePDFDOICite

Abstract

Artificial intelligence is increasingly applied to inverse materials design, including crystals and molecules. For molecular systems, multimodal models that integrate chemical structures with natural language have enabled flexible, instruction-driven reasoning. However, approaches for crystalline materials remain limited by the scarcity and imbalanced bias of available crystal datasets and the weak semantic supervision provided by peer-reviewed literature. We introduce a contrastive language-crystals model (CLaC), a multimodal contrastive learning framework that aligns crystal structures with natural language, pre-trained on property-conditioned synthetic narratives of crystal structure-text pairs generated by language models. Here we show that, compared with models trained on literature-derived text, this property-grounded synthetic supervision enables semantic alignment unachievable with existing academic corpora. Embedding-space analyses reveal structured associations between language and crystal structures. These results establish a foundation for zero-shot, instruction-driven reasoning in crystalline materials. CLaC is a contrastive language-crystals model that aligns crystal structures with natural language using property-conditioned synthetic narratives, enabling zero-shot, instruction driven reasoning unachievable with literature-derived text alone.

Related

  1. Multimodal Representation Learning for Exploring Natural Product Chemical Space
    Tarapong Srisongkram, Supreeya Paiboon, Huynh Anh Duy
    ChemRxivPreprintCC BY-NCGeneral chemistryChemRxiv
  2. Correlative SEM–AFM Characterization of 2D Materials: From Sequential Measurements to In-Situ Multimodal Platforms
    Xuan Wu, Wenping Li, Shu Li +5
    Nanomaterials (Basel, Switzerland)Peer-reviewedCC BYFull textMaterials & polymersEurope PMC
  3. Reading the whole corpus: a reproducible cross-sample Raman workflow for materials characterisation, with a spectral 3D Gaussian splatting representation for hyperspectral maps
    Siddhartha Pahari, Jainish Shailesh Solanki
    ChemRxivPreprintCC BYFull textGeneral chemistryChemRxiv
  4. Domain-Informed AI Multimodal Foundation Model for Materials Science
    Carine de Menezes Rebello, Erick Giovani Sperandio Nascimento, Idelfonso B. R. Nogueira
    ChemRxivPreprintCC BYMaterials & polymersChemRxiv
  5. Fragment-Based Representation Learning for Polymers and Heterogeneous Formulations
    Jianxin He, Subhamoy Mahajan, Nathaniel Straight +1
    ChemRxivPreprintLicence not statedGeneral chemistryChemRxiv

Cite

BibTeXRIS
BibTeX
@article{park2026scaling,
  title = {{Scaling multimodal materials representation learning with synthetic narratives}},
  author = {Yang Jeong Park and Mayank Kumaran and Chia-Wei Hsu and Elsa Olivetti and Ju Li},
  journal = {Nature Communications},
  year = {2026},
  month = aug,
  doi = {10.1038/s41467-026-76378-7},
  url = {https://doi.org/10.1038/s41467-026-76378-7},
  note = {CC BY-NC-ND},
}
RIS
TY  - JOUR
TI  - Scaling multimodal materials representation learning with synthetic narratives
AU  - Yang Jeong Park
AU  - Mayank Kumaran
AU  - Chia-Wei Hsu
AU  - Elsa Olivetti
AU  - Ju Li
PY  - 2026
DA  - 2026/08/26
JO  - Nature Communications
DO  - 10.1038/s41467-026-76378-7
UR  - https://doi.org/10.1038/s41467-026-76378-7
AB  - Artificial intelligence is increasingly applied to inverse materials design, including crystals and molecules. For molecular systems, multimodal models that integrate chemical structures with natural language have enabled flexible, instruction-driven reasoning. However, approaches for crystalline materials remain limited by the scarcity and imbalanced bias of available crystal datasets and the weak semantic supervision provided by peer-reviewed literature. We introduce a contrastive language-crystals model (CLaC), a multimodal contrastive learning framework that aligns crystal structures with natural language, pre-trained on property-conditioned synthetic narratives of crystal structure-text pairs generated by language models. Here we show that, compared with models trained on literature-derived text, this property-grounded synthetic supervision enables semantic alignment unachievable with existing academic corpora. Embedding-space analyses reveal structured associations between language and crystal structures. These results establish a foundation for zero-shot, instruction-driven reasoning in crystalline materials. CLaC is a contrastive language-crystals model that aligns crystal structures with natural language using property-conditioned synthetic narratives, enabling zero-shot, instruction driven reasoning unachievable with literature-derived text alone.
SN  - 2041-1723
ER  -