Atomic Design Transformer: Scaffold-Conditioned 3D Molecule Generation with xTB-Verified Reinforcement Learning
Abstract
We present an autoregressive 3D-molecule generator with SE(3)-invariant tokenization, the Atomic Design Transformer (ADT). ADT places atoms one at a time, autoregressively. SE(3) invariance is achieved by tokenization: each new atom's position is encoded in the local coordinate frame of a previously placed atom. The backbone is a plain causal transformer. The token stream fully specifies a 3D structure together with its chemical-bond graph G, without any bond-order assignment. The model emits heavy-atom skeletons; hydrogens are added by separate learned models before the xTB relaxation. To score generated molecules we introduce the xTB topology-preservation rate (XTP): the fraction of molecules for which an xTB GFN2 relaxation preserves G specified by the token stream. For XTP-accepted molecules we also report the relaxation energy and the root mean square of the atomic displacement (RMSD). We evaluate two ADT models. The first is ADT pretrained on the GEOM-Drugs ≤ 30-heavy-atom dataset; we benchmark scaffold-conditioned 3D generation across seven drug-like scaffolds from the model. It reaches an XTP of ∼55% and a valid-molecule yield Ngen/N of ∼53%, where Ngen/N is the fraction of samples that are distinct, topology-preserving, and RDKit-readable. The second model continues from the first by reinforcement learning against the verifiable xTB reward (RLVR), using no external molecules. RLVR raises XTP to ∼98% and Ngen/N to ∼95%, while approximately preserving the GEOM-Drugs size and composition distributions. Finally, we present an Inverse-Kinematics Transformer that recovers XTP for large molecules, where discretization error accumulates. ADT thus enables direct 3D generation.
Related
- Learning Scattering Amplitudes with Transformer Reinforcement Learning
- Manufacturing-conditioned hierarchical generation of cathode active-material microstructures with global electrode context
- Editable Map-Conditioned Trajectory Generation for Human Mobility Simulation
- Variational Quantum Transformer Architecture for Synthetic Language Generation
- Site-orbit masked twin-branch Transformer for symmetry-constrained crystal generation
BibTeX
@misc{kotani2026atomic,
title = {{Atomic Design Transformer: Scaffold-Conditioned 3D Molecule Generation with xTB-Verified Reinforcement Learning}},
author = {Takao Kotani},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.15918},
eprint = {2607.15918},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.15918},
}
RIS
TY - UNPB
TI - Atomic Design Transformer: Scaffold-Conditioned 3D Molecule Generation with xTB-Verified Reinforcement Learning
AU - Takao Kotani
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2607.15918
UR - https://arxiv.org/abs/2607.15918
AB - We present an autoregressive 3D-molecule generator with SE(3)-invariant tokenization, the Atomic Design Transformer (ADT). ADT places atoms one at a time, autoregressively. SE(3) invariance is achieved by tokenization: each new atom's position is encoded in the local coordinate frame of a previously placed atom. The backbone is a plain causal transformer. The token stream fully specifies a 3D structure together with its chemical-bond graph G, without any bond-order assignment. The model emits heavy-atom skeletons; hydrogens are added by separate learned models before the xTB relaxation. To score generated molecules we introduce the xTB topology-preservation rate (XTP): the fraction of molecules for which an xTB GFN2 relaxation preserves G specified by the token stream. For XTP-accepted molecules we also report the relaxation energy and the root mean square of the atomic displacement (RMSD). We evaluate two ADT models. The first is ADT pretrained on the GEOM-Drugs ≤ 30-heavy-atom dataset; we benchmark scaffold-conditioned 3D generation across seven drug-like scaffolds from the model. It reaches an XTP of ∼55% and a valid-molecule yield N^gen/N of ∼53%, where N^gen/N is the fraction of samples that are distinct, topology-preserving, and RDKit-readable. The second model continues from the first by reinforcement learning against the verifiable xTB reward (RLVR), using no external molecules. RLVR raises XTP to ∼98% and N^gen/N to ∼95%, while approximately preserving the GEOM-Drugs size and composition distributions. Finally, we present an Inverse-Kinematics Transformer that recovers XTP for large molecules, where discretization error accumulates. ADT thus enables direct 3D generation.
ER -