Bridging the Modality Bottleneck in Pathology MIL through Molecularly Guided Feature Adaptation

Abstract

Multiple instance learning (MIL) is the dominant framework for whole-slide image analysis in computational pathology, typically combining a frozen patch encoder, a projection layer, and a slide-level aggregator. While encoders and aggregators have been extensively studied, the projection layer remains a largely morphology-only bottleneck. This limits endpoints such as biomarker status and survival, which are governed by a molecular state that is not fully captured by H&E morphology. We introduce Molecularly Informed Staining Transform (MIST), a plug-in replacement for the MIL projection layer that uses paired spatial transcriptomics only during training to construct virtual molecular stains. MIST clusters gene expression profiles into cross-modal prototypes, anchors them in the frozen foundation model feature space, and uses them to reorganize H&E patch features along molecularly guided axes. It requires no transcriptomics at inference and can be inserted before standard MIL aggregators. We evaluate MIST across 23 downstream tasks and 8 MIL aggregators. MIST improves 240 of 256 configurations over the standard projection layer, with an average gain of +3.5%, observed consistently across endpoint types: +5.2% on survival prediction, +3.3% on tissue subtyping, and +2.6% on biomarker prediction. Ablations confirm that gene-derived prototypes are the primary source of the gains, while spatial, biological, and pathological analyses show that cross-modal prototype affinities capture spatially coherent molecular programs from H&E alone.

EEG Pathology Detection via Clinical Concept Bottleneck Models
M. Radwan, A. Yazidi, P. G. Lind
bioRxiv4 Oct 2026PreprintCC BY-NCFull textNeurosciencebioRxiv
Clinical-Anchored Incremental Survival Learning: A Stability-Constrained Framework for Molecularly Enhanced Survival Prediction in Lung Adenocarcinoma
P. Shringarpure, S. Divakar, S. Khobragade
bioRxiv4 Nov 2026PreprintAll rights reservedFull textBioinformatics & computational biologybioRxiv
945 A molecularly defined polymeric platform for environmentally responsive activation of STING to enhance cancer immunotherapy
Neil C Chada, Karan Arora, John T Wilson +2
Journal for ImmunoTherapy of Cancer6 Nov 2026Peer-reviewedCC BYFull textImmunologyDOAJ
Spatial organization of voltage-gated ion channel expression across molecularly defined neuronal populations in the mouse mammillary bodies
K. S. Springer, F. C. Driver, X. Wang +6
bioRxiv15 Sep 2026PreprintAll rights reservedFull textNeurosciencebioRxiv
Intersectional genetic targeting of molecularly distinct melanin-concentrating hormone neuron subpopulations reveals distinct and overlapping subcircuit architecture
M. S. Antony, P. A. Miller, W. P. Armstrong +10
bioRxiv5 Oct 2026PreprintAll rights reservedFull textNeurosciencebioRxiv

Cite

BibTeX RIS

BibTeX

@misc{xing2026bridging,
  title = {{Bridging the Modality Bottleneck in Pathology MIL through Molecularly Guided Feature Adaptation}},
  author = {Yucheng Xing and Pei Liu and Jingying Ma and Ruping Hong and Jiangdong Qiu and Tianyu Liu and Kai He and Ling Huang and Mengling Feng},
  howpublished = {arXiv},
  year = {2026},
  month = nov,
  doi = {10.48550/arxiv.2605.16392},
  eprint = {2605.16392},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.16392},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - Bridging the Modality Bottleneck in Pathology MIL through Molecularly Guided Feature Adaptation
AU  - Yucheng Xing
AU  - Pei Liu
AU  - Jingying Ma
AU  - Ruping Hong
AU  - Jiangdong Qiu
AU  - Tianyu Liu
AU  - Kai He
AU  - Ling Huang
AU  - Mengling Feng
PY  - 2026
DA  - 2026/11/04
PB  - arXiv
DO  - 10.48550/arxiv.2605.16392
UR  - https://arxiv.org/abs/2605.16392
AB  - Multiple instance learning (MIL) is the dominant framework for whole-slide image analysis in computational pathology, typically combining a frozen patch encoder, a projection layer, and a slide-level aggregator. While encoders and aggregators have been extensively studied, the projection layer remains a largely morphology-only bottleneck. This limits endpoints such as biomarker status and survival, which are governed by a molecular state that is not fully captured by H&E morphology. We introduce Molecularly Informed Staining Transform (MIST), a plug-in replacement for the MIL projection layer that uses paired spatial transcriptomics only during training to construct virtual molecular stains. MIST clusters gene expression profiles into cross-modal prototypes, anchors them in the frozen foundation model feature space, and uses them to reorganize H&E patch features along molecularly guided axes. It requires no transcriptomics at inference and can be inserted before standard MIL aggregators. We evaluate MIST across 23 downstream tasks and 8 MIL aggregators. MIST improves 240 of 256 configurations over the standard projection layer, with an average gain of +3.5%, observed consistently across endpoint types: +5.2% on survival prediction, +3.3% on tissue subtyping, and +2.6% on biomarker prediction. Ablations confirm that gene-derived prototypes are the primary source of the gains, while spatial, biological, and pathological analyses show that cross-modal prototype affinities capture spatially coherent molecular programs from H&E alone.
ER  -

Abstract

Related

Cite