Modelling interpretable patient-level representationsfrom structured and simple multimodal data
- 1Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Warsaw, Poland; Institute of AI for Health, Helmholtz Munich, Neuherberg, Germany
Abstract
Patient cohort profiling increasingly includes structured views for multiple modalities, such as single-cell RNA sequencing, spatial transcriptomics or proteomics, and histology, each providing multiple subobservations per patient, including single cells, spatial spots or patches. To model such data along with simple patient-level views, current multimodal integration methods typically rely on separately precomputed summaries and fail to fully leverage information in structured views. Here we present FACTMx, a variational framework that jointly models structured and simple views to learn interpretable patient-level representations. FACTMx couples latent patient factors with subobservation clustering and per-patient component proportions, enabling direct interpretation and downstream association analyses. The framework supports different structured-view mixture assumptions, including topic- and Gaussian-structured data, while retaining modular encoder-decoder parameterisations. In simulations spanning sparse and dense dependencies and multiple noise regimes, FACTMx improved reconstruction, integration and recovery of structured components relative to previous methods. Applied to non-small cell lung cancer cohorts, FACTMx captured survival-associated latent signals linked to immune microenvironments, gene expression pathways and spatially coherent histological patterns. In a longitudinal coronary syndrome cohort, FACTMx highlighted an outcome-associated axis connected to ejection-fraction change, immune cell states, soluble mediators and cardiac injury markers. These results support joint structured-simple modelling for interpretable multimodal patient stratification.
Related
- An interpretable multimodal deep learning model for physiological stress detection
- SAIL: Sparse Autoencoders for Interpretable Alignment of Human Vision and Multimodal Large Language Models
- Which modelling choices are supported when marine survey data are limited? An evidence map of 95 studies and a benchmark on structured subsamples of a trawl survey
- FloREN: Decoding Immune Regulatory Networks through Interpretable Graph Transformer Patient Representations
- Learning from Adaptive Oscillatory Modes: An Interpretable Synchrosqueezing Transformer for Patient-Independent Seizure Detection
BibTeX
@misc{okszaorzechowski2026modelling,
title = {{Modelling interpretable patient-level representationsfrom structured and simple multimodal data}},
author = {K. Oksza-Orzechowski and M. Lazecka and L. Koperski and D. Wojtowicz and M. Mozejko and D. Schulz and R. Liechti and F. Marzetta and M. Morfouace and H. S. Hong and S. Tissot and B. Bodenmiller and E. Staub and E. Szczurek},
howpublished = {bioRxiv},
year = {2026},
month = aug,
doi = {10.64898/2026.08.31.748244},
url = {https://www.biorxiv.org/content/10.64898/2026.08.31.748244v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Modelling interpretable patient-level representationsfrom structured and simple multimodal data
AU - K. Oksza-Orzechowski
AU - M. Lazecka
AU - L. Koperski
AU - D. Wojtowicz
AU - M. Mozejko
AU - D. Schulz
AU - R. Liechti
AU - F. Marzetta
AU - M. Morfouace
AU - H. S. Hong
AU - S. Tissot
AU - B. Bodenmiller
AU - E. Staub
AU - E. Szczurek
PY - 2026
DA - 2026/08/31
PB - bioRxiv
DO - 10.64898/2026.08.31.748244
UR - https://www.biorxiv.org/content/10.64898/2026.08.31.748244v1
AB - Patient cohort profiling increasingly includes structured views for multiple modalities, such as single-cell RNA sequencing, spatial transcriptomics or proteomics, and histology, each providing multiple subobservations per patient, including single cells, spatial spots or patches. To model such data along with simple patient-level views, current multimodal integration methods typically rely on separately precomputed summaries and fail to fully leverage information in structured views. Here we present FACTMx, a variational framework that jointly models structured and simple views to learn interpretable patient-level representations. FACTMx couples latent patient factors with subobservation clustering and per-patient component proportions, enabling direct interpretation and downstream association analyses. The framework supports different structured-view mixture assumptions, including topic- and Gaussian-structured data, while retaining modular encoder-decoder parameterisations. In simulations spanning sparse and dense dependencies and multiple noise regimes, FACTMx improved reconstruction, integration and recovery of structured components relative to previous methods. Applied to non-small cell lung cancer cohorts, FACTMx captured survival-associated latent signals linked to immune microenvironments, gene expression pathways and spatially coherent histological patterns. In a longitudinal coronary syndrome cohort, FACTMx highlighted an outcome-associated axis connected to ejection-fraction change, immune cell states, soluble mediators and cardiac injury markers. These results support joint structured-simple modelling for interpretable multimodal patient stratification.
ER -