Probabilistic representation learning of baseline transcriptomes enables prediction and pathway-level interpretation of anticancer drug responses
Abstract
Baseline transcriptomes can predict anticancer drug response, but their high dimensionality complicates pathway-level interpretation. We developed a probabilistic representation learning framework using a reusable variational autoencoder (VAE) latent space to prioritize response-associated factors by aggregate importance and drug preferentiality. Across five reference drugs, leakage-free repeated cross-validation showed that gene-level Ridge regression performed best, whereas 64-dimensional latent representations retained predictive signal after compression. In a separate interpretation VAE fitted to 939 cancer cell lines, six aggregate-importance candidates were bootstrap-robust within the fixed representation and were associated with immune/inflammatory, oncogenic, cell-cycle, stress-response, and cellular-processing programs. Drug-preferential candidates lacked factor-level bootstrap robustness, although several pathway associations remained supported after lineage adjustment. Five independent VAE refits showed partial gene-program concordance but seed-sensitive axes, while a principal component analysis baseline supported stable factor selection and pathway annotation. As an exploratory application, baseline transcriptomes from 15 QAL333-tested cell lines were projected into the fixed VAE without encoder retraining. QAL333-prioritized factors were associated with inflammatory, oncogenic, cellular-processing, and microenvironment-related pathways. The two-model zebrafish xenograft comparison remained descriptive and hypothesis-generating. These findings support reusable, stability-aware latent-space analysis as a framework for pathway-level hypothesis generation from high-dimensional transcriptomic drug-response data.
Related
- Alkaloids Reported in the Genus Griffinia (Amaryllidaceae): Phytochemical Evidence and Current Limitations in Anticancer Interpretation
- Nonparametric framework for the definition, adaptive detection and probabilistic interpretation of outliers
- Complex-Time Cosmology: A Three-Sector Model of the Universe and the Probabilistic Interpretation of Cosmic Composition
- Probabilistic representation and limit theorems for particle numbers of quasi-free states
- The Riesz representation in submetric spaces and its probabilistic aspects
BibTeX
@article{kim2026probabilistic,
title = {{Probabilistic representation learning of baseline transcriptomes enables prediction and pathway-level interpretation of anticancer drug responses}},
author = {Hyon Hee Kim and Suji Im and Sumin Kim and Jiyeon Han and Jiyun Kim and Jaehyun Sim and Seyeon Park},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-77428-2},
url = {https://www.nature.com/articles/s41598-026-77428-2},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Probabilistic representation learning of baseline transcriptomes enables prediction and pathway-level interpretation of anticancer drug responses
AU - Hyon Hee Kim
AU - Suji Im
AU - Sumin Kim
AU - Jiyeon Han
AU - Jiyun Kim
AU - Jaehyun Sim
AU - Seyeon Park
PY - 2026
DA - 2026/10/30
JO - Scientific Reports
DO - 10.1038/s41598-026-77428-2
UR - https://www.nature.com/articles/s41598-026-77428-2
AB - Baseline transcriptomes can predict anticancer drug response, but their high dimensionality complicates pathway-level interpretation. We developed a probabilistic representation learning framework using a reusable variational autoencoder (VAE) latent space to prioritize response-associated factors by aggregate importance and drug preferentiality. Across five reference drugs, leakage-free repeated cross-validation showed that gene-level Ridge regression performed best, whereas 64-dimensional latent representations retained predictive signal after compression. In a separate interpretation VAE fitted to 939 cancer cell lines, six aggregate-importance candidates were bootstrap-robust within the fixed representation and were associated with immune/inflammatory, oncogenic, cell-cycle, stress-response, and cellular-processing programs. Drug-preferential candidates lacked factor-level bootstrap robustness, although several pathway associations remained supported after lineage adjustment. Five independent VAE refits showed partial gene-program concordance but seed-sensitive axes, while a principal component analysis baseline supported stable factor selection and pathway annotation. As an exploratory application, baseline transcriptomes from 15 QAL333-tested cell lines were projected into the fixed VAE without encoder retraining. QAL333-prioritized factors were associated with inflammatory, oncogenic, cellular-processing, and microenvironment-related pathways. The two-model zebrafish xenograft comparison remained descriptive and hypothesis-generating. These findings support reusable, stability-aware latent-space analysis as a framework for pathway-level hypothesis generation from high-dimensional transcriptomic drug-response data.
SN - 2045-2322
ER -