Spatially Context-Aware Transformers Facilitate Modeling-Based Anomaly Detection of Subtle Lesions in Brain MRI Images
- 1Ruhr-University Bochum, Center for Protein Diagnostics, Bochum, Germany
Abstract
The detection of small and subtle lesions in high-resolution 3D volumes is a highly relevant, yet far from solved task in biomedical imaging. We here address a specific task in detecting certain types of epileptogenic lesions through our novel semi-supervised spatially context aware transformer (SpyCAT) approach to anomaly detection. SpyCAT is modeling-based in the sense that it builds on specific assumptions that constitute what is normal and what constitues relevant deviations from normality. We explicitly use these assumptions to justify the inductive bias of our anomaly detection approach. The resulting SpyCAT system is patch-based and uses a transformer architecture to process discrete tokens obtained from a vector quantizing variational autoencoder, which produces counterfactual patches through full 3D convolutions of each patch. We evaluate our approach on the grounds of point-annotations of two subtypes of epileptogenic lesions, using validation measures that build on the Metrics Reloaded framework, showing that SpyCAT can reliably identify and localize the lesion types under consideration, and outperforms state-of-the-art reference methods. Our code is available at https://github.com/johannesSX/SpyCAT.
Related
- UAV image segmentation in complex mining areas by integrating spatially adaptive feature modulation and multi-scale context fusion modules
- Continuous spatially informed microenvironment programs show cross-cohort concordance and context-dependent spatial organization in diffuse large B-cell lymphoma
- SpatioCAD: Context-aware graph diffusion model for pinpointing spatially variable genes in heterogeneous tissues
- Modeling spatially dependent count time series data using COM-poisson INGARCH Models
- CONORM: Context-Aware Entity Normalization for Adverse Drug Event Detection
BibTeX
@misc{schwarz2026spatially,
title = {{Spatially Context-Aware Transformers Facilitate Modeling-Based Anomaly Detection of Subtle Lesions in Brain MRI Images}},
author = {J. Schwarz and L. Will and J. Wellmer and A. Mosig},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.14.26363006},
url = {https://www.medrxiv.org/content/10.64898/2026.09.14.26363006v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Spatially Context-Aware Transformers Facilitate Modeling-Based Anomaly Detection of Subtle Lesions in Brain MRI Images
AU - J. Schwarz
AU - L. Will
AU - J. Wellmer
AU - A. Mosig
PY - 2026
DA - 2026/09/14
PB - medRxiv
DO - 10.64898/2026.09.14.26363006
UR - https://www.medrxiv.org/content/10.64898/2026.09.14.26363006v1
AB - The detection of small and subtle lesions in high-resolution 3D volumes is a highly relevant, yet far from solved task in biomedical imaging. We here address a specific task in detecting certain types of epileptogenic lesions through our novel semi-supervised spatially context aware transformer (SpyCAT) approach to anomaly detection. SpyCAT is modeling-based in the sense that it builds on specific assumptions that constitute what is normal and what constitues relevant deviations from normality. We explicitly use these assumptions to justify the inductive bias of our anomaly detection approach. The resulting SpyCAT system is patch-based and uses a transformer architecture to process discrete tokens obtained from a vector quantizing variational autoencoder, which produces counterfactual patches through full 3D convolutions of each patch. We evaluate our approach on the grounds of point-annotations of two subtypes of epileptogenic lesions, using validation measures that build on the Metrics Reloaded framework, showing that SpyCAT can reliably identify and localize the lesion types under consideration, and outperforms state-of-the-art reference methods. Our code is available at https://github.com/johannesSX/SpyCAT.
ER -