Representation Transfer of Foundation Models for Ultra-Widefield Retinal Imaging.
8 affiliations
- 1Institute of Ophthalmology, University College London London EC1V 9EL U.K.
- 2Wake Forest University School of Medicine Winston-Salem NC 27101 USA.
- 3Virginia Tech-Wake Forest University School of Biomedical Engineering and Sciences Blacksburg VA 24061 USA.
- 4Faculty of Electrical Engineering and ComputingUniversity of Zagreb Zagreb 10000 Croatia.
- 5Department of Ophthalmology and Visual SciencesThe University of British Columbia Vancouver BC V5Z 3N9 Canada.
- 6School of Biomedical EngineeringThe University of British Columbia Vancouver BC V6T 1Z3 Canada.
- 7NIHR Biomedical Research CentreMoorfields Eye Hospital NHS Foundation Trust London EC1V 2PD U.K.
- 8Department of Medical Physics and Biomedical EngineeringUniversity College London London WC1E 6BT U.K.
Abstract
Despite the widespread adoption of foundation models as feature extractors for medical imaging, relatively little is understood about how different pretraining strategies influence the transferability of learned representations to weakly supervised ophthalmic imaging tasks. We investigate this question in ultra-widefield (UWF) retinal imaging by evaluating foundation model representations within a patch-based multiple instance learning (MIL) framework for disease classification on UWF images. We compare Vision Transformer encoders pretrained with supervised, Masked Autoencoder (MAE), and self-distillation objectives, while keeping the downstream aggregation architecture unchanged. Within a controlled comparison of ViT-B encoders pretrained on ImageNet-1k, the choice of pretraining objective substantially influenced frozen representation transfer, with supervised and self-distillation-based models outperforming MAE. A contemporary DINOv3 model pretrained at a larger scale achieved the strongest overall performance, with a quadratic weighted kappa of 0.863 for five-class diabetic retinopathy grading, comparable with DINOv1. Attention analysis further revealed distinct patch-aggregation behaviours associated with the different pretrained representations, while partial fine-tuning reduced the performance gap for MAE. These findings suggest that pretraining strategy influences both representation transferability and the subsequent aggregation of patch-level evidence within MIL, resulting in differences in downstream classification performance.
Related
- Test-time neural style transfer for domain harmonization, with application to cone detection in adaptive optics retinal imaging
- Multimodal Imaging of Fundus Abnormalities in Stickler Syndrome: Characterization of 15 Distinct Features Using Ultra-Widefield Swept-Source OCT Angiography
- Explainable foundation model for dementia screening and risk stratification using retinal fundus images
- Retinal layer thickness prediction from smartphone fundus imaging for early glaucoma detection
- Use of a Retinal Phantom to Facilitate Adaptive Optics Scanning Light Ophthalmoscopy Imaging in a Multicenter Clinical Trial
BibTeX
@article{gong2026representation,
title = {{Representation Transfer of Foundation Models for Ultra-Widefield Retinal Imaging.}},
author = {Mingya Alexa Gong and Da Ma and Lovre Antonio Budimir and Ivana Matovinovic and Sven Loncaric and Myeong Jin Ju and Yukun Zhou and Siegfried K Wagner and Pearse A Keane and Marinko V Sarunic},
journal = {IEEE access : practical innovations, open solutions},
year = {2026},
month = sep,
doi = {10.1109/access.2026.3737380},
url = {https://europepmc.org/article/MED/42890659},
note = {CC BY},
}
RIS
TY - JOUR
TI - Representation Transfer of Foundation Models for Ultra-Widefield Retinal Imaging.
AU - Mingya Alexa Gong
AU - Da Ma
AU - Lovre Antonio Budimir
AU - Ivana Matovinovic
AU - Sven Loncaric
AU - Myeong Jin Ju
AU - Yukun Zhou
AU - Siegfried K Wagner
AU - Pearse A Keane
AU - Marinko V Sarunic
PY - 2026
DA - 2026/09/25
JO - IEEE access : practical innovations, open solutions
DO - 10.1109/access.2026.3737380
UR - https://europepmc.org/article/MED/42890659
AB - Despite the widespread adoption of foundation models as feature extractors for medical imaging, relatively little is understood about how different pretraining strategies influence the transferability of learned representations to weakly supervised ophthalmic imaging tasks. We investigate this question in ultra-widefield (UWF) retinal imaging by evaluating foundation model representations within a patch-based multiple instance learning (MIL) framework for disease classification on UWF images. We compare Vision Transformer encoders pretrained with supervised, Masked Autoencoder (MAE), and self-distillation objectives, while keeping the downstream aggregation architecture unchanged. Within a controlled comparison of ViT-B encoders pretrained on ImageNet-1k, the choice of pretraining objective substantially influenced frozen representation transfer, with supervised and self-distillation-based models outperforming MAE. A contemporary DINOv3 model pretrained at a larger scale achieved the strongest overall performance, with a quadratic weighted kappa of 0.863 for five-class diabetic retinopathy grading, comparable with DINOv1. Attention analysis further revealed distinct patch-aggregation behaviours associated with the different pretrained representations, while partial fine-tuning reduced the performance gap for MAE. These findings suggest that pretraining strategy influences both representation transferability and the subsequent aggregation of patch-level evidence within MIL, resulting in differences in downstream classification performance.
SN - 2169-3536
ER -