Enhancing self-supervised representation learning through lightweight learnable data augmentation

Abstract

Data augmentation plays a central role in self-supervised learning, as the quality and diversity of augmented views strongly influence the learned representations. However, most existing self-supervised methods rely on fixed stochastic augmentation pipelines, while more adaptive alternatives often require expensive policy search, adversarial training, or additional optimization procedures. In this paper, we propose a lightweight learnable augmentation framework based on Extreme Learning Machines (ELM) for self-supervised visual representation learning. The proposed module predicts image-dependent transformation parameters and applies them through a differentiable augmentation operator, enabling joint optimization with the representation model while introducing minimal additional computational overhead. The framework is integrated into three representative self-supervised learning methods: SimCLR, BYOL, and SimSiam. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet show that the proposed method consistently improves linear evaluation performance relative to reproduced baselines across most settings. In particular, the method yields notable gains on CIFAR datasets and remains effective on the more challenging Tiny ImageNet benchmark. A per-class difficulty analysis further shows that the proposed augmentation strategy substantially improves performance on hard classes, indicating stronger robustness to challenging categories while maintaining competitive overall performance. In general, the results demonstrate that lightweight learnable augmentation can effectively enhance self-supervised representation learning across different frameworks and datasets.

IFC-HFlowVAE: A self-enhancing generative framework with structural anchoring for imbalanced clinical data augmentation
Lu Yuwen, Shuyu Chen
PLOS One3 Sep 2026Peer-reviewedCC BY 4.0Full textProbability Endocrinology & metabolismPLOS
Dual-domain self-supervised feature alignment via spectral–spatial representation learning for deepfake anomaly detection
Yi He
PLOS One11 Sep 2026Peer-reviewedCC BY 4.0Full textBioinformatics & computational biology ProbabilityPLOS
Enhancing cervical squamous cell carcinoma diagnosis under distribution shift via domain-adaptive representation learning
Zirui Huang, Ye Liu, Saihui Jin +15
iScience16 Oct 2026Peer-reviewedCC BY-NC-ND 4.0Full textOncologyCell PressPMCCrossrefEurope PMC
Semi-supervised clustering with knowledge-guided representation learning in cryo-electron tomography
Mohamad Kassab, Chengzhi Cao, Vincent Yao +3
PLOS Digital Health31 Aug 2026Peer-reviewedCC BY 4.0Full textCell biology Physical chemistryPLOS
MedLP-HAFB-CLIP: Hierarchical Adaptive Large Model With Learnable Medical Prompts for Level II Ultrasound Standard Plane Identification
Jiaxin Cai, Chenquan Dai, Runqing Xiong +4
Ultrasound in Medicine & Biology27 Jun 2026Peer-reviewedLicence not stated10 citedEpidemiology & public healthOpenAlex

Cite

BibTeX RIS

BibTeX

@article{alotaibi2026enhancing,
  title = {{Enhancing self-supervised representation learning through lightweight learnable data augmentation}},
  author = {Mubarakah Alotaibi},
  journal = {Scientific Reports},
  year = {2026},
  month = aug,
  doi = {10.1038/s41598-026-67845-8},
  url = {https://doi.org/10.1038/s41598-026-67845-8},
  note = {CC BY},
}

RIS

TY  - JOUR
TI  - Enhancing self-supervised representation learning through lightweight learnable data augmentation
AU  - Mubarakah Alotaibi
PY  - 2026
DA  - 2026/08/01
JO  - Scientific Reports
DO  - 10.1038/s41598-026-67845-8
UR  - https://doi.org/10.1038/s41598-026-67845-8
AB  - Data augmentation plays a central role in self-supervised learning, as the quality and diversity of augmented views strongly influence the learned representations. However, most existing self-supervised methods rely on fixed stochastic augmentation pipelines, while more adaptive alternatives often require expensive policy search, adversarial training, or additional optimization procedures. In this paper, we propose a lightweight learnable augmentation framework based on Extreme Learning Machines (ELM) for self-supervised visual representation learning. The proposed module predicts image-dependent transformation parameters and applies them through a differentiable augmentation operator, enabling joint optimization with the representation model while introducing minimal additional computational overhead. The framework is integrated into three representative self-supervised learning methods: SimCLR, BYOL, and SimSiam. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet show that the proposed method consistently improves linear evaluation performance relative to reproduced baselines across most settings. In particular, the method yields notable gains on CIFAR datasets and remains effective on the more challenging Tiny ImageNet benchmark. A per-class difficulty analysis further shows that the proposed augmentation strategy substantially improves performance on hard classes, indicating stronger robustness to challenging categories while maintaining competitive overall performance. In general, the results demonstrate that lightweight learnable augmentation can effectively enhance self-supervised representation learning across different frameworks and datasets.
SN  - 2045-2322
ER  -

Abstract

Related

Cite