Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra
Abstract
Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be labelled by an expert. To exploit the rest, we pretrain a 1D Vision Transformer with a masked autoencoder on 223,000 unlabelled spectra. Without labels, its representation recovers the emission-line intensity ratios that physicists use to diagnose the precipitating particles (R^2 0.91 vs. 0.77 for an untrained control) and, under one linear probe, classifies as well as 13 features designed by experts. Fine-tuned, the model outperforms the previous supervised auroral classifier on its own benchmark (macro-AP 88.5 vs. 77.8), reaches 0.870 mAP, and exceeds the same architecture trained from scratch by +0.159 with 10% of the labels; attribution shows that it uses both N2+ bands. Could an existing pretrained model replace it? Two astronomical spectral foundation models and a time-series model transfer according to their spectral window: SpectraFM, trained in the infrared, falls below the untrained control, whereas SpecFormer, trained in the optical, approaches in-domain pretraining without reaching it.
Related
- Self-Supervised Representation Learning for JWST Galaxy Images and Low Resolution Spectra
- Data representation matters: Optimising machine learning for X-ray source classification by leveraging spatio-spectral information
- Data Representation Matters: Optimizing Machine Learning for X-ray Source Classification by Leveraging Spatio-Spectral Information
- DINOspec: Efficient Multimodal Alignment of Vision and Spectral Foundation Models for Astronomy
- The NEMESIS general young stellar object catalogue. I. Supervised classification with deep learning methods
BibTeX
@misc{lain2026selfsupervised,
title = {{Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra}},
author = {Matthieu Le Lain and Gaël Cessateur and Sébastien Lefèvre},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.31206},
eprint = {2609.31206},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.31206},
}
RIS
TY - UNPB
TI - Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra
AU - Matthieu Le Lain
AU - Gaël Cessateur
AU - Sébastien Lefèvre
PY - 2026
DA - 2026/09/25
PB - arXiv
DO - 10.48550/arxiv.2609.31206
UR - https://arxiv.org/abs/2609.31206
AB - Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be labelled by an expert. To exploit the rest, we pretrain a 1D Vision Transformer with a masked autoencoder on 223,000 unlabelled spectra. Without labels, its representation recovers the emission-line intensity ratios that physicists use to diagnose the precipitating particles (R^2 0.91 vs. 0.77 for an untrained control) and, under one linear probe, classifies as well as 13 features designed by experts. Fine-tuned, the model outperforms the previous supervised auroral classifier on its own benchmark (macro-AP 88.5 vs. 77.8), reaches 0.870 mAP, and exceeds the same architecture trained from scratch by +0.159 with 10% of the labels; attribution shows that it uses both N2+ bands. Could an existing pretrained model replace it? Two astronomical spectral foundation models and a time-series model transfer according to their spectral window: SpectraFM, trained in the infrared, falls below the untrained control, whereas SpecFormer, trained in the optical, approaches in-domain pretraining without reaching it.
ER -