Multi-Scale Contrastive Attention for Light-Curve Representation Learning
Abstract
Current and next-generation time-domain surveys demand automated techniques capable of analyzing millions of light curves, observed in multiple filters, without relying on exhaustive human annotation or scarce spectroscopic follow-up. We present Astra-CLR, an attention-based, self-supervised contrastive learning framework which enables the representation of raw light curves into a highly discriminative latent space. Pre-trained on ∼2.1 million unlabeled Zwicky Transient Facility light curves, the framework utilizes partial light curves as input sequences to generate asymmetric, multi-scale temporal views (explicitly contrasting shorter sequences against longer ones) forcing the network to learn a robust "local-to-global" mapping strategy. Furthermore, we introduce a novel multi-view late fusion architecture that extends the model to efficiently handle longer light curves with larger numbers of observations while accommodating the different cadences associated with each filter. The discriminatory power of the resulting representations was evaluated by using them as input to a Multinomial Logistic Regression classifier, trained to identify 12 broad classes of variability. Final accuracy achieved ∼ 0.70. When applying a label-efficient, partial top-layer fine-tuning strategy, the topological structure of the latent space is significantly refined, boosting results to ∼0.77. Astra-CLR is the first publicly available multi-filter time-series Transformer trained exclusively on real ZTF light curves. Results presented here demonstrate that it provides an ideal foundation for the development of end-to-end pipelines, taking into account color evolution and respecting the inhomogeneous nature of astronomical light curve sampling.
Related
- Contrastive Astromer: Two-Stage Contrastive Representation Learning for Astronomical Light Curve Classification
- Lyman Break Galaxy selection and redshift measurement with supervised contrastive learning
- Data-driven characterisation of variable source light curves with multimodal representation learning
- Data representation matters: Optimising machine learning for X-ray source classification by leveraging spatio-spectral information
- Self-Supervised Representation Learning for JWST Galaxy Images and Low Resolution Spectra
BibTeX
@misc{majumder2026multiscale,
title = {{Multi-Scale Contrastive Attention for Light-Curve Representation Learning}},
author = {Torsha Majumder and Konstantin Malanchev and Emille E. O. Ishida},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.31627},
eprint = {2606.31627},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.31627},
}
RIS
TY - UNPB
TI - Multi-Scale Contrastive Attention for Light-Curve Representation Learning
AU - Torsha Majumder
AU - Konstantin Malanchev
AU - Emille E. O. Ishida
PY - 2026
DA - 2026/09/18
PB - arXiv
DO - 10.48550/arxiv.2606.31627
UR - https://arxiv.org/abs/2606.31627
AB - Current and next-generation time-domain surveys demand automated techniques capable of analyzing millions of light curves, observed in multiple filters, without relying on exhaustive human annotation or scarce spectroscopic follow-up. We present Astra-CLR, an attention-based, self-supervised contrastive learning framework which enables the representation of raw light curves into a highly discriminative latent space. Pre-trained on ∼2.1 million unlabeled Zwicky Transient Facility light curves, the framework utilizes partial light curves as input sequences to generate asymmetric, multi-scale temporal views (explicitly contrasting shorter sequences against longer ones) forcing the network to learn a robust "local-to-global" mapping strategy. Furthermore, we introduce a novel multi-view late fusion architecture that extends the model to efficiently handle longer light curves with larger numbers of observations while accommodating the different cadences associated with each filter. The discriminatory power of the resulting representations was evaluated by using them as input to a Multinomial Logistic Regression classifier, trained to identify 12 broad classes of variability. Final accuracy achieved ∼ 0.70. When applying a label-efficient, partial top-layer fine-tuning strategy, the topological structure of the latent space is significantly refined, boosting results to ∼0.77. Astra-CLR is the first publicly available multi-filter time-series Transformer trained exclusively on real ZTF light curves. Results presented here demonstrate that it provides an ideal foundation for the development of end-to-end pipelines, taking into account color evolution and respecting the inhomogeneous nature of astronomical light curve sampling.
ER -