STEM-LM: Spatio-Temporal Ecological Modeling via Masked Language Model for Multi-Species Distribution
- 1University of California, Berkeley
Abstract
Species distribution models (SDMs) are central to biodiversity forecasting and conservation decision-making. As ecological datasets grow in size, dimensionality, and resolution, there is a need for flexible yet scalable SDMs tailored to large-scale species observation data. Inspired by the masked modeling approach in natural language processing and genomics, if we treat each species' presence or absence as a token, and a site's species assemblage together with its spatio-temporal and ecological covariates as analogous to a sentence, we can learn joint co-occurrence structure by reconstructing masked species from their neighboring sites. We propose STEM-LM, a Transformer-based SDM that applies masked language modeling to multi-species distribution modeling with explicit consideration of spatio-temporal structure. By applying dynamic masking rates during training, a single trained model can support both purely spatio-temporal/ecological prediction and prediction conditioned on arbitrary subsets of observed species. On a North American butterfly dataset, a Northeast US demersal fish trawl survey, and a global plant distribution dataset, STEM-LM has the highest discrimination (AUROC) among all compared methods, and its rank-monotonicity (continuous Boyce index) is either the best or close to the best.
Related
- Multi‐Scale Habitat Drivers of Hainan Gibbon Distribution: Integrating Macro‐Environmental and Fine‐Scale Ecological Factors for Targeted Conservation of Threatened Species
- Differential thermal sensitivity may explain the temporal distribution of foraging activity among different-sized workers in a polymorphic ant species
- Distribution patterns of Abietinella species under climate change across multiple spatiotemporal scales based on ecological niche models
- Spatio-temporal modeling of an insect vector distribution
- SAGE: A sampling-aware global evaluation benchmark for species distribution modeling
BibTeX
@misc{li2026stemlm,
title = {{STEM-LM: Spatio-Temporal Ecological Modeling via Masked Language Model for Multi-Species Distribution}},
author = {J. K. Li and W. Lim and R. Rheem and F. Callahan and L. Y. Raskin and M. Lemmon-Kishi and R. Nielsen},
howpublished = {bioRxiv},
year = {2026},
month = may,
doi = {10.64898/2026.05.13.724718},
url = {https://www.biorxiv.org/content/10.64898/2026.05.13.724718v2},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - STEM-LM: Spatio-Temporal Ecological Modeling via Masked Language Model for Multi-Species Distribution
AU - J. K. Li
AU - W. Lim
AU - R. Rheem
AU - F. Callahan
AU - L. Y. Raskin
AU - M. Lemmon-Kishi
AU - R. Nielsen
PY - 2026
DA - 2026/05/13
PB - bioRxiv
DO - 10.64898/2026.05.13.724718
UR - https://www.biorxiv.org/content/10.64898/2026.05.13.724718v2
AB - Species distribution models (SDMs) are central to biodiversity forecasting and conservation decision-making. As ecological datasets grow in size, dimensionality, and resolution, there is a need for flexible yet scalable SDMs tailored to large-scale species observation data. Inspired by the masked modeling approach in natural language processing and genomics, if we treat each species' presence or absence as a token, and a site's species assemblage together with its spatio-temporal and ecological covariates as analogous to a sentence, we can learn joint co-occurrence structure by reconstructing masked species from their neighboring sites. We propose STEM-LM, a Transformer-based SDM that applies masked language modeling to multi-species distribution modeling with explicit consideration of spatio-temporal structure. By applying dynamic masking rates during training, a single trained model can support both purely spatio-temporal/ecological prediction and prediction conditioned on arbitrary subsets of observed species. On a North American butterfly dataset, a Northeast US demersal fish trawl survey, and a global plant distribution dataset, STEM-LM has the highest discrimination (AUROC) among all compared methods, and its rank-monotonicity (continuous Boyce index) is either the best or close to the best.
ER -