LOAM: A family of genomic language models trained on long-read soil metagenomes
- 1Soilytix GmbH
Abstract
Soil ecosystems represent a vast, largely uncharacterised reservoir of microbial diversity. Metagenomic assembly has unlocked access to this resource, and recent advances in long-read sequencing have improved the recovery and quality of microbial genomes from complex samples. While genomic language models have proven highly effective at capturing biological concepts from large sequence datasets, they have predominantly been trained on reference genomes or short-read assemblies. Here, we present LOAM, a family of decoder-only genomic language models ranging from 25 to 624 million parameters and trained exclusively on Oxford Nanopore long-read environmental metagenomes. LOAM model performance scales predictably with model size and training-token budget. Despite a relatively small training sequence corpus, LOAM models outperformed comparably sized models across biological benchmarks, and achieved performance competitive with substantially larger state-of-the-art models trained on much larger datasets. Context-intervention experiments further showed that LOAM models use genomic information over several kilobases, highlighting the potential value of increased contiguity provided by long-read metagenome-assembled genomes. For probe-based benchmark tasks, we systematically evaluated representations across hidden layers and revealed that task-relevant biological information was frequently more linearly accessible from intermediate than final model layers. Finally, we observed that variation in zero-shot variant-effect prediction was strongly associated with the presence of homologous target sequences in the pre-training corpus. Together, these results establish long-read environmental metagenomes as a viable foundation for training competitive genomic language models and demonstrate the importance of both model scale and training-corpus composition for biological generalisation.
Related
- The impact of strain heterogeneity on genomic profiling accuracy in mixed-strain Escherichia coli metagenomes
- Word-Class and Construction-Like Structure Emerges in Neural Successor Representations Trained on Natural Language
- A pilot computer-vision model trained to identify slide-mounted scale insect pests from the family Rhizoecidae
- Machine‐Learning Models Trained on Genomic Summary Statistics Assess Extinction Risk and Recovery Potential of Bottlenecked Populations
- RadMultiBench: A Reproducible Dual-Task Benchmark Exposing the Limits of Pre-Trained Vision-Language Models Across Thoracic and Breast Imaging
BibTeX
@misc{ferry2026loam,
title = {{LOAM: A family of genomic language models trained on long-read soil metagenomes}},
author = {Q. R. Ferry and M. Frank and J. Jehn and M. Schaks and B. R. Steinkraus and T. Rajakumar},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.28.755120},
url = {https://www.biorxiv.org/content/10.64898/2026.09.28.755120v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - LOAM: A family of genomic language models trained on long-read soil metagenomes
AU - Q. R. Ferry
AU - M. Frank
AU - J. Jehn
AU - M. Schaks
AU - B. R. Steinkraus
AU - T. Rajakumar
PY - 2026
DA - 2026/09/28
PB - bioRxiv
DO - 10.64898/2026.09.28.755120
UR - https://www.biorxiv.org/content/10.64898/2026.09.28.755120v1
AB - Soil ecosystems represent a vast, largely uncharacterised reservoir of microbial diversity. Metagenomic assembly has unlocked access to this resource, and recent advances in long-read sequencing have improved the recovery and quality of microbial genomes from complex samples. While genomic language models have proven highly effective at capturing biological concepts from large sequence datasets, they have predominantly been trained on reference genomes or short-read assemblies. Here, we present LOAM, a family of decoder-only genomic language models ranging from 25 to 624 million parameters and trained exclusively on Oxford Nanopore long-read environmental metagenomes. LOAM model performance scales predictably with model size and training-token budget. Despite a relatively small training sequence corpus, LOAM models outperformed comparably sized models across biological benchmarks, and achieved performance competitive with substantially larger state-of-the-art models trained on much larger datasets. Context-intervention experiments further showed that LOAM models use genomic information over several kilobases, highlighting the potential value of increased contiguity provided by long-read metagenome-assembled genomes. For probe-based benchmark tasks, we systematically evaluated representations across hidden layers and revealed that task-relevant biological information was frequently more linearly accessible from intermediate than final model layers. Finally, we observed that variation in zero-shot variant-effect prediction was strongly associated with the presence of homologous target sequences in the pre-training corpus. Together, these results establish long-read environmental metagenomes as a viable foundation for training competitive genomic language models and demonstrate the importance of both model scale and training-corpus composition for biological generalisation.
ER -