Alignment-Aware Genomic Coordinate Encoding for Long-Range DNA Modeling
- 1AI for Life Sciences Lab, Tencent
Abstract
Existing genomic foundation models (GFMs) are adept at capturing local, sequential dependencies in DNA. However, their architectures often struggle to efficiently model gene regulation features, such as sparse motifs and distal regulatory elements that operate beyond linear proximity. Moreover, standard positional encodings fail to fully exploit the reference genome coordinate system. Therefore, we present USM-AA, an alignment-aware genomic foundation model that incorporates reference genome coordinates through a non-contiguous positional encoding, Alignment-Aware Rotary Position Embedding, and an alignment embedding module encoding chromosome, strand, and Compact Idiosyncratic Gapped Alignment Report (CIGAR) information. To improve long-range sequence modeling, USM-AA leverages a reverse-complement masked learning strategy. Pretrained on alignment-aware genomic sequences spanning cross-species evolution and human population variation, USM-AA consistently outperforms existing models across multiple downstream tasks, highlighting the importance of alignment-aware representations for genomic sequence modeling.
Related
- A context-aware temporal modeling through unified multi-scale temporal encoding and hierarchical sequence learning for single-channel EEG sleep staging
- Decoding Alignment without Encoding Alignment: A critique of similarity analysis in neuroscience
- Alignment of visual brain encoding models with individual perceptual similarity judgments
- CLIVeg: Contrastive Image-Vegetation Alignment for Botany-Aware Earth Observation Representations
- Whole-genome alignment of 23 genomes reveals the tempo and genomic determinants of de novo gene birth in the Brassicaceae
BibTeX
@misc{yang2026alignmentaware,
title = {{Alignment-Aware Genomic Coordinate Encoding for Long-Range DNA Modeling}},
author = {X. Yang and C. QIN and F. Wang and B. He and Y. Zhao and J. Wu and Y. Zhang and J. Yao},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.10.757278},
url = {https://www.biorxiv.org/content/10.64898/2026.10.10.757278v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - Alignment-Aware Genomic Coordinate Encoding for Long-Range DNA Modeling
AU - X. Yang
AU - C. QIN
AU - F. Wang
AU - B. He
AU - Y. Zhao
AU - J. Wu
AU - Y. Zhang
AU - J. Yao
PY - 2026
DA - 2026/10/10
PB - bioRxiv
DO - 10.64898/2026.10.10.757278
UR - https://www.biorxiv.org/content/10.64898/2026.10.10.757278v1
AB - Existing genomic foundation models (GFMs) are adept at capturing local, sequential dependencies in DNA. However, their architectures often struggle to efficiently model gene regulation features, such as sparse motifs and distal regulatory elements that operate beyond linear proximity. Moreover, standard positional encodings fail to fully exploit the reference genome coordinate system. Therefore, we present USM-AA, an alignment-aware genomic foundation model that incorporates reference genome coordinates through a non-contiguous positional encoding, Alignment-Aware Rotary Position Embedding, and an alignment embedding module encoding chromosome, strand, and Compact Idiosyncratic Gapped Alignment Report (CIGAR) information. To improve long-range sequence modeling, USM-AA leverages a reverse-complement masked learning strategy. Pretrained on alignment-aware genomic sequences spanning cross-species evolution and human population variation, USM-AA consistently outperforms existing models across multiple downstream tasks, highlighting the importance of alignment-aware representations for genomic sequence modeling.
ER -