PathoBERT: A Hybrid Attention-Based Genomic Language Model for Read-Level Bacterial Pathogenicity Prediction
- 1National Research Centre of Egypt
Abstract
Motivation Although recent deep learning models have achieved promising results on read level classification tasks, their robustness to realistic sequencing conditions, sensitivity to read length, and ability to support reliable genome level inference remain incompletely characterized. Here, we present PathoBERT, a hybrid deep learning framework that integrates a LoRA adapted DNABERT encoder with convolutional feature extraction, a modified Convolutional Block Attention Module (MCBAM), and Multiscale Convolutional Attention (MSCA) for bacterial pathogenicity prediction. Results The model demonstrated near complete strand invariance and maintained robust performance under simulated sequencing errors, highlighting its suitability for real world next generation sequencing applications. The model operates on individual sequencing reads and supports genome level inference through a read-aggregation strategy based on the Pathogenic Fraction (PathFrac), which combines read level predictions using a majority vote framework. At the read level, PathoBERT outperformed DeePaC at short and moderate fragment lengths (100 and 150 bp), achieving peak performance at 150 bp. At the genome level, PathoBERT achieved perfect separation between pathogenic and non pathogenic genomes using PathFrac based aggregation, resulting in perfect classification performance on the evaluated benchmark compared with competing approaches, namely DeePaC and PathogenFinder 2. Representation level analyses further demonstrated that a progressive refinement of pathogen-associated features throughout the architecture, culminating in highly separable and biologically meaningful latent representations within the final attention pooled embedding space. These findings demonstrate that integrating contextual genomic language models with attention guided multiscale feature extraction provides a robust framework for pathogenicity prediction from short read sequencing data.
Related
- AlphaGenome informed machine learning for pathogenicity prediction of germline mutations
- Mechanism-based prediction of insertion-driven high pathogenicity avian influenza virus emergence
- MetaSplice: pathogenicity prediction and ACMG/AMP evidence for intronic variants outside the canonical splice sites
- Advancing genetic pathogenicity prediction with three-dimensional proteoform-phenotype analysis
- Maturational constraints on tracking of temporal attention in infant language acquisition.
BibTeX
@misc{elaarag2026pathobert,
title = {{PathoBERT: A Hybrid Attention-Based Genomic Language Model for Read-Level Bacterial Pathogenicity Prediction}},
author = {S. A. El-aarag and M. Flores and M. E. Hasan and A. E. Hemeida and M. Elhefnawi},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.27.754806},
url = {https://www.biorxiv.org/content/10.64898/2026.09.27.754806v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - PathoBERT: A Hybrid Attention-Based Genomic Language Model for Read-Level Bacterial Pathogenicity Prediction
AU - S. A. El-aarag
AU - M. Flores
AU - M. E. Hasan
AU - A. E. Hemeida
AU - M. Elhefnawi
PY - 2026
DA - 2026/09/27
PB - bioRxiv
DO - 10.64898/2026.09.27.754806
UR - https://www.biorxiv.org/content/10.64898/2026.09.27.754806v1
AB - Motivation Although recent deep learning models have achieved promising results on read level classification tasks, their robustness to realistic sequencing conditions, sensitivity to read length, and ability to support reliable genome level inference remain incompletely characterized. Here, we present PathoBERT, a hybrid deep learning framework that integrates a LoRA adapted DNABERT encoder with convolutional feature extraction, a modified Convolutional Block Attention Module (MCBAM), and Multiscale Convolutional Attention (MSCA) for bacterial pathogenicity prediction. Results The model demonstrated near complete strand invariance and maintained robust performance under simulated sequencing errors, highlighting its suitability for real world next generation sequencing applications. The model operates on individual sequencing reads and supports genome level inference through a read-aggregation strategy based on the Pathogenic Fraction (PathFrac), which combines read level predictions using a majority vote framework. At the read level, PathoBERT outperformed DeePaC at short and moderate fragment lengths (100 and 150 bp), achieving peak performance at 150 bp. At the genome level, PathoBERT achieved perfect separation between pathogenic and non pathogenic genomes using PathFrac based aggregation, resulting in perfect classification performance on the evaluated benchmark compared with competing approaches, namely DeePaC and PathogenFinder 2. Representation level analyses further demonstrated that a progressive refinement of pathogen-associated features throughout the architecture, culminating in highly separable and biologically meaningful latent representations within the final attention pooled embedding space. These findings demonstrate that integrating contextual genomic language models with attention guided multiscale feature extraction provides a robust framework for pathogenicity prediction from short read sequencing data.
ER -