A low-annotation-budget PubMedBERT classifier for chondrogenesis regulator discovery via active learning
- 1University of Liverpool
Abstract
Motivation: Biomedical natural language processing (Bio-NLP) classification tasks are often limited by the cost of manual annotation, especially for specialised extraction problems where labelled corpora are scarce. Active learning can reduce this cost by prioritising informative samples, while large language models (LLMs) avoid task-specific annotation, but at high computational cost. We ask whether an encoder trained via active learning can match LLM performance at a lower computational cost, using identification of chondrogenesis regulators from PubMed abstracts as a test case applicable to other under-annotated Bio-NLP problems. Results: We built an active learning framework to fine-tune PubMedBERT to classify gene mentions as regulators of chondrocyte differentiation, achieving an AUC-ROC of 0.93 using 1,238 annotated sentences. Against open-weight LLMs (Qwen3, Llama-3.1), PubMedBERT matched the top-performing LLMs on AUC-ROC and achieved the highest precision (0.69) of any model, while classifying the full corpus far faster. Applied to 67,111 gene mentions, it identified 1,128 candidate regulators, recovered 79% of Gene Ontology-annotated genes related to chondrocyte differentiation (94 genes recovered), and proposed 981 new candidates. The pipeline is agnostic to the target process, suggesting it could be adapted to identify regulators of other biological processes. Availability and Implementation: Code is available at https://github.com/ChondroTextomics/ALRegulatorDiscovery. Data are available at Zenodo: https://doi.org/10.5281/zenodo.22746090. Final model is available in HuggingFace Hub amav/pubmedbert-chondrogenesis-classifier Supplementary information: Supplementary data are available in a separate file
Related
- Integrating multi-omics with machine learning: from genomic variant functional annotation to pharmacogenomics and biomarker discovery
- From Closed-Set Annotation to In-Context Discovery: Biologically Grounded LLMs for Cellular Profiling
- A formation-response classifier for borehole deviation using conventional well log data
- Enhancing Clinical Classifier Robustness with Deep Representation Learning Based OOD Detection on DNA Methylation Data
- Rigorous nested cross-validation and external generalization-gap analysis of a machine-learning classifier for U-251 glioblastoma cytotoxicity
BibTeX
@misc{anhel2026lowannotationbudget,
title = {{A low-annotation-budget PubMedBERT classifier for chondrogenesis regulator discovery via active learning}},
author = {A.-M. Anhel and M. J. Peffers and D. Young and J. Soul},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.24.754045},
url = {https://www.biorxiv.org/content/10.64898/2026.09.24.754045v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - A low-annotation-budget PubMedBERT classifier for chondrogenesis regulator discovery via active learning
AU - A.-M. Anhel
AU - M. J. Peffers
AU - D. Young
AU - J. Soul
PY - 2026
DA - 2026/09/24
PB - bioRxiv
DO - 10.64898/2026.09.24.754045
UR - https://www.biorxiv.org/content/10.64898/2026.09.24.754045v1
AB - Motivation: Biomedical natural language processing (Bio-NLP) classification tasks are often limited by the cost of manual annotation, especially for specialised extraction problems where labelled corpora are scarce. Active learning can reduce this cost by prioritising informative samples, while large language models (LLMs) avoid task-specific annotation, but at high computational cost. We ask whether an encoder trained via active learning can match LLM performance at a lower computational cost, using identification of chondrogenesis regulators from PubMed abstracts as a test case applicable to other under-annotated Bio-NLP problems. Results: We built an active learning framework to fine-tune PubMedBERT to classify gene mentions as regulators of chondrocyte differentiation, achieving an AUC-ROC of 0.93 using 1,238 annotated sentences. Against open-weight LLMs (Qwen3, Llama-3.1), PubMedBERT matched the top-performing LLMs on AUC-ROC and achieved the highest precision (0.69) of any model, while classifying the full corpus far faster. Applied to 67,111 gene mentions, it identified 1,128 candidate regulators, recovered 79% of Gene Ontology-annotated genes related to chondrocyte differentiation (94 genes recovered), and proposed 981 new candidates. The pipeline is agnostic to the target process, suggesting it could be adapted to identify regulators of other biological processes. Availability and Implementation: Code is available at https://github.com/ChondroTextomics/ALRegulatorDiscovery. Data are available at Zenodo: https://doi.org/10.5281/zenodo.22746090. Final model is available in HuggingFace Hub amav/pubmedbert-chondrogenesis-classifier Supplementary information: Supplementary data are available in a separate file
ER -