Evo 2 as a classification machine: evidence from in-context learning and mechanistic interpretability
- 1European Commission, Joint Research Centre (JRC)
Abstract
In-context learning (iCL) is an emergent capability of Large Language Models (LLMs), allowing them to perform new tasks at inference time using prompt-injected examples. While extensively studied in LLMs, the boundaries of its capabilities and underlying mechanisms remain poorly characterised in genomic Language Models (gLMs). Here, we map the operating regime of Evo 2, a nucleotide-level foundation gLM, across five binary classification tasks spanning biological and artificial sequences. We observe robust iCL on shorter natural sequences (F1=0.902 for miRNA, 0.785 for Toxins), degrading with sequence length, and collapsing at kilobase scale. Strikingly, we find no benefit in model scaling, as the 7B model systematically outperforms the 40B variant. We further find that perplexity, a widely used gLM performance proxy, poorly predicts accuracy. Mechanistic interpretability reconciles these observations: the logit-lens profiling suggests a prediction-generalisation trade-off, while Jacobian Scope indicates models might track prompts' structure rather than the signal-carrying content.
Related
- Effects of within-season temporal aggregation of weather variables on performance and interpretability in rice yield machine learning models
- Mechanistic Interpretability Needs a Logic of Inference
- Mechanistic Interpretability of Protein Language Models Reveals Encoded Structural and Functional Properties of Intrinsically Disordered Proteins
- MarkerScout: A Disease-Agnostic Machine Learning Framework for Biomarker Prediction from Multi-Scale Mechanistic Models
- Integrating Multisource Target Evidence with Machine-Learning Potency Prediction for Alzheimer's Disease Drug Discovery
BibTeX
@misc{agnoletto2026classification,
title = {{Evo 2 as a classification machine: evidence from in-context learning and mechanistic interpretability}},
author = {L. Bertolini Agnoletto and F. Curion and M. Petrillo and G. Leoni and M. Ronco and V. Ruiz Serra and S. Consoli and M. Ceresa},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.30.755330},
url = {https://www.biorxiv.org/content/10.64898/2026.09.30.755330v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Evo 2 as a classification machine: evidence from in-context learning and mechanistic interpretability
AU - L. Bertolini Agnoletto
AU - F. Curion
AU - M. Petrillo
AU - G. Leoni
AU - M. Ronco
AU - V. Ruiz Serra
AU - S. Consoli
AU - M. Ceresa
PY - 2026
DA - 2026/09/30
PB - bioRxiv
DO - 10.64898/2026.09.30.755330
UR - https://www.biorxiv.org/content/10.64898/2026.09.30.755330v1
AB - In-context learning (iCL) is an emergent capability of Large Language Models (LLMs), allowing them to perform new tasks at inference time using prompt-injected examples. While extensively studied in LLMs, the boundaries of its capabilities and underlying mechanisms remain poorly characterised in genomic Language Models (gLMs). Here, we map the operating regime of Evo 2, a nucleotide-level foundation gLM, across five binary classification tasks spanning biological and artificial sequences. We observe robust iCL on shorter natural sequences (F1=0.902 for miRNA, 0.785 for Toxins), degrading with sequence length, and collapsing at kilobase scale. Strikingly, we find no benefit in model scaling, as the 7B model systematically outperforms the 40B variant. We further find that perplexity, a widely used gLM performance proxy, poorly predicts accuracy. Mechanistic interpretability reconciles these observations: the logit-lens profiling suggests a prediction-generalisation trade-off, while Jacobian Scope indicates models might track prompts' structure rather than the signal-carrying content.
ER -