EvSpark: Lossless Speculative Decoding for Hybrid DNA Foundation Models
- 1CreatiPhage Biotechnology Co., Ltd.
Abstract
Hybrid DNA foundation models combine convolutional, recurrent, and attention layers, making speculative decoding more difficult than truncating a KV cache. We present EvSpark, a speculative decoding system for Evo2 that verifies draft blocks in parallel and restores all three classes of inference state by selecting retained intermediate states, without replay. A compact, hidden-state-conditioned drafter proposes each block in one parallel forward pass. On Evo2 7B, a 48-prompt benchmark with three training seeds yields a 2.96-fold speedup on 43 real-sequence prompts and a 3.27-fold speedup including five synthetic controls. Acceleration persists at 262k-token context (1.84-2.43-fold speedup on two bacterial genomes) and over 32k generated tokens. Retraining the same drafter architecture for Evo2 20B and 40B yields 2.18-2.46-fold speedups on real sequences and 2.51-2.78-fold speedups on the full suite. Autoregressive drafter comparisons and batch measurements show why low draft latency, rather than acceptance alone, determines the gain. The method preserves the target distribution. Greedy checks find no non-tie divergences across 48 prompts and six checkpoints; complementary sampling diagnostics quantify decoding fidelity. A drafter trained on 30M supervised positions reaches a 2.82-fold speedup on real sequences, compared with a 2.96-fold speedup at 150M positions. In regulatory-DNA design, EvSpark achieves a median 1.57-fold complete-workflow speedup over a calibrated batched native baseline.
Related
- Lossless Reductions and Hybrid Sampling for Phylogeny Using the Maximum Compatibility Criterion
- Limitations of Genomic Foundation Models for Decoding Regulatory Mechanisms in ALS
- Scaling subjects in cross-modal alignment: video decoding with EEG foundation model
- Decoding inflammation to target hybrid diabetes phenotypes
- LIMBER: A Hybrid RNA Foundation Model for Predicting and Understanding RNA Structure and Function
BibTeX
@misc{ding2026evspark,
title = {{EvSpark: Lossless Speculative Decoding for Hybrid DNA Foundation Models}},
author = {H. Ding and N. Wu and T. Qiu},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.02.749017},
url = {https://www.biorxiv.org/content/10.64898/2026.09.02.749017v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - EvSpark: Lossless Speculative Decoding for Hybrid DNA Foundation Models
AU - H. Ding
AU - N. Wu
AU - T. Qiu
PY - 2026
DA - 2026/09/02
PB - bioRxiv
DO - 10.64898/2026.09.02.749017
UR - https://www.biorxiv.org/content/10.64898/2026.09.02.749017v1
AB - Hybrid DNA foundation models combine convolutional, recurrent, and attention layers, making speculative decoding more difficult than truncating a KV cache. We present EvSpark, a speculative decoding system for Evo2 that verifies draft blocks in parallel and restores all three classes of inference state by selecting retained intermediate states, without replay. A compact, hidden-state-conditioned drafter proposes each block in one parallel forward pass. On Evo2 7B, a 48-prompt benchmark with three training seeds yields a 2.96-fold speedup on 43 real-sequence prompts and a 3.27-fold speedup including five synthetic controls. Acceleration persists at 262k-token context (1.84-2.43-fold speedup on two bacterial genomes) and over 32k generated tokens. Retraining the same drafter architecture for Evo2 20B and 40B yields 2.18-2.46-fold speedups on real sequences and 2.51-2.78-fold speedups on the full suite. Autoregressive drafter comparisons and batch measurements show why low draft latency, rather than acceptance alone, determines the gain. The method preserves the target distribution. Greedy checks find no non-tie divergences across 48 prompts and six checkpoints; complementary sampling diagnostics quantify decoding fidelity. A drafter trained on 30M supervised positions reaches a 2.82-fold speedup on real sequences, compared with a 2.96-fold speedup at 150M positions. In regulatory-DNA design, EvSpark achieves a median 1.57-fold complete-workflow speedup over a calibrated batched native baseline.
ER -