Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting
Abstract
Block drafters propose several tokens in one forward pass, before earlier target tokens are realised. Their rejection mixes two losses: missing within-block path information and imperfect modelling of observable information. Accepted length cannot distinguish them. We separate the two with an information floor, the minimum expected rejection at a specified conditioning order; rejection above this floor is the model gap. Measurements across four domains and four open-weight targets, plus floor estimates on a frontier API target, yield three findings. First, the all-parallel floor reaches $0.286$ at the final slot on Qwen3-4B, limiting even the best proposal to 71% per-slot acceptance. Second, one realised token removes $86−−100\%$ of this floor, a locality also recovered by an independent mutual-information analysis. Third, current drafters remain far above their floors: the final-slot model gap accounts for $43−−64\%$ of DFlash rejection and $85−−92\%$ of DSpark's oracle-conditioned rejection. Guided by this separation of short-range conditioning from proposal quality, we replace a context-independent predecessor correction with a prefix-attention head that reads committed context conditional on the predecessor. With supporting backbone components, the resulting Qwen3-4B drafter improves mean serving accepted length by 3.93% over the released DSpark checkpoint across nine tasks, without increasing the conditioning order. Code and data: https://github.com/tie-pilot-qxw/specfloor; drafter checkpoints: https://huggingface.co/TIE-Pilot/dspark-attnconv-block7-qwen3-4b.
Related
- Randomization Beyond Deterministic Adaptivity in Parallel Sampling
- From Common Information to Canonical Correlation Analysis and Beyond: Bridging Information Theory, Signal Processing, and Machine Learning
- The Pauli Lightcone: Information-Theoretic Error Mitigation Beyond the Autocorrelation
- Beyond Second-Order Information: Paid Logistic Regression with Noisy Features
- Thermal Quantum Sensing: Fisher Information and Work Beyond Gaussian Signals
BibTeX
@misc{qiang2026beyond,
title = {{Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting}},
author = {Xinwei Qiang and Xiang Fang and Chang Chen and Zaifeng Pan and Yue Guan and Yufei Ding},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2608.27339},
eprint = {2608.27339},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.27339},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting
AU - Xinwei Qiang
AU - Xiang Fang
AU - Chang Chen
AU - Zaifeng Pan
AU - Yue Guan
AU - Yufei Ding
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2608.27339
UR - https://arxiv.org/abs/2608.27339
AB - Block drafters propose several tokens in one forward pass, before earlier target tokens are realised. Their rejection mixes two losses: missing within-block path information and imperfect modelling of observable information. Accepted length cannot distinguish them. We separate the two with an information floor, the minimum expected rejection at a specified conditioning order; rejection above this floor is the model gap. Measurements across four domains and four open-weight targets, plus floor estimates on a frontier API target, yield three findings. First, the all-parallel floor reaches $0.286$ at the final slot on Qwen3-4B, limiting even the best proposal to 71% per-slot acceptance. Second, one realised token removes $86−−100\%$ of this floor, a locality also recovered by an independent mutual-information analysis. Third, current drafters remain far above their floors: the final-slot model gap accounts for $43−−64\%$ of DFlash rejection and $85−−92\%$ of DSpark's oracle-conditioned rejection. Guided by this separation of short-range conditioning from proposal quality, we replace a context-independent predecessor correction with a prefix-attention head that reads committed context conditional on the predecessor. With supporting backbone components, the resulting Qwen3-4B drafter improves mean serving accepted length by 3.93% over the released DSpark checkpoint across nine tasks, without increasing the conditioning order. Code and data: https://github.com/tie-pilot-qxw/specfloor; drafter checkpoints: https://huggingface.co/TIE-Pilot/dspark-attnconv-block7-qwen3-4b.
ER -