Skip to content
DraconiticResearch
Search Sources API
190,335 studies31 sourcesUpdated 54 min ago
PreprintarXiv licencearXiv

The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts

  • Aidar Amankulov
  • Denis Mamatin
Open at sourcePDFDOICite

Abstract

Speculative decoding in Mixture-of-Experts (MoE) models faces the problem of unstable verification cost caused by input-dependent expert loading. To study the physics of this process, we formulate speculation-budget selection as an offline Stochastic Shortest Path (SSP) problem over reference sequences and build a diagnostic Oracle that uses counterfactual simulation to account for MoE verification cost. A detailed analysis of the Oracle's decisions on the Qwen3-Coder and EAGLE-3 pairing, in the space of marginal deltas (Delta Space), shows that rejected candidates form a strict linear boundary. This result demonstrates that a complex global optimization is locally governed by a necessary condition balancing marginal cost against expected progress (Δ𝔼[Cost]/Δ𝔼[a]), providing a rigorous mathematical reference point for designing future adaptive online heuristics.

Related

  1. Distributed LLM Inference via Communication-Efficient Speculative Decoding
    Joohyun Shin, Byeonghyeon Park, Keumseo Ryum +1
    arXivPreprintarXiv licenceFull textGeneral mathematicsarXiv
  2. Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds
    Yunxiao Zhao, Changxiao Cai
    arXivPreprintCC BY 4.0Full textStatisticsarXiv
  3. Mixture of Censored Partially Linear Experts Models for Symmetric or Asymmetric Data with or without Outlying Observations: A novel approach with two-piece scale-mixture of normal distributions
    Mohsen Maleki, Geoffrey J. McLachlan
    arXivPreprintCC BY 4.0Full textStatisticsarXiv
  4. Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts
    TrungTin Nguyen, Hien D. Nguyen, Faïcel Chamroukhi +1
    Annals of the Institute of Statistical MathematicsPeer-reviewedCC BYFull text7 citedStatisticsOpenAlex
  5. A Mixture-of-Experts Framework for Multi-Stream Quickest Change Detection
    Wuxia Chen, Arman Adibi, Taposh Banerjee +1
    arXivPreprintCC BY 4.0Full textStatisticsarXiv

Cite

BibTeXRIS
BibTeX
@misc{amankulov2026limits,
  title = {{The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts}},
  author = {Aidar Amankulov and Denis Mamatin},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2609.22156},
  eprint = {2609.22156},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.22156},
}
RIS
TY  - UNPB
TI  - The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts
AU  - Aidar Amankulov
AU  - Denis Mamatin
PY  - 2026
DA  - 2026/09/01
PB  - arXiv
DO  - 10.48550/arxiv.2609.22156
UR  - https://arxiv.org/abs/2609.22156
AB  - Speculative decoding in Mixture-of-Experts (MoE) models faces the problem of unstable verification cost caused by input-dependent expert loading. To study the physics of this process, we formulate speculation-budget selection as an offline Stochastic Shortest Path (SSP) problem over reference sequences and build a diagnostic Oracle that uses counterfactual simulation to account for MoE verification cost. A detailed analysis of the Oracle's decisions on the Qwen3-Coder and EAGLE-3 pairing, in the space of marginal deltas (Delta Space), shows that rejected candidates form a strict linear boundary. This result demonstrates that a complex global optimization is locally governed by a necessary condition balancing marginal cost against expected progress (Δ𝔼[Cost]/Δ𝔼[a]), providing a rigorous mathematical reference point for designing future adaptive online heuristics.
ER  -