Adapting offline models to a streaming context for music source separation
Abstract
Real-time music source separation must satisfy two constraints: a bound on algorithmic latency and a bound on computational cost. Offline separators are usually omitted from real-time comparisons or credited with a latency equal to their full input length. We show that this latency is set by where the output is read, not by the length of the separator's input. An unmodified offline model can therefore run in a streaming setting, without retraining. At each step, the input slides by one STFT hop, and one output hop is read out. The resulting latency can be as low as one STFT hop (23 ms), and the computational cost does not increase as latency shrinks. We identify a theoretical model-dependent latency boundary below which separation quality should drop steeply, and confirm this experimentally across three architectures. At equal algorithmic latency, streamed off-the-shelf checkpoints for HT-Demucs and SCNet match the published results of dedicated real-time models in terms of separation quality. Streamed models remain far less computationally efficient: only HT-Demucs runs faster than real time on a GPU.
Related
- Adapting Linear-Time Architectures for Tabular In-Context Learning
- Tilted Convexity and an Exact Offline-Online Separation
- Adapting to Evolving Requirements: Agentic AI for Retail Supply Chain Operations
- Adapting Pairs Trading to Gambling Markets A Case Study of the U.S. Presidential Election
- Accelerating posterior sampling in Bayesian inverse problems with diffusion models: an online-offline perspective
BibTeX
@misc{sechet2026adapting,
title = {{Adapting offline models to a streaming context for music source separation}},
author = {Dylan Sechet and Marc Evrard and Matthieu Kowalski},
howpublished = {HAL},
year = {2026},
month = sep,
eprint = {2609.35397},
archivePrefix = {arXiv},
url = {https://hal.science/hal-05767849v1},
note = {CC BY-SA 4.0},
}
RIS
TY - UNPB
TI - Adapting offline models to a streaming context for music source separation
AU - Dylan Sechet
AU - Marc Evrard
AU - Matthieu Kowalski
PY - 2026
DA - 2026/09/23
PB - HAL
UR - https://hal.science/hal-05767849v1
AB - Real-time music source separation must satisfy two constraints: a bound on algorithmic latency and a bound on computational cost. Offline separators are usually omitted from real-time comparisons or credited with a latency equal to their full input length. We show that this latency is set by where the output is read, not by the length of the separator's input. An unmodified offline model can therefore run in a streaming setting, without retraining. At each step, the input slides by one STFT hop, and one output hop is read out. The resulting latency can be as low as one STFT hop (23 ms), and the computational cost does not increase as latency shrinks. We identify a theoretical model-dependent latency boundary below which separation quality should drop steeply, and confirm this experimentally across three architectures. At equal algorithmic latency, streamed off-the-shelf checkpoints for HT-Demucs and SCNet match the published results of dedicated real-time models in terms of separation quality. Streamed models remain far less computationally efficient: only HT-Demucs runs faster than real time on a GPU.
ER -