Beyond Class Marginals: Bounding Rehearsal Gaps without Freezing Class Co-occurrence
Abstract
Class-balanced replay controls class frequency but does not determine the interval between successive replay appearances of a class. We study this interval, the rehearsal gap, separately from the class marginal and class co-occurrence, and introduce randomised-pass replay (RPR), which visits each resident class once per shuffled pass. For a fixed set of C resident classes and replay batch size b less than or equal to C, RPR preserves the balanced time-averaged class marginal and bounds every gap by 2*ceil(C/b)-1; a churn-conditional bound applies while the resident set changes. The scheduler uses no future class information and adds no replay examples or forward passes. In a linear-head ER-ACE diagnostic, joint absence from the incoming and replay batches produces a one-sided classifier-bias gradient. Longer absence episodes are associated with larger negative bias displacement, and removing the incoming-loss mask attenuates the scheduling effect. In the primary ER-ACE experiments, RPR improves final average accuracy by 0.72-1.67 percentage points relative to independent class-balanced retrieval under reservoir storage, with positive effects also observed under balanced storage. Pretrained ViTs show positive effects on the tested LT10 streams with small replay batches, while matched larger-batch controls show no material effect. Fixed-cycle and reused-pass controls change more than one temporal statistic, so the experiments do not isolate rehearsal-gap length from all other forms of temporal dependence. The accuracy effects depend on the learner and operating regime.
Related
- Bounding MrP: Subnational public opinion estimation without detailed census data
- Revisiting the Bobkov-Chistyakov-Götze theorem on approximately Gaussian marginals
- How far are d-dimensional copulas with uniform (d−1)-marginals from (total) independence?
- Proper Agnostic Learning of Affine Halfspaces under Log-Concave Marginals
- Gaussian Flow Dynamics: Simulation-Free Neural SDE Learning Beyond One-Time Marginals
BibTeX
@misc{dai2026beyond,
title = {{Beyond Class Marginals: Bounding Rehearsal Gaps without Freezing Class Co-occurrence}},
author = {Congren Dai and Nat Roongjirarat and Fei Ye},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.25735},
eprint = {2609.25735},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.25735},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Beyond Class Marginals: Bounding Rehearsal Gaps without Freezing Class Co-occurrence
AU - Congren Dai
AU - Nat Roongjirarat
AU - Fei Ye
PY - 2026
DA - 2026/09/22
PB - arXiv
DO - 10.48550/arxiv.2609.25735
UR - https://arxiv.org/abs/2609.25735
AB - Class-balanced replay controls class frequency but does not determine the interval between successive replay appearances of a class. We study this interval, the rehearsal gap, separately from the class marginal and class co-occurrence, and introduce randomised-pass replay (RPR), which visits each resident class once per shuffled pass. For a fixed set of C resident classes and replay batch size b less than or equal to C, RPR preserves the balanced time-averaged class marginal and bounds every gap by 2*ceil(C/b)-1; a churn-conditional bound applies while the resident set changes. The scheduler uses no future class information and adds no replay examples or forward passes. In a linear-head ER-ACE diagnostic, joint absence from the incoming and replay batches produces a one-sided classifier-bias gradient. Longer absence episodes are associated with larger negative bias displacement, and removing the incoming-loss mask attenuates the scheduling effect. In the primary ER-ACE experiments, RPR improves final average accuracy by 0.72-1.67 percentage points relative to independent class-balanced retrieval under reservoir storage, with positive effects also observed under balanced storage. Pretrained ViTs show positive effects on the tested LT10 streams with small replay batches, while matched larger-batch controls show no material effect. Fixed-cycle and reused-pass controls change more than one temporal statistic, so the experiments do not isolate rehearsal-gap length from all other forms of temporal dependence. The accuracy effects depend on the learner and operating regime.
ER -