The Price of Scale in Proactive Memory Recall
Abstract
A long-lived assistant that brings up memories without being asked must decide at every turn whether anything in its store is worth an interruption. We show that this decision is a multiple-testing problem whose difficulty grows with the store size N. On streaming replays of LongMemEval, a similarity threshold tuned to 5% false alarms at N=8 reaches 84-100% at N=2048 across embedding models and cue types, and LoCoMo and LLM judges show it too. We prove that a gate which thresholds the best of N independent irrelevant scores at false-alarm rate alpha operates at a per-item false-positive rate of about -ln(1-alpha)/N, which pins its hit rate to the far left of the single-comparison ROC curve and, under regular variation, makes it regularly varying in N. A finite-pool form of this bound predicts the hit rate of a label-tuned threshold on held-out questions to within 6.1 points. At matched false alarms the deployed score recalls 14-26 points more than raw cosine. A cue-swap conformal gate, which ranks the current cue among other cues on the same store, has a finite-sample false-alarm guarantee under exchangeability. With a background drawn from the user's unfiltered history it keeps false alarms at 3-6% on LongMemEval at the hit rate of a label-tuned threshold; on LoCoMo it overshoots or turns conservative depending on where its reference cues come from. In a chronological replay with the user's own earlier messages as references, it speaks on 6-8% of session-opening turns, against 57% for the deployed gate at N=2048, at about 50 ms per decision on one CPU thread. A cross-encoder reranker pays the same price of scale, and on a prospective-memory benchmark, similarity gates recall at most 9% of constraints at controlled false alarms. One human and two LLM annotators judge most false alarms unhelpful for a response. Code and data: https://github.com/RiverChenStillCompiling/price-of-scale
Related
- Computing Multi-Slot First-Price Pacing Equilibria at Scale via Entropic Smoothing
- Spatial Aggregation of ROC and Precision-Recall Curves
- Proactive Incentive Regulation in Multi-Agent Systems with Environmental Feedback
- Optimal Dividends and Proactive Capital Injection over a Finite Horizon
- Partially Observed Mean Field Games Without Perfect Recall: Optimality Conditions and Equilibria
BibTeX
@misc{chen2026price,
title = {{The Price of Scale in Proactive Memory Recall}},
author = {Yichuan Chen and Xin Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.34507},
eprint = {2610.34507},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.34507},
}
RIS
TY - UNPB
TI - The Price of Scale in Proactive Memory Recall
AU - Yichuan Chen
AU - Xin Wang
PY - 2026
DA - 2026/10/29
PB - arXiv
DO - 10.48550/arxiv.2610.34507
UR - https://arxiv.org/abs/2610.34507
AB - A long-lived assistant that brings up memories without being asked must decide at every turn whether anything in its store is worth an interruption. We show that this decision is a multiple-testing problem whose difficulty grows with the store size N. On streaming replays of LongMemEval, a similarity threshold tuned to 5% false alarms at N=8 reaches 84-100% at N=2048 across embedding models and cue types, and LoCoMo and LLM judges show it too. We prove that a gate which thresholds the best of N independent irrelevant scores at false-alarm rate alpha operates at a per-item false-positive rate of about -ln(1-alpha)/N, which pins its hit rate to the far left of the single-comparison ROC curve and, under regular variation, makes it regularly varying in N. A finite-pool form of this bound predicts the hit rate of a label-tuned threshold on held-out questions to within 6.1 points. At matched false alarms the deployed score recalls 14-26 points more than raw cosine. A cue-swap conformal gate, which ranks the current cue among other cues on the same store, has a finite-sample false-alarm guarantee under exchangeability. With a background drawn from the user's unfiltered history it keeps false alarms at 3-6% on LongMemEval at the hit rate of a label-tuned threshold; on LoCoMo it overshoots or turns conservative depending on where its reference cues come from. In a chronological replay with the user's own earlier messages as references, it speaks on 6-8% of session-opening turns, against 57% for the deployed gate at N=2048, at about 50 ms per decision on one CPU thread. A cross-encoder reranker pays the same price of scale, and on a prospective-memory benchmark, similarity gates recall at most 9% of constraints at controlled false alarms. One human and two LLM annotators judge most false alarms unhelpful for a response. Code and data: https://github.com/RiverChenStillCompiling/price-of-scale
ER -