The Role of Working Memory in Reward-based Learning over Long Timeframes: Opponent Bootstrapping and Interference Effects of Working Memory on Incremental Learning
Abstract
Learning from feedback is essential for effective decision-making in daily life. Previous experiments have indicated that feedback-based learning involves multiple processes, such as slower reinforcement learning (RL) and fast working memory (WM). However, most of these studies focused on the dynamics of RL and WM in short timeframes, ranging from a few seconds to minutes, whereas real-life learning may occur over hours, days, or weeks. This study aimed to investigate the interplay between RL and WM in long-term feedback-based learning. Instead of learning back-to-back iterations in quick succession, participants learned one or two iterations per day for 24 days. In this scenario, initial trials each day should isolate non WM decision processes, and WM should primarily assist in subsequent trials, and in low load conditions, for example when a given item was repeated in short succession. There were four key findings. First, WM contributed broadly to learning in trials 2-n of each day, especially in low load conditions, while trial 1 isolated non WM contributions. Second, when WM was available, it produced both immediate and long-term benefits. WM steered choices toward correct responses, allowing more immediate rewards, but also supporting the training of correct stimulus-action associations over time (a bootstrapping effect). Third, both test-phase performance and computational modeling revealed that using WM had an additional deleterious interference effect; trials during which WM was available produced slower RL updating and worse test-phase performance. Fourth, the long-term training benefits of WM overshadowed its interference effect, such that the overall effect of WM on long-term learning remained positive.
Related
- Clinical-Anchored Incremental Survival Learning: A Stability-Constrained Framework for Molecularly Enhanced Survival Prediction in Lung Adenocarcinoma
- Human Category Learning in Reversal Tasks: Dynamic Learning Rates Help Overcome Catastrophic Interference in Connectionist Networks
- Beyond Successful Play: Diagnosing Opponent Modeling in Large Language Models
- Related memories exhibit interference during learning in adults but later retrieval in children
- ACOR: incremental evidence accumulation towards real-time DNA storage reconstruction
BibTeX
@misc{zhang2026role,
title = {{The Role of Working Memory in Reward-based Learning over Long Timeframes: Opponent Bootstrapping and Interference Effects of Working Memory on Incremental Learning}},
author = {Yiwen Zhang and Xitong Chen and Benjamin Rottman and Anne Collins},
howpublished = {PsyArXiv},
year = {2026},
month = oct,
doi = {10.31234/osf.io/vb9u6_v2},
url = {https://osf.io/preprints/psyarxiv/vb9u6_v2/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The Role of Working Memory in Reward-based Learning over Long Timeframes: Opponent Bootstrapping and Interference Effects of Working Memory on Incremental Learning
AU - Yiwen Zhang
AU - Xitong Chen
AU - Benjamin Rottman
AU - Anne Collins
PY - 2026
DA - 2026/10/21
PB - PsyArXiv
DO - 10.31234/osf.io/vb9u6_v2
UR - https://osf.io/preprints/psyarxiv/vb9u6_v2/
AB - Learning from feedback is essential for effective decision-making in daily life. Previous experiments have indicated that feedback-based learning involves multiple processes, such as slower reinforcement learning (RL) and fast working memory (WM). However, most of these studies focused on the dynamics of RL and WM in short timeframes, ranging from a few seconds to minutes, whereas real-life learning may occur over hours, days, or weeks. This study aimed to investigate the interplay between RL and WM in long-term feedback-based learning. Instead of learning back-to-back iterations in quick succession, participants learned one or two iterations per day for 24 days. In this scenario, initial trials each day should isolate non WM decision processes, and WM should primarily assist in subsequent trials, and in low load conditions, for example when a given item was repeated in short succession. There were four key findings. First, WM contributed broadly to learning in trials 2-n of each day, especially in low load conditions, while trial 1 isolated non WM contributions. Second, when WM was available, it produced both immediate and long-term benefits. WM steered choices toward correct responses, allowing more immediate rewards, but also supporting the training of correct stimulus-action associations over time (a bootstrapping effect). Third, both test-phase performance and computational modeling revealed that using WM had an additional deleterious interference effect; trials during which WM was available produced slower RL updating and worse test-phase performance. Fourth, the long-term training benefits of WM overshadowed its interference effect, such that the overall effect of WM on long-term learning remained positive.
ER -