Echoes of Thought: Task Complexity, Inner Speech Analogs, and Prompt Design Shape LLMs’ Response Behavior in the Two-Step Task
Abstract
Large Language Models (LLMs) have shown impressive performance across a wide range of tasks, yet the mechanisms behind their response behavior remain poorly understood. Drawing on insights from cognitive science and the computational framework of model-free and model-based reinforcement learning (RL), we examined conditions under which GPT-3.5 and four open-weight LLMs can exhibit behavioral signatures of model-based RL in the two-step task, a well-established paradigm for studying human learning and decision-making. Across nine simulation studies, we varied the task complexity and prompt design to examine conditions under which previously reported model-based RL signatures could be replicated. We found that GPT-3.5 showed model-based RL signatures in simplified task settings, but that these effects disappeared under de-simplified task versions. Crucially, model-based reasoning signatures could be re-established when incorporating an inner speech-analog prompt, requiring the LLM agent to reflect on previous trials and anticipate consequences of future actions. For the open-weight LLMs tested, we observed diminished signatures of model-based reasoning under both de-simplified and facilitative task conditions. Our fine-grained analyses of behavioral learning dynamics highlight the role of structured reflective and anticipatory linguistic processes in enabling complex response patterns in LLMs and reveal both similarities to and differences from human decision-making patterns.
Related
- Functional response, numerical response, mutual interference and switching behavior of the predatory mite Euseius scutalis (Acari: Phytoseiidae) fed on Bemisia tabaci (Hemiptera: Aleyrodidae)
- Cultural echoes in social bonding: Universal and culture-specific effects of movement synchrony and music on connectedness, likeability, and well-being
- Echoes of Evolution in Holocentridae : Harmony Between Phylogeny, Morphology and Acoustics
- Weathering the Ballot Box: Climate Disasters Response and Electoral Behavior in Colombia
- Sustained Activation of a Defensive Behavior-Related Circuit Elicits a distinct Homeostatic Sleep Response
BibTeX
@misc{usee2026echoes,
title = {{Echoes of Thought: Task Complexity, Inner Speech Analogs, and Prompt Design Shape LLMs’ Response Behavior in the Two-Step Task}},
author = {Franziska Usée and Daniel J. Schad},
howpublished = {PsyArXiv},
year = {2026},
month = oct,
doi = {10.31234/osf.io/mgq9r_v7},
url = {https://osf.io/preprints/psyarxiv/mgq9r_v7/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Echoes of Thought: Task Complexity, Inner Speech Analogs, and Prompt Design Shape LLMs’ Response Behavior in the Two-Step Task
AU - Franziska Usée
AU - Daniel J. Schad
PY - 2026
DA - 2026/10/06
PB - PsyArXiv
DO - 10.31234/osf.io/mgq9r_v7
UR - https://osf.io/preprints/psyarxiv/mgq9r_v7/
AB - Large Language Models (LLMs) have shown impressive performance across a wide range of tasks, yet the mechanisms behind their response behavior remain poorly understood. Drawing on insights from cognitive science and the computational framework of model-free and model-based reinforcement learning (RL), we examined conditions under which GPT-3.5 and four open-weight LLMs can exhibit behavioral signatures of model-based RL in the two-step task, a well-established paradigm for studying human learning and decision-making. Across nine simulation studies, we varied the task complexity and prompt design to examine conditions under which previously reported model-based RL signatures could be replicated. We found that GPT-3.5 showed model-based RL signatures in simplified task settings, but that these effects disappeared under de-simplified task versions. Crucially, model-based reasoning signatures could be re-established when incorporating an inner speech-analog prompt, requiring the LLM agent to reflect on previous trials and anticipate consequences of future actions. For the open-weight LLMs tested, we observed diminished signatures of model-based reasoning under both de-simplified and facilitative task conditions. Our fine-grained analyses of behavioral learning dynamics highlight the role of structured reflective and anticipatory linguistic processes in enabling complex response patterns in LLMs and reveal both similarities to and differences from human decision-making patterns.
ER -