Competitive exploration in structured environments
Abstract
Organisms must constantly weigh exploiting the resources they know against exploring for better ones. When foragers compete, the best behavior for each forager also depends on what its competitors do. How does competition change the strategic balance between exploration and exploitation? Here we address this question in a minimal model: two ε-greedy temporal-difference learners on a one-dimensional lattice, who start at a small reward at one end and are unaware of a larger reward at the other. For a lone forager, the optimal exploration rate increases monotonically with the value of the distant reward, and with its distance up to a critical distance. When two foragers share the lattice and split rewards, the choice of exploration rate becomes a game whose Nash equilibria have both foragers exploring, both staying, or one exploring while the other stays (anti-coordination). Under local best-response dynamics, pairs of foragers often converge on both exploring or both staying, even when anti-coordination would be socially optimal. This social dilemma arises for a wide range of parameters, and it persists even in a simplified model without explicit spatial structure. A minimal, naturalistic signal, in which one forager informs the other that it has found the distant reward, can remove many of the dilemmas in which both foragers explore, and can produce a Nash equilibrium whose collective payoff exceeds that of every outcome available without signaling.
Related
- Joint QTL mapping reveals patterns of pleiotropy across structured environments in budding yeast
- The Spatial Structure of Competitive Interactions: Evidence From Public Procurement Networks in Quebec
- A comparison of female competitive traits: Female aggression peaks at nest building but female song spans multiple contexts in a temperate songbird
- A competitive PDLIM1-USP14 axis destabilizes ACTN4 to disturb the invasive homeostasis of trophoblast in preeclampsia
- Reaching agreement in competitive microbial systems
BibTeX
@misc{wolfe2026competitive,
title = {{Competitive exploration in structured environments}},
author = {James P. Wolfe and Joshua B. Plotkin},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.34643},
eprint = {2610.34643},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.34643},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Competitive exploration in structured environments
AU - James P. Wolfe
AU - Joshua B. Plotkin
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.34643
UR - https://arxiv.org/abs/2610.34643
AB - Organisms must constantly weigh exploiting the resources they know against exploring for better ones. When foragers compete, the best behavior for each forager also depends on what its competitors do. How does competition change the strategic balance between exploration and exploitation? Here we address this question in a minimal model: two ε-greedy temporal-difference learners on a one-dimensional lattice, who start at a small reward at one end and are unaware of a larger reward at the other. For a lone forager, the optimal exploration rate increases monotonically with the value of the distant reward, and with its distance up to a critical distance. When two foragers share the lattice and split rewards, the choice of exploration rate becomes a game whose Nash equilibria have both foragers exploring, both staying, or one exploring while the other stays (anti-coordination). Under local best-response dynamics, pairs of foragers often converge on both exploring or both staying, even when anti-coordination would be socially optimal. This social dilemma arises for a wide range of parameters, and it persists even in a simplified model without explicit spatial structure. A minimal, naturalistic signal, in which one forager informs the other that it has found the distant reward, can remove many of the dilemmas in which both foragers explore, and can produce a Nash equilibrium whose collective payoff exceeds that of every outcome available without signaling.
ER -