Learning to Plan from Random Exploration
Abstract
Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without policy-improvement training? Our random-walk analysis explains what temporal relations contain: short horizons reveal geodesic geometry in the diffusion limit, while longer horizons reveal connectivity between regions before mixing removes these distinctions. We learn these relations with a conditional energy-based model that estimates temporal log-density ratios through horizon-conditioned embeddings. The model is trained on observation pairs by noise-contrastive estimation, without action or reward labels. The planner queries these learned relations at different horizons as it moves toward the goal. At test time, a separate local dynamics model predicts candidate action outcomes, and the temporal model evaluates their progress toward the goal by selecting or aggregating estimated improvements across horizons. The agent executes one action and replans with both models fixed. Experiments demonstrate long-range maze planning from random exploration using states and images. Learned score fields, embedding probes, and planned routes exhibit properties of a multiscale cognitive map. We further demonstrate egocentric navigation from random exploration and manipulation planning from suboptimal data.
Related
- Deep learning methods for stochastic Galerkin approximations of random domain problems
- Robust Iterative Learning Control for Multi-Agent Systems with Random Impulsive Disturbances
- Physics-informed random neural architectures on compact boundaryless multiply connected manifolds for random-operator models in machine learning and uncertainty quantification
- Learning Submanifolds for Subsequent Inference on Random Dot Product Graphs, Part 1: Theory
- A Quantum-Inspired Dequantization Method for Diagonally Weighted Matrix Functions: Application to Learning with Optimized Random Features
BibTeX
@misc{kong2026learning,
title = {{Learning to Plan from Random Exploration}},
author = {Deqian Kong and Guangyan Sun and Sheng Cheng and Sirui Xie and Bo Pang and Jianwen Xie and Tony Geng and Caiwen Ding and Ying Nian Wu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38383},
eprint = {2609.38383},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38383},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Learning to Plan from Random Exploration
AU - Deqian Kong
AU - Guangyan Sun
AU - Sheng Cheng
AU - Sirui Xie
AU - Bo Pang
AU - Jianwen Xie
AU - Tony Geng
AU - Caiwen Ding
AU - Ying Nian Wu
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.38383
UR - https://arxiv.org/abs/2609.38383
AB - Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without policy-improvement training? Our random-walk analysis explains what temporal relations contain: short horizons reveal geodesic geometry in the diffusion limit, while longer horizons reveal connectivity between regions before mixing removes these distinctions. We learn these relations with a conditional energy-based model that estimates temporal log-density ratios through horizon-conditioned embeddings. The model is trained on observation pairs by noise-contrastive estimation, without action or reward labels. The planner queries these learned relations at different horizons as it moves toward the goal. At test time, a separate local dynamics model predicts candidate action outcomes, and the temporal model evaluates their progress toward the goal by selecting or aggregating estimated improvements across horizons. The agent executes one action and replans with both models fixed. Experiments demonstrate long-range maze planning from random exploration using states and images. Learned score fields, embedding probes, and planned routes exhibit properties of a multiscale cognitive map. We further demonstrate egocentric navigation from random exploration and manipulation planning from suboptimal data.
ER -