Plan Before You Trade: Inference-Time Optimization for RL Trading Agents
Abstract
Reinforcement learning agents for portfolio management are typically trained and deployed as static policies, with no mechanism for policy adjustments using price forecasts at inference time. We propose FinPILOT (Financial Plugin Inference-time Learning for Optimal Trading), a plugin inference-time policy optimization framework inspired by Model Predictive Control (MPC). Under the price-taker assumption, portfolio actions do not affect future market prices, allowing a multi-step price forecaster to provide imagined market trajectories without iterative action-conditioned rollouts. At each decision step, FinPILOT uses the forecaster's predicted price trajectory to construct an allocation-based imagined return objective and optimizes the pretrained policy at inference time before executing one step of the trade. Our framework is compatible with any pretrained policy and adapts the policy using the forecaster's predictions without modifying the original RL training pipeline. Evaluated with five pretrained policies (PPO, SAC, A2C, TD3, DDPG) across DJ30 equities and foreign exchange (FX) over three rolling held-out test years (30 settings in total), FinPILOT improves Total Return in 28 of 30 settings and Sharpe ratio in 27 of 30, with gains spanning both stochastic and deterministic policies. Direct single- and multi-period portfolio optimizers supplied with the same forecasts do not reproduce these aggregate gains under our evaluation protocol. Further, using controlled forecasts at calibrated quality levels, we show that both the magnitude and consistency of FinPILOT's performance increase as forecast error is reduced.
Related
- Hypergraph cycle enumeration for efficient inference of planted k-XOR with smooth trade-off
- Privacy-accuracy trade-off optimization in sensitive surveys through regression estimation
- Stock Exchange Trading Optimization Algorithm Improved Based on Information Technology Investment Analysis for Numerical Optimization and Practical Problems
- Trade-off invariance for weighted scalarizations in multi-objective optimization
- Discovery of connectivity-trainability trade-off of IQP circuits for Hamiltonian optimization
BibTeX
@misc{go2026plan,
title = {{Plan Before You Trade: Inference-Time Optimization for RL Trading Agents}},
author = {Eun Go and Rohan Deb and Arindam Banerjee},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.12653},
eprint = {2605.12653},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.12653},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Plan Before You Trade: Inference-Time Optimization for RL Trading Agents
AU - Eun Go
AU - Rohan Deb
AU - Arindam Banerjee
PY - 2026
DA - 2026/10/15
PB - arXiv
DO - 10.48550/arxiv.2605.12653
UR - https://arxiv.org/abs/2605.12653
AB - Reinforcement learning agents for portfolio management are typically trained and deployed as static policies, with no mechanism for policy adjustments using price forecasts at inference time. We propose FinPILOT (Financial Plugin Inference-time Learning for Optimal Trading), a plugin inference-time policy optimization framework inspired by Model Predictive Control (MPC). Under the price-taker assumption, portfolio actions do not affect future market prices, allowing a multi-step price forecaster to provide imagined market trajectories without iterative action-conditioned rollouts. At each decision step, FinPILOT uses the forecaster's predicted price trajectory to construct an allocation-based imagined return objective and optimizes the pretrained policy at inference time before executing one step of the trade. Our framework is compatible with any pretrained policy and adapts the policy using the forecaster's predictions without modifying the original RL training pipeline. Evaluated with five pretrained policies (PPO, SAC, A2C, TD3, DDPG) across DJ30 equities and foreign exchange (FX) over three rolling held-out test years (30 settings in total), FinPILOT improves Total Return in 28 of 30 settings and Sharpe ratio in 27 of 30, with gains spanning both stochastic and deterministic policies. Direct single- and multi-period portfolio optimizers supplied with the same forecasts do not reproduce these aggregate gains under our evaluation protocol. Further, using controlled forecasts at calibrated quality levels, we show that both the magnitude and consistency of FinPILOT's performance increase as forecast error is reduced.
ER -