Online Control via Counterfactual Tracking
Abstract
We study online control of a known linear dynamical system with adversarial costs and bounded disturbances, measuring regret against a general class of benchmark policies. We introduce counterfactual tracking, which separates the challenge of learning from the challenge of controlling the system. An online learner builds a reference trajectory by selecting or averaging the trajectories that the benchmark policies would have generated under the realized costs and disturbances, and a corrective law steers the system toward that reference. Charging each change in the reference its recovery cost (the cost of steering the system onto the new reference) reduces the problem to online learning with switching costs. Conversely, under additional natural assumptions, we show that this reduction is tight: the two problems have the same minimax regret up to a system-dependent factor, uniformly over horizons and policy classes. The reduction gives sharp regret guarantees for policy classes beyond standard finite-memory parameterizations. For a class of N possibly nonlinear or history-dependent policies, it achieves O(√(Tlog N)) regret over T rounds, provided their trajectories remain within a bounded distance of one another and recovery costs are bounded. For the full ℓ1 ball of disturbance-response controllers, it achieves O(√(Tlog T)) regret, which is minimax optimal in T, without assuming a common decay rate for disturbance effects. The framework also improves the best known regret bounds for linear state-feedback policies.
Related
- Tracking a Small Set of Experts in Online Convex Optimization
- Graph-Regularized Synthetic Control for Stable Counterfactual Estimation under Donor Redundancy
- Event-Triggered Funnel Control for Output Reference Tracking of Unknown Nonlinear Systems
- A Monolithic Discontinuous Galerkin Framework for Darcy Optimal Control with Radon-Measure Tracking and Pointwise Control Constraints
- Finite element error estimates for a bilinear optimal control problem with pointwise tracking governed by a semilinear elliptic PDE
BibTeX
@misc{xu2026online,
title = {{Online Control via Counterfactual Tracking}},
author = {Yunzong Xu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2607.13029},
eprint = {2607.13029},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.13029},
}
RIS
TY - UNPB
TI - Online Control via Counterfactual Tracking
AU - Yunzong Xu
PY - 2026
DA - 2026/10/02
PB - arXiv
DO - 10.48550/arxiv.2607.13029
UR - https://arxiv.org/abs/2607.13029
AB - We study online control of a known linear dynamical system with adversarial costs and bounded disturbances, measuring regret against a general class of benchmark policies. We introduce counterfactual tracking, which separates the challenge of learning from the challenge of controlling the system. An online learner builds a reference trajectory by selecting or averaging the trajectories that the benchmark policies would have generated under the realized costs and disturbances, and a corrective law steers the system toward that reference. Charging each change in the reference its recovery cost (the cost of steering the system onto the new reference) reduces the problem to online learning with switching costs. Conversely, under additional natural assumptions, we show that this reduction is tight: the two problems have the same minimax regret up to a system-dependent factor, uniformly over horizons and policy classes. The reduction gives sharp regret guarantees for policy classes beyond standard finite-memory parameterizations. For a class of N possibly nonlinear or history-dependent policies, it achieves O(√(Tlog N)) regret over T rounds, provided their trajectories remain within a bounded distance of one another and recovery costs are bounded. For the full ℓ_1 ball of disturbance-response controllers, it achieves O(√(Tlog T)) regret, which is minimax optimal in T, without assuming a common decay rate for disturbance effects. The framework also improves the best known regret bounds for linear state-feedback policies.
ER -