Learning Changes the World and Then What Was Learned has Changed: Training and Deploying Neural Networks in the Control Loop
Abstract
Neural networks offer critical adaptability for controlling complex systems where nominal modeling proves intractable. However, when offline-trained neural networks are deployed within closed-loop control systems, their actions fundamentally alter the state distribution upon which they were trained. This paper introduces a unified taxonomy to categorize these sequential failure modes: the initial Closed-Loop Identification Error, the deployment-induced Action Error, and the continuous Tracking Error. Through discrete-time sensitivity analysis, this work formalizes how offline-learned parameters locally guarantee a trajectory divergence upon deployment, degrading performance due to local loss geometry. To mitigate this distribution shift, we propose actionable design policies centered on time-scale separated online adaptation. These mechanisms are validated through ablation studies on a nonlinear Duffing oscillator, demonstrating that bounded continuous learning successfully mitigates trajectory divergence whereas static deployments fail, offering concrete guidelines for practitioners deploying machine learning components in safety-critical feedback systems.
Versions
Related
- Fast Learning Rate Transfer in Shallow Linear Networks at Growing Training Horizons
- Integrable building blocks for symplectic neural networks: learned-type layers, fiber-preserving layers, and what a single layer can do
- Lazy training of quantum physics informed neural networks
- Subspace Levenberg Marquardt Algorithms in Training Neural Networks
- Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks
BibTeX
@misc{lippi2026learning,
title = {{Learning Changes the World and Then What Was Learned has Changed: Training and Deploying Neural Networks in the Control Loop}},
author = {Vittorio Lippi},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.22084},
eprint = {2610.22084},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.22084},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Learning Changes the World and Then What Was Learned has Changed: Training and Deploying Neural Networks in the Control Loop
AU - Vittorio Lippi
PY - 2026
DA - 2026/10/15
PB - arXiv
DO - 10.48550/arxiv.2610.22084
UR - https://arxiv.org/abs/2610.22084
AB - Neural networks offer critical adaptability for controlling complex systems where nominal modeling proves intractable. However, when offline-trained neural networks are deployed within closed-loop control systems, their actions fundamentally alter the state distribution upon which they were trained. This paper introduces a unified taxonomy to categorize these sequential failure modes: the initial Closed-Loop Identification Error, the deployment-induced Action Error, and the continuous Tracking Error. Through discrete-time sensitivity analysis, this work formalizes how offline-learned parameters locally guarantee a trajectory divergence upon deployment, degrading performance due to local loss geometry. To mitigate this distribution shift, we propose actionable design policies centered on time-scale separated online adaptation. These mechanisms are validated through ablation studies on a nonlinear Duffing oscillator, demonstrating that bounded continuous learning successfully mitigates trajectory divergence whereas static deployments fail, offering concrete guidelines for practitioners deploying machine learning components in safety-critical feedback systems.
ER -