A simultaneous approach for training neural differential-algebraic systems of equations
- 1Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, USA
- 2Department of Chemical Engineering, Process Systems Engineering (AVT.SVT), RWTH Aachen University, Aachen, Germany
Abstract
Neural differential-algebraic systems of equations (DAEs) are a modeling paradigm where some unknown relationships within a DAE are modeled with a neural network and learned from data. Training neural DAEs is more challenging than training neural ordinary-differential equations (ODEs), particularly for higher-index systems. Existing approaches utilize differentiable pipelines, usually comprising integration, projection, operator splitting, or penalty terms for algebraic constraints. The parameters are then updated in a sequential manner using gradient descent. In this work, we employ the simultaneous approach for DAE-constrained parameter estimation instead. This defines a fully discretized nonlinear programing problem (NLP), whose solution simultaneously obtains the neural network parameters and the trajectories of the corresponding DAE, while enforcing constraint satisfaction at the discretization points. We show that with careful initialization and handling of the neural network terms, this approach can be efficient for smaller-scale problems, including higher-index DAEs. As the number of parameters or the amount of data increase, decomposition strategies are necessary to make this method scalable. We present a bi-level approach, where a tractable, discretized NLP is solved at every iteration of an outer gradient descent method, and the sensitivity of its solution with respect to the neural network parameters is evaluated in an efficient manner. We demonstrate the scalability of this decomposition with respect to the number of parameters and the size of the training data set.
Related
- Chaotic dynamics in blooms of Ostreopsis cf. ovata modelled from environmental time series using ordinary differential equations systems
- Canonical differential equations for Feynman integrals from 𝒜-hypergeometric systems in the Schwinger representation
- Simultaneous energy and entropy stability of volume-penalisation immersed-boundary methods for the compressible Navier--Stokes equations
- Bayesian data-driven discovery of partial differential equations with variable coefficients
- Fast quantum algorithm for differential equations
BibTeX
@article{lueg2026simultaneous,
title = {{A simultaneous approach for training neural differential-algebraic systems of equations}},
author = {Laurens R. Lueg and Victor Manuel Cunha Alves and Daniel Schicksnus and John R. Kitchin and Carl D. Laird and Lorenz T. Biegler},
journal = {Computational Optimization and Applications},
year = {2026},
month = aug,
doi = {10.1007/s10589-026-00823-y},
url = {https://doi.org/10.1007/s10589-026-00823-y},
note = {CC BY},
}
RIS
TY - JOUR
TI - A simultaneous approach for training neural differential-algebraic systems of equations
AU - Laurens R. Lueg
AU - Victor Manuel Cunha Alves
AU - Daniel Schicksnus
AU - John R. Kitchin
AU - Carl D. Laird
AU - Lorenz T. Biegler
PY - 2026
DA - 2026/08/17
JO - Computational Optimization and Applications
DO - 10.1007/s10589-026-00823-y
UR - https://doi.org/10.1007/s10589-026-00823-y
AB - Neural differential-algebraic systems of equations (DAEs) are a modeling paradigm where some unknown relationships within a DAE are modeled with a neural network and learned from data. Training neural DAEs is more challenging than training neural ordinary-differential equations (ODEs), particularly for higher-index systems. Existing approaches utilize differentiable pipelines, usually comprising integration, projection, operator splitting, or penalty terms for algebraic constraints. The parameters are then updated in a sequential manner using gradient descent. In this work, we employ the simultaneous approach for DAE-constrained parameter estimation instead. This defines a fully discretized nonlinear programing problem (NLP), whose solution simultaneously obtains the neural network parameters and the trajectories of the corresponding DAE, while enforcing constraint satisfaction at the discretization points. We show that with careful initialization and handling of the neural network terms, this approach can be efficient for smaller-scale problems, including higher-index DAEs. As the number of parameters or the amount of data increase, decomposition strategies are necessary to make this method scalable. We present a bi-level approach, where a tractable, discretized NLP is solved at every iteration of an outer gradient descent method, and the sensitivity of its solution with respect to the neural network parameters is evaluated in an efficient manner. We demonstrate the scalability of this decomposition with respect to the number of parameters and the size of the training data set.
SN - 0926-6003
SN - 1573-2894
ER -