Implementing neural network mixed-effects models in Template Model Builder (TMB)
Abstract
Neural network mixed-effects models (NMMs) have gained traction by combining the strong representation and predictive power of artificial neural networks with the capacity of mixed-effects modeling to capture complex correlation structures. However, existing estimation approaches rely heavily on manual derivations of objective functions and gradients, which inherently forces simplifying approximations and severely constrains the complexity and accuracy of NMMs. In this work, we introduce a general framework for implementing NMMs using Template Model Builder (TMB). By leveraging automatic differentiation and Laplace approximation, TMB requires users to specify only the negative joint log-likelihood and any regularization terms. The framework automatically integrates out random effects and evaluates the marginal objective function alongside its exact gradients, eliminating the need for manual derivations or ad hoc approximations. We demonstrate the efficiency, flexibility, and statistical performance of TMB-based NMMs across two numerical examples, including an application to monotonic NMMs. Reproducible code is provided to facilitate broader adoption.
Related
- Shallow neural network approximation in mixed Sobolev spaces
- Approximation, Consistency, and Prediction in Generalized Neural Network Mixed Models for Longitudinal and Clustered Data
- Mixed-Frequency Time Series Forecasting via Depth-Separable Neural Networks
- Optimal Selling of Defaultable Assets using the Distribution Builder
- Learning to Solve Mixed-Integer Nonlinear Programming with Neural Networks and Sequential Quadratic Programming Refinement
BibTeX
@misc{zheng2026implementing,
title = {{Implementing neural network mixed-effects models in Template Model Builder (TMB)}},
author = {Nan Zheng and Hoi Yiu Cheung and Vibhu Sharma and James T. Thorson and Noel G. Cadigan},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.31133},
eprint = {2608.31133},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.31133},
}
RIS
TY - UNPB
TI - Implementing neural network mixed-effects models in Template Model Builder (TMB)
AU - Nan Zheng
AU - Hoi Yiu Cheung
AU - Vibhu Sharma
AU - James T. Thorson
AU - Noel G. Cadigan
PY - 2026
DA - 2026/08/31
PB - arXiv
DO - 10.48550/arxiv.2608.31133
UR - https://arxiv.org/abs/2608.31133
AB - Neural network mixed-effects models (NMMs) have gained traction by combining the strong representation and predictive power of artificial neural networks with the capacity of mixed-effects modeling to capture complex correlation structures. However, existing estimation approaches rely heavily on manual derivations of objective functions and gradients, which inherently forces simplifying approximations and severely constrains the complexity and accuracy of NMMs. In this work, we introduce a general framework for implementing NMMs using Template Model Builder (TMB). By leveraging automatic differentiation and Laplace approximation, TMB requires users to specify only the negative joint log-likelihood and any regularization terms. The framework automatically integrates out random effects and evaluates the marginal objective function alongside its exact gradients, eliminating the need for manual derivations or ad hoc approximations. We demonstrate the efficiency, flexibility, and statistical performance of TMB-based NMMs across two numerical examples, including an application to monotonic NMMs. Reproducible code is provided to facilitate broader adoption.
ER -