On architectural choices for interpretability and thermodynamic consistency in Physically Recurrent Neural Networks in the low-data regime
Abstract
In this paper, we unravel the effect of different decoder architectures on the interpretability of the latent space of the Physically Recurrent Neural Network. Particular emphasis is given to a new weight normalization constraint, which acts as a regularization technique and enables robust training in the low-data regime. A brief visual exploration illustrates how these changes impact the latent space and how the fictitious stress can align with the true state of the RVE without explicit training. Reaping the benefits of a meaningful latent space, a case study illustrates how information from the microscopic level can be retrieved and incorporated into a multi-task approach that does not require extra parameters or larger training sets. Another key contribution shows that a specific architectural choice can naturally lead to a thermodynamically consistent formulation. By enforcing an adjoint encoder-decoder structure with positive scalar contributions, this modification ensures energy consistency across scales and non-negative dissipation, leading to even lower training requirements. This alternative completes the study on interpretability, inductive bias, and thermodynamic consistency, and demonstrates that data efficiency can be improved with careful architectural choices rooted in the underlying physics.
Related
- Interpretability for Turing Machines
- Plug-and-Play Methods Provably Converge Even with Improperly Trained Denoisers: Convergence by Architectural Design
- The Metagame of Interpretability and Meta-Attributions
- On the Architectural Complexity of Neural Networks
- An Integrated Framework for Assessing Interpretability in Exploratory Factor Analysis
BibTeX
@misc{maia2026architectural,
title = {{On architectural choices for interpretability and thermodynamic consistency in Physically Recurrent Neural Networks in the low-data regime}},
author = {M. A. Maia and K. A. Meyer and A. M. C. M. van Gils and I. B. C. M. Rocha and F. P. van der Meer},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.04067},
eprint = {2610.04067},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.04067},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - On architectural choices for interpretability and thermodynamic consistency in Physically Recurrent Neural Networks in the low-data regime
AU - M. A. Maia
AU - K. A. Meyer
AU - A. M. C. M. van Gils
AU - I. B. C. M. Rocha
AU - F. P. van der Meer
PY - 2026
DA - 2026/10/02
PB - arXiv
DO - 10.48550/arxiv.2610.04067
UR - https://arxiv.org/abs/2610.04067
AB - In this paper, we unravel the effect of different decoder architectures on the interpretability of the latent space of the Physically Recurrent Neural Network. Particular emphasis is given to a new weight normalization constraint, which acts as a regularization technique and enables robust training in the low-data regime. A brief visual exploration illustrates how these changes impact the latent space and how the fictitious stress can align with the true state of the RVE without explicit training. Reaping the benefits of a meaningful latent space, a case study illustrates how information from the microscopic level can be retrieved and incorporated into a multi-task approach that does not require extra parameters or larger training sets. Another key contribution shows that a specific architectural choice can naturally lead to a thermodynamically consistent formulation. By enforcing an adjoint encoder-decoder structure with positive scalar contributions, this modification ensures energy consistency across scales and non-negative dissipation, leading to even lower training requirements. This alternative completes the study on interpretability, inductive bias, and thermodynamic consistency, and demonstrates that data efficiency can be improved with careful architectural choices rooted in the underlying physics.
ER -