Finite-Sample Limits of Entropy-Based Structure Identification in Discretized Nonlinear Systems
Abstract
Discretization fundamentally limits structure identification in stochastic systems. When system stochasticity exceeds the discretization resolution, entropy-based methods lose their ability to distinguish which input drives the output. We study this in Fuzzy Inductive Reasoning (FIR), a nonparametric framework for learning dynamical systems from discretized measurements, where the choice of input variables determines both predictive accuracy and the interpretability of the learned input--output relationships. Entropy-based selection targets explainability, i.e., identifying which variables causally drive the output, while mean-squared-error-based selection targets prediction. We introduce a resolution-stochasticity ratio that governs when entropy-based selection is reliable. Three results follow. First, entropy-based selection is consistent below this threshold but loses discriminative power above it, regardless of sample size. Second, using the entropy-selected variables for prediction instead of the MSE-selected ones incurs a closed-form excess prediction risk that grows with input complexity and shrinks with sample size. Third, reliable identification of the causally relevant inputs requires data that scales with the number of input combinations and inversely with the strength of the entropy signal. The theory is validated on a two-state Markov model and demonstrated on a distribution grid reliability dataset analyzing the impact of infrastructure investment, where the goal is to explain which investments drive reliability improvements rather than merely predict outcomes.
Related
- Control Realization of Schrödinger Bridges over Discrete-Time Nonlinear Systems via Maximum-Entropy Distribution Steering
- Finite Sample Identification of Analytic Nonlinear Systems
- An arbitrarily high-order accurate and locally energy/entropy-stable space-time method for a class of nonlinear wave systems
- Identification of Nonlinear and Dependent Latent Factor Structure through Clique Search
- Controlled Excitation for Identification of Nonlinear Systems with Saturated Outputs
BibTeX
@misc{shukla2026finitesample,
title = {{Finite-Sample Limits of Entropy-Based Structure Identification in Discretized Nonlinear Systems}},
author = {Pratishtha Shukla and James Nutaro},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.03074},
eprint = {2609.03074},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.03074},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Finite-Sample Limits of Entropy-Based Structure Identification in Discretized Nonlinear Systems
AU - Pratishtha Shukla
AU - James Nutaro
PY - 2026
DA - 2026/09/02
PB - arXiv
DO - 10.48550/arxiv.2609.03074
UR - https://arxiv.org/abs/2609.03074
AB - Discretization fundamentally limits structure identification in stochastic systems. When system stochasticity exceeds the discretization resolution, entropy-based methods lose their ability to distinguish which input drives the output. We study this in Fuzzy Inductive Reasoning (FIR), a nonparametric framework for learning dynamical systems from discretized measurements, where the choice of input variables determines both predictive accuracy and the interpretability of the learned input--output relationships. Entropy-based selection targets explainability, i.e., identifying which variables causally drive the output, while mean-squared-error-based selection targets prediction. We introduce a resolution-stochasticity ratio that governs when entropy-based selection is reliable. Three results follow. First, entropy-based selection is consistent below this threshold but loses discriminative power above it, regardless of sample size. Second, using the entropy-selected variables for prediction instead of the MSE-selected ones incurs a closed-form excess prediction risk that grows with input complexity and shrinks with sample size. Third, reliable identification of the causally relevant inputs requires data that scales with the number of input combinations and inversely with the strength of the entropy signal. The theory is validated on a two-state Markov model and demonstrated on a distribution grid reliability dataset analyzing the impact of infrastructure investment, where the goal is to explain which investments drive reliability improvements rather than merely predict outcomes.
ER -