Revisiting entropy and entropy-related magnitudes of a symbolic sequence and their estimation
Abstract
This article revisits entropy and related magnitudes (entropy density, effective measure complexity, and informational distance) for the analysis of sequential data. We present their mathematical foundations within information theory and ergodic theory, establishing connections between probabilistic Shannon entropy and algorithmic Kolmogorov complexity. A central result, Zivs theorem, demonstrates the asymptotic equivalence between the normalised number of distinct words and entropy density, providing the theoretical basis for estimation via Lempel-Ziv factorisation. We examine the convergence properties of these magnitudes and introduce estimation procedures for finite sequences, including a method for the effective measure complexity and the transfer entropy rate. An open-source software library implementing parallel algorithms for efficient Lempel-Ziv factorisation is presented. The practical utility of the framework is illustrated through three case studies: functional brain connectivity analysis from fMRI data, characterisation of spatiotemporal dynamics in coupled nonlinear oscillators, and electrocardiogram-based classification of atrial fibrillation, demonstrating how entropic measures capture meaningful structural and dynamical properties across diverse domains.
Related
- Bivariate Frank Copula: Some More Results on Point Estimation of the Association Parameter from a Bayesian Perspective and Revisiting the Goodness of Fit Tests with an Application to Model Groundwater Data from Dong Thap, Vietnam
- Li--Yorke Chaos Along Any Infinite Sequence: Relative Mixing, Sofic and Rokhlin Entropy versus Naive Entropy
- Optimal von Neumann Entropy Estimation
- Improved estimation of differential entropy of a multivariate Gaussian distribution
- SENECA: Small-Sample Discrete Entropy Estimation via Self-Consistent Missing Mass
BibTeX
@misc{aragonperez2026revisiting,
title = {{Revisiting entropy and entropy-related magnitudes of a symbolic sequence and their estimation}},
author = {Efren Aragon-Perez and Ania Mesa-Rodriguez and Karel Garcia-Medina and Daniel Estevez-Moya and Amanda Mollineda-Rodriguez and Ernesto Estevez-Rams},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.35121},
eprint = {2610.35121},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.35121},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Revisiting entropy and entropy-related magnitudes of a symbolic sequence and their estimation
AU - Efren Aragon-Perez
AU - Ania Mesa-Rodriguez
AU - Karel Garcia-Medina
AU - Daniel Estevez-Moya
AU - Amanda Mollineda-Rodriguez
AU - Ernesto Estevez-Rams
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.35121
UR - https://arxiv.org/abs/2610.35121
AB - This article revisits entropy and related magnitudes (entropy density, effective measure complexity, and informational distance) for the analysis of sequential data. We present their mathematical foundations within information theory and ergodic theory, establishing connections between probabilistic Shannon entropy and algorithmic Kolmogorov complexity. A central result, Zivs theorem, demonstrates the asymptotic equivalence between the normalised number of distinct words and entropy density, providing the theoretical basis for estimation via Lempel-Ziv factorisation. We examine the convergence properties of these magnitudes and introduce estimation procedures for finite sequences, including a method for the effective measure complexity and the transfer entropy rate. An open-source software library implementing parallel algorithms for efficient Lempel-Ziv factorisation is presented. The practical utility of the framework is illustrated through three case studies: functional brain connectivity analysis from fMRI data, characterisation of spatiotemporal dynamics in coupled nonlinear oscillators, and electrocardiogram-based classification of atrial fibrillation, demonstrating how entropic measures capture meaningful structural and dynamical properties across diverse domains.
ER -