Sharp High-Entropy Bounds for Sums of Independent Discrete Random Variables
Abstract
Sharp high-entropy lower bounds for the entropy of a sum were known for identically distributed summands in torsion-free abelian groups and in prime cyclic groups. For arbitrary independent summands, Gavalakis, Goh and Kontoyiannis obtained an additive constant of 1/8 and conjectured that the sharp constant is 1/2. We prove that independent discrete random variables X,Y with finite Shannon entropies satisfy H(X+Y)≥ (H(X)+H(Y))/2+1/2−o(1) in every torsion-free abelian group as maxH(X),H(Y)→∞. The same conclusion holds in the prime cyclic group 𝔽p when both maxH(X),H(Y) and log2 p−maxH(X),H(Y) tend to infinity. We give explicit error bounds in both settings. The proof extracts a component with paired point probabilities while controlling the entropy of the remainder independently of its support. Discrete rearrangement and uniform perturbation then transfer the continuous entropy power inequality to this component. In prime cyclic groups, an additional estimate controls the entropy lost under modular reduction. Binomial distributions show that the constant 1/2 is optimal.
Related
- Majorization, Entropy and Concentration Inequalities for Discrete α-Concave Random Variables
- A Khintchine type inequality for non-independent random variables
- A weak invariance principle for triangular arrays of independent random variables in some Besov spaces
- Permutations from ranking independent random variables
- Extending Pathwise Gradients to Discrete Random Variables via Finite-Order Relaxation
BibTeX
@misc{wang2026sharp,
title = {{Sharp High-Entropy Bounds for Sums of Independent Discrete Random Variables}},
author = {Haoran Wang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.21459},
eprint = {2609.21459},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.21459},
}
RIS
TY - UNPB
TI - Sharp High-Entropy Bounds for Sums of Independent Discrete Random Variables
AU - Haoran Wang
PY - 2026
DA - 2026/09/18
PB - arXiv
DO - 10.48550/arxiv.2609.21459
UR - https://arxiv.org/abs/2609.21459
AB - Sharp high-entropy lower bounds for the entropy of a sum were known for identically distributed summands in torsion-free abelian groups and in prime cyclic groups. For arbitrary independent summands, Gavalakis, Goh and Kontoyiannis obtained an additive constant of 1/8 and conjectured that the sharp constant is 1/2. We prove that independent discrete random variables X,Y with finite Shannon entropies satisfy H(X+Y)≥ (H(X)+H(Y))/2+1/2−o(1) in every torsion-free abelian group as maxH(X),H(Y)→∞. The same conclusion holds in the prime cyclic group 𝔽_p when both maxH(X),H(Y) and log_2 p−maxH(X),H(Y) tend to infinity. We give explicit error bounds in both settings. The proof extracts a component with paired point probabilities while controlling the entropy of the remainder independently of its support. Discrete rearrangement and uniform perturbation then transfer the continuous entropy power inequality to this component. In prime cyclic groups, an additional estimate controls the entropy lost under modular reduction. Binomial distributions show that the constant 1/2 is optimal.
ER -