Skip to content
DraconiticResearch
Search Sources API
203,234 studies31 sourcesUpdated just now
PreprintCC BY 4.0arXiv

The Entropy of Floating-Point Numbers

  • Sultan M. Daniels
  • Samuel H. D'Ambrosia
  • Michael R. DeWeese
  • Anant Sahai
Open at sourcePDFDOICite

Abstract

Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a uniformly quantized random variable. Our approximation uncovers a different quantity that provides this link for floating-point quantization. Additionally, we prove that the entropy of a floating-point quantized random variable is approximately unchanged under scaling. Closed-form expressions for the floating-point entropy of common distributions are provided and compared to exact results.

Related

  1. The Rise of Low Precision Floating Point
    Guy Lemieux, Michael L. Overton
    arXivPreprintarXiv licenceFull textNumerical analysis & scientific computingarXiv
  2. Performance Evaluation of an Adaptive Quadrature and a Double Exponential Formula Using Arbitrary-Precision Floating-Point Arithmetic
    Tomonori Kouya
    arXivPreprintCC BY 4.0Full textNumerical analysis & scientific computingarXiv
  3. Floating-point autotuning with customized precisions
    Xinye Chen, Fabienne Jézéquel, Thibault Hilaire
    ACM Transactions on Mathematical SoftwarePreprintCC BY 4.0Full textNumerical analysis & scientific computingHALarXiv
  4. Analysis and Detection of Stagnation in Floating-Point Summation
    Mantas Mikaitis, William Woolfenden
    arXivPreprintCC BY 4.0Full textNumerical analysis & scientific computingarXiv
  5. FPScan: An Automated Constraint-Based Analyzer for Floating-Point Anomaly Detection
    Julien Bortolussi, Dorra Ben Khalifa, Pierre-Loïc Garoche
    arXivPreprintCC BY-NC-SA 4.0Full textNumerical analysis & scientific computingarXiv

Cite

BibTeXRIS
BibTeX
@misc{daniels2026entropy,
  title = {{The Entropy of Floating-Point Numbers}},
  author = {Sultan M. Daniels and Samuel H. D'Ambrosia and Michael R. DeWeese and Anant Sahai},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2605.11546},
  eprint = {2605.11546},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.11546},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - The Entropy of Floating-Point Numbers
AU  - Sultan M. Daniels
AU  - Samuel H. D'Ambrosia
AU  - Michael R. DeWeese
AU  - Anant Sahai
PY  - 2026
DA  - 2026/09/02
PB  - arXiv
DO  - 10.48550/arxiv.2605.11546
UR  - https://arxiv.org/abs/2605.11546
AB  - Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a uniformly quantized random variable. Our approximation uncovers a different quantity that provides this link for floating-point quantization. Additionally, we prove that the entropy of a floating-point quantized random variable is approximately unchanged under scaling. Closed-form expressions for the floating-point entropy of common distributions are provided and compared to exact results.
ER  -