Skip to content
DraconiticResearch
Search Sources API
185,139 studies31 sourcesUpdated 47 min ago
PreprintarXiv licencearXiv

The Rise of Low Precision Floating Point

  • Guy Lemieux
  • Michael L. Overton
Open at sourcePDFDOICite

Abstract

Floating point computing has dominated scientific computations since the 1950s because of its versatile ability to approximate real numbers with both sufficiently high precision and sufficiently great range. Since 1985, two specific formats have become standard across the industry, one with 32 bits which is adequate for many purposes and one with 64 bits which allows numbers with both much higher precision and much greater range for applications where these are needed. However, during the past decade, deep learning has come to dominate computation, and narrower floating point formats with lower precision and reduced range have steadily become more common. In this article we discuss this phenomenon, why it is happening, how various 16-bit, 8-bit and even 4-bit floating point formats have been developed, and efforts to standardize these formats.

Related

  1. Performance Evaluation of an Adaptive Quadrature and a Double Exponential Formula Using Arbitrary-Precision Floating-Point Arithmetic
    Tomonori Kouya
    arXivPreprintCC BY 4.0Full textNumerical analysis & scientific computingarXiv
  2. The Entropy of Floating-Point Numbers
    Sultan M. Daniels, Samuel H. D'Ambrosia, Michael R. DeWeese +1
    arXivPreprintCC BY 4.0Full textGeneral mathematicsarXiv
  3. Floating-point autotuning with customized precisions
    Xinye Chen, Fabienne Jézéquel, Thibault Hilaire
    ACM Transactions on Mathematical SoftwarePreprintCC BY 4.0Full textNumerical analysis & scientific computingHALarXiv
  4. Analysis and Detection of Stagnation in Floating-Point Summation
    Mantas Mikaitis, William Woolfenden
    arXivPreprintCC BY 4.0Full textNumerical analysis & scientific computingarXiv
  5. FPScan: An Automated Constraint-Based Analyzer for Floating-Point Anomaly Detection
    Julien Bortolussi, Dorra Ben Khalifa, Pierre-Loïc Garoche
    arXivPreprintCC BY-NC-SA 4.0Full textNumerical analysis & scientific computingarXiv

Cite

BibTeXRIS
BibTeX
@misc{lemieux2026rise,
  title = {{The Rise of Low Precision Floating Point}},
  author = {Guy Lemieux and Michael L. Overton},
  howpublished = {arXiv},
  year = {2026},
  month = nov,
  doi = {10.48550/arxiv.2611.00301},
  eprint = {2611.00301},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00301},
}
RIS
TY  - UNPB
TI  - The Rise of Low Precision Floating Point
AU  - Guy Lemieux
AU  - Michael L. Overton
PY  - 2026
DA  - 2026/11/01
PB  - arXiv
DO  - 10.48550/arxiv.2611.00301
UR  - https://arxiv.org/abs/2611.00301
AB  - Floating point computing has dominated scientific computations since the 1950s because of its versatile ability to approximate real numbers with both sufficiently high precision and sufficiently great range. Since 1985, two specific formats have become standard across the industry, one with 32 bits which is adequate for many purposes and one with 64 bits which allows numbers with both much higher precision and much greater range for applications where these are needed. However, during the past decade, deep learning has come to dominate computation, and narrower floating point formats with lower precision and reduced range have steadily become more common. In this article we discuss this phenomenon, why it is happening, how various 16-bit, 8-bit and even 4-bit floating point formats have been developed, and efforts to standardize these formats.
ER  -