Skip to content
DraconiticResearch
Search Sources API
203,952 studies31 sourcesUpdated 47 min ago
PreprintCC BY 4.0Zenodo

LLM compression measurements: lossless and lossy compression on top of existing KV-cache and weight formats (RTX 3090, llama.cpp)

  • NeoXider
Open at sourceDOICite

Abstract

Measurements of how much compression can be added on top of the KV-cache formats (F16, Q8_0, Q4_0) and weight formats (BF16, Q8_0, Q4_K_M) used by llama.cpp, on one RTX 3090. A measurement study with mostly negative results; no new method is proposed. Includes the paper, all measurement records (R1-R87), llama.cpp patches, scripts and result summaries. All speed windows were shared with desktop applications; server-GPU rows are analytic projections.

Related

  1. Options for Compression of radio interferometry data: lossy compression of visibilities and lossless compression of uv-visibility grids for the MHONGOOSE survey
    Richard Dodson, Jonghwan Rhee, W. J. G. de Blok +6
    arXivPreprintCC BY 4.0Full textInstrumentation & methodsarXiv
  2. Lossless compression of protein databases for efficient and accurate metagenomic sequence classification with Centrifuger
    L. Song
    bioRxivPreprintCC BY-NDFull textBioinformatics & computational biologybioRxiv
  3. Structure-Aware Lossless Compression for Cell-Level BESS Monitoring Data
    Tianle Liu, Tengjiao Lu, Changlin Yang +3
    BatteriesPeer-reviewedCC BY 4.0Electrochemistry & energyCrossref
  4. The Invocation Bottleneck in LLM-Based Lossless Compression: What Can Be Removed and What Must Be Traded
    Zhe Yan, Yihao Liu, Antonio Boffa +1
    arXivPreprintarXiv licenceFull textGeneral mathematicsarXiv
  5. A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation
    Rahul Krishnan, Volker Schulz
    arXivPreprintCC BY 4.0Full textOptimisation & controlarXiv

Cite

BibTeXRIS
BibTeX
@misc{neoxider2026compression,
  title = {{LLM compression measurements: lossless and lossy compression on top of existing KV-cache and weight formats (RTX 3090, llama.cpp)}},
  author = {NeoXider},
  howpublished = {Zenodo},
  year = {2026},
  month = oct,
  doi = {10.5281/zenodo.23578725},
  url = {https://zenodo.org/records/23578725},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - LLM compression measurements: lossless and lossy compression on top of existing KV-cache and weight formats (RTX 3090, llama.cpp)
AU  - NeoXider
PY  - 2026
DA  - 2026/10/22
PB  - Zenodo
DO  - 10.5281/zenodo.23578725
UR  - https://zenodo.org/records/23578725
AB  - Measurements of how much compression can be added on top of the KV-cache formats (F16, Q8_0, Q4_0) and weight formats (BF16, Q8_0, Q4_K_M) used by llama.cpp, on one RTX 3090. A measurement study with mostly negative results; no new method is proposed. Includes the paper, all measurement records (R1-R87), llama.cpp patches, scripts and result summaries. All speed windows were shared with desktop applications; server-GPU rows are analytic projections.
ER  -