AVENC: Compressing and Streaming Meteorological Archives via GRIB-Native Video Encoding
Abstract
The growing volume of meteorological data challenges the storage, distribution and accessibility of high-resolution forecasts, reanalyses and ensembles. We introduce AVENC, an open-source, GRIB-compatible toolchain that represents sequences of meteorological fields as video streams within self-contained MP4 files. Built on ecCodes and FFmpeg, AVENC exploits spatial and temporal redundancy while retaining the scientific metadata required to reconstruct standard GRIB messages. It integrates with existing numerical modelling workflows without modifying model interfaces and supports selective HTTP byte-range access, lazy xarray loading and conversion to other scientific formats. We evaluate AVENC in three complementary applications: driving nested 800 m and 160 m atmospheric downscaling simulations with reconstructed boundary conditions, preserving spatial and temporal variability in ERA5-Land wind fields, and retaining probabilistic information in a 51-member ECMWF ensemble. Compression ratios range from approximately 3-fold with reversible coding of quantized video samples to more than 650-fold with aggressive lossy encoding. A high-fidelity configuration reduces the original GRIB data by factors of 17-35 while producing limited changes in the evaluated rainfall patterns, wind-field diagnostics and ensemble spread. Even stronger compression can retain much of the large-scale meteorological structure and ensemble variability, but may introduce substantial localized errors, alter temporal spectra and affect nonlinear model responses. AVENC combines configurable numerical fidelity with GRIB interoperability and selective data access. Rather than prescribing a universal acceptable compression level, it provides reconstruction and comparison tools to help users select suitable configurations for their applications. Video-based representations thus offer a practical route towards more compact, accessible and potentially higher-frequency meteorological datasets.
Related
- The Sedimentary Archives of Ancient Purandarapura
- Dual Nd isotopic archives in fossil teeth reveal bottom-current control of rare-earth element enrichment in deep-sea sediments
- Geochemical clustering reveals cryptic eruptions and tephra correlations from Antarctic fumarolic ice-cave archives
- Characteristics and simulation of ozone pollution in Haikou City under combined effects of meteorological factors and pollutants
- A support vector machine classifier of PM₂.₅-based air quality index categories from co-pollutants and meteorological variables in Ahvaz, Iran
BibTeX
@misc{filippi2026avenc,
title = {{AVENC: Compressing and Streaming Meteorological Archives via GRIB-Native Video Encoding}},
author = {Jean-Baptiste Filippi},
howpublished = {HAL},
year = {2026},
month = oct,
url = {https://hal.science/hal-05847010v1},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - AVENC: Compressing and Streaming Meteorological Archives via GRIB-Native Video Encoding
AU - Jean-Baptiste Filippi
PY - 2026
DA - 2026/10/26
PB - HAL
UR - https://hal.science/hal-05847010v1
AB - The growing volume of meteorological data challenges the storage, distribution and accessibility of high-resolution forecasts, reanalyses and ensembles. We introduce AVENC, an open-source, GRIB-compatible toolchain that represents sequences of meteorological fields as video streams within self-contained MP4 files. Built on ecCodes and FFmpeg, AVENC exploits spatial and temporal redundancy while retaining the scientific metadata required to reconstruct standard GRIB messages. It integrates with existing numerical modelling workflows without modifying model interfaces and supports selective HTTP byte-range access, lazy xarray loading and conversion to other scientific formats. We evaluate AVENC in three complementary applications: driving nested 800 m and 160 m atmospheric downscaling simulations with reconstructed boundary conditions, preserving spatial and temporal variability in ERA5-Land wind fields, and retaining probabilistic information in a 51-member ECMWF ensemble. Compression ratios range from approximately 3-fold with reversible coding of quantized video samples to more than 650-fold with aggressive lossy encoding. A high-fidelity configuration reduces the original GRIB data by factors of 17-35 while producing limited changes in the evaluated rainfall patterns, wind-field diagnostics and ensemble spread. Even stronger compression can retain much of the large-scale meteorological structure and ensemble variability, but may introduce substantial localized errors, alter temporal spectra and affect nonlinear model responses. AVENC combines configurable numerical fidelity with GRIB interoperability and selective data access. Rather than prescribing a universal acceptable compression level, it provides reconstruction and comparison tools to help users select suitable configurations for their applications. Video-based representations thus offer a practical route towards more compact, accessible and potentially higher-frequency meteorological datasets.
ER -