Options for Compression of radio interferometry data: lossy compression of visibilities and lossless compression of uv-visibility grids for the MHONGOOSE survey
Abstract
Next generation radio astronomy telescopes are challenging existing data reduction paradigms. With ever more antennas, larger bandwidths, and sometimes multiple primary beams, they often generate more observed data products than can readily be stored long-term. Thus, data storage becomes a major cost driver and processing constraint. In this paper, we test two methods of addressing this problem: grid-stacking, a two-stage lossless compression solution; and the lossy compression of the raw visibilities before traditional processing. To demonstrate these solutions we utilised a deep imaging pipeline based on software for the ASKAP telescope, ASKAPSoft, but applied to a strong source (NGC1566) from the deep MeerKAT HI spectral line project, MHONGOOSE. The grid-stacking solution reproduces the spectrum from traditional processing to within better than 0.7%, and also allows for the reconstruction of other weighting scales without significant computing costs. In comparison, image-stacking also reproduces the spectrum from the traditional processing, to within better than 3% but with worse image residuals in the cube. The lossy compression, even at a near ten-fold reduction in file size, reproduces the spectra almost perfectly (to better than ~0.01% in all cases). Thus both compression methods are promising solutions, and we discuss considerations for their application.
Versions
Related
- The origin and distribution of diffuse HI in the close interacting MHONGOOSE galaxy systems
- The origin and distribution of diffuse HI in the close interacting MHONGOOSE galaxy systems
- IViS: Interferometric Visibility-domain inversion Software. A GPU-accelerated Python framework for joint deconvolution with ASKAP
- Accurate dynamical mass and orbit determination of the M dwarf binary DG CVn using astrometric very long baseline interferometry
- Asteroseismology and interferometry of the F7V spectroscopic binary χ Draconis A in the TESS CVZ
BibTeX
@misc{dodson2026options,
title = {{Options for Compression of radio interferometry data: lossy compression of visibilities and lossless compression of uv-visibility grids for the MHONGOOSE survey}},
author = {Richard Dodson and Jonghwan Rhee and W. J. G. de Blok and Martin Meyer and Alexander Williamson and Daniel Mitchell and Kristóf Rozgonyi and María J. Rioja and Pascal J. Elahi},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.25988},
eprint = {2609.25988},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.25988},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Options for Compression of radio interferometry data: lossy compression of visibilities and lossless compression of uv-visibility grids for the MHONGOOSE survey
AU - Richard Dodson
AU - Jonghwan Rhee
AU - W. J. G. de Blok
AU - Martin Meyer
AU - Alexander Williamson
AU - Daniel Mitchell
AU - Kristóf Rozgonyi
AU - María J. Rioja
AU - Pascal J. Elahi
PY - 2026
DA - 2026/09/22
PB - arXiv
DO - 10.48550/arxiv.2609.25988
UR - https://arxiv.org/abs/2609.25988
AB - Next generation radio astronomy telescopes are challenging existing data reduction paradigms. With ever more antennas, larger bandwidths, and sometimes multiple primary beams, they often generate more observed data products than can readily be stored long-term. Thus, data storage becomes a major cost driver and processing constraint. In this paper, we test two methods of addressing this problem: grid-stacking, a two-stage lossless compression solution; and the lossy compression of the raw visibilities before traditional processing. To demonstrate these solutions we utilised a deep imaging pipeline based on software for the ASKAP telescope, ASKAPSoft, but applied to a strong source (NGC1566) from the deep MeerKAT HI spectral line project, MHONGOOSE. The grid-stacking solution reproduces the spectrum from traditional processing to within better than 0.7%, and also allows for the reconstruction of other weighting scales without significant computing costs. In comparison, image-stacking also reproduces the spectrum from the traditional processing, to within better than 3% but with worse image residuals in the cube. The lossy compression, even at a near ten-fold reduction in file size, reproduces the spectra almost perfectly (to better than ~0.01% in all cases). Thus both compression methods are promising solutions, and we discuss considerations for their application.
ER -