{
  "abstract": "We consider the problem of estimating a high-dimensional covariance matrix from a very limited number of samples. This problem is ubiquitous in computational fluid dynamics, where a small number of fluid snapshots must be used to construct a Gramian matrix determining a reduced-order model, as well as in computational geoscience, where a small ensemble of Earth system forecasts must be used to estimate the covariance matrix associated with the forecast uncertainty. It is common practice to regularize the small-sample covariance by imposing a \"localization\" structure that enforces a physically realistic correlation length scale, imposing a sparsity constraint, \"shrinking\" towards a prescribed target, or attenuating small correlations. We propose an alternate technique that regularizes the small-sample covariance by imposing hierarchical rank structure. Compared to regularization methods that assume sparsity such as spatial localization, hierarchical rank structure accommodates a wider range of covariance matrices, roughly corresponding to situations where long-range correlations vary more smoothly than short-range ones. It also results in a data-sparse matrix format that permits highly efficient matrix-vector products. We present theory and algorithms which show how to efficiently estimate a high-dimensional, hierarchically rank structured covariance matrix from limited samples. Through an error analysis and numerical experiments with a variety of model problems, we demonstrate that these techniques are effective at reducing sampling errors, and that in many cases they achieve smaller estimation error than conventional techniques.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2609.09944",
  "author_total": 3,
  "authors": [
    {
      "name": "Robin Armstrong"
    },
    {
      "name": "Anil Damle"
    },
    {
      "name": "Samuel E. Otto"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/bdensbxq5r9dr.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2609.09944",
  "fulltext_url": null,
  "id": "bdensbxq5r9dr",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2609.09944",
  "language": null,
  "licence": "LicenseRef-arXiv-nonexclusive-1.0",
  "licence_label": "arXiv licence",
  "licence_source": "article",
  "licence_url": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "cs.NA"
    ],
    [
      "arxiv",
      "math.NA"
    ],
    [
      "arxiv",
      "stat.CO"
    ],
    [
      "openalex",
      "1703|Matrix Theory and Algorithms"
    ]
  ],
  "openalex": "W7212178800",
  "pdf_url": "https://arxiv.org/pdf/2609.09944",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-09",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/bdensbxq5r9dr.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:20:30Z",
      "native_id": "2609.09944",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2609.09944"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "arxiv:math.NA",
      "subfield": "numerical-analysis"
    },
    {
      "field": "mathematics",
      "origin": "arxiv:stat.CO",
      "subfield": "statistics"
    }
  ],
  "title": "Estimating Hierarchically Rank Structured Covariance Matrices",
  "title_text": "Estimating Hierarchically Rank Structured Covariance Matrices",
  "type": "preprint",
  "updated_at": "2026-10-30T12:20:36Z",
  "url": "https://research.draconitic.com/paper/bdensbxq5r9dr",
  "venue": "arXiv",
  "versions": []
}