Matrix-Aware Proper Scoring Rules and Significance Testing for Correlation and Covariance Forecasts in Python
Abstract
Forecasting a correlation or covariance matrix is common in risk management and portfolio construction, but evaluating such a forecast correctly is not routine: naive matrix-comparison metrics are not proper scoring rules, walk-forward evaluation windows are easy to overlap with the estimation window in ways that silently leak information, and significance testing on serially dependent forecast-error sequences needs machinery few analysts implement from scratch. corrscore is a Python package that provides matrix-aware implementations of two established proper scoring rules for this setting -- the energy score and the variogram score -- dispatched across a closed-form tractability spectrum (point, discrete-mixture, and isotropic-Gaussian-mixture forecasts are scored exactly; a general Monte Carlo ensemble falls back to sampling), a geometry-aware variant of the variogram score built from the affine-invariant distance on the correlation manifold, a zero-overlap-by-construction walk-forward backtest harness, and a bundled significance-testing suite (circular block bootstrap, the Diebold-Mariano test, and the Model Confidence Set). We describe the package's design, its point of departure from the existing scoringRules and properscoring packages, and walk through a complete worked example.
Related
- Do I Even Need a Multilevel Model? Negligible Effect Significance Testing for Intraclass Correlation Coefficients
- Stress-Testing Dynamical and Generative Downscaling Using Subseasonal Extreme Precipitation Forecasts
- An F-Type Normal-Reference Calibration for High-Dimensional Covariance Testing
- On the nineteenth-century origins of significance testing and p-hacking
- High-Dimensional Two-Sample Covariance Testing with Null-Preserving Transformations
BibTeX
@misc{nguyen2026matrixaware,
title = {{Matrix-Aware Proper Scoring Rules and Significance Testing for Correlation and Covariance Forecasts in Python}},
author = {Vinh Nguyen},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.01648},
eprint = {2609.01648},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.01648},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Matrix-Aware Proper Scoring Rules and Significance Testing for Correlation and Covariance Forecasts in Python
AU - Vinh Nguyen
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2609.01648
UR - https://arxiv.org/abs/2609.01648
AB - Forecasting a correlation or covariance matrix is common in risk management and portfolio construction, but evaluating such a forecast correctly is not routine: naive matrix-comparison metrics are not proper scoring rules, walk-forward evaluation windows are easy to overlap with the estimation window in ways that silently leak information, and significance testing on serially dependent forecast-error sequences needs machinery few analysts implement from scratch. corrscore is a Python package that provides matrix-aware implementations of two established proper scoring rules for this setting -- the energy score and the variogram score -- dispatched across a closed-form tractability spectrum (point, discrete-mixture, and isotropic-Gaussian-mixture forecasts are scored exactly; a general Monte Carlo ensemble falls back to sampling), a geometry-aware variant of the variogram score built from the affine-invariant distance on the correlation manifold, a zero-overlap-by-construction walk-forward backtest harness, and a bundled significance-testing suite (circular block bootstrap, the Diebold-Mariano test, and the Model Confidence Set). We describe the package's design, its point of departure from the existing scoringRules and properscoring packages, and walk through a complete worked example.
ER -