Backtesting Probability-of-Default Grades When Defaults Are Correlated: Exact One-Factor Acceptance Intervals, a Variance Bound on the Information in a Single Year, and an Application to 30,000 Credit-Card Accounts
- 1Former Officer, CGGB
Abstract
Banks test their probability-of-default (PD) grades by comparing the default rate in each grade with the assigned PD, usually by a binomial or Jeffreys test that treats borrowers as independent. Defaults are not independent: a common economic factor moves all of them. I prove that in the one-factor Gaussian model the variance of a grade's default rate equals the binomial variance plus a term equal to the variance of the conditional PD, which does not shrink with the number of borrowers. The information in one year of data is therefore capped: with an asset correlation of 0.04, the Basel value for credit cards, a grade of any size is worth at most 40 to 85 independent borrowers at the PDs studied here. As a consequence, the binomial test rejects a correct PD in 40% to 89% of years for grades of 1,000 to 10,000 borrowers, and its size tends to 1 as the grade grows. Detecting a PD understated by 25% needs 20 years of data at a 5% PD and 36 years at 1%. I apply the tests to a seven-grade logistic PD model fitted to the UCI credit-card default data (30,000 accounts, hold-out AUC 0.787). Within the single observation period the hold-out default counts agree with the grade PDs (joint chi-square p = 0.116). I explain why that within-period test is valid but says nothing about calibration in future years.
Related
- Beyond technology acceptance: AI-application readiness and continuance intention among Chinese university students—a measurement-aware secondary analysis
- Precision follows the decision: how well an acceptance probability is known for mean-spread release rules
- A review of older adults’ acceptance of health monitoring tools: application status, theoretical acceptance models, and user-related and context-level influencing factors
- Exact Common Information and Exact Channel Synthesis for Correlated Discrete Sources
- The concept of quasi-correlated absorption spectra and its application in atmospheric remote sensing
BibTeX
@misc{aguru2026backtesting,
title = {{Backtesting Probability-of-Default Grades When Defaults Are Correlated: Exact One-Factor Acceptance Intervals, a Variance Bound on the Information in a Single Year, and an Application to 30,000 Credit-Card Accounts}},
author = {Manoj Kumar Aguru},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23653368},
url = {https://zenodo.org/records/23653368},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Backtesting Probability-of-Default Grades When Defaults Are Correlated: Exact One-Factor Acceptance Intervals, a Variance Bound on the Information in a Single Year, and an Application to 30,000 Credit-Card Accounts
AU - Manoj Kumar Aguru
PY - 2026
DA - 2026/10/25
PB - Zenodo
DO - 10.5281/zenodo.23653368
UR - https://zenodo.org/records/23653368
AB - Banks test their probability-of-default (PD) grades by comparing the default rate in each grade with the assigned PD, usually by a binomial or Jeffreys test that treats borrowers as independent. Defaults are not independent: a common economic factor moves all of them. I prove that in the one-factor Gaussian model the variance of a grade's default rate equals the binomial variance plus a term equal to the variance of the conditional PD, which does not shrink with the number of borrowers. The information in one year of data is therefore capped: with an asset correlation of 0.04, the Basel value for credit cards, a grade of any size is worth at most 40 to 85 independent borrowers at the PDs studied here. As a consequence, the binomial test rejects a correct PD in 40% to 89% of years for grades of 1,000 to 10,000 borrowers, and its size tends to 1 as the grade grows. Detecting a PD understated by 25% needs 20 years of data at a 5% PD and 36 years at 1%. I apply the tests to a seven-grade logistic PD model fitted to the UCI credit-card default data (30,000 accounts, hold-out AUC 0.787). Within the single observation period the hold-out default counts agree with the grade PDs (joint chi-square p = 0.116). I explain why that within-period test is valid but says nothing about calibration in future years.
ER -