Calibrating Classifiers Across Covariates: Hierarchical Vocal Density Estimation for Acoustic Monitoring at Scale
- 1University of Pittsburgh
Abstract
Passive acoustic sensors and machine learning classifiers offer a scalable approach to monitoring wildlife populations. These classifiers typically detect whether a species is present in a short time-window of audio; ecologists, however, want ecologically meaningful measures such as indices of site-level abundance. Vocal density, the proportion of time-windows containing a vocalization, is a useful abundance proxy. Recovering vocal density from classifiers requires calibrating their outputs against expert-labeled data. Because classifier performance varies between sites, a single global calibration yields overconfident site-level estimates, yet calibrating every site independently requires prohibitive labeling effort. Estimating site-level vocal density at the scale of modern sensor networks demands a label-efficient alternative. We present a Bayesian hierarchical extension of Platt scaling, a commonly used calibration method, that calibrates classifiers at the site level while borrowing strength across sites. Where site-level covariates are available, a Gaussian-process prior lets the model learn how calibration varies across covariate space. Averaging calibrated per-clip probabilities across a site's recordings yields a posterior distribution over vocal density, and we show how to carry that uncertainty into downstream regressions of vocal density on environmental covariates. Across simulations and two fully annotated field datasets from Hawai'i and the Pacific Northwest, our model attained desired credible-interval coverage of vocal density estimates with narrower intervals than independent site-level calibration at equal labeling effort, and lower mean squared error. Global calibration, by contrast, failed to attain coverage except when simulated sites were genuinely homogeneous. Unlike site-level calibration, our model enables calibration at sites with zero labeled data, and incorporating covariates further improved precision when site heterogeneity was covariate-driven. Applied to a 283-site dataset in Pennsylvania with only two labeled clips per site, we recovered known habitat associations for the declining Wood Thrush (Hylocichla mustelina). Our method provides a principled, label-efficient route from bioacoustic classifier scores to site-level abundance indices with well-quantified uncertainty, making rigorous ecological inference feasible at the scale at which acoustic sensor networks are now deployed.
Related
- Built-in Selection Bias in Proportional Hazards Models with Omitted Covariates: Simulation Evidence and Alternative Approaches
- Orientation detection for self-calibrating neuromorphic systems
- Ensemble-based support vector machine classifiers as an efficient tool for quality assessment of beef fillets from electronic nose data
- MAXWELL: Calibrating the probabilistic outputs of protein language models to the mutation-induced stability change landscape
- Sexing carnivores from faecal metagenomes: calibrating Rx for stone marten and red fox
BibTeX
@misc{freelandhaynes2026calibrating,
title = {{Calibrating Classifiers Across Covariates: Hierarchical Vocal Density Estimation for Acoustic Monitoring at Scale}},
author = {L. Freeland-Haynes and M. J. Weldy and A. K. Navine and P. J. Hart and J. Kitzes and T. Denton},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.08.747031},
url = {https://www.biorxiv.org/content/10.64898/2026.09.08.747031v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Calibrating Classifiers Across Covariates: Hierarchical Vocal Density Estimation for Acoustic Monitoring at Scale
AU - L. Freeland-Haynes
AU - M. J. Weldy
AU - A. K. Navine
AU - P. J. Hart
AU - J. Kitzes
AU - T. Denton
PY - 2026
DA - 2026/09/08
PB - bioRxiv
DO - 10.64898/2026.09.08.747031
UR - https://www.biorxiv.org/content/10.64898/2026.09.08.747031v1
AB - Passive acoustic sensors and machine learning classifiers offer a scalable approach to monitoring wildlife populations. These classifiers typically detect whether a species is present in a short time-window of audio; ecologists, however, want ecologically meaningful measures such as indices of site-level abundance. Vocal density, the proportion of time-windows containing a vocalization, is a useful abundance proxy. Recovering vocal density from classifiers requires calibrating their outputs against expert-labeled data. Because classifier performance varies between sites, a single global calibration yields overconfident site-level estimates, yet calibrating every site independently requires prohibitive labeling effort. Estimating site-level vocal density at the scale of modern sensor networks demands a label-efficient alternative. We present a Bayesian hierarchical extension of Platt scaling, a commonly used calibration method, that calibrates classifiers at the site level while borrowing strength across sites. Where site-level covariates are available, a Gaussian-process prior lets the model learn how calibration varies across covariate space. Averaging calibrated per-clip probabilities across a site's recordings yields a posterior distribution over vocal density, and we show how to carry that uncertainty into downstream regressions of vocal density on environmental covariates. Across simulations and two fully annotated field datasets from Hawai'i and the Pacific Northwest, our model attained desired credible-interval coverage of vocal density estimates with narrower intervals than independent site-level calibration at equal labeling effort, and lower mean squared error. Global calibration, by contrast, failed to attain coverage except when simulated sites were genuinely homogeneous. Unlike site-level calibration, our model enables calibration at sites with zero labeled data, and incorporating covariates further improved precision when site heterogeneity was covariate-driven. Applied to a 283-site dataset in Pennsylvania with only two labeled clips per site, we recovered known habitat associations for the declining Wood Thrush (Hylocichla mustelina). Our method provides a principled, label-efficient route from bioacoustic classifier scores to site-level abundance indices with well-quantified uncertainty, making rigorous ecological inference feasible at the scale at which acoustic sensor networks are now deployed.
ER -