Estimator Precision and Selection for Up-the-Ramp Sampling with a High-Read-Noise InGaAs Camera
Abstract
The noise properties of an InGaAs camera vary with gain, requiring a trade-off between implementation complexity and precision. Using generalized least squares (GLS) with known covariance as the precision benchmark, we examine the applicability and precision penalties of ordinary least squares (OLS) and correlated double sampling (CDS) at different noise operating points. We define η=fT/σR2, where f is the total pixel count rate, T the fitting span, and σR the single-read noise. For constant flux, uniform sampling, and independent white read noise plus Poisson noise, established finite-sampling variance expressions give equal OLS and CDS variances at η=10 for N≥4 reads; OLS is favored below this boundary and CDS above it. Analytical and Monte Carlo comparisons for three gain modes quantify the precision loss of the better simplified estimator relative to GLS. In high-gain on-sky data, choosing OLS at 20 s and CDS at 90 s increases the background residual standard deviation by 3.55% and 3.01%, respectively, relative to two-iteration feasible GLS (FGLS). Photometry of 21 common sources does not clearly resolve most estimator differences. The observed OLS-CDS crossover occurs earlier than the ideal prediction, indicating that temporal covariance beyond the ideal model also affects the selection boundary. Observational validation is limited to high gain; medium- and low-gain choices rely on theory and simulations using calibrated parameters. OLS or CDS can replace GLS with a small precision penalty at suitable operating points. The ideal criterion provides an initial selection rule, whose practical boundary requires validation against observed temporal covariance.
Related
- Improving exoplanet mass characterisation with Bayesian model selection using the Learned Harmonic Mean Estimator
- Stellar masses and ages in Gaia Data Release 4 from the Final Luminosity Age Mass Estimator algorithm
- ECLAIRs: The SVOM High-energy Transient Trigger Camera
- Adaptive Model-Switch Thermodynamic Integration: A Bayes Factor Estimator
- Filter and baffle design for the short-wavelength camera on SOLAR-C EUVST
BibTeX
@misc{meng2026estimator,
title = {{Estimator Precision and Selection for Up-the-Ramp Sampling with a High-Read-Noise InGaAs Camera}},
author = {Wei-Jie Meng and Fang-Yu Xu and Zhen-Yu Jin and Guang-Qian Liu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.27742},
eprint = {2610.27742},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.27742},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Estimator Precision and Selection for Up-the-Ramp Sampling with a High-Read-Noise InGaAs Camera
AU - Wei-Jie Meng
AU - Fang-Yu Xu
AU - Zhen-Yu Jin
AU - Guang-Qian Liu
PY - 2026
DA - 2026/10/20
PB - arXiv
DO - 10.48550/arxiv.2610.27742
UR - https://arxiv.org/abs/2610.27742
AB - The noise properties of an InGaAs camera vary with gain, requiring a trade-off between implementation complexity and precision. Using generalized least squares (GLS) with known covariance as the precision benchmark, we examine the applicability and precision penalties of ordinary least squares (OLS) and correlated double sampling (CDS) at different noise operating points. We define η=fT/σ_R^2, where f is the total pixel count rate, T the fitting span, and σ_R the single-read noise. For constant flux, uniform sampling, and independent white read noise plus Poisson noise, established finite-sampling variance expressions give equal OLS and CDS variances at η=10 for N≥4 reads; OLS is favored below this boundary and CDS above it. Analytical and Monte Carlo comparisons for three gain modes quantify the precision loss of the better simplified estimator relative to GLS. In high-gain on-sky data, choosing OLS at 20 s and CDS at 90 s increases the background residual standard deviation by 3.55% and 3.01%, respectively, relative to two-iteration feasible GLS (FGLS). Photometry of 21 common sources does not clearly resolve most estimator differences. The observed OLS-CDS crossover occurs earlier than the ideal prediction, indicating that temporal covariance beyond the ideal model also affects the selection boundary. Observational validation is limited to high gain; medium- and low-gain choices rely on theory and simulations using calibrated parameters. OLS or CDS can replace GLS with a small precision penalty at suitable operating points. The ideal criterion provides an initial selection rule, whose practical boundary requires validation against observed temporal covariance.
ER -