Testing for Unobserved Heterogeneity in Censored Duration Models: EM Approach
Abstract
Ignoring unobserved heterogeneity in duration models biases parameter estimates and invalidates inference, but testing for it is non-regular: the null hypothesis lies on the boundary of the parameter space and some parameters are unidentified under the null. These features render standard asymptotic theory inapplicable. This paper develops an EM test for unobserved heterogeneity in censored Weibull duration models, building on the EM approach of Li, Chen, and Marriott (2009). The test statistic has an asymptotic null distribution equal to the square of max{0, N(0,1)}, hence critical values require neither simulation nor bootstrap, and the test accommodates covariate-dependent censoring of arbitrary form. Monte Carlo simulations compare the EM test with the likelihood ratio test (LRT) of Cho and White (2010), information matrix tests, and Lagrange multiplier tests. The EM test has empirical size close to the nominal level for sample sizes of 500 or more, where the LRT remains markedly conservative and the other tests over-reject. Its size-adjusted power is comparable to that of the LRT and higher than that of the other tests. Because size adjustment requires knowledge of the data-generating process and is unavailable in practice, the EM test attains higher power than the LRT in most designs as the tests would actually be applied. In an application to the Stanford Heart Transplant data, the EM test rejects homogeneity in every covariate specification, whereas the LRT's conclusion depends on a user-chosen set of admissible parameter values and on the specification.
Related
- Two-level Factorial Life Testing with Type-II Censored Data
- Practical Guidelines for Goodness-of-Fit Testing for Right-Censored Data
- Testing Treatment Effect Heterogeneity with Functional Outcomes
- Robust Design and Evaluation of Predictive Algorithms under Unobserved Confounding
- Unobserved Consideration and Consumer Welfare
BibTeX
@misc{kasahara2026testing,
title = {{Testing for Unobserved Heterogeneity in Censored Duration Models: EM Approach}},
author = {Hiroyuki Kasahara and Hirokazu Matsuyama and Katsumi Shimotsu and Shota Takeishi},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.36824},
eprint = {2609.36824},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.36824},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Testing for Unobserved Heterogeneity in Censored Duration Models: EM Approach
AU - Hiroyuki Kasahara
AU - Hirokazu Matsuyama
AU - Katsumi Shimotsu
AU - Shota Takeishi
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.36824
UR - https://arxiv.org/abs/2609.36824
AB - Ignoring unobserved heterogeneity in duration models biases parameter estimates and invalidates inference, but testing for it is non-regular: the null hypothesis lies on the boundary of the parameter space and some parameters are unidentified under the null. These features render standard asymptotic theory inapplicable. This paper develops an EM test for unobserved heterogeneity in censored Weibull duration models, building on the EM approach of Li, Chen, and Marriott (2009). The test statistic has an asymptotic null distribution equal to the square of max{0, N(0,1)}, hence critical values require neither simulation nor bootstrap, and the test accommodates covariate-dependent censoring of arbitrary form. Monte Carlo simulations compare the EM test with the likelihood ratio test (LRT) of Cho and White (2010), information matrix tests, and Lagrange multiplier tests. The EM test has empirical size close to the nominal level for sample sizes of 500 or more, where the LRT remains markedly conservative and the other tests over-reject. Its size-adjusted power is comparable to that of the LRT and higher than that of the other tests. Because size adjustment requires knowledge of the data-generating process and is unavailable in practice, the EM test attains higher power than the LRT in most designs as the tests would actually be applied. In an application to the Stanford Heart Transplant data, the EM test rejects homogeneity in every covariate specification, whereas the LRT's conclusion depends on a user-chosen set of admissible parameter values and on the specification.
ER -