A taxonomy of reliability indices for cluster-level constructs measured by multiple individual-level indicators.
- 1University of Amsterdam, Amsterdam, The Netherlands.
Abstract
Generalizability theory (GT) provides a variety of intraclass correlation coefficients (ICCs) to quantify reliability across different conditions of measurement-for example, scale and interrater reliability quantify generalization across items and raters, respectively. In a multilevel modelling (MLM) context, an ICC is often used to quantify the proportion of a subject-level variable's variance that is attributable to cluster-level differences (e.g., between schools or neighbourhoods). When data from cluster members are their ratings of a group-level characteristic, the ICC is an inter-rater reliability coefficient, interpreted as the degree to which observations of a cluster generalize across members of that cluster. Structural equation modelling (SEM) is becoming a popular framework for modelling GT, and the 'omega' indices of composite reliability-which are functions of SEM parameters-are special cases of GT (i.e., how measurements of a construct generalize across indicators). The merger of MLM and SEM (i.e., ML-SEM) provides a unique opportunity to explore how well measurements of cluster-level constructs generalize across multiple measurement conditions (indicators, cluster members, or both). A taxonomy of ICCs is explicated to appropriately quantify reliability across aspects of a procedure for measuring cluster-level constructs, adding value to the limited ICCs currently considered in ML-SEM applications.
Related
- Structural taxonomy and complex irreducible representations of non-abelian groups of order p^4
- Mathematical Programming in Machine Learning and Artificial Intelligence: A Unified Taxonomy of Models and Applications
- Operational Regimes in Non-Convex Optimization: A Multiplier-Based Taxonomy
- Scientific Data Analysis for Class-Informatics in Computational Taxonomy
- Penguin data reanalyzed via Computational Taxonomy
BibTeX
@article{jorgensen2026taxonomy,
title = {{A taxonomy of reliability indices for cluster-level constructs measured by multiple individual-level indicators.}},
author = {Terrence D Jorgensen and Suzanne Jak},
journal = {The British journal of mathematical and statistical psychology},
year = {2026},
month = oct,
doi = {10.1111/bmsp.70074},
url = {https://europepmc.org/article/MED/42872674},
note = {CC BY},
}
RIS
TY - JOUR
TI - A taxonomy of reliability indices for cluster-level constructs measured by multiple individual-level indicators.
AU - Terrence D Jorgensen
AU - Suzanne Jak
PY - 2026
DA - 2026/10/14
JO - The British journal of mathematical and statistical psychology
DO - 10.1111/bmsp.70074
UR - https://europepmc.org/article/MED/42872674
AB - Generalizability theory (GT) provides a variety of intraclass correlation coefficients (ICCs) to quantify reliability across different conditions of measurement-for example, scale and interrater reliability quantify generalization across items and raters, respectively. In a multilevel modelling (MLM) context, an ICC is often used to quantify the proportion of a subject-level variable's variance that is attributable to cluster-level differences (e.g., between schools or neighbourhoods). When data from cluster members are their ratings of a group-level characteristic, the ICC is an inter-rater reliability coefficient, interpreted as the degree to which observations of a cluster generalize across members of that cluster. Structural equation modelling (SEM) is becoming a popular framework for modelling GT, and the 'omega' indices of composite reliability-which are functions of SEM parameters-are special cases of GT (i.e., how measurements of a construct generalize across indicators). The merger of MLM and SEM (i.e., ML-SEM) provides a unique opportunity to explore how well measurements of cluster-level constructs generalize across multiple measurement conditions (indicators, cluster members, or both). A taxonomy of ICCs is explicated to appropriately quantify reliability across aspects of a procedure for measuring cluster-level constructs, adding value to the limited ICCs currently considered in ML-SEM applications.
SN - 0007-1102
SN - 2044-8317
ER -