Skip to main content
Article
"Rassling the hog": the influence of correlated item error on internal consistency, classical reliability, and congeneric reliability
Applied Psychological Measurement (2005)
  • Joseph F. Lucke, State University of New York at Buffalo
Abstract

The properties of internal consistency ($\alpha$), classical reliability ($\rho$), and congeneric reliability ($\omega$) for a composite test with correlated item error were analytically investigated. Possible sources of correlated item error are contextual effects, item bundles, and item models that ignore additional attributes or higher-order attributes. The relation between reliability and internal consistency is determined by the deviance from true-score equivalence. Reliability (classical or congeneric) is internal consistency plus the relative deviance from true-score equivalence. The influence of correlated item error on $\alpha$, $\rho$, and $\omega$ is conveyed strictly through the total item error covariance. As the total item error covariance increases, $\rho$ and $\omega$ decrease but $\alpha$ increases. The necessary and sufficient condition for $\alpha$ to be a lower bound to $\rho$ and to $\omega$ is that the total item error covariance not exceed the deviance from true-score equivalence. Coefficient $\alpha$ will uniformly exceed $\rho$ or $\omega$ in true-score equivalent tests with positively correlated item error. The factor analytic item model with specific factors alters some of these relationships. Correlated item error in this model can cause $\alpha$ to exceed $\omega$ but not $\rho$ or to exceed both $\omega$ and $\rho$. Positively correlated specific factors can further contribute to $\alpha$'s exceeding $\omega$. The compound symmetric item error covariance matrix is sufficient to study effects of correlated item error. Contrary to the standard practice in test development, $\alpha$ cannot be assumed to be a lower bound to $\rho$ or $\omega$. For congeneric tests, a better approach is to fit an item model and estimate $\omega$ directly.

Keywords
  • classical reliability; coefficient alpha; coefficient omega; compound symmetry; congeneric reliability; correlated item error; deviance from true-score equivalence; internal consistency; reliability; total item error covariance; true-score equivalence; cronbach;
Publication Date
2005
Citation Information
Joseph F. Lucke. ""Rassling the hog": the influence of correlated item error on internal consistency, classical reliability, and congeneric reliability" Applied Psychological Measurement Vol. 29 Iss. 2 (2005)
Available at: http://works.bepress.com/joseph_lucke/13/