The impact of misspecifying cross-classified random effects models in cross-sectional and longitudinal multilevel data: a Monte Carlo study

Date

2009-05-15

Authors

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Cross-classified random effects models (CCREMs) are used in the analyses of cross-sectional and longitudinal multilevel data that are not strictly hierarchical. Because of the complexity of this technique, many researchers simply ignore the cross-classified structures of their data and use hierarchical linear models. The study simulated crosssectional and longitudinal multilevel data with cross-classified structures and examined the impact of misspecifying CCREMs on parameter and standard error estimates in these data. The dissertation consists of two studies. Study One examines cross-sectional multilevel data and Study Two examines longitudinal multilevel data. In Study One, three-level cross-classified data were generated. Two random factors were crossed at either the top level or the intermediate level. It was found that ignoring a crossed random factor causes the variance of the remaining crossed factor and the adjacent levels to be overestimated. The fixed effects themselves are unbiased; however, the standard errors associated with the fixed effects are biased. When the ignored crossed factor is at the top level, the standard error of the intercept is underestimated whereas the standard error of the regression coefficients associated with the covariate of the intermediate level and the remaining crossed factor are overestimated. When the ignored crossed factor is at the intermediate level, only the standard error of the regression coefficients associated with the covariate of the bottom level is overestimated. In Study Two, longitudinal multilevel data were generated mirroring studies in which students are measured repeatedly and change schools over time. It was found that when the school level is modeled hierarchically above the student level rather than as a crossed factor, part of the variance at the school level is added to the student level, causing underestimation of the school-level variance and overestimation of the studentlevel variance and covariance. The standard errors of the intercept and the regression coefficients associated with the school-level predictors are underestimated, which may cause spurious significance for results. The findings of the dissertation enhanced our understanding of the functioning of CCREMs in both cross-sectional and longitudinal multilevel data. The findings can help researchers to determine when CCREMs should be used and to interpret their results with caution when they misspecify CCREMs.

Description

Citation