A Monte Carlo Investigation of Three Different Estimation Methods in Multilevel Structural Equation Modeling Under Conditions of Data Nonnormality and Varied Sample Sizes

The purpose of the study was to examine multilevel regression models in the context of multilevel structural equation modeling (SEM) in terms of accuracy of parameter estimates, standard errors, and fit indices in normal and nonnormal data under various sample sizes and differing estimators (maximum likelihood, generalized least squares, and weighted least squares). The finding revealed that the regression coefficients were estimated with little to no bias among the study design conditions investigated. However, the number of clusters (group level) appeared to have the greatest impact on bias among the parameter estimate standard errors at both level-1 and level-2. In small sample sizes (i.e., 300 and 500) the standard errors were negatively biased. When the number of clusters was 30 and cluster size was held at 10, the level-1 standard errors were biased downward by approximately 20% for the maximum likelihood and generalized least squares estimators, while the weighted least squares estimator produced level-1 standard errors that were negatively biased by 25%. Regarding the level-2 standard errors, the level-2 standard errors were biased downward by approximately 24% in nonnormal data, especially when the correlation among variables was fixed at .5 and kurtosis was held constant at 7. In this same setting (30 clusters with cluster size fixed at 10), when kurtosis was fixed at 4 and the correlation among variables was held at .7, both the maximum likelihood and generalized least squares estimators resulted in standard errors that were biased downward by approximately 11%. Regarding fit statistics, negative bias was noted among each of the fit indices investigated when the number of clusters ranged from 30 to 50 and cluster size was fixed at 10. The least amount of bias was associated with the maximum likelihood estimator in each of the data normality conditions examined. As sample size increased, bias decreased to near zero when the sample size was equal to or greater than 1,500 with similar results reported across estimation methods. Recommendations for the substantive researcher are presented and areas of future research are presented.