Investigating the construct validity of the reading comprehension section of the College English Test in China : a structural equation modeling approach

The College English Test (CET) in China is the largest language test in the world. The number of CET test-takers has steadily increased from 100,000 for its first administration in 1987 to 13 million in 2006. CET scores are used to draw inferences about the test-takers’ English as a foreign language proficiency as well as their specific skills in listening, speaking, reading, and writing. To justify the inferences drawn from test scores, evidence from a variety of sources should be constantly collected (Cronbach & Meehl, 1955; Messick, 1992; Chapelle, 1998; Bachman, 2000; Weir, 2005). Despite the large-scale and high-stakes nature of the CET and the importance of test validation, studies on the quality of the CET are scarce. This study aims to examine the construct validity of the reading comprehension section of the CET by modeling the internal relationships between test-takers’ scores on the CET reading section and their underlying reading abilities. Six components have been chosen as observed variables of the latent variable of reading ability, namely, word recognition efficiency, working memory, semantic knowledge, syntactic knowledge, discourse knowledge, and metacognitive reading skills. A pseudowords identification task programmed by the DMDX computer software, a revised version of Daneman & Carpenter’s (1980) sentence reading span working memory test, Meara & Milton’s (2002) Yes/No vocabulary tests, the syntactic test used in Shiotsu & Weir’s (2007) study, Abeywickrama’s (2007) discourse knowledge test, and a revised version of Phakiti’s (2008) strategy use questionnaire were utilized to measure these six observed variables. A total of 181 Chinese undergraduates participated in the study. With a baseline confirmatory factor model of reading ability and the CET scores, a structural model was analyzed. The results indicated that the path from reading ability to test performance was .75 and the squared regression coefficient of test performance was .56, which implied that participants’ test performance was strongly underlined by their actual reading ability. Therefore, the scores on the CET reading section are largely justifiable for use in drawing inferences about participants’ reading ability. Implications for validation research and reading instruction were also explored.