179 / 2018-03-29 20:32:46

Linking the TEM-8 Summary Writing to the China's Standards of English

Summary writing; CSE; TEM-8; linking

全体主题 > Standards in validating language assessments

The establishment of China's Standards of English (CSE) smoothes the way for the recognition of diverse language qualifications across the country, and points out the necessity of improving the consistency, coherence and efficiency of China's English language education practices at all stages (Jin, 2017). Meanwhile, integrated tasks have become a topic of heated debates in language assessment. The past decade has witnessed the advent of integrated writing tasks in large-scale language tests in China, for example, Test for English Majors Band-8 (TEM-8), with inadequate validity evidence though (Zhang and Zeng, 2009). Aligning tests that incorporate integrated writing tasks to the CSE can help test users and decision makers interpret test scores in terms of the CSE levels, explore the underlying construct, and enhance test use.
This study thus aims to link TEM-8 to part of the descriptors assigned at CSE levels following the methodology recommended in the Council of Europe's Manual (2003): familiarization, specification, standard setting, and validation. Using a mixed-methods approach, the study was conducted to determine which level on the CSE the TEM-8 summary writing measures and how to validate the linkage. During the initial phase, the study aligned the TEM-8 summary writing with the CSE through content analysis, including selecting the CSE sub-scales appropriate for the TEM-8 summary writing. In the standardization phase, we adopted an examinee-centered method, i.e. the correlation of TEM-8 results with teacher judgments on the same group of examinees was used as an external criterion measure. With regard to empirical validation phase, forty-five fourth year undergraduate students specializing in English from a comprehensive university in Eastern China were firstly invited to finish the TEM-8 summary writing of the year 2018 and within two weeks, they were invited to complete one out of three forms of equated questionnaires, the bulk of which was comprised of CSE descriptors; whereas, two teachers working with those examinees were asked to assess them using the global scales of Written Production (Kecker and Eckes, 2010). The data collected through the questionnaire survey and teacher judgments were analyzed using such software as Winsteps 3.91.0 (Linacre, 2013) and IBM SPSS 21.0 (IBM, 2012).
On the whole, the linkage of the TEM-8 summary writing with the level 7 of the relevant CSE scales was confirmed. Findings yielded from this study have further implications not only for future CSE linking studies, but more importantly, for research focusing on the construct of integrated writing tasks.