167 / 2018-03-27 10:20:22

Developing data-driven standards for adapting source texts of academic reading tests

data-driven standards,adapting source texts,validation,methodological implications

全体主题 > Principles and methodology of developing standards for language assessment

While source texts must be adapted for inclusion in reading tests, the adaptation practice has depended primarily on the intuition and experience of item writers, leading to considerable variation in text complexity of adapted texts (Green & Hawkey, 2012). In this connection, this study reports on a case study using a data-driven approach (Jin & Lu, 2017) to develop standards for adapting source texts of academic reading tests in the Chinese context. The case study included three stages. The first stage was a needs analysis. A group of item writers were invited to participate in a semi-structured interview to elicit their perceptions, finding that lexical coverage and syntactic complexity were the two major concerns in which the item writers needed the most support when adapting academic texts. Based on the needs analysis, the second stage established specific standards for item writers in conducting lexical and syntactic modifications, including linguistic annotations and reference ranges. The third stage carried out think-aloud protocols of another group of item writers in the process of adapting academic texts while employing the standards established. Overall, this case study has provided empirical evidence in the validation of developing data-driven standards for adapting source texts. Methodological implications are also given to create standards for adapting other source materials in language tests.

References
Green, A., & Hawkey, R. (2012). Re-fitting for a different purpose: A case study of item writer practices in adapting source texts for a test of academic reading. Language Testing, 29, 109–129.
Jin, T., & Lu, X. (2017). A data-driven approach to text adaptation in teaching material preparation: Design, implementation, and teacher professional development. TESOL Quarterly. Advance online publication. https://doi.org/10.1002/tesq.434