150 / 2018-02-16 14:19:06

Investigating the consequential validity of the Test of English for Academic Purposes (TEAP) Test in the Japanese high school context

washback,consequential validity,TEAP,EIKEN,Japan,High school,university entrance exam

全体主题 > Standards in validating language assessments

University entrance exams in Japan have been widely criticized for their design and content, and their lack of alignment with the national course of study at Japanese high schools. To address these issues, a new four skills test, the Test of English for Academic Purposes (TEAP), has been collaboratively designed by Japanese and U.K. institutions.
Adopting the socio-cognitive model as the framework for test validation (Weir, 2005), studies have provided a priori evidence for the test's construct, cognitive and scoring validities (Nakatsuhara, 2014; Taylor, 2014; Weir, 2014). However, since the test's launch in 2015, no validation studies have been conducted to assess its consequential validity. Such posteriori validity evidence is essential because the TEAP test is explicitly designed to promote positive washback to teaching and learning at high schools (Green, 2014). The aim of the present study is to investigate whether this is being achieved in terms of student's learning.
To investigate washback on learning, we conducted a comprehensive online survey with 50 student participants at a Japanese high school in Tokyo and assessed the content and methods of their English study. We then introduced the TEAP test and participants prepared individually (i.e., in a non-instructed context) for approximately six months. Participants then took the TEAP test and completed a second survey, which allowed us to compare learning behavior prior to and after the introduction of the test. Interviews were conducted with five participants before and after the test to investigate language learner behavior and perceptions.
The survey and interview data revealed that learners were motivated to study and perceived the test as challenging but achievable and, in many cases, important for their future. Although these conditions are conducive for washback (Green, 2007), we found that TEAP had less washback than anticipated, especially on productive skills. This was mainly due to the innovative approach to language teaching undertaken at the school, which focused on developing students' productive skills. Consequently, and contrary to expectations, participants were strongest at speaking and writing according to scores from the TEAP test and they reported focusing less on these skills during their test preparation. Nevertheless, the data revealed numerous signs of positive washback from the test on students' learning behavior.
In sum, this study provides evidence for the consequential validity of the TEAP test and highlights the importance of the micro-context (i.e., the school) in determining the intensity and nature of washback.