246 / 2018-04-15 18:28:13

The reconceptualization and redesign of assessment scoring rubrics for university-level ESL courses

writing accessment; analytic scoring rubrics;the Standards

全体主题 > Principles and methodology of developing standards for language assessment

The English Language Centre (ELC) at the Hong Kong Polytechnic University delivers English language enhancement subjects to over 6000 undergraduate students each academic year. Students are assessed continuously through in-class and out-of-class written and spoken assessment tasks supplemented with online independent learning activities. Teachers assess students’ assignments using scoring rubrics within an assessment system comprising online standardisation, double-marking and grade moderation to ensure fairness. The rubrics are designed to be used by teachers and are not given to students, though teachers may show them to students so that they are aware of the criteria and standards. From 2018-19, the ELC will use the scoring rubrics with students as a teaching and learning as well as an assessment tool. This has required a reconceptualization and redesign of the rubrics.

This revision of the rubrics has been done through a genre-specific process in which initial drafts of one writing (a for-against essay) and one speaking (an academic presentation) rubric were drawn up by the Centre’s Assessment Team. These drafts were then used as models for subject coordinators to work from to develop rubrics applying to the genres assessed in their subjects. The new draft rubrics were trialed first by teachers in a standardisation session before being further revised for use with students in small cohorts. In each part of the process, observations were made and both qualitative data (e.g. teachers’ approaches, opinions and thinking processes) and quantitative data (analytic ratings on the different criteria for use in standard setting) were recorded to both facilitate the revision but also to enable the Team to explore a number of issues related to the measurement of ESL writing and speaking ability at undergraduate level:

1. How to design learner-friendly rubrics that adhere to existing standards which have been internalized by teachers over a long period of time.
2. How to ensure that the standards are adhered to once the revised rubrics go into use.
3. How the revised rubrics can help learners better understand what is required of them in writing and speaking assignments.

In this presentation, the speakers will describe the rubric revision process and share their experiences of working with their colleagues on this complex and at times contentious process. Their findings and experiences will be of interest to university language teachers and assessment designers and administrators, particularly in light of moves towards increased learner participation in the assessment process.