Skip to main content
  • Research Article
  • Open access
  • Published:

When Raters Talk, Rubrics Fall Silent

Abstract

The research reported here suggests that raters, when involved in writing assessment, are more concerned with their own criteria to set a basis for their judgment rather than the standards provided by scale descriptors. This study sampled think aloud of eight raters who scored 15 essays in accord with Test of Written English (TWE) holistic scoring guide. Verbal report data indicated that just less than five percent of the statements made by the raters are related to the issues assessed in TWE. These findings background the utility of holistic rating scale descriptors, foregrounding the raters' descriptors-independent judgments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Masoumeh Ahmadi Shirazi.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Shirazi, M.A. When Raters Talk, Rubrics Fall Silent. Language Testing in Asia 2, 123 (2012). https://doi.org/10.1186/2229-0443-2-4-123

Download citation

  • Published:

  • DOI: https://doi.org/10.1186/2229-0443-2-4-123

Keywords