Language Testing in Asia

Table 2 Reliability as a function of number of raters

From: Rater reliability and score discrepancy under holistic and analytic scoring of second language writing

No. of raters	Holistic scoring		Analytic scoring
No. of raters	Relative	Absolute	Relative	Absolute
1	0.82	0.78	0.83	0.82
2	0.90	0.88	0.91	0.90
3	0.93	0.92	0.94	0.93
14	0.98	0.98	0.99	0.98

This is the condition in the present study.
Note. The last row with 14 raters shows the results for the condition in the present study

Back to article page