Rater reliability and score discrepancy under holistic and analytic scoring of second language writing

Language Testing in Asia

Table 3 Reliability for the components under analytic scoring

Decision to be made	No. of raters	Task	Grammar	Mechanics	Vocabulary	Structure
Relative	1	0.77	0.73	0.72	0.75	0.70
	2	0.87	0.85	0.84	0.85	0.83
	3	0.91	0.89	0.88	0.90	0.88
	4	0.93	0.92	0.91	0.92	0.90
Absolute	1	0.72	0.72	0.70	0.73	0.69
	2	0.84	0.83	0.82	0.84	0.82
	3	0.89	0.88	0.87	0.89	0.87
	4	0.91	0.91	0.90	0.91	0.90