The analysis of marking reliability through the approach of gauge repeatability and reproducibility (GR&R) study: a case of English-speaking test

Language Testing in Asia

Table 3 The data from the case of three examiners marking 30 clips twice

Recording clips	Examiner 1			Examiner 2			Examiner 3			Clip Average
Recording clips	Trial 1	Trial 2	R	Trial 1	Trial 2	R	Trial 1	Trial 2	R	Clip Average
1	16	17	1	16	16	0	9	11	2	14.17
2	11	12	1	14	12	2	8	8	0	10.83
3	16	16	0	14	15	1	12	12	0	14.17
4	13	15	2	14	14	0	11	12	1	13.17
5	15	15	0	18	14	4	12	14	2	14.67
6	13	14	1	14	11	3	7	10	3	11.50
7	16	18	2	16	18	2	9	13	4	15.00
8	12	11	1	13	13	0	6	10	4	10.83
9	16	15	1	14	10	4	10	13	3	13.00
10	16	15	1	10	11	1	8	9	1	11.50
11	15	11	4	12	10	2	8	9	1	10.83
12	11	12	1	8	10	2	7	7	0	9.17
13	15	15	0	6	11	5	5	7	2	9.83
14	13	12	1	10	11	1	6	5	1	9.50
15	15	15	0	15	13	2	15	16	1	14.83
16	14	15	1	12	12	0	9	9	0	11.83
17	16	14	2	6	13	7	14	14	0	12.83
18	13	12	1	11	12	1	6	6	0	10.00
19	13	12	1	6	11	5	9	13	4	10.67
20	17	17	0	16	17	1	15	16	1	16.33
21	13	11	2	10	9	1	5	6	1	9.00
22	16	16	0	12	12	0	12	11	1	13.17
23	16	18	2	14	16	2	15	15	0	15.67
24	13	12	1	8	8	0	6	6	0	8.83
25	15	16	1	14	14	0	9	14	5	13.67
26	12	11	1	11	11	0	5	9	4	9.83
27	17	17	0	18	16	2	14	17	3	16.50
28	15	13	2	14	14	0	11	14	3	13.50
29	12	10	2	8	12	4	10	12	2	10.67
30	15	14	1	8	15	7	10	15	5	12.83
Column average	14.183		1.10	12.383		1.97	10.267		1.80