Skip to main content

Advertisement

Table 4 Inter-rater agreement between Raters 1 and 2 in assessing speaking on the CEFR scales

From: Evaluating CEFR rater performance through the analysis of spoken learner corpora

(Dis)agreement between R1 and R2 Range Accuracy Fluency Phonological control Coherence Holistic score Mean
Exact agreement 0 38 34 46 35 34 44 39
Adjacent agreement 1 51 53 42 50 52 46 49
Discrepancy by two sub-bands 2 8 10 9 15 11 9 10
Discrepancy by three sub-bands 3 3 3 3 0 3 1 2
Total 100 100 100 100 100 100 100
Spearman’s rho 0.882 0.887 0.851 0.877 0.864 0.893 0.875
Jagged score 21 23 25 31 25 22 25