From: Evaluating CEFR rater performance through the analysis of spoken learner corpora
(Dis)agreement between R1 and R2 | Range | Accuracy | Fluency | Phonological control | Coherence | Holistic score | Mean |
---|---|---|---|---|---|---|---|
Exact agreement 0 | 38 | 34 | 46 | 35 | 34 | 44 | 39 |
Adjacent agreement 1 | 51 | 53 | 42 | 50 | 52 | 46 | 49 |
Discrepancy by two sub-bands 2 | 8 | 10 | 9 | 15 | 11 | 9 | 10 |
Discrepancy by three sub-bands 3 | 3 | 3 | 3 | 0 | 3 | 1 | 2 |
Total | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
Spearman’s rho | 0.882 | 0.887 | 0.851 | 0.877 | 0.864 | 0.893 | 0.875 |
Jagged score | 21 | 23 | 25 | 31 | 25 | 22 | 25 |