Table 8 Distribution of (dis)agreement between Raters 1 and 2 across CEFR levels
From: Evaluating CEFR rater performance through the analysis of spoken learner corpora
Rater 1's (dis)agreement with Rater 2 across CEFR levels | Rater 2's (dis)agreement with Rater 1 across CEFR levels | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
CEFR Levels | Samples | Discrepancy (sub-bands) | Total | CEFR Levels | Samples | Discrepancy (sub-bands) | Total | ||||
0 | 1 | 2 or 3 | 0 | 1 | 2 or 3 | ||||||
B1 | 13 | 3 | 9 | 1 | 13 | B1 | 11 | 3 | 7 | 1 | 11 |
23% | 69% | 8% | 100% | 27% | 64% | 9% | 100% | ||||
B2 | 37 | 14 | 21 | 2 | 37 | B2 | 49 | 14 | 28 | 7 | 49 |
38% | 57% | 5% | 100% | 29% | 57% | 14% | 100% | ||||
C1 | 48 | 25 | 16 | 7 | 48 | C1 | 38 | 25 | 11 | 2 | 38 |
52% | 33% | 15% | 100% | 66% | 29% | 5% | 100% | ||||
C2 | 2 | 2 | 0 | 0 | 2 | C2 | 2 | 2 | 0 | 0 | 2 |
100% | 0% | 0% | 100% | 100% | 0% | 0% | 100% | ||||