Skip to main content

Table 4 Inter-rater agreement between Raters 1 and 2 in assessing speaking on the CEFR scales

From: Evaluating CEFR rater performance through the analysis of spoken learner corpora

(Dis)agreement between R1 and R2

Range

Accuracy

Fluency

Phonological control

Coherence

Holistic score

Mean

Exact agreement 0

38

34

46

35

34

44

39

Adjacent agreement 1

51

53

42

50

52

46

49

Discrepancy by two sub-bands 2

8

10

9

15

11

9

10

Discrepancy by three sub-bands 3

3

3

3

0

3

1

2

Total

100

100

100

100

100

100

100

Spearman’s rho

0.882

0.887

0.851

0.877

0.864

0.893

0.875

Jagged score

21

23

25

31

25

22

25