Skip to main content

Table 9 Multilevel model results of the five writing quality scores in Study 2

From: Validity evidence of Criterion® for assessing L2 writing proficiency in a Japanese university context

 

Error per 100 words

No. of words

No. of words per sentence

No. of transitional words per 100 words

No. of organization elements

 

Null + time + class

Null + time + class

Null + time + class

Null

Null

Fixed effects

Level 1 (n = 243)

Coefficient (SE)

Coefficient (SE)

Coefficient (SE)

Coefficient (SE)

Coefficient (SE)

 Intercept (γ00)—initial status

4.48*** (0.56)

208.65*** (11.72)

12.73*** (0.40)

3.65*** (0.12)

4.18*** (0.03)

 Time (γ10)—rate of change

−0.65 (0.39)

34.17*** (5.71)

2.08*** (0.43)

--

--

Level 2 (n = 81)

 Class (γ01)—initial status

1.11*** (0.26)

−8.32 (4.45)

−0.29 (0.16)

--

--

 Class (γ11)—rate of change

−0.28 (0.17)

−4.24 (2.16)

−0.44** (0.15)

--

--

Random effects

Level 1 (n = 243)

 Within-student variance (r)

4.45

962.55

3.31

1.25

0.33

Level 2 (n = 81)

 Between-student variance (u0)

7.56

2361.17

2.38

0.70

0.0002

 Between-student variance (u1)

2.64

253.08

1.67

--

--

 Chi-square (u 0 ; df)

246.09*** (79)

319.44*** (79)

150.75*** (79)

216.74*** (80)

71.86 (80)

 Chi-square (u1; df)

176.98*** (79)

123.59*** (79)

162.57*** (79)

--

--

 Intraclass correlation

--

--

--

0.36

0.0006

Reliability

 Intercept (β0)

0.67

0.75

0.46

0.63

0.00

 Time (β1)

0.54

0.35

0.50

--

--

Model fit

 Deviance (# of estimated parameters)

1193.68 (8)

2555.23 (8)

1132.04 (8)

821.30 (3)

422.00 (3)

 AIC

1209.68

2571.23

1148.04

827.30

428.00

  1. Note. **p < .01; ***p < .001. The design effect for each null model is 1.46, 2.04, 1.58, 1.72, and 1.00 (see Table 8 Note.) They all exceeded 1 and thus indicate the need to use multilevel models