Skip to main content

Table 1 Previous studies comparing Criterion®-generated scores before and after L2 instructions

From: Validity evidence of Criterion® for assessing L2 writing proficiency in a Japanese university context

Study

Context; variable

Instruction length (Nested?)

Prompts of essays compared

Use of Criterion® holistic scores

Analysis of writing quality (Use of Criterion® and/or other measures)

Statistical analysis (time points)

Results of comparison between pretest and posttest

Ohta (2008a) [43]

University in Japan; instruction

16 weeks (yes)

Different

Yes

Yes (Criterion®)

t-test (2)

Higher scores & longer essays for students with TOEFL ITP® scores of 500 or above

Ohta (2008b) [25]

University in Japan; instruction

16 weeks (yes)

Different

No

Yes (Criterion® + other measures)

t-test (2)

Longer essays; better organization

Hosogoshi et al. (2012) [74a]

University in Japan; peer feedback + revision

2 weeks (yes)

Same: first vs. revised essays

Yes

Yes (Criterion® + other measures)

ANOVA (2)

Higher scores; fewer mechanical errors; longer essays; more syntactic complexity, better organization

Tajino et al. (2011) [97]

University in Japan; instruction

2 weeks (unknown)

Different

Yes

No

t-test (2)

Higher scores

Li et al. (2015) [70]

University in the U.S.; instruction

Unknown (no)

Same: first vs. revised essays

No

Yes (Criterion®)

t-test (2)

Fewer errors (with some variations among prompts and teachers)

Current study (Study 2) [81]

University in Japan; instruction

28 weeks (yes)

Different

Yes

Yes (Criterion®)

Multilevel modeling (3)

 
  1. Note. Nested? = Yes, if there is a nested structure in the data in terms of the classes or grades to which the participants belonged. [ ] = N. aThey selected and analyzed those whose holistic scores increased by one or more point out of six