Table 7 A summary of the partial validity argument for the OSWT with the assumptions, backing, and the judgment of the degree of support
From: Validity of score interpretations on an online English placement writing test
Assumption | Backing | Degree of support |
|---|---|---|
1. Test-takers hold self-confidence in CBW tests regardless of L2 writing ability. | 1-1. A two-way group-independence chi-square analysis revealed there was no significant difference in test-takers’ self-confidence in CBW tests between the higher- and lower-level groups with a zero effect size (χ2 = 0.00, df = 1, p = .996, ω = .00), meaning the two groups held an identical level of self-confidence in CBW tests. The thematic analysis indicated a majority of the test-takers from the two groups held self-confidence for similar reasons (e.g., convenience in writing). A small number of the test-takers did not hold such self-confidence for almost the same reasons (e.g., unfamiliarity with CBW). | Fully supported |
1-2. An independent-samples t-test showed there was no significant difference between the higher- and lower-level groups in test-takers’ preferences for CBW tests, t = 1.17, p = .24, df = 95 with a small-sized effect (d = 0.24), which suggests that test-takers tended to prefer CBW tests regardless of L2 writing ability. | Fully supported | |
1.2. OSWT performance is significantly different depending on test-takers’ L2 writing ability and preferences for CBW tests regardless of L2 writing ability. | 2-1. An independent-samples t-test showed there was a significant difference between the higher- and lower-level groups in the OSWT scores with a medium-sized effect (t = 2.61, p = .01, df = 95, d = 0.53), which indicates the OWST is a measure of L2 writing ability that could differentiate the test-takers according to the levels of L2 writing ability. | Fully supported |
2-2. A simple linear regression revealed that, in the higher-level group, test-takers’ preferences for CBW tests were a significant predictor of the OSWT scores, t = 2.954, p = .005. However, test-takers’ preferences did not significantly contribute to OSWT scores in the lower-level group, t = −0.362, p = .719. The finding means that test-takers’ preferences significantly contributed to OWST scores only in the higher-level group. | Partially supported |