Table 1 Research design
From: Developing and evaluating a dynamic assessment of listening comprehension in an EFL context
Research questions | Participants | Data | Analysis |
|---|---|---|---|
a. To what extent do estimates of test-taker ability and item difficulty vary across static and dynamic listening? | 60 test-takers who sat for both tests: Static and dynamic | a. A one-hour achievement static LC test | Quantitative analysis Analysis of test scores: |
b. Is there any bias interaction of rater by test type, rater by test-taker and test-taker by test type? | 11 raters who scored both tests (in the static test, raters are referred to as “raters”; while they are referred to as “mediators” in the dynamic test. | Forty items (5 tasks with 8 items each) Item type: Gap filling, MC, information transfer, true/false statements and following instructions | - FACETS: Ability estimates and item difficulty |
c. What are the mediators’ and test-takers’ perceptions of both modes of assessment? | b. A forty-five minute progress dynamic test | - Bias analyses of rater by test type, rater by test-taker and test-taker by test type | |
Fourteen items Pre-testing phase (10 minutes): Wh, guessing, matching | |||
While-testing phase (20 minutes): Wh (x2), summarizing (x2), MC, true/false, guessing | Qualitative analysis | ||
Post-testing phase (15 minutes): MC, picture reordering, summarizing, making inference | Analysis of the interview data | ||
c. Interview with raters and test-takers in both test modes | - Interview analysis: Perceptions of and attitudes towards the dynamic test |