The role of reading and writing in summarization as an integrated task

Abstract

In order to complete summarization tasks L2 learners need to demonstrate both reading and writing abilities. Thus, understanding the role of these abilities is necessary for interpreting performance on such tasks. This study examined the role of reading and writing in summarization tasks completed by 64 Chinese college students. It was found word- and sentence-level strategies were most commonly used for reading the source texts. Writing strategies played a more important role than reading in such tasks. General English proficiency was found to have exerted no significant effect on the use of both reading and writing strategies, and participants’ reading and writing abilities only made moderate contribution to their summarization performance respectively. The theoretical, empirical, practical, and pedagogical implications of the study were then discussed.

Background

Summarization has a natural appeal for communicative language testing because of its approximation to target language use (Cohen 1994). This meets the need of language testing for task authenticity, which looks at the degree of correspondence of the characteristics of a given language test task to the features of a TLU task^a (Bachman & Palmer 1996). The need for authenticity, as well as other merits that integrated assessment enjoys, gives rise to the use of such tasks that many would argue have recently been “reinstated and revitalized” (Yu 2013, p.110). As an integrated assessment, summarization has been used in high-stakes international language tests, such as the new TOEFL (Yu 2009). However, this type of task is much under researched in the field of language testing, especially those involving extended writing (Yu 2007). And, as different skills are generally involved in summarization and other integrated tasks, interpreting task performance raises questions about the role each skill plays as well as the interaction among them (Plakans 2009). To shed some light on these issues and also increase our understanding of how learners approach these tasks, the present study looks at the role of reading and writing in test takers’ completion of summarization tasks.

Literature review: reading, writing, and summarizing

In order to complete summarization tasks L2 learners need to demonstrate both their reading and writing skills. Studies suggest that summarizing is intimately linked to reading comprehension in that the writer must, first of all, make sense of the target text (Kintsch & van Dijk 1978). Because of this, it has been frequently used as a measure of reading comprehension for both research and testing purpose (e.g., Cohen 1994; Yu 2008). However, Taylor (1986) has argued that summarizing “requires certain written language skills which are apart from and may be more complex than mere reading skills” (p. 206). Studies have shown that instructing students in summarization not only enhances their writing ability (Taylor & Beach 1984), but can also facilitate learning (Brown et al. 1981).

The role of reading and writing in summarization has been explored by Cohen (1994) and Sarig (1993), who examined the processes and strategies students engage in reading and writing, and by Asención Delaney (2008) and Yu (2008), who examined the relationship between reading, writing, and summarizing. Cohen (1994) asked five Portuguese EFL students to summarize English texts. Test takers provided verbal report protocols which were analyzed for instances of cognitive processes involved in reading the source text and in writing the summary. It was found the most successful strategy user used at least six strategies to her benefit. With respect to reading the source text, she used effectively technical facilitation strategies (underlining discourse markers and words to look up, and circling pronominal referents), clarification and simplification strategies. With respect to writing, she used technical facilitation strategies (providing a detailed answer to include the main ideas) and metacgnitive monitoring.

Sarig (1993) examined the process of how an Israeli student of high English proficiency composed study-summaries. It was found when reading transforming was the most important strategy category for the student, followed by moderate linking, revising, and clarifying. When writing, revising was the most important activity, followed by low-intensity transforming, linking, and clarifying strategies. Composing a study-summary was found to be an interactive writing endeavor with considerably more work done on comprehension than on writing.

Asención Delaney (2008) explored the relationship between summarization and L2 reading and writing abilities. The reading and writing measures were a reading test (the Nelson-Denny Reading Test), and a writing test adapted from the one Khaldieh (2000) used. Pearson product–moment correlation analysis suggested there were low positive correlation coefficients between the writing measure and the summary writing (r = .20), and it was not significant. However, summary writing significantly related to the reading measure, though weakly (r = .28). The researcher concluded that the reading-to-write ability seems to be a unique construct weakly associated with reading for comprehension and disassociated from writing an essay without background reading support.

Yu (2008) examined the differential effects of the use of the two languages (English and Chinese) on summarization as a measure of reading comprehension. He found English and Chinese summarization together can only account for a small proportion of the variances in TOEFL reading comprehension. Similar to what was found in above-mentioned Asención Delaney (2008) study, TOEFL reading comprehension was the statistically significant predictor for both English and Chinese summarization performances, while the predictability of English writing was not significant.

Most of these studies, however, used summarization either as a reading measure (Cohen 1994; Yu 2008) or as a study tool (Sarig 1993). The only one (Asención Delaney 2008) that treated summarization as an integrated writing task explored the relation between reading, writing, and summarization with correlation analysis. Thus the current research literature are in need of more studies that specifically investigate what processes and strategies test takers employ in reading and writing, and what is the relation between these abilities and task performance in summarization as an integrated measure. Given this discrepancy, the present study examined the two important language abilities in summarization to improve our understanding of the construct the task assesses. The study was guided by the following three research questions:

• 1) What processes and strategies do test takers engage in reading the source text in summarization tasks, and what role does reading play in such tasks?

• 2) What processes and strategies do test takers engage for producing own text in summarization tasks, and what role does writing play?

• 3) What is the relationship between reading, writing, and summarization performance?

Methods

Participants and their language abilities

A sample of 64 EFL learners was drawn from an undergraduate program in a Chinese university. Generally speaking, the sample was at intermediate level of English proficiency according to their NMET (Chinese national matriculation English test) score (mean = 95.4 on a 0 – 150 scale). In order to meet the needs of statistical analyses that aimed at discovering the relationship between summarization performance and English reading and writing abilities, an English proficiency test, College English Test Band 4 (CET-4), was administered to participants. Scores in CET4 Reading and Writing (inter-rater reliability 0.635) were used as a measure of their English reading and writing respectively.

Summarization tasks

Two source texts were chosen for use in the summarization task. They (both were about 900 words) were taken from a college English textbook so that they were fit for testing the participants in terms of topics and difficulties. One text describes how a medical worker shares the last moment of a patient who has two impressively graceful hands. The other looks into the question of iron and how exercise affects its levels in the human body. An analytical scoring rubric was used to evaluate participants’ summary scripts (Appendix 1). The rubric contained four components addressing different aspects of summarization abilities: Main Idea Coverage, Integration, Language Use, and Source Use. The students were asked to complete the tasks under examination conditions. The task instruction stated students should read the text first, and then write an English summary for about 130 words without copying the source.

Procedure

This study proceeded in two sessions. The first session involved four participants who provided think-aloud protocols while summarizing. Immediately after each of the two summarization tests, they participated in a one-to-one interview session to talk about their test-taking process and strategies they used. All the think-aloud and interview sessions were audio-recorded upon permission, the data were then transcribed, segmented, and coded into strategies.

In order to improve coding validity and reliability, cautions were exercised to make sure that only those strategies that have appeared in existing literature were used to assign names to segments of think-aloud protocols, though in some cases, minor changes were made or new strategy names were given to fit the practical situation. In this regard, extensive review of related literature were conducted with special attention to strategy use in reading (e.g., Anderson et al. 1991; Cohen & Upton 2007; Plakans 2009), writing (e.g., Baker & Boonkit 2004), integrated reading/writing (e.g., Asención 2004; Plakans 2009; Yang 2009), and summarizing (e.g., Kim 2001; Kirkland & Saunders 1991). When coding finished, a second coder was asked to code 10% of each of the 8 protocols, the agreement percentage was respectably high for each protocol (above 0.81). Data were analyzed in NVivo7, a software package from Qualitative Research Software used for coding, searching data, and modeling.

Based on the analysis of the think-aloud and interview data, a questionnaire with 5-point Likert scale items was developed to explore the use of various reading and writing strategies in summarization tasks (Additional file 1). This questionnaire was distributed to the remaining 60 participants in session two immediately after they have finished each of the two summarization tests. Students’ summary scripts were scored by three raters then averaged to yield a composite score for each participant. Analysis showed the inter-rater reliability estimates of the ratings were fairly high for both summarization tasks (in one task α = .804, and in the other α = .802).

Results

Reading and writing strategies used in summarization tasks

A. Think-aloud data

a. Reading and writing process

Analyses of participants’ think-aloud and interview data showed two stages appeared along the process of the read-to-write summarization task: first reading, then, writing (see Table 1). Under these two general processes, seven stages of the test taking process were identified based on participants’ think-aloud and interview data. Each of them was assigned with names representing the purpose or goal of a particular stage. The reading process subsumes the activity of ‘Assess task’, ‘Reflect on reading topic’ and ‘Read text’. Participants used a variety of strategies in reading, the source text in particular, the process of which was predominantly linear because most of the time participants moved from one sentence to the next. However, they were also observed to occasionally sum up what they have read, be it a portion or a paragraph, which might be influenced by the task goal. After reading was completed, participants started a writing process that subsumed the activities of ‘Work out text thesis and major ideas’, ‘Put pen to paper’ and ‘Assess task fulfillment’. Throughout this phase, participants refer to the text now and then which showed heavy reliance on source.

Table 1 Type and count of strategies used in summarization process

As Table 1 shows, altogether 70 strategies were identified in the summarization process, they were operating along the stages of the test taking process to help achieve the purpose or goal of a particular stage. Overall the 70 strategies occurred 1705 times. Among them 36 were used for the purpose of reading and 34 for writing. Altogether the reading strategies occurred 897 times while the writing 808 times. 30 (42.3%) strategies were employed for the purpose of reading the source text, making it the largest category. The second largest strategy category was ‘Put pen to paper’ (21, 29.6%). These two categories also topped the list in terms of strategy count. The occurrence of ‘Read text’ was 827 (48.5%) and that for ‘Put pen to paper’ was 514 (30.1%). Table 9 in Appendix 2 sums up participants’ strategy use for the two tasks with information about strategy description, category (to which it belonged), source (where it came from) and frequency counts (the number of times a certain strategy occurred).

b. Use of reading and writing strategies

The most frequently used reading strategies were identify and skip unknown word, reprocess information to clarify meaning, and reread for clarification, all of which occurred 100 times in the summarizing process of the two tasks. The strategies used only once for the purpose of reading were check inference, and reread instruction. Check inference was done when participants were not sure about their inference making. This did not happen frequently, as in most cases, participants read slowly which precluded the needs to check the inferences they might have made.

The writing strategy used most frequently by participants was plan content from source which occurred 127 times during the entire test taking process. The least employed writing strategies were reorganize source pattern, and edit for punctuation, both of which were employed only once.

B. Questionnaire survey

Examining the responses to the questionnaire items helped us to see the patterns of reading and writing strategies in the summarization tasks completed by a larger sample. Participants’ ratings of each of the questionnaire items were averaged to produce a composite score. Understanding of the average scores was informed by the SILL (Strategy Inventory for Language Learning) profile of results developed by Oxford (1990). Based on this profile, a key (Table 2) was constructed to aid interpretation of item ratings.

Table 2 Key to understanding average scores of questionnaire items

a. Measurement of frequency of reading strategy

The descriptive statistics of reading strategy use in the questionnaire were presented in Table 10 in Appendix 2, which shows that participants tend to agree on the use of various reading strategies. Mean rating of the reading strategies hardly reached the lower end of the high level in the Key shown in Table 2 which indicated participants often used these strategies in summarization. Strategy 42 (use phonological cues) and 44 (note and analyze sentence structure) received the lowest rating, and participants used these strategies very occasionally according to the key.

Strategy 21 (read title) was rated the highest, which might be the result of mismatch between the item intended inquiry and participants’ perceived meaning, who might have considered it more as a Yes or No item than a frequency item. Apart from this one, strategy number 40 (use context clues to interpret meaning) and strategy 41 (use context clues to predict meaning) were rated the highest.

b. Measurement of frequency of writing strategy

The full results of writing strategy use in the questionnaire were presented in Appendix 2 (Table 11). Mean rating of the writing strategies in summarization suggested overall participants often or sometimes used these strategies while summarizing source texts according to the Key. The highest rated variable was strategy 48 (identify topic sentence of paragraph) which was the most efficient strategy considered by participants in summary writing. Strategy 53, 57, 64, 67, 74 and 75 were rated below the middle point of 3 across tasks, with strategy 75 (use phonological cues) being the least applied strategy, the occurrence of which was at the lower end of medium level (sometimes used) according to the key.

Relationship between strategy use, summarization performance, and English proficiency

The statistical analyses exploring the relationship between the use of reading and writing strategies and other variables were performed in two steps. In the first step, the grouping variable was participants’ summarization performance, and in the second participants’ English proficiency as measured by CET4 was entered as the grouping variable.

A. Statistics by success of summarization performance

For this analysis, the 60 students were put into two groups of equal size according to their summary scores (t = −10.875, df = 58, p = .000): higher score summarizers (n = 30) and lower scorers (n = 30). The effects of grouping on the use of summarization reading and writing strategies were modeled with MANOVA. Table 3 presents the distributions of the dependent variables indicating all the variables were normally distributed (i.e., skewness and kurtosis statistics were near zero).

Table 3 Distributions for reading and writing strategy use variables

For this MANOVA analysis, Box’s Test of Equality of Covariance Matrices demonstrated that the data had homogeneity of variance. Levene’s test of equality of error variances also indicated that the homogeneity of variance was not violated in the dataset. The multivariate tests of significance indicated that there was a statistically significant multivariate effect for success levels (F = 4.118, p = .021, η² = .126) though its contribution to the model was very modest (Pillai’s Trace = .126, Wilks Lambda = .874).

The fact that the summarizing performance only contributed moderately to the model might be due to the reality there were factors other than these strategies (e.g., language ability, test method effects and error of measurement) that could be used to explain the test score (see Bachman 1990). Table 4 presents the significance of F test for the grouping variable by summarization performance.

Table 4 MANOVA results for success levels by summarization performance

In this model, the grouping variable (i.e., summarizing performance) did not significantly affect reading strategies, but did so for writing strategies. The value of partial eta square offered a direct view of the effect size on the dependent variables, which showed a hardly moderate effect on reading strategies and a large effect on writing strategies according to Cohen (1988) criteria (.01 = small effect, .06 = moderate effect, .14 = large effect). These seemed to show that 1) there was no significant relationship between level of summarization performance and the use of reading strategies; 2) the use of writing strategies had a positive relationship to the summary writing performance, or, higher score summarizers employed significantly more frequent use of summarization writing strategies (mean = 3.570) than the lower scorers (mean =3.225).

B. Statistics by success of CET4 performance

The relationship between strategy use, summarization performance and summarizers’ English proficiency was also probed with statistical analysis. The students were put into two groups according to their CET4 performance (t = −4.861, df = 58, p = .000): more proficient (n = 30) and less proficient (n = 30). Then MANOVA was conducted with participants’ CET4 scores as the independent variables, their strategy use and summarization performance as the dependent variables. Table 5 presents the distributions of the dependent variables indicating all the variables were normally distributed (i.e., skewness and kurtosis statistics were near zero).

Table 5 Distributions for summarization performance and strategy use variables

In this analysis, Box’s Test of Equality of Covariance Matrices demonstrated that the data had homogeneity of variance. The multivariate tests of significance indicated that there was a statistically significant multivariate effect for success levels (F = 8.283, p = .000, η² = .307) though its contribution to the model is very modest (Pillai’s Trace = .307, Wilks Lambda = .693).

Table 6 presents the significance of F test for the grouping variable by CET4 performance. It suggested the independent variable had insignificant effect on both reading and writing strategy use (η² = .004 in both cases), but the effect on summarization performance was very large according to Cohen (1988) criteria. These seemed to show that 1) there was no significant relationship between participants’ level of English proficiency and the use of reading and writing strategies; 2) participants’ English proficiency had a positive relationship to their summarization performance, or, more proficient language learners performed better (mean =9.789) than the less proficient (mean =7.633) in the summarization tasks.

Table 6 MANOVA results for success levels by CET performance

Relationships between language abilities and summarization performance

The relationships between various language abilities and summarization performance were examined to see to what extent English reading and writing affected students’ English summarization performance. To this end, a series of correlation analysis was first performed and the results were presented in Table 7. It showed the relationship among the three measures were significant with the coefficients ranging from low to moderate level according to Connolly and Sluckin (1957) criteria.

Table 7 Correlation between language abilities and summarization performance

Data were then analyzed using regression analyses (enter method), prior to which, 5 outliers were identified and excluded by examining Mahalanobis distance statistics, thus 55 participants remained at last. Assumptions were checked by looking at the tolerance and VIF collinearity statistics and condition index, with no significant violations noted. A regression model was constructed with CET4 reading^b and writing scores as two independent variables and summarization performance represented by the average of the summary scores in the two tests as the dependent variable (see Table 8).

Table 8 Summarization performance and language abilities in reading and writing

Overall, the model can significantly predict summary writing performance, and English reading and writing together accounted for 40.2% of the total variance in the model. English writing can significantly predict summarization performance, while the predictability of English reading was insignificant. CET4 reading and writing only explained a small proportion of the variance in summary writing respectively. However, students’ writing ability contributed much more than their reading ability (r² = .233 vs. .036) to the total variance in their summary writing.

Discussion

Use of reading and writing strategies in summarization

Results of the present study showed among the 70 strategies identified in the summarization process, most were used for reading source texts and writing summaries, which reflect the importance of these skills in the said tasks. The three most frequently used reading strategies were identify and skip unknown word, reprocess information to clarify meaning, and reread for clarification. These strategies point to the significance of bottom-up reading processes focused on word- and sentence-level comprehension. This conforms to what has been shown by numerous researchers that an integrated comprehension of a text relies heavily on the fluid, accurate, and efficient application of bottom-up processes (e.g., Aebersold & Field 1997; Anderson & Pearson 1984; Carrell 1984; Rumelhart 1980; Stanovich 1980, as cited in Rupp et al. 2006). The writing strategy used most frequently by participants was plan content from source, which indicates the heavy reliance on source text in summary writing task. This is maybe due to the nature of the task of summary writing which confines ideas for writing within the limit of the source text.

The least employed strategies were check inference, reread instruction, reorganize source pattern, and edit for punctuation, with the former two being reading strategies while the latter two writing strategies. The scarce use of these strategies might be interpreted this way. For check inference, participants might think it unnecessary as they were found reading slowly and carefully in the test-taking process, thus there was little need to check what they might have inferred about the text. Another possibility is that they might wish to save time to finish reading the entirety of the text so as to complete the task under a simulated testing condition. For reread instruction, participants might have understood the instruction well and there was no need to confirm it.

The little use of reorganize source pattern can be explained from two perspectives. First, writers tend to “appropriate the global organizational patterns of the source to frame the summary” (Spivey 1990, p.265). Thus the scarce use of the said strategy reflects such a tendency among participants to follow the structure of the source text when writing summaries. Second, reorganization of source text was considered a higher level summarization strategy (Hidi & Anderson 1986), which might not be readily stored in the repertoire of strategies of the language learners at intermediate level such as the participants in the present study.

The low occurrence of the strategy edit for punctuation seems to show much less attention has been given to punctuation than to other aspects in writing, such as grammar and vocabulary. This seems to reflect a pair of connected realities that punctuation is only considered as secondary to other linguistic features in writing and that it has not been paid sufficient attention on the part of both teachers and students in classroom education in China. Review of the scoring rubric of CET-4/6 writing test indicates no consideration has been given to punctuation. This ignorance is likely to spread to classroom as one can infer from the wash back effects of a large scale test. Also from the researcher’s own experience as a college English teacher, punctuation had only been treated as trivia, if it was indeed mentioned, in the evaluation criteria of English writings of college students in assessment practices.

However, the above findings concerning the frequency of strategy use should be accepted with caution, as there was a great diversity among participants in terms of test-taking approaches and the sample size of participants was small, the estimate of frequencies may be biased. Thus the pattern of reading and writing strategy use of a larger sample was then investigated with a questionnaire developed based on the think-aloud and interview data. For use of reading strategies, strategy number 40 (use context clues to interpret meaning) and strategy 41 (use context clues to predict meaning) were rated the highest, which showed the high degree of reliance on context in the process of meaning construction in reading. Strategy 42 (use phonological cues) and 44 (note and analyze sentence structure) received the lowest rating, which indicated relatively less attention was directed to these two aspects of smaller unit of text variables as far as this sample of participants was concerned.

For writing strategies, the highest rated variable was strategy 48 (identify topic sentence of paragraph) which was considered the most efficient strategy by participants in summary writing. This is quite natural, as the summarization tasks require test takers to present the most important ideas of a text to the readers, and the topic sentence represents the main idea of a paragraph. Strategy 53, 57, 64, 67, 74 and 75 were rated below the average mean of 3, with strategy 75 (use phonological cues) being the least applied strategy, which was employed when writers attempted using advanced or unfamiliar vocabulary in summary writing. It might be that students felt reluctant to use unfamiliar words at the risk of making mistakes in spelling, which was considered a fundamental ability in English learning in China. Another possible reason is that examinees’ need for using phonological cues decreases as they are normally asked to keep quiet in a testing context.

The relationship between summarization strategy and performance, and summarizers’ English proficiency were investigated with statistical measures (i.e., MANOVA). It was found the use of writing strategies had a positive relationship to summarization performance. The fact that no significant relationship was observed between frequency of reading strategy use and summarization performance might suggest that success in this task is more related to how test takers write than how they read. After all, it is the written product that helps make inference about a student’s summarization ability. The study also found a very small effect of English proficiency on the use of reading and writing strategies in summarization tasks. These suggest the summarization strategies might be “task-limited procedures” (Alexander et al. 1998, p.132) which are referred to as “task-specific strategies” (Pressley et al. 1989).

Relationship between reading, writing, and summarization performance

In terms of the relationship between reading, writing, and summarization performance, the difference in percentage distribution of strategies (more reading strategies than writing were employed) agrees with, to some extent, previous findings in relation to the role of reading and writing in summarization. In Yu (2008) study, for example, regression analysis showed that TOEFL reading contributed more than writing to students’ summarization performances. Sarig (1993) found composing a study-summary entailed considerably more work done on comprehension than on writing.

Previous studies (e.g., Asención Delaney 2008; Yu 2008) have found weak association both between summarization and reading and between summarization and writing. This was also the case in this study, as regression analysis showed CET4 reading and writing only made moderate contribution to the variance in summarization respectively. However, regression analyses also showed English writing ability can significantly predict summary writing performance, yet English reading cannot. These are, in a way, in agreement with what is revealed in MANOVA analysis about the relationship between summarization performance and strategy use mentioned above. These findings are also similar to what was found by Watanabe (2001). His study showed that the writing performance on the independent task was a stronger predictor of score on the reading-to-write tasks, and that the predictive power of the reading test was more likely due to general language proficiency rather than reading ability.

CET4 reading and writing together accounted for 40.2% of the total variance in summary writing. Findings concerning the individual as well as collective contribution of these abilities to summarization performance also, to some extent, contradicted with some previous studies in this line of research (e.g., Asención Delaney 2008; Yu 2008). For instance, Yu (2008) found TOEFL reading and English writing can only account for a small proportion (7%) of the variances in summarization performance, and compared with English writing, TOEFL reading was the best statistically significant predictor of summarization performance. The causes leading to such a contradiction might be that, on the one hand, different measures were used to test reading and writing abilities, on the other, summarization task served different purposes in these studies. In Yu (2008) study, summarization was used as a measure of reading comprehension, and apart from English summary, the participants in his study were also required to summarize in their native language (i.e., Chinese), while in the present study it was examined as a measure of read-to-write integrated test task.

Conclusions

To complete summarization tasks, respondents need both reading and writing skills. Thus examining the role of these skills offers insight into the task construct and informs interpretation of task performance. Overall, the findings of the present study confirm the importance of the roles reading and writing play in summarization, though writing made a greater contribution to task performance. The results also show while participants’ use of writing strategies has a significant correlation with their summarization performance, its relation to their general English proficiency is weak.

These findings have theoretical, empirical, practical, and pedagogical implications. In terms of theoretical implications, this study revealed what roles reading and writing play and what are their relationships in summarization tasks. In some instances the so-called receptive and productive skills operated simultaneously. For example, in the selecting process, participants made plans for writing based on their understanding of text segments (when they used the strategy plan content from source). Thus, examining participants’ summarization process, overall, reveals how writing involves reading and vice versa. As Fitzgerald and Shanahan (2000) have concluded, one can hardly view reading and writing as stand-alone processes. The findings also indicate that the view of shared knowledge in reading and writing (Tierney & Shanahan 1991) can justifiably account for reading-writing connections in an English language test using these modalities. In the process of summarization, students were observed to orchestrate their reading and writing skills through deeper analysis, synthesis, and application of personal judgment.

In terms of empirical implication, the analysis of participants’ think-aloud data generated a pool of strategies for integrated summarization tasks, which may be useful in future research investigating this type of task. Strategy research is relevant to integrated read-to-write tasks as research on test taking strategies is an important aspect of test validation (Cohen 2006; Xi 2008).

The practical implications are related to summarization task design, construct definition and rubric development in particular. The present study shows that test-takers were actually engaged in the strategies proposed in the literature of integrated writing, which provides validity evidence for the said task. It sheds some light on the construct of integrated writing task, which has not been seriously examined (Hirvela 2004). Thus the use of these reading and writing strategies should be considered while defining the construct of the summarization task.

This study also helps verify the roles reading and writing perform in integrated summarization tasks. This verification is meaningful if related to, among other considerations, the caution issued against a danger of “muddied measurement” (Urquhart & Weir 1998, p. 121; Weir 2005, p. 101) in terms of the confounding effects of reading and writing abilities on summarization performance (Alderson et al. 1995). The results seem to suggest the role of writing is relatively more significant than reading in summarization tasks. The relative significance between reading and writing should be reflected in scoring process, suggesting a need of assigning more weight to writing over reading in the scoring rubric, which may lead to better and more precise score interpretation.

The study also has some pedagogic implications. The findings showed that general English proficiency had a very small effect on the reading and writing strategies used in summarization. This seems to suggest that these strategies, or at least some of them, are, to a certain extent, different from those used in the tasks in which the linguistic skills are assessed separately. They, therefore, need special training in academic writing courses. This raises new requirements in classroom teaching, learning and practicing which should be designed in a way that is expected to prepare students to meet the demands in integrated assessment.

Endnotes

^aBachman and Palmer (1996) believe that in order to be useful, any given language test must be developed with a specific purpose, a particular group of test takers and a specific language use domain (i.e. situation or context in which the test taker will be using the language outside of the test itself) in mind. They refer to this domain as a ‘target language use’, or TLU, domain, and the tasks in the TLU domain as ‘TLU tasks’.

^bWhen this article was written, CET4 reading consists of two parts: Skimming & Scanning, and Reading in Depth which contains two sub-components (i.e. banked cloze and multiple choice items).

Appendix 1 Scoring scale

(1) Main Idea Coverage

• 5 EXCELLENT: A response has complete coverage of main ideas.

• 4 VERY GOOD: A response has coverage of most main ideas.

• 3 GOOD: A response has moderate coverage of main ideas.

• 2 MODERATE: A response has some coverage of main ideas.

• 1 POOR: A response has coverage of very few ideas.

• 0 NO: A response has no coverage of main ideas.

(2) Integration

• 5 EXCELLENT: A response rearranges the order of the statements logically, displays excellent examples of integration and connectives, and demonstrates global interpretation of the source text.

• 4 VERY GOOD: A response rearranges the order of the statements logically, displays good examples of integration and connectives, and demonstrates global interpretation of the source text.

• 3 GOOD: A response rearranges the order of the statements logically, displays moderate examples of integration and connectives, and demonstrates global interpretation of the source text.

• 2 MODERATE: A response basically follows the order of source text with few cases of re-ordering and integration, and is not global in the interpretation of the source text.

• 1 POOR: A response follows the original order of the statements in the source text, shows rare instance of proper integration and connectives, and is not global in their interpretation of the source text.

• 0 NO: A response has no instances of integration or connectives at all.

(3) Language Use

• 5 EXCELLENT: A response displays consistent facility in the use of language, demonstrating syntactic variety, appropriate word choice; it is within the word limit as required.

• 4 VERY GOOD: A response displays facility in the use of language, demonstrating syntactic variety and range of vocabulary, though it will probably have occasional noticeable minor errors in structure, or word form that do not interfere with meaning; it is basically within the word limit.

• 3 GOOD: A response demonstrates inconsistent facility in sentence formation and word choice that may result in lack of clarity and occasionally obscure meaning; and/or it exceeds the word limit to a noticeable degree.

• 2 MODERATE: A response has a noticeably inappropriate choice of words or word forms, an accumulation of errors in sentence structure and/or usage; and/or it exceeds the word limit to a large degree.

• 1 POOR: A response has serious and frequent errors in sentence structure or usage, the text shows a lack of control of vocabulary and/or grammar; and/or it exceeds the word limit to a large degree.

• 0 NO: A response is totally incomprehensible due to language errors, or because the response is left blank.

(4) Source Use

• 5 EXCELLENT: A response is predominantly in the summarizers’ own words and sentence structures, in addition to the accurate use of the information from the source text.

• 4 VERY GOOD: A response is mostly in the summarizers’ own words and sentence structures, in addition to the accurate use of the information from the source text.

• 3 GOOD: A response is basically in the summarizers’ own words and sentence structures, in addition to appropriate use of information from the source text.

• 2 MODERATE: A response has some use of the summarizers’ own words and sentence structures, in addition to the adequate use of the information from the source text.

• 1 POOR: A response is predominately verbatim copying the source text.

• 0 NO: A response demonstrates completely verbatim copying from the source text.

Appendix 2

Tables of strategy use (Tables 9, 10 and 11).

Table 9 Strategy category, description, source and count

Table 10 Mean of reading strategies in summarization

Table 11 Mean of writing strategies in summarization