Open Access

The role of reading and writing in summarization as an integrated task

Language Testing in Asia20144:3

https://doi.org/10.1186/2229-0443-4-3

Received: 17 February 2014

Accepted: 15 April 2014

Published: 9 June 2014

Abstract

In order to complete summarization tasks L2 learners need to demonstrate both reading and writing abilities. Thus, understanding the role of these abilities is necessary for interpreting performance on such tasks. This study examined the role of reading and writing in summarization tasks completed by 64 Chinese college students. It was found word- and sentence-level strategies were most commonly used for reading the source texts. Writing strategies played a more important role than reading in such tasks. General English proficiency was found to have exerted no significant effect on the use of both reading and writing strategies, and participants’ reading and writing abilities only made moderate contribution to their summarization performance respectively. The theoretical, empirical, practical, and pedagogical implications of the study were then discussed.

Keywords

Summarization performance Reading Writing

Background

Summarization has a natural appeal for communicative language testing because of its approximation to target language use (Cohen 1994). This meets the need of language testing for task authenticity, which looks at the degree of correspondence of the characteristics of a given language test task to the features of a TLU taska (Bachman & Palmer 1996). The need for authenticity, as well as other merits that integrated assessment enjoys, gives rise to the use of such tasks that many would argue have recently been “reinstated and revitalized” (Yu 2013, p.110). As an integrated assessment, summarization has been used in high-stakes international language tests, such as the new TOEFL (Yu 2009). However, this type of task is much under researched in the field of language testing, especially those involving extended writing (Yu 2007). And, as different skills are generally involved in summarization and other integrated tasks, interpreting task performance raises questions about the role each skill plays as well as the interaction among them (Plakans 2009). To shed some light on these issues and also increase our understanding of how learners approach these tasks, the present study looks at the role of reading and writing in test takers’ completion of summarization tasks.

Literature review: reading, writing, and summarizing

In order to complete summarization tasks L2 learners need to demonstrate both their reading and writing skills. Studies suggest that summarizing is intimately linked to reading comprehension in that the writer must, first of all, make sense of the target text (Kintsch & van Dijk 1978). Because of this, it has been frequently used as a measure of reading comprehension for both research and testing purpose (e.g., Cohen 1994; Yu 2008). However, Taylor (1986) has argued that summarizing “requires certain written language skills which are apart from and may be more complex than mere reading skills” (p. 206). Studies have shown that instructing students in summarization not only enhances their writing ability (Taylor & Beach 1984), but can also facilitate learning (Brown et al. 1981).

The role of reading and writing in summarization has been explored by Cohen (1994) and Sarig (1993), who examined the processes and strategies students engage in reading and writing, and by Asención Delaney (2008) and Yu (2008), who examined the relationship between reading, writing, and summarizing. Cohen (1994) asked five Portuguese EFL students to summarize English texts. Test takers provided verbal report protocols which were analyzed for instances of cognitive processes involved in reading the source text and in writing the summary. It was found the most successful strategy user used at least six strategies to her benefit. With respect to reading the source text, she used effectively technical facilitation strategies (underlining discourse markers and words to look up, and circling pronominal referents), clarification and simplification strategies. With respect to writing, she used technical facilitation strategies (providing a detailed answer to include the main ideas) and metacgnitive monitoring.

Sarig (1993) examined the process of how an Israeli student of high English proficiency composed study-summaries. It was found when reading transforming was the most important strategy category for the student, followed by moderate linking, revising, and clarifying. When writing, revising was the most important activity, followed by low-intensity transforming, linking, and clarifying strategies. Composing a study-summary was found to be an interactive writing endeavor with considerably more work done on comprehension than on writing.

Asención Delaney (2008) explored the relationship between summarization and L2 reading and writing abilities. The reading and writing measures were a reading test (the Nelson-Denny Reading Test), and a writing test adapted from the one Khaldieh (2000) used. Pearson product–moment correlation analysis suggested there were low positive correlation coefficients between the writing measure and the summary writing (r = .20), and it was not significant. However, summary writing significantly related to the reading measure, though weakly (r = .28). The researcher concluded that the reading-to-write ability seems to be a unique construct weakly associated with reading for comprehension and disassociated from writing an essay without background reading support.

Yu (2008) examined the differential effects of the use of the two languages (English and Chinese) on summarization as a measure of reading comprehension. He found English and Chinese summarization together can only account for a small proportion of the variances in TOEFL reading comprehension. Similar to what was found in above-mentioned Asención Delaney (2008) study, TOEFL reading comprehension was the statistically significant predictor for both English and Chinese summarization performances, while the predictability of English writing was not significant.

Most of these studies, however, used summarization either as a reading measure (Cohen 1994; Yu 2008) or as a study tool (Sarig 1993). The only one (Asención Delaney 2008) that treated summarization as an integrated writing task explored the relation between reading, writing, and summarization with correlation analysis. Thus the current research literature are in need of more studies that specifically investigate what processes and strategies test takers employ in reading and writing, and what is the relation between these abilities and task performance in summarization as an integrated measure. Given this discrepancy, the present study examined the two important language abilities in summarization to improve our understanding of the construct the task assesses. The study was guided by the following three research questions:
  1. 1)

    What processes and strategies do test takers engage in reading the source text in summarization tasks, and what role does reading play in such tasks?

     
  2. 2)

    What processes and strategies do test takers engage for producing own text in summarization tasks, and what role does writing play?

     
  3. 3)

    What is the relationship between reading, writing, and summarization performance?

     

Methods

Participants and their language abilities

A sample of 64 EFL learners was drawn from an undergraduate program in a Chinese university. Generally speaking, the sample was at intermediate level of English proficiency according to their NMET (Chinese national matriculation English test) score (mean = 95.4 on a 0 – 150 scale). In order to meet the needs of statistical analyses that aimed at discovering the relationship between summarization performance and English reading and writing abilities, an English proficiency test, College English Test Band 4 (CET-4), was administered to participants. Scores in CET4 Reading and Writing (inter-rater reliability 0.635) were used as a measure of their English reading and writing respectively.

Summarization tasks

Two source texts were chosen for use in the summarization task. They (both were about 900 words) were taken from a college English textbook so that they were fit for testing the participants in terms of topics and difficulties. One text describes how a medical worker shares the last moment of a patient who has two impressively graceful hands. The other looks into the question of iron and how exercise affects its levels in the human body. An analytical scoring rubric was used to evaluate participants’ summary scripts (Appendix 1). The rubric contained four components addressing different aspects of summarization abilities: Main Idea Coverage, Integration, Language Use, and Source Use. The students were asked to complete the tasks under examination conditions. The task instruction stated students should read the text first, and then write an English summary for about 130 words without copying the source.

Procedure

This study proceeded in two sessions. The first session involved four participants who provided think-aloud protocols while summarizing. Immediately after each of the two summarization tests, they participated in a one-to-one interview session to talk about their test-taking process and strategies they used. All the think-aloud and interview sessions were audio-recorded upon permission, the data were then transcribed, segmented, and coded into strategies.

In order to improve coding validity and reliability, cautions were exercised to make sure that only those strategies that have appeared in existing literature were used to assign names to segments of think-aloud protocols, though in some cases, minor changes were made or new strategy names were given to fit the practical situation. In this regard, extensive review of related literature were conducted with special attention to strategy use in reading (e.g., Anderson et al. 1991; Cohen & Upton 2007; Plakans 2009), writing (e.g., Baker & Boonkit 2004), integrated reading/writing (e.g., Asención 2004; Plakans 2009; Yang 2009), and summarizing (e.g., Kim 2001; Kirkland & Saunders 1991). When coding finished, a second coder was asked to code 10% of each of the 8 protocols, the agreement percentage was respectably high for each protocol (above 0.81). Data were analyzed in NVivo7, a software package from Qualitative Research Software used for coding, searching data, and modeling.

Based on the analysis of the think-aloud and interview data, a questionnaire with 5-point Likert scale items was developed to explore the use of various reading and writing strategies in summarization tasks (Additional file 1). This questionnaire was distributed to the remaining 60 participants in session two immediately after they have finished each of the two summarization tests. Students’ summary scripts were scored by three raters then averaged to yield a composite score for each participant. Analysis showed the inter-rater reliability estimates of the ratings were fairly high for both summarization tasks (in one task α = .804, and in the other α = .802).

Results

Reading and writing strategies used in summarization tasks

A. Think-aloud data

a. Reading and writing process
Analyses of participants’ think-aloud and interview data showed two stages appeared along the process of the read-to-write summarization task: first reading, then, writing (see Table 1). Under these two general processes, seven stages of the test taking process were identified based on participants’ think-aloud and interview data. Each of them was assigned with names representing the purpose or goal of a particular stage. The reading process subsumes the activity of ‘Assess task’, ‘Reflect on reading topic’ and ‘Read text’. Participants used a variety of strategies in reading, the source text in particular, the process of which was predominantly linear because most of the time participants moved from one sentence to the next. However, they were also observed to occasionally sum up what they have read, be it a portion or a paragraph, which might be influenced by the task goal. After reading was completed, participants started a writing process that subsumed the activities of ‘Work out text thesis and major ideas’, ‘Put pen to paper’ and ‘Assess task fulfillment’. Throughout this phase, participants refer to the text now and then which showed heavy reliance on source.
Table 1

Type and count of strategies used in summarization process

Process

Reading

Writing

Total

AT

RRT

RT

WTT

WMI

PPP

ATF

 

Type

4

2

30

1

8

21

4

70

Count

54

16

827

17

188

514

89

1705

Note: AT = assess task, RRT = reflect on reading topic, RT = read text, WTT = work out text thesis, WMI = work out major ideas, PPP = put pen to paper, ATF = assess task fulfillment.

As Table 1 shows, altogether 70 strategies were identified in the summarization process, they were operating along the stages of the test taking process to help achieve the purpose or goal of a particular stage. Overall the 70 strategies occurred 1705 times. Among them 36 were used for the purpose of reading and 34 for writing. Altogether the reading strategies occurred 897 times while the writing 808 times. 30 (42.3%) strategies were employed for the purpose of reading the source text, making it the largest category. The second largest strategy category was ‘Put pen to paper’ (21, 29.6%). These two categories also topped the list in terms of strategy count. The occurrence of ‘Read text’ was 827 (48.5%) and that for ‘Put pen to paper’ was 514 (30.1%). Table 9 in Appendix 2 sums up participants’ strategy use for the two tasks with information about strategy description, category (to which it belonged), source (where it came from) and frequency counts (the number of times a certain strategy occurred).

b. Use of reading and writing strategies

The most frequently used reading strategies were identify and skip unknown word, reprocess information to clarify meaning, and reread for clarification, all of which occurred 100 times in the summarizing process of the two tasks. The strategies used only once for the purpose of reading were check inference, and reread instruction. Check inference was done when participants were not sure about their inference making. This did not happen frequently, as in most cases, participants read slowly which precluded the needs to check the inferences they might have made.

The writing strategy used most frequently by participants was plan content from source which occurred 127 times during the entire test taking process. The least employed writing strategies were reorganize source pattern, and edit for punctuation, both of which were employed only once.

B. Questionnaire survey

Examining the responses to the questionnaire items helped us to see the patterns of reading and writing strategies in the summarization tasks completed by a larger sample. Participants’ ratings of each of the questionnaire items were averaged to produce a composite score. Understanding of the average scores was informed by the SILL (Strategy Inventory for Language Learning) profile of results developed by Oxford (1990). Based on this profile, a key (Table 2) was constructed to aid interpretation of item ratings.
Table 2

Key to understanding average scores of questionnaire items

Frequency

Description

Range

High

Always or almost always used

4.5 to 5.0

Often used

3.5 to 4.4

Medium

Sometimes used

2.5 to 3.4

Low

occasionally used

1.5 to 2.4

Never or almost never used

1.0 to 1.4

Note. Adapted from Oxford 1990.

a. Measurement of frequency of reading strategy

The descriptive statistics of reading strategy use in the questionnaire were presented in Table 10 in Appendix 2, which shows that participants tend to agree on the use of various reading strategies. Mean rating of the reading strategies hardly reached the lower end of the high level in the Key shown in Table 2 which indicated participants often used these strategies in summarization. Strategy 42 (use phonological cues) and 44 (note and analyze sentence structure) received the lowest rating, and participants used these strategies very occasionally according to the key.

Strategy 21 (read title) was rated the highest, which might be the result of mismatch between the item intended inquiry and participants’ perceived meaning, who might have considered it more as a Yes or No item than a frequency item. Apart from this one, strategy number 40 (use context clues to interpret meaning) and strategy 41 (use context clues to predict meaning) were rated the highest.

b. Measurement of frequency of writing strategy

The full results of writing strategy use in the questionnaire were presented in Appendix 2 (Table 11). Mean rating of the writing strategies in summarization suggested overall participants often or sometimes used these strategies while summarizing source texts according to the Key. The highest rated variable was strategy 48 (identify topic sentence of paragraph) which was the most efficient strategy considered by participants in summary writing. Strategy 53, 57, 64, 67, 74 and 75 were rated below the middle point of 3 across tasks, with strategy 75 (use phonological cues) being the least applied strategy, the occurrence of which was at the lower end of medium level (sometimes used) according to the key.

Relationship between strategy use, summarization performance, and English proficiency

The statistical analyses exploring the relationship between the use of reading and writing strategies and other variables were performed in two steps. In the first step, the grouping variable was participants’ summarization performance, and in the second participants’ English proficiency as measured by CET4 was entered as the grouping variable.

A. Statistics by success of summarization performance

For this analysis, the 60 students were put into two groups of equal size according to their summary scores (t = −10.875, df = 58, p = .000): higher score summarizers (n = 30) and lower scorers (n = 30). The effects of grouping on the use of summarization reading and writing strategies were modeled with MANOVA. Table 3 presents the distributions of the dependent variables indicating all the variables were normally distributed (i.e., skewness and kurtosis statistics were near zero).
Table 3

Distributions for reading and writing strategy use variables

Variable

Mean

SD

Skewness

Kurtosis

Median

Mode

Sum-read strategy

3.5295

0.50549

.327

.108

3.5200

3.81

Sum-write strategy

3.3973

.50119

0.033

-.362

3.4000

3.17

For this MANOVA analysis, Box’s Test of Equality of Covariance Matrices demonstrated that the data had homogeneity of variance. Levene’s test of equality of error variances also indicated that the homogeneity of variance was not violated in the dataset. The multivariate tests of significance indicated that there was a statistically significant multivariate effect for success levels (F = 4.118, p = .021, η2 = .126) though its contribution to the model was very modest (Pillai’s Trace = .126, Wilks Lambda = .874).

The fact that the summarizing performance only contributed moderately to the model might be due to the reality there were factors other than these strategies (e.g., language ability, test method effects and error of measurement) that could be used to explain the test score (see Bachman 1990). Table 4 presents the significance of F test for the grouping variable by summarization performance.
Table 4

MANOVA results for success levels by summarization performance

DV

df

Error df

F

p

Partial η2

Sum-read strategy

1

58

3.134

.082

.051

Sum-write strategy

1

58

7.927

.007

.120

In this model, the grouping variable (i.e., summarizing performance) did not significantly affect reading strategies, but did so for writing strategies. The value of partial eta square offered a direct view of the effect size on the dependent variables, which showed a hardly moderate effect on reading strategies and a large effect on writing strategies according to Cohen (1988) criteria (.01 = small effect, .06 = moderate effect, .14 = large effect). These seemed to show that 1) there was no significant relationship between level of summarization performance and the use of reading strategies; 2) the use of writing strategies had a positive relationship to the summary writing performance, or, higher score summarizers employed significantly more frequent use of summarization writing strategies (mean = 3.570) than the lower scorers (mean =3.225).

B. Statistics by success of CET4 performance

The relationship between strategy use, summarization performance and summarizers’ English proficiency was also probed with statistical analysis. The students were put into two groups according to their CET4 performance (t = −4.861, df = 58, p = .000): more proficient (n = 30) and less proficient (n = 30). Then MANOVA was conducted with participants’ CET4 scores as the independent variables, their strategy use and summarization performance as the dependent variables. Table 5 presents the distributions of the dependent variables indicating all the variables were normally distributed (i.e., skewness and kurtosis statistics were near zero).
Table 5

Distributions for summarization performance and strategy use variables

Variable

Mean

SD

Skewness

Kurtosis

Median

Mode

Sum-read strategy

3.5295

0.50549

.327

.108

3.5200

3.81

Sum-write strategy

3.3973

.50119

0.033

-.362

3.4000

3.17

Summarization performance

8.7102

2.25027

0.401

−0.476

8.333

6

In this analysis, Box’s Test of Equality of Covariance Matrices demonstrated that the data had homogeneity of variance. The multivariate tests of significance indicated that there was a statistically significant multivariate effect for success levels (F = 8.283, p = .000, η2 = .307) though its contribution to the model is very modest (Pillai’s Trace = .307, Wilks Lambda = .693).

Table 6 presents the significance of F test for the grouping variable by CET4 performance. It suggested the independent variable had insignificant effect on both reading and writing strategy use (η2 = .004 in both cases), but the effect on summarization performance was very large according to Cohen (1988) criteria. These seemed to show that 1) there was no significant relationship between participants’ level of English proficiency and the use of reading and writing strategies; 2) participants’ English proficiency had a positive relationship to their summarization performance, or, more proficient language learners performed better (mean =9.789) than the less proficient (mean =7.633) in the summarization tasks.
Table 6

MANOVA results for success levels by CET performance

DV

df

Error df

F

p

Partial η2

Sum-read strategy

1

58

.228

.635

.004

Sum-write strategy

1

58

.204

.653

. 004

Summarization performance

1

58

23.625

.000

.289

Relationships between language abilities and summarization performance

The relationships between various language abilities and summarization performance were examined to see to what extent English reading and writing affected students’ English summarization performance. To this end, a series of correlation analysis was first performed and the results were presented in Table 7. It showed the relationship among the three measures were significant with the coefficients ranging from low to moderate level according to Connolly and Sluckin (1957) criteria.
Table 7

Correlation between language abilities and summarization performance

 

CET4 reading

CET4 writing

Summary writing

CET4 reading

1.000

  

CET4 writing

.389(**)

1.000

 

Summary writing

.411(**)

.605(**)

1.000

n = 60; **Correlation is significant at the 0.01 level (2-tailed).

Data were then analyzed using regression analyses (enter method), prior to which, 5 outliers were identified and excluded by examining Mahalanobis distance statistics, thus 55 participants remained at last. Assumptions were checked by looking at the tolerance and VIF collinearity statistics and condition index, with no significant violations noted. A regression model was constructed with CET4 readingb and writing scores as two independent variables and summarization performance represented by the average of the summary scores in the two tests as the dependent variable (see Table 8).
Table 8

Summarization performance and language abilities in reading and writing

DV

IV

R2

F

sig.

Standardized coefficients Beta

t

sig.

Summarization performance*

CET write

.402

17.478

.000

.524

4.502

.000

 

CET read

   

.207

1.777

.081

*Average of scores of two summarization tests.

Overall, the model can significantly predict summary writing performance, and English reading and writing together accounted for 40.2% of the total variance in the model. English writing can significantly predict summarization performance, while the predictability of English reading was insignificant. CET4 reading and writing only explained a small proportion of the variance in summary writing respectively. However, students’ writing ability contributed much more than their reading ability (r2 = .233 vs. .036) to the total variance in their summary writing.

Discussion

Use of reading and writing strategies in summarization

Results of the present study showed among the 70 strategies identified in the summarization process, most were used for reading source texts and writing summaries, which reflect the importance of these skills in the said tasks. The three most frequently used reading strategies were identify and skip unknown word, reprocess information to clarify meaning, and reread for clarification. These strategies point to the significance of bottom-up reading processes focused on word- and sentence-level comprehension. This conforms to what has been shown by numerous researchers that an integrated comprehension of a text relies heavily on the fluid, accurate, and efficient application of bottom-up processes (e.g., Aebersold & Field 1997; Anderson & Pearson 1984; Carrell 1984; Rumelhart 1980; Stanovich 1980, as cited in Rupp et al. 2006). The writing strategy used most frequently by participants was plan content from source, which indicates the heavy reliance on source text in summary writing task. This is maybe due to the nature of the task of summary writing which confines ideas for writing within the limit of the source text.

The least employed strategies were check inference, reread instruction, reorganize source pattern, and edit for punctuation, with the former two being reading strategies while the latter two writing strategies. The scarce use of these strategies might be interpreted this way. For check inference, participants might think it unnecessary as they were found reading slowly and carefully in the test-taking process, thus there was little need to check what they might have inferred about the text. Another possibility is that they might wish to save time to finish reading the entirety of the text so as to complete the task under a simulated testing condition. For reread instruction, participants might have understood the instruction well and there was no need to confirm it.

The little use of reorganize source pattern can be explained from two perspectives. First, writers tend to “appropriate the global organizational patterns of the source to frame the summary” (Spivey 1990, p.265). Thus the scarce use of the said strategy reflects such a tendency among participants to follow the structure of the source text when writing summaries. Second, reorganization of source text was considered a higher level summarization strategy (Hidi & Anderson 1986), which might not be readily stored in the repertoire of strategies of the language learners at intermediate level such as the participants in the present study.

The low occurrence of the strategy edit for punctuation seems to show much less attention has been given to punctuation than to other aspects in writing, such as grammar and vocabulary. This seems to reflect a pair of connected realities that punctuation is only considered as secondary to other linguistic features in writing and that it has not been paid sufficient attention on the part of both teachers and students in classroom education in China. Review of the scoring rubric of CET-4/6 writing test indicates no consideration has been given to punctuation. This ignorance is likely to spread to classroom as one can infer from the wash back effects of a large scale test. Also from the researcher’s own experience as a college English teacher, punctuation had only been treated as trivia, if it was indeed mentioned, in the evaluation criteria of English writings of college students in assessment practices.

However, the above findings concerning the frequency of strategy use should be accepted with caution, as there was a great diversity among participants in terms of test-taking approaches and the sample size of participants was small, the estimate of frequencies may be biased. Thus the pattern of reading and writing strategy use of a larger sample was then investigated with a questionnaire developed based on the think-aloud and interview data. For use of reading strategies, strategy number 40 (use context clues to interpret meaning) and strategy 41 (use context clues to predict meaning) were rated the highest, which showed the high degree of reliance on context in the process of meaning construction in reading. Strategy 42 (use phonological cues) and 44 (note and analyze sentence structure) received the lowest rating, which indicated relatively less attention was directed to these two aspects of smaller unit of text variables as far as this sample of participants was concerned.

For writing strategies, the highest rated variable was strategy 48 (identify topic sentence of paragraph) which was considered the most efficient strategy by participants in summary writing. This is quite natural, as the summarization tasks require test takers to present the most important ideas of a text to the readers, and the topic sentence represents the main idea of a paragraph. Strategy 53, 57, 64, 67, 74 and 75 were rated below the average mean of 3, with strategy 75 (use phonological cues) being the least applied strategy, which was employed when writers attempted using advanced or unfamiliar vocabulary in summary writing. It might be that students felt reluctant to use unfamiliar words at the risk of making mistakes in spelling, which was considered a fundamental ability in English learning in China. Another possible reason is that examinees’ need for using phonological cues decreases as they are normally asked to keep quiet in a testing context.

The relationship between summarization strategy and performance, and summarizers’ English proficiency were investigated with statistical measures (i.e., MANOVA). It was found the use of writing strategies had a positive relationship to summarization performance. The fact that no significant relationship was observed between frequency of reading strategy use and summarization performance might suggest that success in this task is more related to how test takers write than how they read. After all, it is the written product that helps make inference about a student’s summarization ability. The study also found a very small effect of English proficiency on the use of reading and writing strategies in summarization tasks. These suggest the summarization strategies might be “task-limited procedures” (Alexander et al. 1998, p.132) which are referred to as “task-specific strategies” (Pressley et al. 1989).

Relationship between reading, writing, and summarization performance

In terms of the relationship between reading, writing, and summarization performance, the difference in percentage distribution of strategies (more reading strategies than writing were employed) agrees with, to some extent, previous findings in relation to the role of reading and writing in summarization. In Yu (2008) study, for example, regression analysis showed that TOEFL reading contributed more than writing to students’ summarization performances. Sarig (1993) found composing a study-summary entailed considerably more work done on comprehension than on writing.

Previous studies (e.g., Asención Delaney 2008; Yu 2008) have found weak association both between summarization and reading and between summarization and writing. This was also the case in this study, as regression analysis showed CET4 reading and writing only made moderate contribution to the variance in summarization respectively. However, regression analyses also showed English writing ability can significantly predict summary writing performance, yet English reading cannot. These are, in a way, in agreement with what is revealed in MANOVA analysis about the relationship between summarization performance and strategy use mentioned above. These findings are also similar to what was found by Watanabe (2001). His study showed that the writing performance on the independent task was a stronger predictor of score on the reading-to-write tasks, and that the predictive power of the reading test was more likely due to general language proficiency rather than reading ability.

CET4 reading and writing together accounted for 40.2% of the total variance in summary writing. Findings concerning the individual as well as collective contribution of these abilities to summarization performance also, to some extent, contradicted with some previous studies in this line of research (e.g., Asención Delaney 2008; Yu 2008). For instance, Yu (2008) found TOEFL reading and English writing can only account for a small proportion (7%) of the variances in summarization performance, and compared with English writing, TOEFL reading was the best statistically significant predictor of summarization performance. The causes leading to such a contradiction might be that, on the one hand, different measures were used to test reading and writing abilities, on the other, summarization task served different purposes in these studies. In Yu (2008) study, summarization was used as a measure of reading comprehension, and apart from English summary, the participants in his study were also required to summarize in their native language (i.e., Chinese), while in the present study it was examined as a measure of read-to-write integrated test task.

Conclusions

To complete summarization tasks, respondents need both reading and writing skills. Thus examining the role of these skills offers insight into the task construct and informs interpretation of task performance. Overall, the findings of the present study confirm the importance of the roles reading and writing play in summarization, though writing made a greater contribution to task performance. The results also show while participants’ use of writing strategies has a significant correlation with their summarization performance, its relation to their general English proficiency is weak.

These findings have theoretical, empirical, practical, and pedagogical implications. In terms of theoretical implications, this study revealed what roles reading and writing play and what are their relationships in summarization tasks. In some instances the so-called receptive and productive skills operated simultaneously. For example, in the selecting process, participants made plans for writing based on their understanding of text segments (when they used the strategy plan content from source). Thus, examining participants’ summarization process, overall, reveals how writing involves reading and vice versa. As Fitzgerald and Shanahan (2000) have concluded, one can hardly view reading and writing as stand-alone processes. The findings also indicate that the view of shared knowledge in reading and writing (Tierney & Shanahan 1991) can justifiably account for reading-writing connections in an English language test using these modalities. In the process of summarization, students were observed to orchestrate their reading and writing skills through deeper analysis, synthesis, and application of personal judgment.

In terms of empirical implication, the analysis of participants’ think-aloud data generated a pool of strategies for integrated summarization tasks, which may be useful in future research investigating this type of task. Strategy research is relevant to integrated read-to-write tasks as research on test taking strategies is an important aspect of test validation (Cohen 2006; Xi 2008).

The practical implications are related to summarization task design, construct definition and rubric development in particular. The present study shows that test-takers were actually engaged in the strategies proposed in the literature of integrated writing, which provides validity evidence for the said task. It sheds some light on the construct of integrated writing task, which has not been seriously examined (Hirvela 2004). Thus the use of these reading and writing strategies should be considered while defining the construct of the summarization task.

This study also helps verify the roles reading and writing perform in integrated summarization tasks. This verification is meaningful if related to, among other considerations, the caution issued against a danger of “muddied measurement” (Urquhart & Weir 1998, p. 121; Weir 2005, p. 101) in terms of the confounding effects of reading and writing abilities on summarization performance (Alderson et al. 1995). The results seem to suggest the role of writing is relatively more significant than reading in summarization tasks. The relative significance between reading and writing should be reflected in scoring process, suggesting a need of assigning more weight to writing over reading in the scoring rubric, which may lead to better and more precise score interpretation.

The study also has some pedagogic implications. The findings showed that general English proficiency had a very small effect on the reading and writing strategies used in summarization. This seems to suggest that these strategies, or at least some of them, are, to a certain extent, different from those used in the tasks in which the linguistic skills are assessed separately. They, therefore, need special training in academic writing courses. This raises new requirements in classroom teaching, learning and practicing which should be designed in a way that is expected to prepare students to meet the demands in integrated assessment.

Endnotes

aBachman and Palmer (1996) believe that in order to be useful, any given language test must be developed with a specific purpose, a particular group of test takers and a specific language use domain (i.e. situation or context in which the test taker will be using the language outside of the test itself) in mind. They refer to this domain as a ‘target language use’, or TLU, domain, and the tasks in the TLU domain as ‘TLU tasks’.

bWhen this article was written, CET4 reading consists of two parts: Skimming & Scanning, and Reading in Depth which contains two sub-components (i.e. banked cloze and multiple choice items).

Appendix 1 Scoring scale

(1) Main Idea Coverage

  1. 5

    EXCELLENT: A response has complete coverage of main ideas.

     
  2. 4

    VERY GOOD: A response has coverage of most main ideas.

     
  3. 3

    GOOD: A response has moderate coverage of main ideas.

     
  4. 2

    MODERATE: A response has some coverage of main ideas.

     
  5. 1

    POOR: A response has coverage of very few ideas.

     
  6. 0

    NO: A response has no coverage of main ideas.

     

(2) Integration

  1. 5

    EXCELLENT: A response rearranges the order of the statements logically, displays excellent examples of integration and connectives, and demonstrates global interpretation of the source text.

     
  2. 4

    VERY GOOD: A response rearranges the order of the statements logically, displays good examples of integration and connectives, and demonstrates global interpretation of the source text.

     
  3. 3

    GOOD: A response rearranges the order of the statements logically, displays moderate examples of integration and connectives, and demonstrates global interpretation of the source text.

     
  4. 2

    MODERATE: A response basically follows the order of source text with few cases of re-ordering and integration, and is not global in the interpretation of the source text.

     
  5. 1

    POOR: A response follows the original order of the statements in the source text, shows rare instance of proper integration and connectives, and is not global in their interpretation of the source text.

     
  6. 0

    NO: A response has no instances of integration or connectives at all.

     

(3) Language Use

  1. 5

    EXCELLENT: A response displays consistent facility in the use of language, demonstrating syntactic variety, appropriate word choice; it is within the word limit as required.

     
  2. 4

    VERY GOOD: A response displays facility in the use of language, demonstrating syntactic variety and range of vocabulary, though it will probably have occasional noticeable minor errors in structure, or word form that do not interfere with meaning; it is basically within the word limit.

     
  3. 3

    GOOD: A response demonstrates inconsistent facility in sentence formation and word choice that may result in lack of clarity and occasionally obscure meaning; and/or it exceeds the word limit to a noticeable degree.

     
  4. 2

    MODERATE: A response has a noticeably inappropriate choice of words or word forms, an accumulation of errors in sentence structure and/or usage; and/or it exceeds the word limit to a large degree.

     
  5. 1

    POOR: A response has serious and frequent errors in sentence structure or usage, the text shows a lack of control of vocabulary and/or grammar; and/or it exceeds the word limit to a large degree.

     
  6. 0

    NO: A response is totally incomprehensible due to language errors, or because the response is left blank.

     

(4) Source Use

  1. 5

    EXCELLENT: A response is predominantly in the summarizers’ own words and sentence structures, in addition to the accurate use of the information from the source text.

     
  2. 4

    VERY GOOD: A response is mostly in the summarizers’ own words and sentence structures, in addition to the accurate use of the information from the source text.

     
  3. 3

    GOOD: A response is basically in the summarizers’ own words and sentence structures, in addition to appropriate use of information from the source text.

     
  4. 2

    MODERATE: A response has some use of the summarizers’ own words and sentence structures, in addition to the adequate use of the information from the source text.

     
  5. 1

    POOR: A response is predominately verbatim copying the source text.

     
  6. 0

    NO: A response demonstrates completely verbatim copying from the source text.

     

Appendix 2

Tables of strategy use (Tables 9, 10 and 11).
Table 9

Strategy category, description, source and count

Category

Description

Sourcea

Countb

Assess task

Aware of task goal

5

8

 

Aware of task requirement

6

40

 

Read instruction

5

5

 

Reread instruction

1

1

Assess task fulfillment

Check requirement compliance

4

6

 

Compare own writing to source

3

4

 

Evaluate own writing

4

11

 

Read own writing

8

68

Put pen to paper

Edit for content

6

28

 

Edit for expression

4

16

 

Edit for grammar

6

18

 

Edit for punctuation

1

1

 

Edit for vocabulary

7

10

 

Identify problem in writing

6

38

 

Lift sentence from source

6

13

 

Manipulate information identified as important

3

8

 

Paraphrase selected portion

6

20

 

Plan content from source

8

127

 

Plan language

6

39

 

Plan the process

5

17

 

Recall grammatical rule

4

7

 

Recall phrase or expression

4

12

 

Recall word memory

4

13

 

Reorganize source pattern

1

1

 

Reread portion for use in writing

7

48

 

Revise plan

7

31

 

Scan for specific word to use in writing

4

10

 

Scan text for information to use in writing

7

38

 

Set organizational frame to meet own rhetorical needs

7

19

Read text

Check inference

1

1

 

Compare related information within text

3

7

 

Confirm inference

3

6

 

Disconfirm inference

5

9

 

Establish connections between propositions

4

8

 

Affirm understanding

5

12

 

Identify and skip unknown word

8

100

 

Identify problem in understanding

8

69

 

Infer phrase meaning

5

14

 

Infer sentence meaning

5

17

 

Infer word meaning

7

46

 

Integrate what is already known with text idea

5

27

 

Interpret content meaning with own words

7

60

 

Make inference based on understanding

3

8

 

Moderate understanding

8

58

 

Preview the text

2

2

 

Recognize sentence structure

3

20

 

Refer to previous part

5

6

 

Reflect on particular phrase or expression

5

10

 

Reflect on particular word

8

91

 

Reprocess information to clarify meaning

8

100

 

Reread for clarification

8

100

 

Skip unknown phrase

2

2

 

Skip unknown portion

1

2

 

Use context clues to interpret meaning

5

10

 

Use context clues to interpret phrase

2

3

 

Use context clues to predict content meaning

5

7

 

Use linguistic knowledge to guess phrase meaning

3

5

 

Use linguistic knowledge to guess word meaning

4

15

 

Use phonological cues

4

12

Reflect on reading topic

Read title

8

9

 

Reread title

6

7

Work out major ideas

Assess importance of idea for use in writing

6

36

 

Extrapolate based on understanding

5

15

 

Identify and learn key words in text

4

17

 

Integrate detail of what is read

6

25

 

Recall idea

6

34

 

Recognize text structure

4

30

 

Skim for gist

5

17

 

Work out main idea of paragraph

4

14

Work out text thesis

 

6

17

aSource – number of think-aloud protocols where a certain strategy is used. Four participants did two tasks, so the total number of source is 8.

bCount–frequency counts of a certain strategy occurred in the 8 sources.

Table 10

Mean of reading strategies in summarization

No. Strategy description

Mean

21 Read title

4.486

22 Establish connections between propositions

3.459

23 Identify and skip unknown word

3.765

24 Identify problem in understanding

3.288

25 Reflect on particular word/phrase/expression

3.281

26 Infer word/phrase/sentence meaning

3.685

27 Moderate understanding

3.578

28 Compare related information within text

3.355

29 Interpret content meaning with own words

3.688

30 Affirm understanding

3.581

31 Check inference

3.581

32 Translate/process text content for understanding

3.230

33 Recall word meaning

3.198

34 Use linguistic knowledge to guess word/phrase meaning

3.657

35 Refer to previous part

3.754

36 Dis/confirm inference

3.334

37 Reread for clarification

3.495

38 Integrate what is already known with text idea

3.681

39 Skip unknown phrase/portion

3.707

40 Use context clues to interpret meaning

3.859

41 Use context clues to predict meaning

3.892

42 Use phonological cues

2.437

43 Extrapolate based on understanding

3.611

44 Note and analyze sentence structure

2.625

45 Note and analyze text structure

3.121

46 Identify and learn key words in text

3.524

General mean

3.495

S.D.

0.402

Table 11

Mean of writing strategies in summarization

No. Strategy description

Mean

48 Identify topic sentence of paragraph

4.017

49 Recall idea

3.875

50 Reprocess information to clarify meaning

3.549

51 Assess importance of idea for use in writing

3.703

52 Work out main idea of paragraph

3.600

53 Set organizational frame to meet own rhetorical needs

2.834

54 Reorganize source pattern

3.097

55 Compare own writing to source

3.109

56 Edit for content

3.000

57 Edit for language, including grammar, expression, vocabulary and punctuation

2.887

58 Work out text thesis

3.757

59 Identify problem in writing

3.040

60 Integrate detail of what is read

3.207

61 Lift sentence from source

3.120

62 Manipulate information identified as important, such as delete unnecessary information or condense ideas

3.835

63 Paraphrase selected portion

3.629

64 Plan the process

2.966

65 Plan content and language

3.069

66 Recall word memory

3.177

67 Recall grammatical rule

2.828

68 Recall phrase or expression

3.086

69 Reread portion for use in writing

3.709

70 Reread title

2.977

71 Scan text for information to use in writing

3.618

72 Scan for specific word to use in writing

3.412

73 Skim for gist

3.469

74 Revise plan

2.760

75 Use phonological cues

2.572

76 Read own writing

3.452

General mean

3.288

S.D.

0.390

Declarations

Acknowledgements

This study was supported by a project of Beijing Institute of Fashion Technology (Project No. PTTBIFT).

Authors’ Affiliations

(1)
Foreign Language Department, Beijing Institute of Fashion Technology

References

  1. Aebersold J, Field M: From reader to reading teacher: issues and strategies for second language classrooms. Cambridge: Cambridge University Press; 1997.Google Scholar
  2. Alderson JC, Clapham C, Wall D: Language test construction and evaluation. Cambridge: Cambridge University Press; 1995.Google Scholar
  3. Alexander PA, Graham S, Harris KR: A perspective on strategy research: progress and prospects. Educational Psychology Review 1998, 10(2):129–154. 10.1023/A:1022185502996View ArticleGoogle Scholar
  4. Anderson R, Pearson P: A schema-theoretic view of basic processes in reading comprehension. In Handbook of Reading Research. Edited by: Pearson PD. New York: Longman; 1984:255–291.Google Scholar
  5. Anderson NJ, Bachman L, Perkins K, Cohen A: An exploratory study into the construct validity of a reading comprehension test: triangulation of data sources. Language Testing 1991, 8(1):41–66. 10.1177/026553229100800104View ArticleGoogle Scholar
  6. Asención Y Unpublished PhD thesis. In Validation of Reading-To-Write Assessment Tasks Performed by Second Language Learners. Northern Arizona University; 2004.Google Scholar
  7. Asención Delaney Y: Investigating the reading-to-write construct. Journal of English for Academic Purposes 2008, 7(3):140–150. 10.1016/j.jeap.2008.04.001View ArticleGoogle Scholar
  8. Bachman LF: Fundamental Considerations in Language Testing. Oxford: Oxford University Press; 1990.Google Scholar
  9. Bachman LF, Palmer A: Language Testing in Practice. Oxford: Oxford University Press; 1996.Google Scholar
  10. Baker W, Boonkit K: Learning Strategies in Reading and Writing: EAP Contexts. RELC Journal 2004, 35(3):299–328. 10.1177/0033688205052143View ArticleGoogle Scholar
  11. Brown AL, Campione JC, Day JD: Learning to learn: on training students to learn from texts. Educational Researcher 1981, 10(2):14–21. 10.3102/0013189X010002014View ArticleGoogle Scholar
  12. Carrell P: Schema theory and ESL reading: classroom implications and applications. The Modern Language Journal 1984, 68(4):332–343. 10.1111/j.1540-4781.1984.tb02509.xView ArticleGoogle Scholar
  13. Cohen J: Statistical Power Analysis for The Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum; 1988.Google Scholar
  14. Cohen AD: English for academic purposes in Brazil: The use of summary tasks. In From testing to assessment: English as an international language. Edited by: Hill C, Parry K. London: Longman; 1994:174–204.Google Scholar
  15. Cohen AD: The Coming of Age of Research on Test-Taking Strategies. Language Assessment Quarterly 2006, 3(4):307–331. 10.1080/15434300701333129View ArticleGoogle Scholar
  16. Cohen AD, Upton TA: ‘I want to go back to the text’: Response strategies on the reading subtest of the new TOEFL®1. Language Testing 2007, 24(2):209–250. 10.1177/0265532207076364View ArticleGoogle Scholar
  17. Connolly TG, Sluckin W: Statistics for The Social Sciences. London: Cleaver-Hume Press; 1957.Google Scholar
  18. Fitzgerald J, Shanahan T: Reading and writing relations and their development. Educational Psychologist 2000, 35(1):39–50. 10.1207/S15326985EP3501_5View ArticleGoogle Scholar
  19. Hidi S, Anderson V: Producing written summaries: task demands, cognitive operations, and implications for instruction. Review of Educational Research 1986, 56(4):473–493. 10.3102/00346543056004473View ArticleGoogle Scholar
  20. Hirvela A: Connecting Reading and Writing in Second Language Writing Instruction. Ann Arbor, MI: The University of Michigan Press; 2004.Google Scholar
  21. Khaldieh S: Learning strategies and writing processes of proficient vs. less-proficient learners of Arabic. Foreign Language Annals 2000, 33(5):522–533. 10.1111/j.1944-9720.2000.tb01996.xView ArticleGoogle Scholar
  22. Kim SA: Characteristics of EFL readers’ summary writing: a study with Korean university students. Foreign Language Annals 2001, 34(6):569–581. 10.1111/j.1944-9720.2001.tb02104.xView ArticleGoogle Scholar
  23. Kintsch W, van Dijk TA: Toward a model of text comprehension and production. Psychological Review 1978, 85(5):363–394.View ArticleGoogle Scholar
  24. Kirkland MR, Saunders MAP: Maximizing student performance in summary writing: managing cognitive load. TESOL Quarterly 1991, 25(1):105–121. 10.2307/3587030View ArticleGoogle Scholar
  25. Oxford R: Language learning strategies: What every teacher should know?. Boston, MA: Heinle & Heinle, Inc.; 1990.Google Scholar
  26. Plakans L: The role of reading strategies in integrated L2 writing tasks. Journal of English for Academic Purposes 2009, 8(4):252–266. 10.1016/j.jeap.2009.05.001View ArticleGoogle Scholar
  27. Pressley M, Goodchild F, Fleet J, Zajchowski R, Evans ED: The challenges of classroom strategy instruction. The Elementary School Journal 1989, 89(3):301–342. 10.1086/461578View ArticleGoogle Scholar
  28. Rumelhart DE: Schemata: the building blocks of cognition. In Theoretical issues in reading comprehension. Edited by: Spiro RJ, Bruce BC, Brewer BW. Hillsdale, NJ: Erlbaum; 1980:33–58.Google Scholar
  29. Rupp AA, Ferne T, Choi H: How assessing reading comprehension with multiple-choice questions shapes the construct: a cognitive processing perspective. Language Testing 2006, 23(4):441–474. 10.1191/0265532206lt337oaView ArticleGoogle Scholar
  30. Sarig G: Composing a study summary: A reading-writing encounter. In Reading in the composition classroom: Second language perspectives. Edited by: Carson JG, Leki I. New York: Newbury House; 1993.Google Scholar
  31. Spivey N: Transforming texts. Written Communication 1990, 7(2):256–287. 10.1177/0741088390007002004View ArticleGoogle Scholar
  32. Stanovich K: Toward an interactive-compensatory model of individual differences in the development of reading fluency. Reading Research Quarterly 1980, 16(1):32–71. 10.2307/747348View ArticleGoogle Scholar
  33. Taylor KK: Summary writing by young children. Reading Research Quarterly 1986, 21(2):193–208. 10.2307/747845View ArticleGoogle Scholar
  34. Taylor BM, Beach RW: The effects of text structure instruction on middle-grade students’ comprehension and production of expository text. Reading Research Quarterly 1984, 19(2):134–146. 10.2307/747358View ArticleGoogle Scholar
  35. Tierney RJ, Shanahan T: Research on the reading–writing relationship: Interactions, transactions, and outcomes. In The handbook of Reading Research Vol.2. Edited by: Barr R, Kamil ML, Mosenthal P, Pearson PD. New York: Longman; 1991:246–280.Google Scholar
  36. Urquhart AH, Weir CJ: Reading in a Second Language: Process, Product and Practice. London: Longman; 1998.Google Scholar
  37. Watanabe Y Unpublished PhD thesis. In Read-to-write tasks for the assessment of second language academic writing skills: investigating text features and rater reactions. the University of Hawaii; 2001.Google Scholar
  38. Weir CJ: Language testing and validation: An evidence-based approach. Basingstoke: Palgrave Macmillan; 2005.Google Scholar
  39. Xi X: Methods of test validation. In Encyclopedia of Language and Education: Vol. 7. Language Testing and Assessment. 2nd edition. Edited by: Shohamy E, Hornberger NH. New York: Springer Science + Business Media LLC; 2008:177–196.Google Scholar
  40. Yang H Unpublished PhD thesis. In Exploring the Complexity of Second Language Writers’ Strategy Use and Performance on an Integrated Writing Test through Structural Equation Modeling and Qualitative Approaches. University of Texas at Austin; 2009.Google Scholar
  41. Yu G: Students’ voices in the evaluation of their written summaries: Empowerment and democracy for test takers? Language Testing 2007, 24(4):539–572. 10.1177/0265532207080780View ArticleGoogle Scholar
  42. Yu G: Reading to summarize in English and Chinese: A tale of two languages? Language Testing 2008, 25(4):521–551. 10.1177/0265532208094275View ArticleGoogle Scholar
  43. Yu G: The shifting sands in the effects of source text summarizability on summary writing. Assessing Writing 2009, 14(2):116–137. 10.1016/j.asw.2009.04.002View ArticleGoogle Scholar
  44. Yu G: From integrative to integrated language assessment: Are we there yet? Language Assessment Quarterly 2013, 10(1):110–114. 10.1080/15434303.2013.766744View ArticleGoogle Scholar

Copyright

© Li; licensee Springer. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.