Different dimensions of teacher and peer assessment of EFL learners’ writing: descriptive and narrative genres in focus

Abstract

The present study explored the extent to which teacher assessment (TA) and peer assessment (PA) differ in terms of magnitude and patterns of distribution across English as a Foreign Language (EFL) learners’ descriptive and narrative writing performance. Twenty Persian-speaking EFL learners were non-randomly selected and voluntarily participated in a 12-session writing course at a private language school in Iran. Their performance on descriptive and narrative writing tasks was subjected to PA and TA sequentially. The West Virginia Department of Education descriptive writing rubric (with five components of organization, development, sentence structure, word choice and grammar, and mechanics) and Smarter Balanced narrative writing rubric (with five categories of narrative focus, organization, elaboration of narrative, language and vocabulary, and conventions) were adopted to schematize and analyze the distribution of the TA and PA comments. The results of frequency analysis indicated that TA far outnumbered PA on both descriptive and narrative genres of writing. Furthermore, on both descriptive and narrative writings, TA and PA commentaries were local in scope, form-focused, fluctuating, and inconsistent. Also, the distribution of TA and PA comments mainly focused on the conventions of narrative writings, while other macro-components of narrative writing had a steady and depleted pattern. The statistical results confirmed the significance of the observed differences between the number and the nature of TA and PA on descriptive and narrative genres of writing. The researchers made their concluding remarks on the probable causes of observed diversities, imposed limitations of the study, and a number of topics for future research.

Introduction

Classroom assessment has been known as an immediate means to attest student learning and progress (Zhao & Liao, 2021). Over the last decades, a paradigm shift was seen in teaching practitioners’ tendency, from using assessment to grant credits and grades to students (Liu & Huang, 2020) to a more integrated endeavor of investment on students optimized learning (Zhao & Zhao, 2020). Classroom writing assessment in English as a Foreign Language (EFL) contexts, however, is still dominated by language teachers’ use of writing tasks to make summative judgments about student performance (Wang, Lee, & Park, 2020). Given the potential of assessment to “scaffold” the students writing, it seems that assessment should be re-conceptualized as the central pillar of classroom writing.

At the heart of assessment for learning (AfL) (William, 2018), teacher assessment (TA) and peer assessment (PA) are both known as instructional methodologies and “dialogic learner-centered” assessment tools of writing (Carless et al., 2011; Tian & Zhou, 2020; Yu, 2020). Calling for further research, previous studies on EFL writing have demonstrated controversies over the alignment of TA and PA from different perspectives, such as the plainness of language in peer feedback relative to teacher corrective feedback (Mao & Crosthwaite, 2019), relative success of PA over TA in raising L2 writers’ evaluative judgment (Min, 2016), or “the capability to make decision[s] about the quality of work of self and others” (Tai, Ajjawi, Boud, Dawson, & Panadero, 2018, p. 471) and substantial role of PA in the formation of scholarly community of students in their genre-based writing courses (Yu, 2020). On the other hand, several studies on EFL writing argued the efficacy of TA from an achievement angle (Yu & Lee, 2016), where students progress was purely evaluated through analysis of their pre- and post-test gain scores. They encouraged training students to active peer feedback (Mohamadi, 2018) and demanded future research on alternative assessment modalities.

Several studies on genre-based writing assessment in EFL/ESL contexts favored the strict and formal teacher corrective feedback and undermined PA practice for its incompatibility to the exam-oriented educational systems, due to its time-consuming and yet less effective nature (Birjandi & Hadidi Tamjid, 2012; Zhao, 2018). Embedded as a complement to TA routine in writing courses, PA has been understudied as an effective and collaborative learning and assessment opportunity in L2 classroom setting (Dressler, Chu, Crossman, & Hilman, 2019; Pourdana, Nour, & Yousefi, 2021) which highly requires further examination. In the same vein, with the emergence of genre-based assessment of writing in L2 context and its challenges and controversies, several studies examined how L2 learners performance on different writing genres might be affected by such contextual and individual factors as L2 learners’ limited language proficiency (Lee, 2017), concerns over hurting their peer’s ‘face’ (Yu & Lee, 2016), reachability of the target audience (Sari, 2019), or academic context of writing (Hyland, 2003); yet what has left uncovered and under-researched is how genre-based assessment of L2 writing could moderate such reported effects (Yu & Lee, 2016). Last but not the least, research literature on assessing writing securitized it from either linguistic/form-focused perspective, confirming the positive role of writing in improving such qualities in language performance as accuracy (Liao, 2016), fluency (Plakans, Gebril, & Bilki, 2019), and complexity (Lahuerta, 2018; Zenouzagh, 2020), or from psycholinguistic/meaning-focused perspective, supporting the influential role of writing assessment in incitement of such affective factors as anxiety (Fathi & Khodabakhsh, 2020) or demotivation (Yu, Jiang, & Zhou, 2020). Hence, little attention has been paid to genre-based assessment approach to L2 writing performance.

Missing in the research studies is also the expansion of the sociocultural theory (SCT) (Vygotsky, 1980) into a multi-perspective approach to writing assessment, which emphasizes both the assessment process and product, while pays special attention to who (teacher, peer and/or self) would participate in the assessment practice (Ataie-Tabar, Zareian, Amirian, & Adel, 2019). Although the research studies frequently reported the merits of genre-based assessment approach to developing L2 learner writing achievement (Chen & Su, 2012), only a few studies examined the nature and magnitude of teacher and peer commentaries on genre-based writing in L2 environment (Hyatt, 2005; Mirador, 2000; Yelland, 2011).

This study intended to fill the void by adopting a learner-oriented sociocultural approach to genre-based assessment of writing in EFL context, investigating the parallel potentials of TA and PA on descriptive and narrative writing genres. The researchers’ goal for choosing descriptive genre of writing was to examine the EFL learners’ ability in describing facts, figures, and flowcharts in written mode. Likewise, the researchers’ goal for choosing narrative genre of writing was to monitor EFL leaners’ writing performance on recounting facts, and reporting step-wise processes. In the following section, a critical review of the studies on teacher and peer feedback to assess writing is presented. Next, the objectives of the study are restated, followed by the procedure of data collection. Then, the statistical results are discussed and consolidated with previous research findings. Finally, drawn conclusion, implications and suggestions for further research are presented in light of limitations of the study.

Literature review

Teacher versus peer assessment

As a core component in every EFL program, teaching-to-write has intrigued many L2 teachers and practitioners. In the same vein, learning-to-write is a necessity which boosts student access to teacher assessment and its constructive impacts. Therefore, on occasions where giving feedback to students is not part of language teachers’ formal practice or when language learners fail to intake the received feedback on their writing, reaching the intended goals of writing in EFL seems an impossibility (Agbayahoun, 2016).

A considerable body of research in EFL writing has documented the importance of teacher assessment (TA). As a pedagogical genre, TA is manipulated to transfer a heavy informational load to the students, to provide comments on the form and content of a written text and to encourage students to develop their writing (Karimi & Asadnia, 2015; Sermsook, Liamnimitr, & Pochakorn, 2017). The crucial role of TA is commonly acknowledged in language classroom where full access to target language is scarce and where a product-oriented approach towards assessing writing is formally adopted. Agbayahoun (2016)’s research on EFL student writing in the Republic of Benin indicated that although TA promoted students’ language accuracy and mechanics of writing, they favored teacher comments more on “the content” of their written products. Similarly, in another experimental study by Luquin and Garcia Mayo (2021) on teacher written feedback in terms of providing written models—the native-like texts to which young learners can compare their own writing—the researchers analyzed children’s pair talk for evidence of any content and linguistic problems. It was found that the students looked more for content-related than linguistic features at the comparison stage. Moreover, these children actively incorporated teacher comments on mechanics and discourse-related features into their rewriting stage (Luquin & Garcia Mayo, 2021), relative to the control group who self-revised their written scripts.

As an alternative assessment genre, peer assessment (PA) has received special attention by education researchers over the past decades. PA—also known as peer evaluation, peer feedback, peer review, or peer critique—refers to “a communication process through which learners enter a dialog related to their own and peer performance” (Liu & Carless, 2006, p. 280). Despite ample theoretical and empirical support, the effectiveness of peer feedback in various learning environments has remained disputable (Chang, Lee, Tang, & Hwang, 2021). In teacher-centered educational systems, where students often display low ability in self-study, in autonomous problem-solving, and in adapting effective learning strategies, PA can be their remedy (Deng & Carless, 2010). Collaborating in PA, students can generate opportunities to discuss their written texts (Yu & Lee, 2016), improve the quality of their writing (Zhao, 2014), promote their motivation to write (Shih, 2011), and enhance their critical thinking (Joordens, Pare, & Pruesse, 2009).

Although numerous studies reported the educational benefits of PA, some research results suggested that it might hardly be useful as formal assessment in real language classrooms (Brown, 2004; Li, 2017; Liu & Li, 2014; Pope, 2001). In fact, there is little agreement over the adoptability of PA as an alternative assessment to TA. Chang et al. (2021)’s meta-analysis of 20 peer assessment articles and their 340 follow-up studies reported the heterogeneous effect size values for efficacy of PA. In one semester-long intervention study of the efficacy of PA on improving English writing of 70 sophomore Chinese students, and promoting their learning autonomy, Shen, Bai, and Xue (2020) concluded that PA enhanced the students’ learner autonomy in a considerable way. They believed that learner dependency on the teacher feedback was noticeably reduced and the students’ confidence in their learning ability was boosted as a result of their active PA practice. Several research findings reported significant contribution of PA in L2 contexts, in terms of better understanding of the learning objectives (Hattie & Timperley, 2007), regulating learners own performance (Colognesi & Deschepper, 2018), and increasing student metacognitive self-awareness (Lu & Law, 2011). Matsuno (2009) also suggested that PA practice remains challenging and engaging in L2 learning contexts and would cause constructive interactions among language learners.

From a statistical perspective, Cheng and Warren (1999) reported a strong interrelation between teacher raters and peer raters, depending on the target tasks and the context of language use (Jafarpur, 1991; Patri, 2002; Saito & Fujita, 2004). They concluded that proper training could improve the quality and magnitude of PA commentaries in a substantial way (Saito, 2008). Moreover, Jones and Alcock (2014) obtained a high validity and inter-rater reliability measures for PA practice, asking students to compare their written scripts with their peers, and concluded that the students could easily become effective peer assessors. On the other hand, in a comparative study of TA and PA with 38 EFL university students academic writing, Azarnoosh (2013) implemented a learner attitude questionnaire and a friendship bias analysis with a pretest/posttest writing research design. She observed no significant differences between the impacts of PA and TA on student academic writing. Likewise, no friendship bias was detected in student PA comments, yet a positive change in students’ attitudes towards PA emerged.

Descriptive and narrative genres of writing

The concept of genre-based assessment of L2 writing in terms of student performance on writing life narrative or describing a procedure in a research report has remained under-researched (Jaubert, 2007). A number of studies have focused on genre-related teacher assessment of L2 writing performance in colleges and universities (Dolz & Gagnon, 2008; Freedman & Medway, 1994; Horverak, 2016; Hyland, 2003, 2007). In addition, the formative assessment of genre-based writing, such as expository, narrative, and descriptive essays, was reported as a constant challenge to teacher assessment (Dumais, Lafontaine, & Pharand, 2017). Teachers often “have difficulty, resistance in carrying out certified, and even formative evaluation of pupils’ language production” (Gagnon, De Pietro, & Fisher, 2017, p. 16).

By definition, “genre” not only refers to different types of literary texts, such as poetry or drama, but also to “the conventional and recurring patterns of everyday, academic and literary texts within a particular culture” (Derewianka, 2003, p. 133). Members of a particular social and cultural community are expected to recognize the generic language conventions in various situations and be able to respond by using the appropriate genre. Spoken or written, genres are often identified according to five primary social purposes of recount or description, narrative, information report, instruction, explanation, and expository (Swales, 1990).

In general, descriptive genre gives detailed attributes to a person, place, or event. Those entities should be described in such a way that the reader can picture the topic and enter the writer’s experience. Descriptive genre is also considered as a way to improve other genres of writing, such as analytical and argumentative or perhaps as a dominant strategy to depict an image of what something looks like (Birjandi & Hadidi Tamjid, 2012). In descriptive writing, the writer introduces the intended theme in the topic sentence before describing it further through the supporting sentences. Topic sentence often contains “enumeration,” and the use of enumerators notifies what pieces of information is being presented in the supporting sentences to the readers. Characteristics, parts, aspects, layers, formats, habits, behaviors, and facets are the most frequently used enumerators in descriptive genre of writing.

One of the well-known rubrics for assessing descriptive writing has been developed by West Virginia Department of Education (WVDE) (2011). WVDE rubric consists of five categories of organization, development, sentence structure, word choice and grammar, and mechanics. “Organization” entails a clear and logical progression of ideas in a descriptive writing. “Development” is the clear focus maintained by the writer for the intended audience, by the strong use of examples, relevant details, analogies, and illustrations. “Sentence structure” refers to the use of well-constructed sentences with varied structures. “Word choice and grammar” is the writer’s preference for vivid words and grammatical phrases. Finally, “mechanics” refers to the systematic use of punctuation, capitalization, and grammar in writing (NBCT Office of Assessment West Virginia Department of Education, 2015).

Narrative genre, as the art or gift of storytelling, is made through every moment of our lives so that we make narration plenty of times every day (Abbott, 2002). Lou-Conlin (1998) defined written narrative as a gradual mode of development through which the writer expresses the chronological sequence of ideas and events. Narration is mostly done with the purpose of maintaining the readers’ interest in the course of actions to depict a given event or personal experience narrative (PEN) (Labov, 1997). To fulfill this purpose, the narrator needs to illustrate the timeline of the experience with some extraordinary meaning or significance to both the reader and the writer.

Developed by Tulare County Office of Education in 2014, Smarter Balanced narrative writing rubric has been widely used for assessment of written narratives. The rubric identifies five components of narrative focus, organization, elaboration of narrative, language and vocabulary and conventions. “Narrative focus” refers to the effective establishment of a setting, narrator and/or characters. “Organization” entails creating an effective plot which demands unity and completeness. “Elaboration of narrative” is the thorough and effective elaboration using details and dialogs. “Language and vocabulary” means deliberate choice of words and structures that expresses experiences or events. Finally, “convention” indicates the effective and consistent use of punctuation, capitalization, and spelling (Smarter Balanced Assessment Consortium, 2012).

To echo the importance of genre-based writing assessment in EFL contexts, a number of studies documented how writer performance has been modified by corrective feedback on various writing genres, such as argumentation, narration and analysis. Zareei (2009) studied the impact of the genre-based TA on the quality and quantity of 140 Asian EFL learners writing of the letters of application. She reported that while the writing accuracy and complexity were significantly improved, the quantity (i.e., fluency) in students writing performance remained unaffected. Likewise, Rezvani, Aqdam, and Saeidi (2014) supported the positive role of teacher genre-based formative assessment on the achievement of advanced EFL learners in an experimental pretest/posttest research study. In another study, the impact of the genre-based TA on Colombian EFL students’ argumentative writing was examined through questionnaires, semi-structured interviews, class recordings, and students’ artifacts which indicated a significant enhancement in student self-confidence and their highly positive attitudes towards writing (Chala Bejarano & Chapetón, 2013). Genre-based assessment was also reported to provide “scaffolding” opportunities through dialogic interactions by the students which further supported students understanding of writing as a process (Yu, 2020).

A number of recent studies have explored the role of genre-based peer feedback in L2 writing development by comparing it to teacher corrective feedback, in reducing treatable language errors of EFL learners (Diab, 2015), and L2 learners’ perceived importance of peer feedback to their academic writing development (Seror, 2011). In another study, Yang et al. (2006) reported that the EFL learners used 90% of teacher corrective commentaries in revising their expository essays, while they incorporated only 67% of peer reviews. They commented on TA as “very useful” and “informative,” while being critical of PA as “inaccurate,” “confusing,” or “judgmental.” Xu and Liu (2010) compared the nature of peer and teacher assessment of EFL student writing and concluded that PA mostly focused on micro-level language features, while TA addressed the form, content, and mechanics of writing. Overall, research reports suggested that both TA and TA would have significant roles to play in L2 learner writing improvement. However, few studies elaborated on the nature of observed differences between PA and TA comments on the EFL genre-based writing performance.

This study

To bridge the gap in the research literature, this study was intended to explore two dimensions of the teacher and peer assessment, in terms of the relative quantity of the TA and PA comments, and the nature of differences in TA and PA comments on EFL student descriptive and narrative writing performance. To fulfill the objectives of the study, three research questions were raised, as following:

• (1) How teacher assessment and peer assessment differ in their impacts on EFL learners’ descriptive writing?

• (2) How teacher assessment and peer assessment differ in their impacts on EFL learners’ narrative writing?

• (3) Are the differences between teacher assessment and peer assessment impacts on EFL learners’ descriptive and narrative writing significant?

Method

Participants

A sample of 20 EFL learners (15 male and 5 female) who enrolled at a private language institute in Karaj, Iran, took part voluntarily in this study. A non-random purposive sampling method was used in this study (Ames et al., 2019), as selecting a representative group of EFL learners with adequate experience in genre-based writing in English was the researchers’ plan. Normally, there is no systematic emphasis on genre-based writing in curricular programs at private language institutes in Iran. However, the participants in this study were already required to submit at least 10 writing tasks, such as writing an invitation card, a letter of application or diaries in previous courses as partial requirement of their text book Top Notch 3 (Saslow & Ascher, 2006). Their experience in learning English was between 3 to 6 years (m = 5) and their ages ranged from 17 to 27 (m = 20.25). The participants’ language proficiency level at the moment of conducting this research was intermediate (30-37, B1 in OPT ranking system). Additionally, two MA graduates of English language teaching (ELT) took part in this research as the EFL teacher and the co-raters of the participants’ descriptive and narrative writing. Both had at least 7 years of teaching experience at private language schools in Iran. Table 1 summarizes the demographic information of the EFL learner participants.

Table 1 EFL learners’ demographic information

Instruments

Oxford placement test (OPT)

Developed by Oxford University Press and University of Cambridge Local Examinations Syndicate (CLES), OPT (Version 1, 2001) has been calibrated against the English proficiency levels provided by Common European Framework of Reference (CEFR) on a six-level assessment scale, ranged from A1 (lowest level) to C2 (highest level). In this study, OPT was used as a placement test aiming at measuring the selected participants’ knowledge of general English. The participants were required to answer the 60 multiple-choice items in the test in 45 min. Their language proficiency level was determined as intermediate (30-37, B1 in OPT ranking system) (Cronbach’s α = .812).

West Virginia Department of Education (WVDE) descriptive writing rubric

In order to assess the participants’ descriptive writing in both TA and PA practice, the well-known West Virginia Department of Education rubric (2011) for descriptive writing was adopted. The reason behind adopting the WVDE descriptive writing rubric was its user-friendliness and creditability. WVDE rubric is well-known for meeting the criteria to assess English descriptive writing, by defining rigorous and clear-cut score bands to ensure an accurate representation of student genre-based writing performance (NBCT Office of Assessment West Virginia Department of Education, 2015). WVDE descriptive writing rubric consists of five categories of “organization, development, sentence structure, word choice and grammar, and mechanics,” within 1–6 band scores, ranging from 1 (minimal) to 6 (exemplary) spectrum (Additional file 1: Appendix A)

Smarter balanced narrative writing rubric

Known as a public agency, the Smarter Balanced Assessment Consortium (SBAC) has created a global digital library of formative assessment tools to support teachers and students. Developed by Tulare County Office of Education (CA, USA) in 2014, Smarter Balanced narrative writing rubric has since been used to assess language learners’ written narrative performance. The Smarter Balanced assessment system includes a comprehensive suite of standard summative and formative assessment tools that save writing teachers time (Smarter Balanced Assessment Consortium, 2012). The logic behind selecting this rubric in this study was its clarity and user-friendliness which made it more accessible to peer assessment. Smarter Balanced narrative writing rubric consists of five categories of “narrative focus, organization, elaboration of narrative, language and vocabulary and conventions.” The rubric has a 1 to 5 band scores, ranging from 0 (no evidence of the ability to write a narrative) to 5 (meeting all the criteria of writing a real or imagined narrative) (Additional file 1: Appendix B).

Descriptive and narrative writing elicitation tasks

The designed interventions in this study lasted for 12 sessions in six consecutive weeks. The participants received six descriptive (on the odd sessions) and six narrative (on the even sessions) writing elicitation tasks. The arrangement and topics of writing tasks are summarized in Table 2.

Table 2 Topics and arrangement of the descriptive and narrative writing elicitation tasks

Data collection procedure

Prior to the experiment, all the participants were asked to sign a consent form for their volunteer participation in the study. Data collection procedure was conducted in the following steps:

• Two days before the writing course began, an OPT Version 1 (2001) was administered as the placement test for the purpose of normalizing the participants for their language proficiency level. Counting on the OPT scores, 20 participants were selected out of 31 whose scores were within the band scores of 30–37 (B1, in OPT ranking scale).

• A day before the experiment, the researchers provided the participants with an 8-h tutorial on the elements of descriptive and narrative genres of writing as well as the two selected rubrics for genre-based writing assessment. The participants were instructed on the nature of peer feedback in terms of selective comments, evaluation, or suggestions they could provide on each other’s writing. They were informed that PA was not substituted for TA, and they would receive teacher comments on their writing every session.

• A topic familiarity checklist was prepared by the researchers and distributed among the participants, a day before the experiment. The purpose was to select and incorporate 12 most favorable topics into descriptive and narrative writing elicitation tasks.

• The 6-week writing course was divided in to six odd sessions devoted to descriptive writing tasks and six even sessions on narrative writing tasks. The participants were assigned into pairs who were required to individually write a 200 to 250-word paragraph on the assigned topics, followed by reading peer’s written draft and giving feedback. They were allowed to consult the WVDE descriptive writing rubric, and Smarter Balanced narrative writing rubric, in the PA process. Drafting and PA lasted for 60 min.

• At the end of every session, the teacher collected the writings for her formative assessment. The participants were asked to revise/re-write their first draft according to the received PA and TA before submitting the final draft in the following session. As an in-class post-writing task, the students received no grade or bonus for their PA practice.

• Teacher assessment was carried out collaboratively by the two raters in the study (see Additional file 1: Appendices C and D for selected samples of descriptive and narrative writing, along with TA and PA comments). The possible controversies in rating written drafts were discussed and resolved upon every occasion. The inter-rater reliability indices were calculated for a variety of ratings in this study (Cronbach’s α = 0.881, reliability index for descriptive writing tasks; Cronbach’s α = 0.781, reliability index for narrative writing tasks; Cronbach’s α = 0.681, reliability index for TA and PA ratings of descriptive writing tasks; Cronbach’s α = 0.901, reliability index for TA and PA of narrative writing tasks). All values of Cronbach’s alpha represented strong agreement and statistical significant (p < .05).

Coding system

As indicated in Table 3, the TA and PA comments on student descriptive writing were cracked down into 11 themes with reference to five categories in WVDE descriptive writing rubric. Along the retrieved themes, examples of TA and PA comments were given in Table 3.

Table 3 The analytical scheme for classifying TA and PA on descriptive writing tasks

Similarly, TA and PA commentaries on narrative writing tasks were schematized based on the five categories in Smarter Balanced narrative writing rubric. They were further decoded into 11 themes in Table 4. The themes and examples of TA and PA comments were given in Table 4.

Table 4 The analytical scheme for classifying TA and PA on narrative writing tasks

Results

The IBM SPSS 23 was employed for statistical analysis of the collected data. Every comment, correction, or suggestion made by the teacher or peers on the written drafts was considered as one “feedback point” and the total feedback points for every draft of writing were calculated. Therefore, the observed decrease in teacher and peer feedback points was interpreted as the student progress in their writing. A descriptive statistical analysis was conducted with the PA and PA counts on participants’ descriptive writings, produced on the six odd sessions. As it can be seen in Table 5, the mean scores were much higher in TA than in PA, ranging from m = 12.80 on session 1 to m = 24.40 on session 9. On the other hand, PA had its minimum mean score on session 3 (m = 5.40) and the highest mean score on session 9 (m = 13.00). Regarding the measures of standard deviation, PA enjoyed smaller standard deviations (from SD = .96 on session 1 to SD = 4.49 on session 9) than the TA (from SD = 1.90 on session 1 to SD = 5.93 on session 5) which signaled less variance in PA comments on descriptive writings.

Table 5 Descriptive statistics of TA and PA on descriptive writings

Figure 1a, b displays the distribution patterns of TA and PA comments on participants’ descriptive writing, following five components in WVDE descriptive writing rubric. As it can be seen in Fig. 1, both TA and PA focused on the component of “mechanics” of writing at the expense of other components of “organization” or “development.” In both TA and PA cases, a fluctuating pattern of comments can be seen along the treatment sessions, with a decrease of the feedback points on all components of descriptive writing by final session of the treatment.

Fig. 1

TA (a) and PA (b) patterns of distribution on descriptive writings

Figure 2 illustrates the observed difference between the total TA and PA feedback points on student descriptive writing. As it can be seen, along the six sessions of treatment, the sum of TA feedback points has been growing up from m = 12.80 on session 1 to m = 24.40 on session 9, following a pullback on the last session (m = 15.20). On the other hand, with more fluctuation and irregularities, PA feedback points began to rise from m = 7.80 on session 1 to m = 13.00 on session 9 and a noticeable regression to m = 6.00 on session 11.

Fig. 2

Magnitude of TA and PA on descriptive writing

TA and PA feedback points on participants’ descriptive writing showed similarities and differences in their magnitude and patterns of distributions. The quantity of the feedback points in TA exceeded PA on all five components of descriptive writing in WVDE rubric, throughout the six sessions of treatment. Moreover, both TA and PA feedback points followed similar fluctuating pathways, paying laser-focus to the form-focused component of “mechanics of writing” and marginal attention to other four global components such as “organization” or “development.”

Another descriptive statistics was carried out with TA and PA feedback points on student narrative writing, produced and collected on the six even sessions. As it can be seen in Table 6, the mean scores were higher in TA than in PA, ranging from m = 11.80 on session 8 to m = 24.80 on session 10. PA also had its smallest mean score (m = 7.20) on session 12 and the highest mean score (m = 15.40) on session 10. Once again, PA enjoyed smaller measures of standard deviation (from SD = 5.06 on session 8 to SD = 10.92 on session 10) than TA (from SD = 5.84 on session 8 to SD = 14.37 on session 10), which suggested less variance in PA comments on narrative writings.

Table 6 Descriptive statistics of TA and PA on narrative writings

Figure 3a, b reexamines the distribution of TA and PA feedback points on students narrative writing, following five components in Smarter Balanced narrative writing rubric. Both TA and PA feedback points were divided into two areas, so that the majority of TA and PA comments escalated the “conventions” of narrative writing, while minimum attention was paid to other macro-components such as “narrative focus” or “organization” of narrative. While both TA and PA comments showed a noticeable pattern of rise and fall on “narrative convention,” the feedback points on other components of narrative writing remained marginal and steady from the beginning to the end of treatment sessions.

Fig. 3

TA (a) and PA (b) patterns of distribution on narrative writings

Figure 4 illustrates the observed difference between the total TA and PA feedback points on student narrative writing. The patterns of TA and PA distribution were less divergent and more proportional on student narrative writing. Through the six sessions of treatment, the quantity of TA feedback points has been sinking from m = 15.00 on session 2 to m = 11.80 on session 10, rising up to m = 14.37 on session 10 and falling to m = 16.18 on the last session. With less irregularities, PA began from m = 8.60 on session 2, sliding down to m = 5.80 on session 8, ebbing to m = 15.40 on session 10, and flowing to m = 7.20 on session 12.

Fig. 4

Magnitude of TA and PA on narrative writings in quantity

The distribution of TA and PA feedback points on participants’ narrative writing showed similarities than differences. The feedback points of TA outnumbered the PA on the five components of narrative writing, on all six sessions of treatment. Meanwhile, both TA and PA moved along a similar snaking pathway, paying their highest attention to “convention” of narrative writing, while equally focused less on the other global components of written narrative.

To compare the observed differences in TA and PA feedback points on descriptive and narrative writings, the researchers conducted the cross-tabulation of TA and PA feedback points (i.e., group) on the components (i.e., selected criteria), indicated in WVDE descriptive writing rubric and Smarter Balanced narrative writing rubric (Table 7). It was followed by a Pearson chi-square test to determine the statistical significance of the present divergence in the distribution of TA and PA feedback points on descriptive and narrative writings (Table 8).

Table 7 Cross-tabulation of components in descriptive/narrative writings x TA/PA

Table 8 Pearson chi-square test of TA and PA x descriptive and narrative writing

Table 7 is horizontally divided into components of descriptive writing (upper) and narrative writing (lower), while it is vertically partitioned into TA and PA feedback points and percentages in the two right columns. The overall magnitude of the TA and PA feedback points for both descriptive and narrative writing indicated remarkable convergence:

• On the student descriptive writing, both TA and PA feedback points were maximum on “mechanics of writing” (n = 158 and n = 80, respectively) and minimum on “organization” (n = 53 and n = 17, respectively).

• On student narrative writing, both TA and PA total feedback points were maximum on “conventions of narrative writing” (n = 267 and n = 190, respectively) and minimum on “elaboration of narrative” (n = 16 and n = 9, respectively).

Table 7 also summarized the total number of TA feedback points (n = 968, 68%) which was more than twice the size of PA feedback points (n = 440, 31%). The proportions of TA and PA feedback points across descriptive and narrative writings were statistically probed with a Pearson chi-square.

As illustrated in Table 8, the value of Pearson chi-square was statistically significant (X² = 50.000; df = 19; p = .000 < .05, Cramer’s V = .331), which indicated the contingency of the results. In other words, the observed differences between TA and PA comments were meaningful and statistically significant on both descriptive and narrative writing genres.

Discussion

Language assessment plays a critical role in facilitating students’ learning process and learning outcomes. The new paradigm of assessment for learning (AfL) conceptualizes alternative assessment as a teacher-student or student-student “dialogic process.” To observe the differences in the quality and quantity of teacher assessment (TA) and peer assessment (PA), the researchers in this study aimed at investigating a genre-based assessment of EFL learners’ descriptive and narrative writing. The discussion of the first and second objectives of the study which separately explored the possible differences between TA and PA on the EFL learners’ descriptive and narrative writing can be divided into two aspects of the magnitude of TA and PA and their pattern of distribution for five components in WVDE descriptive writing rubric and five components in Smarter Balanced narrative writing rubric. Statistical frequency analysis showed that while the average TA was twice more than PA in quantity, both TA and PA comments were distributed unevenly and unsystematically along the 12 treatment sessions. Moreover, it was found that the majority of TA and PA comments were microscopic, form-focused, and more targeted “the mechanics or conventions of writing” than any other macroscopic and content-based writing criteria, such as “organization or narrative focus.”

The findings implied that despite the predictable insufficient skills and knowledge of genre-based writing, the students could engage in assessing their peers’ work and critically evaluate the quality of their writing from a descriptive and narrative-genre perspective. Moreover, the similarities between TA and PA distribution suggested that the participants’ peer feedback was based on a constant reference to the multiple components in the selected writing rubrics and their close observation of the received teacher feedback. The findings were consistent with Man, Xu, and O’Toole (2018) and Yu (2020) who reported L2 student systematic access to a variety of resources in preparing their PA, including writing rubrics, peer interaction, teacher’s anchor writing, and teacher-generated written feedback. The findings provided further supports for the socio-cultural nature of PA. They also indicated that the students paid unequal attention to the components of genre-based writing and provided more form-focused than content-oriented feedback. This unbalanced and fluctuating nature of PA comments was also noticed by Yang and Carless (2013). They reported the participants’ low skills in genre-based writing as the main cause for such irregularities in their peer feedback. The students’ lack of agency to assess their peers’ language production or to generate evaluative judgment on genre-based writing was also speculated in Yu (2020).

The third objective in this study further probed different distributions of TA and PA comments on descriptive and narrative genres of writing. Significance of the observed statistical interactions was confirmed with (a) the cross-tabulation of the 10 criteria (five categories of descriptive writing, and five categories of narrative writing) in two modes of assessment (TA and PA) and (b) running Pearson chi-square contingency test. In other words, the existing difference in TA and PA nature and number was meaningful and would cause different impacts on students’ genre-based writing.

The researchers hypothesized two causes for such discrepancies, including (1) the inefficient teacher’s how-to-do tutorial on peer assessment, which made it challenging to the students, and (2) the teacher’s limited supervision over the students’ inductive ability to infer and apply the criteria of descriptive and narrative writing to their peer feedback. In a similar case study on face-to-face PA, Farahani, Nemati, and Montazer (2019) argued that the “illegitimacy” and relatively low weight that the peer feedback commonly carried in the eyes of the students often made them too critical to cast comments to their peers or too defensive to accept and apply comments from peers. Furthermore, the students’ concern over the effects their feedback would have on their peers’ feelings and emotions (i.e., the sociocultural aspect of feedback) could be another cause for their underperformance (Yu, 2020). Arguing the simplicity, inaccuracy, and uncertainty of PA comments, Lam (2010) also took a speculative approach to PA and the incorporation of PA comments by the students into their genre-based descriptive or persuasive tasks.

In this study, the components of “mechanics” in descriptive writing and “convention” in narrative writing, which received major focus in TA and PA, entailed the guidelines to preserve the accuracy in punctuation, spelling, and capitalization. Therefore, the current researchers assumed that both TA and PA comments were more microscopic and form-focused than global and content-oriented. The disproportionality of TA and PA comments could optimistically signal the participants’ confident performance on the global criteria of genre-based writing, at the cost of a more form-focused and stylistic approach to their production. From a different perspective, Yu (2020) argued that the participants would likely adopt a “passive stance” and have a tendency to focus more on the language form, rather than the genre-based conventions in their written and oral peer feedback. Farahani et al. (2019) also speculated an unwanted synergy or a “sequential skill acquisition” (Schunk & Zimmerman, 1997) between TA and PA, as a consequence of students’ close “observation and emulation” of the teacher corrective feedback to increase their evaluative judgments in PA practice. The findings of this study corroborated in to a number of other experimental research as far as the differences between teacher assessment and peer assessment are concerned (Sermsook et al., 2017; Tai, Lin, & Yang, 2015; Wang, 2014; Zarifi, 2017). In addition, the findings were in line with a number of studies that supported the positive roles of both TA and PA in directing learners’ attention to different quality aspects of their writing process (Bitchener, 2008; Diab, 2015; Li, Link, & Hegelheimer, 2015; Rassaei, 2013; Yilmaz, 2013).

Conclusion

The findings of this study enriched our understanding of the distribution and disposition of genre-based teacher and peer assessment in EFL context. This study simulated a “multi-perspective” assessment setting in which the solid and outcome-oriented teacher assessment was integrated into dynamic and community-oriented peer assessment in EFL learning environment. The efficacy of each mode of assessment can be predicted with reference to the extent of teacher-student sociocultural interactions, type of language tasks, and the purpose of the course (Farahani et al., 2019). Besides the critical lead of TA as a standard practice, PA as a shared responsibility will help students adopt a more proactive role to reflect on their own writing and others, to evaluate their performance against a well-defined rubric, to re-draft and revise it, and to learn and improve their own work.

As pointed out by Brammer and Rees (2007), the success of implementing peer feedback heavily relies on how well the assessment criteria and a sense of shared community are built by the teacher in the classroom. Therefore, teachers themselves need systemic training on encouraging PA in a regular and more informative fashion, supplementing their formal and summative assessment. In addition, PA can be more effective when face-to-face peer feedback is accompanied with computer- or mobile-mediated modes of communication (CMC) which enable the language students with synchronous modality of peer feedback. Hence, the language teachers are recommended to explicitly model the procedure of peer assessment, the assessment scale as well as the modem technologies to the students long enough to reassure their acquired skills.

Attempts were made in this study to eliminate the lapses in data collection, content analysis, and data interpretation. However, there are a few limitations imposed on the researchers’ drawn conclusions. In this study, 20 EFL learners at the intermediate level of language proficiency were selected with a non-random purposive sampling method. Consequently, the findings in this study should be generalized with adequate cautions in similar context of L2 learning, or with L2 learners at different language proficiency profiles. Still, the findings in this study on how genre-based TA and PA might affect the EFL learners writing can enrich the language teachers’ understanding in various educational settings. Secondly, the researchers narrowed their scope down to general TA and PA practices in this study and made no distinction across the types of TA and PA feedback, such as clarification, correction, problem identification, or explanation (Hanjani & Li, 2014) on descriptive and narrative writings. Future researchers are highly recommended to take such strategies into consideration and add more depth and dimensions to their work. Thirdly, the current researchers collected the data exclusively from written TA and PA. Yet, adding the records of face-to-face feedback could log richer data from the students’ interactions in pairs or in groups while using (non)verbal modes of communication. Last but not least, before partaking this study, the selected participants had already received formal instructions to genre-based writing in previous EFL courses. Therefore, their writing experience might have compromised their current writing performance to the extent that would affect the generalizability of the results in this study.