Enhancing learning potential score in EFL listening comprehension and self-regulation through self-regulated dynamic assessment procedures

Abstract

This study aimed to integrate self-regulation strategies into dynamic assessment procedures of listening comprehension in an EFL setting and examine the possibility of raising EFL students’ listening comprehension and self-regulation skills. In addition, it explored the possibility of enhancing EFL learners’ potential scores in listening comprehension and self-regulation through applying self-regulation activities as a tool for motivating learners while being assessed. The assessment procedure was based on Vygotsky’s notion of the zone of proximal development and self-regulated intervention. The participants were 49 Iranian EFL learners in three groups: a control group, which received the institute’s regular instructional activities; a comparison group, which received dynamic assessment (DA); and an experimental group which received self-regulated dynamic assessment (SR-DA) procedures, in the form of an intervention focusing on cognitive, emotional, and behavioral self-regulation states of learners. Results of the study pointed to the potential of SR-DA for enhancing the students’ learning potential over and beyond that which is available from the DA (as offered to the comparison group) and the static testing (as offered to the control group). The results have important implications for a theoretical understanding of the mechanisms through which EFL learners develop necessary self-regulation skills in the EFL context.

Introduction

Self-regulation defined as “self-generated thoughts, feelings, and actions that are planned and cyclically adapted to the attainment of personal goals” (Zimmerman, 2000, p. 14) has recently attracted a substantial amount of attention in various academic domains (e.g., McClelland, Acock, Piccinin, Rhea, & Stallings, 2013; Park & Kim, in press) and in the second/foreign language context (Bai & Wang, in press; Tsuda & Nakata, 2013; Uztosun, 2020; Zeng & Goh, 2018). In addition, the possibility of its development in an EFL context has been investigated and proposed in a number of studies (Bai & Wang, in press; Nakata, 2016; Zeng & Goh, 2018). However, the results of the interventions on the learners’ L2 performance and self-regulation have been assessed through statistic assessment procedures and the dynamic nature of learning has been neglected. Thus, it seems that the sociocultural perspective can be an alternative framework for implementing self-regulation in educational settings.

According to Vygotsky (1998), dynamic assessment, as an assessment approach which is basically rooted in the sociocultural theory and specifically in Vygotsky’s notion of zone of proximal development (henceforth, ZPD), seeks to diagnose abilities that are matured as well as those that are still in the process of maturing. He believed that human mental functions develop as a result of participation in social activities mediated by others and cultural artifacts (Vygotsky, 1978). Applying Vygotsky’s ZPD framework, a number of researchers (Abdolrezapour, 2017; Kozulin & Garb, 2002; Lantolf & Poehner, 2004; Poehner, 2009; Yang & Qian, 2020) have focused on its pedagogical applications in applied linguistics. The main goal of such studies has been determining the learners’ potential level besides their actual level to show the imperfection of the standard tests designed exclusively to measure the learners’ current performance level. However, the role of dynamic assessment (DA) with a focus on the learners’ cognitive, emotional, and behavioral self-regulation states in the foreign language learning context has been a slightly neglected area of inquiry. Hence, the present study intends to examine how the integration of self-regulation into DA procedures can affect foreign language learners’ self-regulation and strategy use to learn and how this intervention can affect their L2 listening performance. In addition, the possibility of enhancing the learners’ potential scores in self-regulation and listening comprehension through self-regulated dynamic assessment (SR-DA) was the other concern of the present study. As a result, attempts were made to answer the following research questions in this study:

• 1- Is there any significant difference between SR-DA, DA, and control groups in terms of listening comprehension scores?

• 2- Is there any significant difference between SR-DA, DA, and control groups in terms of SR scores?

• 3- Is there any significant difference between SR-DA, DA, and control groups in terms of listening comprehension potential scores?

• 4- Is there any significant difference between SR-DA, DA, and control groups in terms of SR potential scores?

Review of literature

Dynamic assessment in an L2 context

According to Vygotsky and his colleagues, conventional testing approaches do not reveal an individual’s full range of abilities and there might be significant differences between the performances of two persons with similar abilities in traditional testing and their performances after providing them with mediation in dynamic assessment. Following this line, an increasing number of psychologists and educators recognized the potential relevance of ZPD for assessment, and consequently more and more systematic procedures integrating mediation into assessment emerged (see Sternberg & Grigorenko, 2002; Tiekstra, Minnaert, & Hessels, 2016 for a review).

Despite the robust literature on DA in psychology, there was scant attention in the second/foreign language learning context till the twenty-first century. Nevertheless, recently, we are witnessing a growing body of literature using it in applied linguistics context, especially after the seminal works of Lantolf and Poehner (e.g., Lantolf & Poehner, 2004; Poehner & Lantolf, 2005). Most of the previous studies focused on such factors as providing various types of feedback (Haerazo, Davin, & Sagre, 2019), teaching different skills such as reading (Abdolrezapour, 2017; Yang & Qian, 2017, 2020), listening (Hidri, 2014) and writing (Vakili & Ebadi, in press) as well as subskills such vocabulary (Rassaei, 2020). In most of the DA studies, there is a collaborative effort by the instructor and learner to help the learner perform more capably. The instructor first measures the learner’s unassisted performance and, based on the information attained, the required support to effect improved performance is provided.

Tiekstra et al. (2016), in a literature review of DA procedures, scrutinized the learning phases of the testing procedures in terms of the presence of cognitive, metacognitive, or motivational strategies as a goal of the examiner’s activities. The authors found that most tests included a learning phase oriented solely towards cognitive strategies. A few of them included metacognitive strategies besides cognitive ones with some activities aimed at planning behavior or evaluating one’s own activities and motivational factors never played an explicit major role in the learning phase as described in their learning phase. However, among the few studies with DA procedures attending learners’ motivation besides cognition, Abdolrezapour (2017), Abdolrezapour, Tavakoli, and Ketabi (2013), and Kazemi, Bagheri, and Rassaei (2020) can be referred. Considering the necessity to attend to learners’ emotional states and Vygotsky’s (1998) proposal of the link between cognition and emotion, Abdolrezapour (2017) applied an emotionally loaded dynamic assessment procedure in a reading course with Iranian EFL learners and found that it significantly affected the learners’ level of performance in reading tasks. In addition, Borkowski, Carothers, Howard, Schatz, and Farris (2007) discussed the importance of integrating metacognitive skills — especially self-regulation — into a dynamic assessment framework and they considered it as a bridge between assessment, education, and contextualized learning. Therefore, it is important for the teachers to provide instructional support and establish an appropriate classroom environment to foster the students’ development of cognitive, emotional, and behavioral self-regulation states.

Self-regulation

Self-regulation involves the ability to recognize, regulate, and express emotions in accordance with situational demands. According to Zimmerman (2011), students are self-regulated “to the degree they are meta-cognitively, motivationally, and behaviorally active participants in their own learning processes” (Zimmerman, 2011, p. 49). Following Pintrich, Smith, Garcia, and McKeachie (1993), in this study, self-regulation will be examined in terms of four affective and cognitive factors, namely: (a) metacognitive strategies which entail the “use of strategies that help students control and regulate their own cognition” (Pintrich et al., 1993, p. 803), (b) cognitive strategies defined as the “use of basic and complex strategies for the processing of information from texts and lectures” (Pintrich et al., 1993, p. 802), (c) perceived self-efficacy defined as “people’s beliefs about their capabilities to produce designated levels of performance that exercise influence over events that affect their lives” (Bandura, 1994, p. 71), and (d) intrinsic value which refers to the importance that student’s place on their classwork, the challenges, and goals they set for themselves and their intrinsic interest in classwork (Pintrich & De Groot, 1990).

The general belief that self-regulation can be enhanced through providing appropriate support to learners in self-regulated settings (e.g., Chien, 2019; Pahigiannis & Glos, 2020; Zimmerman, 2011) is in line with sociocultural theories of (language) learning (e.g., Lantolf & Poehner, 2004; Lantolf & Thorne, 2006) and has encouraged the students’ training in self-regulation based on the notion of scaffolding within the learners’ zone of proximal development (ZPD) (see Vygotsky, 1978). Self-regulation has attracted the interest of many professional disciplines and there is abundant evidence pointing to its determining role in important real-life outcomes. It has been known to be greatly linked to long-term educational outcomes (McClelland et al., 2013), one’s ability to choose, organize, and create environments that are optimal for learning (Kolovelonis & Goudas, 2013), protection against the consequences of adverse experiences (Banyard, Hamby, & Grych, 2017), and higher physical health in childhood and adolescence (Bub, Robinson, & Curtis, 2016).

The existing literature has a lot to offer in terms of the positive correlation between self-regulation and academic achievement (Park & Kim, in press; Yumusak, Sungur, & Cakiroglu, 2007), the efficacy of pedagogical uses of self-regulation activities in learning contexts, design of self-regulated activities, and the possibility of enhancing individuals’ self-regulation level (see Murray, Rosanbalm, & Christopoulos, 2016, for a comprehensive review). And as there is a substantial body of literature on self-regulation in educational settings, it has received abundant attention in the second/foreign language acquisition contexts (Bai & Guo, in press; Bai & Wang, in press). Such studies point to the fact that self-regulated learners are more likely to have greater language learning achievements. Research has also revealed that self-regulated learning (SRL) is intertwined with many internal and sociocultural factors (e.g., Tsuda & Nakata, 2013). On the other hand, decades of research literature on self-regulated enhanced learning have acknowledged the positive effect of peer relations in self-regulation development and further academic gains (Murray et al., 2016; Park & Kim, in press).

Listening comprehension

Despite the common agreement among many language teachers and researchers over the primacy of listening compared to other language skills (Asher, 1969; Vandergrift, 1997, to name but a few), there is a lot of evidence that this skill is still undervalued. Most studies on listening pedagogy confirm the fact that teachers generally follow a testing model, through which after performing listening tasks, teachers check the answers and provide little information regarding how to improve their listening (Field, 2008).

Nevertheless, after the proposal of the three types of learning strategies (namely, cognitive, metacognitive, and socio-affective) by O’Malley and Chamot (1990), a number of attempts have been conducted focusing on different skills and strategies applied by EFL learners. One of the pioneering attempts on listening strategies applied by adult ESL learners was carried out by Murphy (1985), who suggested that effective listeners used a wider variety of strategies and engaged in more active interaction with the text. This finding was later supported by Vandergrift (1997), who pointed that not only successful listeners used more strategies than those of lower levels; they also applied more metacognitive strategies, such as comprehension monitoring, problem identification, and selective attention. Socio-affective strategies (such as questioning for clarification, cooperation, lowering anxiety, self-encouragement, and taking emotional temperature) also increased by course level. Therefore, there is a need to teach such strategies to improve learners’ listening and support for its possibility comes from Vandergrift (2002), who found that reflection on the processes of listening can help students develop metacognitive knowledge and they might achieve greater success on these types of L2 listening tasks. According to the literature, good listeners are those self-regulated learners who apply different strategies before, during, and after the task and they know that they are better listeners because of their active and strategic listening.

On the other hand, there is good evidence pointing to the effectiveness of interlocutors’ emotional and affective involvement in their listening performance (Abdolrezapour & Ghanbari, in press; Rost, 2011), which may raise or lower their anxiety and self-confidence and consequently, their motivation and interest in participation. The relation between foreign language anxiety and listening performance was also approved in a number of investigative attempts such as Mills, Pajares, and Herron (2006) and Zhang (2013). Thus, following Zeng and Goh (2018), who found that a self-regulated learning approach was influential in developing L2 listening, in this attempt, the SR-DA procedure was used as a strategy to modify the learners’ listening performance through the assistance provided to them, to lower their anxiety by developing their SR, and subsequently to investigate the effectiveness of such intervention in their learning potential.

Learning potential score

One of the earliest works on the application of DA in educational settings was Budoff and Friedman (1964) who measured the learning potential. Considering the fact that children from immigrant, minority, and low socio-economic status backgrounds were disadvantaged by traditional IQ tests, Budoff applied learning potential as an alternative indicator of abilities not based on a one-session measure of independent performance but based on their responses to intervention. Following a pretest-intervention-posttest procedure, Budoff tracked changes in the individuals’ scores, arguing that those whose scores improved after the intervention had high learning potential. Later, Kozulin and Garb (2002), building on the work of Budoff, devised a formula to operationalize student learning potential that differentiated between high and low learning potential students. They noted that some students with high and low learning potential obtained the same scores in the pretest, showing that DA adds important information that remains hidden in static testing. Previous studies provided evidence for the effectiveness of DA in estimating the students’ learning potential (Cioffi & Carney, 1983), and the activation of specific strategies that may help children overcome reading comprehension difficulties (Kletzien & Bednar, 1990). Also, Poehner and Lantolf (2013) suggest that learning potential score (LPS) does capture some feature of learning, at least as it occurs within the context of the test.

Following the previous studies which aimed to provide a theoretical-practical framework to link self-regulated learning activities in listening tasks and those which aimed to apply dynamic assessment procedures to improve L2 listening performance, and considering Borkowski et al.’s (2007) proposal of the integration of self-regulation strategies in DA procedures, this study seeks to follow four interrelated hypotheses: (a) SR-DA learners’ listening scores are comparatively more than the DA and control groups, (b) SR-DA learners’ SR scores are comparatively more than the DA and the control groups, (c) SR-DA learners’ listening potential scores are comparatively more than the DA and control groups, and (d) SR-DA learners’ SR potential scores are comparatively more than the DA and the control groups.

Method

In this study, a quantitative approach was implemented to assess the effects of three approaches to teaching EFL listening comprehension on learners’ SR and listening performance. To this aim, we had three groups of learners going through different interventions in the course of eight weeks. Figure 1 shows the different stages of the study.

Fig. 1

Design of the study

Participants

The sample of this study consisted of 49 Persian native speakers with the age range of 14 to 17 years old (M = 15.48, SD = 2.21). They were all female and all of them participated in a private language learning institute. They mostly had studied English for about three years in that institute. All the participants were currently taking the intermediate EFL course offered by the institution. However, to ascertain their homogeneity in terms of proficiency prior to the experiment, the Oxford Placement Test 2 (Allan, 1992) was used and the results confirmed their initial homogeneity. The participants were in three intact classes taught by the same instructor. One class was taken as the experimental condition with 16 students; the comparison condition involved 16 students; and the control condition had 17 students. All ethical protocols were followed in conducting the research. The participants were informed that their participation in the study was entirely voluntary, and they were assured of the confidentiality of their responses. Also, consent forms to participate were obtained from the students’ parents.

Instruments

For the purpose of the present study, a number of instruments were used which will be described in order:

Listening comprehension test

In order to find the appropriate listening material to assess the subjects’ listening comprehension ability prior to and after the experiment, a series of 20 potentially useful texts was rated by a group of four colleagues. Teachers were asked to listen to each text once and immediately afterward, independently ranked it on a level of difficulty from one to five. On the basis of these responses, 10 authentic oral texts (each comprising 6 questions) at the intermediate level were chosen and two listening comprehension tests, each comprising 30 items, were prepared, one serving as the pretest and the other for the posttest. The listening test included several passages, each one closely related to the life experience and interests of the adolescents but with no special focus on the SR factors to eliminate the possible effect of over-performance by the experimental group, each followed by a few questions assessing listening ability in various forms, for example, multiple-choice, short answer, and true/false. The reliability (Cronbach’s α) of the pretest was 0.84 and for the posttest, it was 0.87.

Self-regulation

One of the most widely used instruments in assessing learners’ self-regulation strategies is the self-report questionnaire (Roth, Ogrin, & Schmitz, 2016) and various questionnaires have been developed and validated to measure self-regulation in different language skills such Bai’s (2015) Questionnaire on SRL strategy use in English writing; Tseng, Dörnyei, and Schmitt’s (2006) self-regulating capacity in vocabulary learning scale; and Uztosun’s (2020) self-regulated motivation for improving speaking English scale. In this study, questionnaire items were adapted from established questionnaires. It corresponded to four affective and cognitive factors, namely metacognitive strategies, cognitive strategies, self-efficacy, and intrinsic value as proposed by Pintrich et al. (1993) and consisted of 52 items to be rated on a six-point Likert scale, with 1 indicating “strongly disagree” and 6 indicating “strongly agree.” One of the most widely used instruments designed to measure self-regulated learning is the Motivated Strategies for Learning Questionnaire (MSLQ) (Jackson, 2018; Roth et al., 2016). In this study, it was used to measure learners’ intrinsic value (e.g., “I like what I am learning in this class”; 9 items), self-efficacy (e.g., “Compared with other students in this class I expect to do well”; 9 items) and cognitive strategy use (“When I study for a test, I try to put together the information from class and from the book”; 13 items). As for metacognitive strategy use, following Zeng and Goh (2018), we decided to apply one of the most widely used instruments designed specifically for listening comprehension, i.e., the metacognitive awareness listening questionnaire (MALQ), to assess the L2 learners’ metacognitive awareness about listening strategy use and themselves as L2 listeners (Vandergrift, Goh, Mareschal, & Tafaghodtari, 2006). This 21-item scale yields scores on five factors of metacognitive awareness about L2 listening, namely problem-solving, planning and evaluation, mental translation, directed attention, and person knowledge. In an exploratory factor analysis of a large international sample of Iranian EFL students, Vandergrift et al. (2006) demonstrated that the psychometrics of MALQ enjoys validity and reliability. Also, internal consistencies (Cronbach’s alphas) were calculated for each of the factors, based on the responses provided by participants in the pretest and posttest. In this study, internal reliability estimates were satisfactory; 0.78 for the six problem-solving items, 0.82 for the five planning and evaluation items, 0.81 for the three mental translation items, 0.84 for the four directed attention items, and 0.83 for the three person knowledge items, respectively. In addition, the Cronbach’s alpha for the three factors of self-regulation were: 0.87 for intrinsic value, 0.89 for self-efficiency, and 0.91 for cognitive strategy use.

Procedures

The test of listening comprehension and self-regulation questionnaire were administered to check the equality of the three conditions in terms of listening comprehension and self-regulation prior to the experiment. Then, the learners went through an intensive 80-hr program presented in four sessions over eight weeks. The participants had about ten hours of English per week - 3 h for listening activities, and 7 h for speaking, reading, and writing. There were 32 sessions between the pre and post-tests, each lasting 150 min. This intervention was integrated into the institute’s syllabi, after ensuring the permission and support of the institute and the teacher. The subjects of the three conditions were exposed to listening activity tasks; those in the experimental group had self-regulated dynamic assessment (SR-DA) intervention, while subjects in the comparison group (DA) went through dynamic assessment procedures of listening comprehension, whereas the control group went through the regular institutional procedures with texts of the same level. Listening tasks and activities designed for all groups, including the control group, focused on the basic strategies applied by successful listeners including the following: (a) reflective listening to comprehend meaning and content, (b) reflective listening to clarify feelings, (c) formulating hypotheses to develop high inferencing capacities, and (d) listening to nonverbal cues. In each session, the instructor provided some information regarding the importance of developing good listening habits to all three groups, and listening was mostly practiced in situations that were challenging. Finally, posttest measures (listening comprehension and MALQ) were administered to the three groups.

Experimental group activities

Dynamic assessment intervention

This study applied a cumulative interactionist approach to DA as the format of DA was not prescribed and the mediation was directed to an individual student making an error while the peers were listening to the exchange and benefiting from it (Poehner, 2009). On the whole, we had the dynamic assessment of listening tests in six sessions, consecutively between the pretest and the posttest. The first time, there was no intervention after the test which helped the instructor develop an understanding of the learner’s baseline performance in listening. Here, the instructor provided a checklist for listening comprehension ability on the board and asked the learners to self-assess their listening ability against the checklist and talk about their strengths and weaknesses. The checklist included different levels of listening comprehension provided by Field (2013) such as input decoding, lexical search, and parsing. Also, additional information about technical terms was given to the learners.

This type of listening test was administered five more times, with 5 different passages, each at learners’ listening level, and they were mostly based on either the comprehension of main ideas or focused on understanding the implicit meaning in a conversation. After each test, there was the intervention procedure in which the teacher used the incorrect responses provided by the learner as a source of information for the listening difficulties she experienced. Then, elaborated feedback about the learner’s performance on the listening test (e.g., what aspects were correct or not) was given and some guiding questions were posed to help the learners find the correct answer.

Self-regulated dynamic assessment (SR-DA)

On the basis of the hypothesis that test performance would be improved by combining a “psychology tool” and a “stimulus” with an assessment task (Brown, Campione, Reeve, Ferrara, & Palincsar, 1991; Brown & Ferrara, 1985), self-regulated dynamic assessment (SR-DA) procedures were developed. The difference between SR-DA and DA lies in the type of intervention provided to the learners. After each test (similar to the DA group we had 6 tests, the first was used to gain some information about the learners’ current level), we had the SR intervention procedures, which were designed to give the learners elaborated feedback about the performance in the preceding section, pose guiding questions, encourage task involvement, make intrinsic motivation, regulate examinee’s behavior, and aimed at focusing their attention on the emotional and cognitive aspects of the task.

As for the SR intervention, Zimmerman’s (2000) cyclical model was followed. This model proposes three sequential phases named forethought, performance, and self-reflection for the learners. In the forethought phase, learners performed task analysis. In the performance phase, they had the listening tasks and they were guided on how to self-control and self-observe while doing the tasks. In fact, they were made aware of various strategies which were applicable to complete the task, and gained metacognitive awareness. Finally, in the self-reflection phase, occurring after the task, they had to judge their performance.

Self-regulated intervention was presented to the learners as follows:

• 1. Introduction. The teacher introduced the notion of self-regulation, its importance in one’s life, and modeled it through providing different examples in different stories. Then, she had students practice it in the class and had a group discussion about this strategy and about its use in different learning situations.

• 2. Self-efficacy. To nurture the learners’ self-efficacy, the teacher first provided them with a moderately difficult listening task which was found to be slightly above the students’ current ability level (its difficulty was checked in previous assessment settings with learners of the same level) and provided feedback if required and guided the learners through the task to motivate them to make their best effort. In some cases, the teacher acted as a model. If one student failed to answer a question, she was told that she has the ability to do the task and the failure was due to not spending enough time on the task or not applying the listening strategy (not because of the lack of language knowledge). Then, the teacher drew learners’ attention to their successful performances made. They were told that they should develop their own internal standards for performance evaluation.

• 3. Intrinsic value. To raise the learners’ interest and enjoyment, the teacher focused on tasks related to learners’ prior knowledge and experiences which captured their attention to make the learning meaningful and engage them in personally relevant tasks. In some cases, they were given choices about the subject of the listening task. And the instructor ensured that the task, while being challenging, is neither difficult nor too easy.

• 4. Cognitive strategies. It should be pointed that while doing listening tasks, cognitive and metacognitive strategies were taught to all three groups (i.e., SR-DA, DA, and control groups) but more emphasis was put on these two strategies in the SR-DA group. Thus, various tasks with various levels of difficulty were provided to learners to teach cognitive strategies such as predicting content, linguistic inferencing, listening to the familiar words, and known topics and listening for redundancies such as repeated words.

• 5. Metacognitive strategies. To raise the learners’ level of metacognitive awareness, the teacher incorporated pre-listening and post-listening activities, during which they were instructed to apply planning, monitoring, and evaluating strategies. In the pre-listening stage, the teacher instructed learners to be prepared for what they would hear and what they are expected to do. Then, during the listening activity, learners were asked to monitor their comprehension and make decisions about the type of strategy appropriate for the task. Finally, the teacher encouraged learners to self-evaluate and assess the effectiveness of the strategies used.

• 6. Group discussion. Learners were engaged in a group discussion about their self-regulated language learning experiences and the strategies they applied to decrease their listening anxiety.

• 7. Handling relationships and caring and compassion practice. To raise learners’ social skill level, they were invited to listen to the interactions between people and talk about the behaviors which get a positive response from others. In addition, learners were guided to care for themselves and other individuals in their immediate environment including their parents, friends, and siblings.

Listening comprehension tasks and evaluation criteria

A number of listening tests (in multiple-choice format) chosen from available standardized tests of listening (e.g., TOEFL) were selected and given to the learners. All the texts used in this study were similar in terms of length and difficulty level. After the completion of listening tasks, the DA and SR-DA teachers gave learners support and feedback whenever they needed. The instructor herself interrupted them at various points to ask questions, offer suggestions and provide help when necessary or to make general comments.

Data collection and analysis

Learners’ self-regulation and listening comprehension abilities were assessed using the quantitative methods. The data sources were SR scores (obtained from adding the four subscales) and scores obtained in the listening tasks both in pretest and posttest. Then, the learners’ learning potential scores (LPS) in listening comprehension and self-regulation were calculated using the following formula (adopted from Kozulin & Garb, 2002):

$$ \mathrm{LPS}=\left[\left({\mathrm{S}}_{\mathrm{post}}-{\mathrm{S}}_{\mathrm{pre}}\right)/\operatorname{Max}\ \mathrm{S}\right]+{\mathrm{S}}_{\mathrm{post}}/\operatorname{Max}\ \mathrm{S}=\left(2{\mathrm{S}}_{\mathrm{post}}-{\mathrm{S}}_{\mathrm{pre}}\right)/\operatorname{Max}\ \mathrm{S} $$

where S _post and S _pre are posttest (or mediated) and pretest (or actual) scores and Max S is the maximum obtainable score.

Results and discussion

In this study, there was an attempt to explore the feasibility of the development and implementation of SR-DA procedures in the EFL context and examine the combined effects of self-regulated activities and dynamic assessment on EFL learners’ listening comprehension. To do so, we had three groups with three different listening tasks during the course, one receiving the SR-DA treatment known as SR-DA group, the second group (i.e., the DA group) received the DA treatment and the third group (i.e., the control group) had the regular EFL listening tasks applied in the language institute. In addition, the study focused on learners’ learning potential scores in listening comprehension and self-regulation. Based on this, a series of quantitative analyses were performed to investigate the effectiveness of the intervention on learners’ listening and SR as well as their learning potential scores.

Listening comprehension

First, mean scores and SDs were calculated for listening pretest and posttest scores of the learners in three groups (see Table 1). The results of ANOVAs proved that there were no statistically significant differences in the case of pretests, with F = 0.134 and p = 0.785 (p > .05).

Table 1 Descriptive and inferential statistics on listening pretest

However, based on the descriptive and inferential statistics provided in Table 2, the SR-DA group performed significantly different from the other two groups in listening comprehension posttest; SR-DA > DA, SR-DA > control group (CG), and the DA group differed significantly from the CG, DA > CG.

Table 2 Descriptive and inferential statistics on listening posttest

Thus, the first hypothesis, i.e., SR-DA learners’ listening scores are comparatively more than the DA and control groups was confirmed. It should be noted once again that the listening posttest was very different from the listening tasks of the intervention provided to the SR-DA group to ensure that the test did not bias in favor of this group and to test the strategy transfer to a different task. Thus, we can claim that learners’ engagement in self-regulated strategies affected their listening performance. This finding was in line with the existing literature (Zeng & Goh, 2018) which pointed to the effectiveness of the self-regulated approach for developing L2 listening and the studies (e.g., Vandergrift, 1997, 2002) which confirmed that reflection on the processes of listening can help students develop metacognitive knowledge and it might result in greater success in L2 listening tasks. In addition, the higher scores gained by the DA group confirm Hidri’s (2014) findings in that similar to his study, the DA approach in our study helped the instructor locate the learners’ weaknesses and it provided insights into learners’ cognitive and metacognitive processes and it ultimately led to improved listening performance by this group in this posttests.

The foreign language setting is accompanied by various positive and negative feelings. One’s success in this setting requires handling such emotions in the most efficient way. Following the literature (Abdolrezapour & Ghanbari, in press; Zeng & Goh, 2018), extensive exposure of learners to self-regulated tasks and activities would result in enhanced social development and a better adjustment in challenging tasks. Thus, one reason for the higher success of SR-DA group can be attributed to the affective involvement of L2 listeners. In fact, considering the close relationship between learners’ emotional states and their cognitive functioning, their increased engagement in emotional activities helped them perform better in listening tasks which require both lower and higher-level processing.

To check the learners’ improvement in self-regulation level, their posttest scores were compared to their pretests, and the results are discussed in the following section.

Self-regulation

Table 3 shows pretest and posttest scores obtained by each group in each subscale of SR. As can be seen, the control group presented lower scores in most subscales of the posttests compared to other groups, despite their initial similar performance in pretests, except for mental translation and person knowledge. Based on the descriptive statistics, the SR-DA group performed differently from the other two groups in most SR factors (i.e., intrinsic value, self-efficacy, cognitive strategies, problem-solving, planning and evaluation, and directed attention); SR-DA > DA, SR-DA > CG, and the DA group differed significantly from the CG in the same subscales; DA > CG.

Table 3 Descriptive statistics on pretest scores of SR subscales

Thus, the second hypothesis, i.e., SR-DA learners’ SR scores are comparatively more than the DA and the control groups, is confirmed for most subscales. The evidence for the possibility of increasing SR with its subsequent effect on the learners’ academic performance (listening performance in the current attempt) comes from previous studies (Bai & Wang, in press; Chien, 2019; Murray et al., 2016; Pahigiannis & Glos, 2020). Thus, the extensive exposure of participants in both DA groups to activities focusing on SR and their interaction with the physical and social environment (as in our case, interactions with teachers, peers, and parents) resulted in greater motivational beliefs and achievement in SR, which was in accordance with previous studies (Murray et al., 2016; Nakata, 2016; Zeng & Goh, 2018). These findings add further support to Borkowski et al.’s (2007) suggestion of the integration of SR into DA procedures and Brown et al.’s hypothesis that test performance would be improved by combining a “psychology tool” and a “stimulus” with an assessment task (Brown et al., 1991; Brown & Ferrara, 1985).

SR-DA’s higher grades in self-regulation and listening performance were in line with Bai and Wang’s (in press) claim regarding the effectiveness of self-regulation strategies, which help learners effectively monitor and expend efforts to regulate their learning, on achieving better English language learning results. The authors proposed that nurturing intrinsic value, and self-efficacy can foster students’ self-regulated learning.

In addition, our results pointed to an increase in the SR-DA and DA groups’ level of self-efficacy. Several studies pointed to the possibility of increasing self-efficacy and its further influence in language learning (e.g., Bai & Guo, in press) and listening performance (Mills et al., 2006); in this regard, our results confirm the literature. Moreover, our study lends further support to previous findings regarding the possibility of increasing learners’ level of intrinsic value. Bai and Wang (in press) claim that learners with higher intrinsic value “are more likely to control their efforts and attention in the face of distractions and setbacks, and make plans for their own learning” (p. 14); thus, SR-DA higher attainment in intrinsic value may directly or indirectly have resulted in higher scores in listening posttests. In this regard, we can claim that our findings are in accordance with Noels, Vargas Lascano, and Saumure (2019) who pointed to the importance of the learner’s positive intrinsic motivation, and Deci and Ryan (2000), who proposed that higher intrinsic value would help learners be less disturbed by distractions and they tend to focus on academic tasks for a longer time.

The results are also aligned with the literature (Tanewong, 2019; Vandergrift & Tafaghodtari, 2010), which pointed to the effectiveness of strategy instruction in raising EFL learners’ listening performance. Listeners in the SR-DA group demonstrated overall improvement in the five metacognitive factors (i.e., problem-solving, planning and evaluation, mental translation, directed attention, and person knowledge) compared to their counterparts in the DA group and the DA group showed better performance compared to the control group. Thus, teaching behavioral, cognitive, and metacognitive strategies results in self-regulated listeners who can better process and perceive the aural input. As for mental translation, it was found that SR-DA and DA learners resorted to this strategy less; however, some still required to translate some parts and this refers to their increased understanding which was in line with previous studies (Tanewong, 2019; Vandergrift & Tafaghodtari, 2010).

Also, following Vygotsky’s concept of ZDP (Vygotsky, 1978) that put emphasis on what a child may potentially become rather than what (s)he is, these results show that paying attention to the learners’ self-regulation results in a more extensive ZPD than the one reached through pure DA procedures. Accordingly, under self-regulated DA procedures, the learners achieve markedly superior results in both tests (SR and listening). In addition, the higher achievement of the DA group in posttests in comparison to the control group confirmed Vygotsky’s idea that a learner’s academic potential can be enhanced beyond what standardized tests measure.

The main reason for using the DA approach in assessing one’s SR and integrating self-regulation into DA of listening comprehension in this study is the importance given by DA proponents to learner-environment interactions and the incorporation of students’ cognitive, emotional, and motivational states into the instructional activities in order to keep students engaged, increase their interest, and presumably maximize their learning (Borkowski et al., 2007; Tiekstra et al., 2016). To the best of our knowledge, affect and motivation have not received much attention in DA research (with the exception of Abdolrezapour, 2017 and Kazemi et al., 2020) and more specifically in L2 DA research, especially from an empirical perspective. Thus, the inventory of SR-DA mediational strategies presented in this study extends our understanding of how to attend to learners’ emotional states in addition to cognitive status in DA procedures and breaks new ground for the integration of SR into DA procedures of listening assessment. In accord with our hypothesis, the study showed that in addition to the direct effect of SR-DA intervention on the learners’ self-regulation and their higher ability to manage positive and negative emotions, it indirectly affected their performance in listening tasks, through its impact on adaptation in diverse intellectually complex situations.

Learning potential scores

In order to find the students’ learning potential, the formula proposed by Kozulin and Garb (2002) (LPS = (2S _post − S _pre)/Max S) was applied. Figure 2 shows the learners’ LPS in the listening skill.

Fig. 2

Distribution of learning potential scores in listening for each group

As can be seen in the figure, learners’ LPS ranged from 0.63 to 1.37 and the SR-DA group had the highest scores, which according to Kozulin and Garb (2002) can be called the high learning potential group. In this way, the third hypothesis, which posed SR-DA learners’ listening potential scores are comparatively more than the DA and control groups, is confirmed.

And Fig. 3 contains learners’ potential scores in self-regulation, according to which, the SR-DA group, once again, received the highest scores.

Fig. 3

Distribution of learning potential scores in SR for each group

So, our fourth hypothesis (i.e., SR-DA learners’ SR potential scores are comparatively more than the DA and control groups) is approved. According to Vygotsky, performance tests that include individuals’ ZPD help instructors recognize how close learners are to successful independent functioning. The significant difference between the listening performance and SR scores of the three groups obtained in the previous subsection and high LPSs reported above both approve the effectiveness of the intervention and confirm the fact that the participants responded well to the mediation.

Information obtained with regard to learners’ LPS can be used to create opportunities to develop their performance ability. For example, the learners who were in the average range in the pretest and got low learning potential scores should be through some activities which help them learn better in future activities. While, those who had scored the same in the pretest (were on the average level), but showed high learning potential, can be given some more challenging tasks. For those who scored poorly both on the pretest and learning potential, some extra activities might be designed.

Conclusion

The present study made two major contributions: (a) it provided evidence for the possibility of improving EFL students’ listening comprehension and self-regulation skills, and (b) it showed the potential of SR-DA for enhancing EFL learners’ potential scores in listening comprehension and self-regulation. Accordingly, it expanded the knowledge base concerning the relationship between SR and academic performance in general and L2 listening comprehension in particular. The findings also provide educationalists with further examination of whether SR is important to academic success. Such an analysis has implications for government policy linked to the teaching of self-regulatory skills. Initiatives that help to enhance SR (such as the intervention proposed in this study) might be integrated into existing curricula to offer educators opportunities to improve educational achievement. As for DA practitioners, learners’ efforts to become more independent, their responsiveness to the mediation, and their self-regulation should be considered as important as the quality of mediation in ZPD interactions. Thus, educational settings should provide information on various ways and strategies to foster learners’ self-efficacy, intrinsic value, and use of cognitive and metacognitive strategies. Language teachers must introduce their students to various language learning strategies in order to facilitate the learning process and improve language performances.

As the current study shows, self-regulation plays an important role in the language learning process and predicts listening performance. Thus, language instructors are suggested to enhance learners’ awareness of the importance of self-regulated learning, and design methods and procedures to first introduce various SRL strategies, then guide them to apply such strategies. They can help learners promote self-efficacy through providing a performance model and helping learners develop their own self-evaluation standards. As for intrinsic value, language tasks should be interesting, challenging, and meaningful and they should have a choice in the type of task and activity. In addition, they should be guided on ways to regulate their efforts by reminding them of the fact that learning a language needs persistence and effort.

The current study is not without its limitations, though every attempt was made to avoid some of the design, measurement, and analytical flaws. First, the participants in this study were female subjects from a private institute which might delimit the generalizability of the study. Future attempts can strengthen the generalizability by conducting similar research with male and female participants studying English at both private and public language classes. Second, although the implementation of the SR approach in an EFL classroom may prove to be incentive and useful for the students, teachers may lag behind so as to terminate the formal syllabus in the due time. In addition, applying a qualitative data collection approach such as interviews and reflective diaries might also provide more in-depth information on the learners’ perceptions of the intervention and details about the motivational variables, learners’ cognitive, metacognitive and self-regulatory strategies. Finally, to expand the use of DA and self-regulatory activities in educational settings, the provision of a more comprehensive mediational strategies inventory can also be a productive venue for future research. The inventory presented in our study only concerned listening comprehension. It can be refined to be applicable for teaching/assessing other language skills as well.