Assessing interactional metadiscourse in EFL writing through intelligent data-driven learning: the Microsoft Copilot in the spotlight

Abstract

The purpose of the current study was twofold: examining the efficacy of data-driven learning (DDL) (hands-on and hands-off approaches) in the realization of interactional metadiscourse markers (IMMs) among English as a foreign language (EFL) learners and analyzing the learners’ perceptions of DDL. The participants consisted of 93 male and female advanced language learners randomly assigned to one of the three groups: hands-on, hands-off, and control. Throughout the duration of treatment lasting for 10 sessions, the hands-on group employed the use of Microsoft Copilot, artificial intelligence (AI) chatbot, on a computer screen to discuss and explore IMMs, but the hands-off group was exposed to IMMs through written texts that were physically printed on paper and articles to be examined through AntConc concordancing program. The control group received conventional instructional techniques including reading assigned course materials. The findings from a one-way analysis of covariance (ANCOVA) procedure indicated that both experimental groups outperformed the control group in the posttest of realizing and identifying IMMs. However, the post hoc comparisons showed statistically significant differences between the hands-on and hands-off groups, with the hands-on group performing more successfully in identifying IMMs. The results of the questionnaire data revealed that all the learners had positive perception of DDL. The results of the current study suggest using both hands-on and hands-off DDL methods helps learners develop their writing performance through metadiscourse realization.

Introduction

Metadiscourse is usually defined as “meanings other than propositional ones” (Hyland, 2019, p. 21). Hyland (2019) proposed an interpersonal model of metadiscourse, which involves two dimensions of interaction: interactive and interactional (see Table 1). The former concerns a writer’s awareness of the audience’s knowledge and interests, while the latter focuses on how writers engage with readers through various IMMs. IMMs are key aspects of academic writing that establish the dynamic relationship between writers and readers (Izquierdo & Pérez Blanco, 2023). Authors utilize a variety of linguistic resources not only to convey their positions and arguments but also to establish a connection with their audience. By employing IMMs, writers can effectively engage readers, guide them through the text, and shape their understanding of the content presented (Hyland & Jiang, 2022). Non-native advanced EFL learners occasionally overuse IMMs in their written work (Paltridge & Prior, 2024).

Table 1 Interpersonal metadiscoursein Hyland’s model (Hyland, 2019 , p. 58)

DDL is an approach to integrate corpora into second language (L2) instruction and learning. In the present study, the researchers have used two DDL approaches, namely hands-on and hands-off (Römer, 2008) to examine interactional meatadiscourse. The hands-on approach involves direct DDL, where learners independently utilize corpora or concordancing software to explore language usage patterns (Vyatkina, 2024). By contrast, the hands-off approach is an indirect procedure, where educators analyze a collection of texts, identify the most commonly used words or structures related to a specific language feature, adapt and adjust the resources accordingly, and provide learners with customized paper-based concordance materials created by the teacher (Karpenko-Seccombe, 2024).

Using DDL, learners use corpus data for knowledge-construction purposes. Through such activities as concordancing, they draw on linguistic evidence to observe, analyze, evaluate, and test hypotheses, leading to informed conclusions. DDL, focusing on data discovery, aligns with emerging skills and personalized trends in language learning. It offers adaptability and potential for individualized learning experiences. This study focuses on how DDL can help advanced EFL learners improve their writing by employing IMMs. The research aims to explore whether DDL has a positive impact on realizing IMMs in writing tasks for EFL learners. If found successful, this method will enhance English language teaching and learning strategies, particularly in addressing challenges related to metadiscourse realization in language production. The findings from this study may lead to the integration of DDL into language teaching approaches and curriculum design to enhance writing quality among EFL learners.

This study compares the impact of hands-on and hands-off approaches of DDL on advanced EFL learners’ employment of IMMs. Furthermore, the research aims to assess learners’ attitudes towards Microsoft Copilot and concordancing for second language IMMs and what their perceptions are through questionnaires administered in the final sessions. In what follows, we first review previous literature, explaining the theoretical framework used in the present study, introducing and conceptually defining interactional metadiscourse, followed by the empirical studies done in this on EFL writing, and elaborating on the two types of DDL employed in this study. Next, the methodology for the present study, including the participants, instrumentation, treatment intervention, and data analysis, is fully described. Then, the findings of the study are presented, discussed in light of the theoretical model used in the study, and compared with those of previous studies. Finally, the paper ends with concluding remarks, delineating limitations and pedagogical implications of the study. The following two research questions are, therefore, formulated to achieve the goals of the present study.

• 1. Do hands-on and hands-off DDL approaches have varying effects on Iranian advanced EFL learners’ realization of IMMs in writing performance?

• 2. What do the learners think about hands-on and hands-off DDL in terms of realizing IMMs in writing performance?

Literature review

Theoretical framework of the study

The DDL approach incorporates elements of the inductive approach, discovery learning, and the noticing hypothesis (Crosthwaite, 2024). The inductive approach involves focusing on the structures being learned and formulating the underlying patterns. Teachers use inductive methods to teach linguistic items, which enable learners to make observations and form generalizations. This learner-centered method involves hypothesis formation and testing, leading to increased participation and understanding (Baepler et al., 2023; Kowshik, 2024). DDL includes inductive learning for autonomous self-directed learning in education. The next principle related to DDL is discovery learning, promoting autonomous self-directed learning (Ellis et al., 2020; Kruk & Peterson, 2020). This principle helps learners improve their language skills by enhancing their awareness and providing opportunities for exploration and discovery in a more engaging and effective way than traditional classroom methods. Discovery learning is considered to be an instructional approach that involves learners in interacting with their environment. It is centered on learners developing skills through observation, inference, and prediction, with teachers supporting their inquiry. It emphasizes hands-on experiences and encourages learners to take an active role in their education. This approach is commonly used in communicative language teaching, highlighting the importance of students actively participating in their learning journey. Discovery-based learning benefits learners by promoting active engagement, motivation, autonomy, responsibility, independence, creativity, and problem-solving skills, and offering a customized learning experience (Bremner et al., 2022; Jiménez Raya & Manzano Vázquez, 2022; Lestari et al., 2021).

As for noticing hypothesis, Schmidt (1990) stated that input does not become intake for language learning unless consciously registered. Learners first notice an external feature, allowing working memory to process it and transfer it to long-term memory. Comparing the noticed feature to previous knowledge leads to recognizing gaps (Lennon, 2020). According to the hypothesis, learning requires noticing to some extent as working memory mediates between the outside world and long-term memory. Subliminal or subconscious learning is deemed impossible by Schmidt. Noticing is influenced by the frequency and salience of the feature in the input, learner’s readiness, and task demands. The hypothesis suggests that conscious effort is essential for language learning. Researchers found that feedback improved students’ oral performance and error correction in language learning (Kartchava, 2019). The noticing hypothesis suggests that increased awareness benefits learners, which can be facilitated through input enhancement and input flooding.

DDL-centered treatments involve learners focusing on linguistic data to generalize observed rules, reducing cognitive workload, and allowing focus on language use (Chen et al., 2019; Yu & Lowie, 2020). While beneficial for nurturing induction, it may be challenging for low-level learners according to scholars (Qin & Stapleton, 2023; Zhang, 2022). Despite this limitation, it is generally effective in developing language skills through active participation and observation. DDL is an inductive approach involving five steps: observing, inferring, formulating, predicting, and communicating. By analyzing corpus data, learners act as language researchers, engaging in discovery learning to enhance language skills (Lin, 2021). The connection between discovery learning and DDL is significant, emphasizing the importance of having a certain level of linguistic knowledge for effective participation (Mirzaee et al., 2015; Rets, 2017). DDL activities promote language awareness through analyzing recurring linguistic features, ultimately enhancing language proficiency (Daskalovska, 2015; Lusta et al., 2023). This aligns with the noticing hypothesis, where heightened language awareness facilitates the transformation of input into output, supported by empirical evidence validating the efficacy of DDL in improving language acquisition (Karpenko-Seccombe, 2024).

In this study, we adhered to metadiscourse by investigating IMMs through DDL and Microsoft Copilot AI. Theoretically, the integration of Microsoft Copilot for enhancing metadiscourse realization can be situated within a framework encompassing two methods. The first method is related to the concept of metadiscourse as a writer-reader interaction proposed by Hyland (2019), who argues that writers attempt to organize the expressed information; therefore, it influences the audience to accept the stated ideas and arguments. Microsoft Copilot supports interactions by responding to user inquiries on various writing aspects, offering suggestions upon request, and serving as an on-demand support tool. The second method is Barrot's (2023) perspective on Microsoft Copilot AI as a dependable writing tool capable of providing immediate feedback to learners as they advance through various writing stages. The current study was based on these two methods, aiming to use Microsoft Copilot AI as a tool incorporating hands-on and hands-off DDL approaches, and evaluate their influence on students’ IMMs realization in academic writing performance.

Interactional metadiscourse

Writers use interactional metadiscourse (IM) to provide commentary on their messages. As Hyland (2019) stated, IM includes components such as self-mentions, hedges, boosters, attitude markers, and engagement markers, which help involve the reader in the text. Through the incorporation of IM, writers engage readers in the discourse and convey their perspective on the discussed content. Table 1 offers information on the different types of IM.

To date, the primary focus of metadiscourse research has been on academic writing, particularly research articles (RAs) (Li et al., 2023; Pearson & Abdollahzadeh, 2023; Qiu et al., 2024). For instance, bibliometric methodologies were employed to monitor shifts in the field of EAP research and identify the predominant themes, key contributors, and significant publications within the discipline spanning a period of four decades (e.g., Hyland et al., 2022). Ädel (2023) examined MMs through a move-based framework, aiming to enhance our understanding of the nature and functions of metadiscourse for a more comprehensive insight into its usage. Liu and Tseng (2021) investigated potential variations in the utilization of hedges and boosters within the discussion sections of RAs, whereas Dontcheva-Navratilova (2021) delved into the examination of engagement within RAs across the fields of linguistics and economics.

Various linguistic resources were utilized across diverse genres to organize and shape interactions. Yang (2021), for example, compared IM in advisory letters from the Chinese governmental and healthcare entities in the midst of the COVID-19 pandemic. The main aim of the research was to clarify the pragmatic role of metadiscourse in enhancing reader engagement during crisis situations. Similarly, Shen and Tao (2021) focused on articles related to COVID-19 for their research dataset, exploring the occurrence rate of stance markers in medical articles and opinion columns in press. Disseminating scientific information to diverse audiences and entailing the challenge of tailoring the communication to individuals who may lack a similar academic background indicate that the utilization of both TEDx Talks and YouTube science dissemination videos plays a crucial role in implementing effective engagement strategies (Bernad-Mechó & Valeiras-Jurado, 2023). Moreover, some research endeavors have explored the realization of metadiscourse across a range of genres, including reputable newspapers (Chen & Li, 2023), instructional manuals (Herriman, 2022), and advertisements (AI-Subhi, 2022). These investigations have provided insights into the functioning of metadiscourse as rhetorical instruments within distinct registers. Furthermore, few experimental studies have sought to differentiate between the realization of metadiscourse using different teaching methodologies (e.g., Esfandiari & Allaf-Akbary, 2024a), indicating that English language learners with different personality traits employ metadiscourse features distinctively to enrich the creation of instructional materials.

El-Dakhs et al. (2022) investigated the effects of explicit versus implicit instruction on the utilization of IMMs by 120 Arab female EFL undergraduate learners in their written work through a mixed-methods design. The results of the study indicated a positive, though limited, effect of both explicit and implicit instruction on the use of self-mentions, appeals to shared knowledge, directives, and questions. Regarding the participants’ perceptions of the instructional methods, both explicit and implicit instructions were found to be beneficial, although the learners reported having trouble applying the methods to learn the concepts because of the demands of the task difficulty.

In academic writing, the act of simplifying or reformulating can be viewed as an important tool for improving clarity and argumentation. Studies indicate that writers’ decision on the number and type of markers they use is influenced by their level of expertise and the area of their study. For this purpose, Triki (2024) analyzed 90 articles and book chapters written by six top linguists and found that successful authors in linguistics did not all follow the same practices and had diverse choices. There was no clear pattern of increasing or decreasing use of exemplification and reformulation as writers progressed in their careers, challenging the idea that experience directly impacted the frequency of these strategies. The study suggested that individual style preferences within a discourse community should be considered, highlighting the need for a deeper understanding of writing expertise and disciplinarity.

Izquierdo and Pérez Blanco (2023) analyzed IMMs in promotional texts, focusing on the informational-persuasive discourse subgenre. Due to limited research in this area, they used move analysis to fill the gap and compared English and Spanish using an ad hoc corpus of online tea descriptions. By manually tagging the corpus, they identified interactional markers establishing a direct relationship between language producers and receivers. The study found that the use of these markers varied across moves and languages, with more persuasive or instructional moves containing more commentary markers. English emphasized self-mentions, while Spanish used inclusive language like “we” more frequently.

Although metadiscourse does not possess a monolithic, definitive conceptual framework, it remains a complex concept that has been extensively explored (Hyland & Jiang, 2022). This research underscores IMMs as pivotal components in academic communication, as they enable writers to signal their presence, discuss and establish understanding assertions, and engage their audience. Consequently, empirical investigations often lack a robust methodology aligned with the realization of metadiscourse. Within the scope of this study’s objectives, metadiscourse is delineated as linguistic features within written texts that do not directly contribute to the primary message but assist readers in structuring, interpreting, and assessing the information presented (Qui et al., 2024).

Generative artificial intelligence

The swift progress of AI has transformed education, significantly affecting teaching and learning methodologies. Artificial intelligence, a field of computer science, allows machines to imitate human cognition. AI technologies in education have significant potential to revolutionize personalized learning experiences (Hwang et al., 2023). The incorporation of AI into education has attracted significant interest from global researchers and educators (Chen et al., 2022; Fitria, 2023). Aldosari (2020) defines AI as an intelligent system executing various tasks efficiently.

AI is utilized in education, imitating human decision-making and improving language proficiency. A wide array of AI-driven language learning applications is available on both computers and mobile devices, aiding individuals in their pursuit of language acquisition. These resources provide significant assistance in enhancing diverse language learning competencies (Fang et al., 2023). A significant amount of research has been conducted to investigate the impact of AI-supported language learning tools on the language proficiency of learners studying English (Schmidt-Fajlik, 2023; Xu et al., 2022).

Microsoft is distinctly equipped to provide enterprise-grade artificial intelligence through its Copilot System. Copilot extends beyond the integration of OpenAI’s ChatGPT within Microsoft 365, representing a comprehensive solution for organizational needs (Panini, 2024). As an AI-powered tool, Microsoft Copilot can be effectively employed in language learning environments to assist learners in enhancing both their language acquisition abilities and specific sub-skills associated with language mastery. This advanced processing and orchestration system functions unobtrusively to combine the features of large language models, such as GPT-4, with Microsoft 365 applications and organizational data contained within the Microsoft Graph. This integration is now accessible to all users through natural language interfaces (Scholl, 2024).

Hands-on and hands-off DDL

DDL is an instructional approach in language learning that prioritizes the learner’s active engagement by providing extensive exposure to authentic language examples. This method supports learners in independently identifying linguistic patterns and rules through their own discovery process (Crosthwaite, 2024). Corino and Onesti (2019) viewed DDL as an innovative method of enhancing awareness of grammatical rules among language learners, encouraging them to actively engage in their learning process. DDL serves as a crucial corrective role. Learners can utilize this approach to compare their own learning with data generated by proficient (native) writers or by referring to a learner corpus that highlights annotated errors. DDL approach offers the added benefit of incorporating an element of discovery, potentially enhancing the motivation and enjoyment of the learning process, and it underscores the concept of discovery learning (Lin, 2021). This approach focuses on the importance of language input to immerse learners in enhanced corpora-based databases. By doing so, both educators and learners are able to access a diverse range of target language structures within authentic contextualized examples. This is particularly valuable as it can be difficult to recreate such an immersive learning experience in traditional classroom settings (Lusta et al., 2023).

DDL is a student-centered language learning approach that utilizes authentic examples to help learners explore linguistic patterns independently. The direct engagement of learners with DDL through computer screens, referred to as hands-on DDL, has been advocated more strongly than the indirect approach, which includes paper-based or hands-off DDL. This preference arises from the observation that learners tend to retain information more effectively when they have independently discovered it through inductive reasoning (Vyatkina, 2024). The incorporation of AI into language education, as demonstrated by platforms such as Microsoft Copilot, has generated considerable discussion. Since its launch, Microsoft Copilot has emerged as a vital resource in educational settings, and its continuous development is unmistakable (Pan, 2024). Consequently, understanding the influence and educational possibilities of AI is of utmost importance.

Previous research on DDL in language learning

Previous research has identified numerous helpful and favorable outcomes of DDL in language learning. Pérez-Paredes et al. (2019) carried out a research study focused on developing a mobile application for language learning, utilizing publicly accessible natural language processing resources. The application was tested to assess the attitudes and perceptions of various groups of language learners throughout Europe. The findings indicated a predominantly favorable assessment of the DDL instant and personalized feedback, as well as its direct access to a diverse array of tools. When designing tasks for mobile-assisted language learning, smartphone hardware limitations and diverse handheld devices were not considered. It was needed to ensure users would maximize the benefits. However, the researchers should have selected a multidisciplinary approach for understanding language learning theory and application development intricacies.

Yao (2019) conducted a study, the primary aim of which was twofold: Firstly, to investigate whether a statistically significant difference exists between the DDL approach to vocabulary acquisition and more conventional methods, such as the dictionary approach; and secondly, to assess students’ attitudes towards DDL activities. To achieve these objectives, a quasi-experimental longitudinal design was implemented, involving a comparison between two groups of Chinese students learning Spanish (experimental group, n = 16; control group, n = 16). The findings from the immediate posttest demonstrated that the DDL approach was more effective than traditional methods for learning Spanish vocabulary. Additionally, results from the delayed posttest showed that the DDL group continued to outperform the control group. Moreover, a questionnaire administered to the experimental group supported the positive outcomes of the tests, revealing that participants generally preferred DDL and expressed a favorable attitude towards its future use in Spanish language learning. The experiment included 32 university students with upper-intermediate Spanish proficiency. Not only was the sample size small, but the results may also not be generalized to Spanish learners with different proficiency levels. Therefore, other studies are required to assess DDL impact on various proficiency levels.

Saeedakhtar et al. (2020) compared hands-on and hands-off DDL for realizing verb-preposition collocations in Iranian English learners. Sixty female pre-intermediate language learners were divided into three groups: hands-on, hands-off, and control. The hands-on group used concordancing tools on a computer screen, while the hands-off group received paper-based collocations. Both experimental groups received instructional dialogue, while the control group used conventional methods. The hands-on and hands-off groups outperformed the control group on the posttest, with the hands-on group showing better retention in the delayed posttest. The findings of the research indicated that even low-level learners experienced advantages from practical engagement in DDL. Participants viewed DDL favorably for learning collocations, as revealed by the questionnaire results. Since only female learners were recruited in the study, the findings may not apply to male learners. By focusing on verb-preposition collocations, the study's scope was limited to those specific pairings, affecting the generalizability to other types of collocations like verb-noun or adjective-noun.

Sun and Hu (2020) examined the effectiveness of direct and indirect approaches to DDL in helping Chinese learners improve their use of hedges in English writing. The study used a randomized design with pretest, posttest, and delayed posttest assessments. One group received direct DDL instruction involving online corpus searches, while the other group received indirect instruction through paper-and-pen tasks. Results showed the strengths and weaknesses of the two approaches. The indirect DDL intervention significantly enhanced the students’ frequency of hedge usage in the immediate posttest; however, this effectiveness diminished substantially by the time of the delayed posttest. Additionally, a questionnaire survey indicated participants’ positive attitudes towards using corpora in teaching and their awareness of the benefits and challenges of DDL in language learning. Limited exposure to corpus tools in the study may hinder the full potential of direct DDL. The potential benefits of using corpus tools and concordance work must be further explored through research on their extended usage.

Crosthwaite and Steeples (2022) conducted a six-month DDL experiment in an all-girls secondary school in Australia, focusing on the passive voice constructions in scientific research reports for a physical science class. Pre/posttests assessed learners’ knowledge and use of the passive voice, as well as their autonomous use of corpora in research reports. Learners’ perceptions of corpora and DDL were collected through questionnaires and interviews immediately and three months after training. The results showed that corpus consultation improved the use of passive voice in science writing for pre-tertiary learners. However, there were preferences and criticisms of corpus tools, functions, and usage, with weak post-training uptake. Overall, the findings suggest positive implications for DDL with younger learners.

In another study, Zare et al. (2022) employed a quasi-experimental design featuring a comparison group with pretest and posttest measures. Ninety-six university students majoring in English were evenly allocated into two distinct groups: a comparison group and an intervention group. Subsequently, these groups were subjected to a placebo and a treatment condition, respectively. The placebo consisted of 12 one-hour sessions of conventional explicit instruction delivered by a qualified instructor, focusing on identifying key points in academic English lectures. In contrast, the treatment involved 12 one-hour sessions utilizing concordancing techniques with the AntConc software for the same purpose. Utilizing questionnaires assessing foreign language anxiety and enjoyment, along with free-response surveys, the results indicated that the concordancing approach did not yield any statistically significant differences in foreign language anxiety among the students. Conversely, participants reported that the DDL method involving concordancing was less enjoyable compared to the traditional explicit instruction provided by the teacher.

Muftah (2023) examined the impact of DDL activities on Arab EFL writing in the medium term. A quasi-experimental design and interviews were used to gather data from 64 Arab EFL undergraduate students. British national corpus (BNC) web was used for DDL, and the findings were compared with Sketch Engine. The experimental group showed improved writing fluency and consistency in the posttest compared to the control group, who used Sketch Engine. However, there was no significant difference in writing complexity between the groups. Students had a positive view of using BNCweb, despite the difficulties of incorporating corpora into the writing process. One of the limitations of the study was that the effectiveness of integrating multiple reference materials in the writing process was not investigated.

The review of previous research highlights certain patterns in metadiscourse. In the last two decades, researchers have significantly enhanced our understanding of metadiscourse. Researchers have examined metadiscourse in both written and spoken language across various academic and professional genres. A significant shortcoming of most of the previous studies is that their predominant focus was on descriptive analyses of IMMs, neglecting mixed-methods studies including both experimental and descriptive ones. In order to offer further insight into how effective hands-on and hands-off DDL methods on language learning, the current research examined the impact of hands-on and hands-off DDL on the realization of IMMs among advanced EFL learners.

Method

Participants

A group of language learners comprising 120 Iranian EFL learners, aged between 23 and 36, was initially chosen for this study. The selection process utilized convenience sampling, as described by Dörnyei and Dewaele (2022), and was based on the participants’ availability. The learners were selected from the University of Mohaghegh Ardabili, which is known for its advanced EFL programs. To ensure the participants’ proficiency in English, the Michigan Test of English Language Proficiency (MTELP) test was done. After considering the test results, the researchers reduced the number of participants to 93 individuals. These participants were advanced Bachelor of Arts (BA) students, consisting of both males and females. It is crucial to mention that most of the participants’ first language was Turkish. Due to the university’s policy of not permitting the allocation of learners into separate groups, it was mandatory for the participants to attend the English language institute at Iranian institute, Ardabil Branch, where they were assigned into two distinct experimental groups and a control group.

Instruments

Data collection for the study was conducted using the specified instruments. Further details regarding these assessments are provided.

Michigan test of English language proficiency

MTELP consists of three sections presented in a multiple-choice format. This reputable test comprises 40 questions related to grammar presented in a conversational style, 40 questions focusing on vocabulary through synonyms or sentence completion, and 20 questions dedicated to reading comprehension. The entire test lasted for 100 min. Those learners who obtained a score of more than 70% were classified as advanced language learners (Phakiti, 2003). The extensive research conducted by some scholars (e.g., Johnson & Lim, 2009) has further confirmed the great reliability and validity of this standardized test. However, in order to assess its reliability in the current study’s context, the KR-21 formula was employed, revealing a reliability index of 0.81.

Microsoft copilot

Microsoft Copilot utilizes AI technology to enhance productivity, foster creativity, and improve understanding of information through a straightforward chat interface. It is a chatbot built on the foundation of ChatGPT (Flowerdew, 2024). It is an AI-powered chatbot designed to help users with a variety of tasks. Microsoft Copilot processes user input, understands their inquiries, and provides pertinent and supportive responses through a conversational chat interface. It employs natural language processing to comprehend user input and offer assistance based on the conversation’s context (Stratton, 2024). This chatbot provided the learners with some explanations and answers to the prompts related to the realization of IMMs.

Concordancing software

The AntConc concordancing program (version 3.5.7) was utilized to identify the IMMs occurring most frequently. This freeware corpus analysis instrument offers a comprehensive textual analysis for researchers. The software displays the most commonly used words and phrases. Learners were instructed to input only the desired target IMMs in the search term field, located at the bottom of the software interface, and subsequently initiate the search process.

Likert scale questionnaire on hands-on and hands-off learning

This questionnaire was researcher-made, consisting of 16 (originally 20 items reduced to 16 after determining problematic items) items given to all the participants in both groups after administering the posttest. The purpose was to seek the participants’ opinions on the treatment. The questionnaire included two subsections, the first section of which carried ten items. It was to determine participants’ background knowledge of IMMs, and the second section with six items was to elicit participants’ opinions on hands-on and hands-off instructions. Since the questionnaire was researcher-made, it was needed to pilot the items. Two of the items were changed during the piloting phase since the learners asserted that they were not clear about them. One expert in the field scrutinized the questionnaire to guarantee the content validity. Cronbach’s alpha reliability was estimated and turned out to be 0.72.

An exploratory factor analysis was run to guarantee the construct validity of the instrument (Table 2). Regarding two experimental groups (62 participants), the researchers conducted an analysis on an Oblimin rotation of the collected responses. Kaiser–Meyer–Olkin (KMO) examined sampling adequacy. KMO value was 0.69 and Bartlett’s test was significant (p = 0.00, < 0.05).

Table 2 Exploratory factor analysis for hands-on and hands-off learning questionnaire

As shown in Table 3, seven components met Kaiser’s criterion (an eigenvalue of 1 or more). These components explained a total of 75.81% of the variance. Displayed in the scree plot (Fig. 1), two components were put out in the analysis output. Comparatively, two factors above the elbow received the eigenvalues above 2 with the highest eigenvalue scored as 6.11.

Table 3 Factor extraction total variance explained

Fig. 1

The Distribution of the Extracted Factors

The two extracted factors explained 41.29% of the whole variance. Since this contribution was considered below, we decided to remove items to improve the efficiency of the quality of the questionnaire. To determine problematic items, the component matrix was scrutinized to pinpoint the items that influenced variations within each component. Some items with cross-loadings were reviewed. Four items with cross-loadings below 0.20 were stood out of the set.

Fig. 2

Prompt 1: What is Metadiscourse in Writing Performance?

Procedure

Having administered the MTELP, the researchers selected 93 advanced individuals as main participants of the study. Then, they were assigned into a hands-on group, a hands-off group, and a control group (31 participants each). Prior to the main study, a writing performance assessment was given as a pretest to all participants to measure their writing skills. During this test, all participants were assigned four separate topics (unemployment, pollution, education, and industry) to compose written responses on. Each individual was instructed to produce a piece of writing consisting of a minimum of two paragraphs (250 words each) for each topic, with the aim of assessing their knowledge and realization of IMMs. The pretest result indicated that participants had difficulty in using and realizing IMMs correctly. Following the pretest, participants in both groups experienced ten treatment sessions. Each session lasted 60 min twice a week. In the first session, through focusing on sample sentences as shown below, the researchers provided more elaboration on IMMs.

• Sample sentence 1: In my opinion, chocolate is the best flavor of ice cream. (Hedges)

• Sample sentence 2: Imagine that you have just won a million pounds. (Engagement markers)

Twenty scientific ESP articles were chosen from two prestigious journals, namely Journal of Pragmatics and Journal of English for Academic Purposes (each ten articles). In each treatment session, two scientific articles were examined to teach the learners how to identify IMMs. The hands-on group was taught how to run Microsoft Copilot using AI technology. The researchers started to follow brainstorming through asking the learners questions about metadiscourse in general and IMMs in particular. This was also followed by giving some prompts to Microsoft Copilot about the same questions (Figs. 2, 3, 4, and 5). The different prompts given to Microsoft Copilot were as follows:

• 1. What is metadiscourse in writing performance?

• 2. What are the different categories of IM?

• 3. What are the common adverbs showing “hedges” in IM?

• 4. Find boosters in the following paragraph.

Microsoft Copilot gave learners the IMMs which were highlighted. It also gave learners sample paragraphs with highlighted IMMs. Using Microsoft Copilot, the learners in the hands-on group received lots of examples regarding IMMs. Microsoft Copilot itself suggested follow-up questions on IMMs for the learners. Moreover, the researchers asked the learners to brainstorm verbs showing IMMs to report results and then we taught them how they might examine for these verbs using Microsoft Copilot. Needless to say, Microsoft Copilot provided great answers for the learners.

Fig. 3

Prompt 2: What Are the Different Categories of IM?

On the other hand, the hands-off group received an explicit instruction of metadiscourse through reading different text types and determining the IMMs. That is, IMMs were italicized and underlined in the printed form. They also received a long list of IMMs and were required to search in the articles through the AntConc concordancing software. For instance, the hands-off group was presented with the word “show” as a booster in the list of IMMs, and they identified it through AntConc concordancing software (Fig. 6). When each session finished, both groups received two paragraphs to identify IMMs they had been taught differently in that session. That is, the hands-off group identified IMMs without referring to the printed list of IMMs and just by running AntConc concordancing software and the hands-on identified them with the Microsoft Copilot. The target IMMs were taught to the control group using conventional instruction.

Fig. 4

Prompt 3: What Are the Common Adverbs Showing Hedges in IM?

In the posttest, all participants were required to write two paragraphs for each of the topics similar to the pretest. The posttest was a parallel form of the pretest. The posttest was paralleled with the pretest. The inter-rater reliability method, as indicated by the Pearson coefficient between the two raters’ scores, yielded reliability values of 0.78 for the pretest and 0.83 for the posttest of the study. After the treatment phase of the study, the researchers provided the participants in both experimental groups with a questionnaire through which the researchers received participants’ viewpoints and opinions about the treatment.

Fig. 5

Prompt 4: Find Boosters in the Following Paragraph

Scoring writing performance was subjective due to the multiple correct ways of writing each paragraph. Thus, an inter-rater scoring procedure was implemented. Two raters evaluated the participants’ writing performance based on their correct use of IMMs. The raters were required to focus on the extent to which the learners used hedges, boosters, attitude markers, self-mentions, and engagement markers properly. We followed this strategy to help the raters avoid using subjective ratings.

Data analysis

A one-way analysis of covariance (ANCOVA) was employed to analyze the data. Before running a one-way ANCOVA, its assumptions were checked. Then, the responses from the survey questionnaire on learners’ perceptions of DDL were examined through frequency values.

Fig. 6

The Concordance Lines for IMMs

Results

Investigation of the first research question

The first research question examined the impact of two distinctive DDL techniques on EFL learners’ realization of IMMs. Table 4 displays the descriptive statistics for learners’ realization of IMMs in the hands-on group and hands-off group. As shown in Table 4, the three groups had approximately the same average score for metadiscourse use in the pretest. The mean posttest score improved significantly compared to the pretest for both hands-on and hands-off groups.

Table 4 Descriptive statistics of IMMs on pretest and posttest across three groups

Fig. 7

Scatter Plots of IMMs and the Covariate for Each of the Groups

Prior to running a one-way ANCOVA, all assumptions were checked (Dikilitas & Reynolds, 2022). Each one-way ANCOVA analysis included only a single covariate, rendering the assumption of correlation among covariates irrelevant. The reliability of the covariates was checked by examining Cronbach’s Alpha. The findings indicated that the covariate was measured reliably with a coefficient of r = 0.79. As shown in Fig. 7, the relationship is linear, meaning that the assumption of linearity was verified. No significant interaction was found between the pretest and group, F _(1,92) = 2.20, p > 0.05, so it supported the assumption of homogeneity of regression slopes (Table 5).

Table 5 Tests of between-subjects effects for realization of IMMsto check homogeneity of regression slopes

After verifying the assumptions, a one-way ANCOVA was employed to explore the effect of DDL on Iranian advanced EFL learners’ realization of IMMs. The study treated the implementation of hands-on and hands-off as two levels of the independent variable, with the use of IMMs in writing as the dependent variable. Participants’ scores on the pretest were utilized as a covariate. The analysis of the ANCOVA is displayed in Table 6.

Table 6 Tests of between-subjects effects for realization of IMMs in writing

The primary findings from the one-way ANCOVA analysis, F _(2,92) = 168.04, p < 0.05), as presented in Table 5, indicate that there are statistically significant variances observed across the three groups in terms of realization of IMMs on the posttest, after adjusting for the influence of the pretest scores. This implies that DDL techniques have varying effects on the learners’ realization of IMMs. Furthermore, the strength of association indicates that 79% of the total variance in the dependent variable (realization of IMMs) is accounted for by the independent variable (DDL approach).

Statistically significant differences were observed among the three groups in pairwise comparisons, as indicated in Table 7. As in Table 4, it was shown that regarding the realization of IMMs hands-on group benefitted from DDL more than the other two groups.

Table 7 Test of between-groups differences for realization IMMS in writing

Investigation of the second research question

The second research question examined advanced EFL leaners’ perception of two approaches of DDL in terms of realizing IMMs in writing performance. The findings presented in Table 8 revealed that 92% of the learners in the hands-on group and 77.4% of the learners in the hands-off group had not previously focused on IMMs in their speaking and writing. Notably, 80.5% of the learners in the hands-on group and 41.9% of the learners in the hands-off group stated that they intended to pay attention to IMMs moving forward. Furthermore, 83.7% of the learners in the hands-on group and 83.8 of the learners in the hands-off group believed that IMMs are not adequately covered in textbooks. The study revealed that 80.6% of the learners in the hands-on group and 64.4% of the learners in the hands-off group expressed their readiness to realize IMMs effortlessly after the study had been finalized.

Table 8 The experimental groups’ perception questionnaire

In the second part of the questionnaire, consisting of six items, the researchers investigated the shift in learners’ attitudes towards DDL throughout the study. According to Table 9, 35.4% of the learners in the hands-on group and 45.1% of the learners in the hands-off group encountered difficulties in using DDL during the initial session. However, 57.9% of the learners in the hands-on group and 41.8% of the learners in the hands-off group attained the requisite skills to utilize DDL effectively in the last two sessions of treatment. The findings revealed that 99.9% of the learners in the hands-on group and 77.3% of the learners in the hands-off group believed that if IMMs are repeated through DDL, their realization of IMMs develops. Additionally, 83.7% of the learners in hands-on group and 48.3% of the learners in hands-off group acknowledged that DDL motivated them to realize IMMs.

Table 9 Responses from experimental groups to the second section of the questionnaire

Discussion

This research sought to evaluate the outcomes of utilizing hands-on and hands-off DDL techniques on the realization of IMMs in advanced EFL learners’ writing performance. The findings indicated that both approaches of DDL led to an improvement in the learners’ realization of IMMs in the posttest. However, the hands-on group performed better than the hands-off group regarding metadiscourse realization. The findings align with prior research (e.g., Corino & Onesti, 2019; Lin, 2021; Zhang, 2022) which showed the effectiveness of both forms of DDL in teaching language materials. The finding of this study is in conflict with that of Mirzaee et al. (2015), claiming that hands-on DDL could be beneficial for learners at lower proficiency levels.

Why both DDL groups outperformed the control group appears to be attributed to both DDL interventions adhering to the same core academic percepts. Specifically, both interventions are characterized by student-centered learning activities that emphasize language discovery (Bremner et al., 2022). The result of the current study contradicts that of Saeedakhtar et al. (2020), stating that learners with low language proficiency benefitted from hands-on DDL. Moreover, our results support that of Yao (2019), pointing out that advanced learners take advantage of hands-on DDL more than do lower-level learners. Moreover, Daskalovska (2015) stated that high-proficiency learners benefited more from hands-on DDL than low-proficiency learners. The findings of this study support that of Sun and Hu (2020), claiming that DDL treatment improved learners’ hedge use on immediate posttest, but effectiveness decreased on delayed posttest, based on the study results. Our findings align with those of Rets (2017), focusing on the enduring impact of hands-on DDL. The findings lend support to the theoretical claims linked to DDL, such as the noticing hypothesis and autonomous learning. DDL aims to raise learners’ awareness by utilizing input-receiving techniques. By employing these strategies, DDL effectively highlights language elements, making them prominent enough to capture the learners’ attention (Crosthwaite, 2024). The enhancement of learner autonomy, believed to be closely linked with DDL, can be supported through guided induction (Jiménez Raya & Manzano Vázquez, 2022). Furthermore, the inclusion of practical DDL experiences can offer learners valuable chances to engage in learning through application (Lusta et al., 2023; Yu & Lowie, 2020).

The hands-on group that engaged in practical activities had the opportunity to directly interact with the DDL, leading to a higher probability of recognizing distinctions between individuals’ own knowledge and the knowledge specific to the target language. The hands-on group demonstrated superior performance compared to the hands-off group, due to ascribed to the distinctiveness of concordance analysis and the elevated motivation levels of the participants (Baepler et al., 2023). The result of the study is consistent with that of Muftah (2023), asserting that the experimental group, using BNCweb, showed improved writing fluency and consistency in the posttest compared to the control group utilizing Sketch Engine. Virtually, all learners in the research indicated in the questionnaire that the use of concordancing offered them a distinct learning exposure. The current research is supported by Crosthwaite and Steeples (2022), concluding that there is a positive outlook on implementing DDL with younger learners. The study provides a successful model for managing and integrating DDL interventions in a context where secondary content teachers are the main stakeholders, rather than just applied linguists.

The second purpose was to examine learners’ perspectives on two distinct approaches to DDL in achieving IMMs, along with any changes in their attitudes over time. The findings revealed that the majority of students exhibited favorable perceptions towards both types of DDL. This supports Chen et al. (2019), affirming teachers' perceptions of the challenges in using corpus tools and their willingness to incorporate DDL in future teaching. They expressed the belief that encountering the same IMMs repeatedly in authentic examples made it easier for them to realize them. Almost most of the learners reported a decrease in their anxiety related to using DDL throughout the study. This specific belief is in contrast with that of Zare et al. (2022), finding no significant difference in foreign language anxiety among students using concordancing. However, students found a DDL approach with concordancing less enjoyable than traditional explicit instruction. The results underscored the importance of teachers in fostering a supportive and engaging learning environment, with students showing a preference for teacher-centered classes. The conclusions drawn from this study support those of Pérez-Paredes et al. (2019), indicating that mobile DDL for language learning was viewed favorably, primarily because of the immediate and tailored feedback, as well as the availability of diverse tools that were integrated into the learning process. Study results show learners find Microsoft Copilot important in successful DDL experiences. Teachers need basic training in corpora to create a DDL-friendly environment.

The evaluation of Microsoft Copilot provided valuable insights into users’ perceptions of DDL technology. This role of AI in language learning aimed to enhance English writing skills by providing context-driven information based on metadiscourse analysis. The tool utilized learner language input and engaged with language data through a DDL inquiry methodology, highlighting the potential benefits of incorporating such technology in language learning. The use of DDL can improve language sensitivity, pattern detection, word frequency awareness, and learning skills in texts. The DDL supports personalized and autonomous learning, aligning with AI trends promoting individualized learning rather than traditional structured courses. This work serves as an initial step in advancing knowledge and creating guidelines for developing and utilizing DDL in generative AI. We recommend modifying hands-on DDL instructional designs along a spectrum of learner autonomy, from teacher-led activities to learner-centered, corpus-browsing projects based on study findings.

The findings provide evidence that aligns with the theoretical frameworks related to DDL, particularly the noticing hypothesis through input enhancement and the concept of autonomous learning (Kartchava, 2019). Hands-on DDL effectively raised the learners’ awareness, rendering IMMs prominent enough to capture their attention. Developing learner autonomy through hands-on DDL can be achieved through guided induction or dialogue. Proficient learners in the online realization of IMMs tended to participate in autonomous browsing, leading to discovery learning. This aligns with Ellis et al. (2020), suggesting self-discovered information is better retained than information simply told to learners. AI-enhanced discovery learning served as an effective instrument that facilitated student engagement in discovery-oriented educational experiences (Lestari et al., 2021). The inductive approach prioritizes learners in their educational process. In essence, the inductive teaching method enables learners to acquire knowledge through active participation. The findings of the current study showed that the learners independently realized significant concepts of IMMs through engaging in observation, posing questions, and hands-on activities. The realization of IMMs was fundamentally on the process of discovery.

Conclusion and implications

Regarding the results obtained from the current research, it can be inferred that two distinct approaches to DDL can be prominent in enhancing EFL learners’ realization of IMMs. The findings revealed that both experimental groups achieved comparable results in realizing IMMs during the posttest due to various factors such as the impact of novelty, the effectiveness of concordancing, and the learners’ level of enthusiasm.

This research is constrained by certain limitations, such as the utilization of a limited number of participants, lacking delayed posttest to determine learners’ learning of IMMs, focusing on only IMMs rather than interactive metadiscourse. Furthermore, all participants were selected from individuals with advanced levels of language proficiency. In addition, this research utilized a questionnaire to gather information on participants’ attitudes towards DDL. Alternative methods, such as think-aloud techniques and interviews, can also be utilized to uncover learners’ attitudes. Future research should examine the particular mechanisms underlying the effectiveness of AI, focusing on the types of AI provided and their effect on metadiscourse use in writing process. Additionally, exploring the effect of individual learner characteristics, such as learning style preferences, on the effectiveness of AI interventions may deepen our understanding of how AI leads to metadiscourse realization in writing development.

Notwithstanding the limitations, the study suggests some implications. We argue that language learning tools driven by AI can be instrumental for EFL learners, significantly improving their language acquisition journey. Such tools offer a more interactional experience, which in turn cultivates greater motivation and eagerness for continued language study. AI-enhanced platforms like Microsoft Copilot allow learners to engage in language practice at their own pace and from any location. This level of accessibility encourages learners to assume responsibility for their educational journey, fostering a sense of independence and self-management. Moreover, the findings highlight the significant impact of AI-driven language learning tools in EFL classrooms, focusing on students’ ability to foster personalized and adaptive educational experiences. Learners engaging with interactive AI systems benefit from prompt feedback, constructive evaluations, and varied sentence constructions, which promote ongoing development and bolster their confidence in language use. This immediate assistance cultivates a supportive and motivating educational environment, thereby enhancing the overall learning experience. Additionally, the incorporation of AI into language education opens up exciting opportunities for further investigation and innovation in the realm of language teaching methodologies. Syllabus designers can use AI technology to place instructional focus on the discourse markers that have a higher effect on learners’ writing products. Moreover, such AI as Microsoft Copilot can help syllabus designers to design some materials on metadiscourse to be discussed by EFL learners. As AI technology progresses, educators and scholars can investigate novel approaches to harness its potential for improving language acquisition outcomes and teaching strategies.