Enhancing language skills and student engagement: investigating the impact of Quizlet in teaching Chinese as a foreign language

Abstract

This research explores the use of the Quizlet application in teaching Chinese as a foreign language, employing a mixed methods approach. The participants of this study were 60 international students who majored in Chinese at a university in China. The students were divided into two groups and received instruction through traditional teaching methods and Quizlet-based teaching activities, respectively. The pre-and post-performance tests were used to collect quantitative data on students’ Chinese listening, speaking, reading, and writing scores, while the open-ended questions were used to measure qualitative data on student engagement. The integration of Quizlet is found to enhance Chinese listening, speaking, and reading skills, promoting student engagement through behavioral, emotional, and cognitive dimensions. However, its influence on writing skills is less significant, warranting further investigation. The results of this study endorse Quizlet as a valuable tool for language instruction, highlighting its potential for facilitating Chinese language education and inspiring innovative teaching methods.

Introduction

As one of the six official languages of the United Nations, Chinese is playing a more and more important role in facilitating communication among countries worldwide. By the end of 2021, more than 180 nations and areas have implemented worldwide Chinese education, and nearly 200 million people worldwide are studying and utilizing Chinese collectively (Ministry of Foreign Affairs of China, 2022). The proportion of Chinese learning is increasing year by year in world language learning. However, it is difficult to learn a language, especially a foreign language with a construction that differs from one’s first language. There are various factors to take into account, ranging from the necessity to pay attention to speaking in the appropriate tones to the challenging aspect of writing (Basith et al. 2019). For instance, Chinese characters must be taught as a unit rather than as separate parts because they include sound (tone), form, and meaning. There are four standard tones in Mandarin Chinese (Lee and Kalyuga, 2011): flat, high-level, rising, falling, and high-falling. The tone is used to discern meaning in Chinese, although in many other languages, it is used to indicate emphasis or emotion. Besides, the relationship between Hanzi (Chinese Characters) and Chinese culture and history is another obstacle (Alshammari, 2020).

Teachers of Chinese are coming up with increasingly creative ways to impart the language both within and outside of the classroom (Sumo and Bah, 2021). The general trend is to integrate technology into teaching Chinese as a foreign language (CFL), which is driven by the growth of information technology and the rising demand for Chinese education. The development of information technology and the advent of the 5 G era provide a strong guarantee for teaching CFL. Mobile-assisted language learning (MALL) is a prominent subject of discussion in the area of second language acquisition (Nurazizah et al. 2019). MALL refers to a strategy for acquiring a second or foreign language via mobile devices, either formally or informally (McCarty et al. 2017). It has received substantial attention in the field of foreign language teaching and learning in recent years. According to Chen and Kessler (2013), because mobile devices have become inseparable communication instruments in people’s social lives, one can see them as promising and practical resources for acquiring a second language (L2). According to Rosell-Aguilar (2018), mobile devices have introduced novel opportunities for instruction and education within the classroom setting.

The Quizlet application is a popular example. It is a mobile and web-based learning tool that uses learning modules made up of ideas and explanations or descriptions. These modules are given to students using a variety of learning techniques such as flashcards, games, collaborative projects, and quizzes that assist them in grasping numerous topics, particularly languages, and vocabulary (Montaner-Villalba, 2019). This platform has been used by over a quarter of all high school students in the USA. As of December 2021, Quizlet has provided over 500 million flashcard sets created by over 60 million monthly active users, who use the application for everything from math tests to medical exams to vocabulary quizzes (Matthew, 2021). Quizlet’s primary goal is to help people improve their linguistic skills. According to Hikmah (2019), Quizlet can be used to improve one’s language skills in the areas of listening, speaking, reading, and writing. Users can practice their listening skills by listening to audio that contains the definition of a word. In addition, users can practice their pronunciation using flashcards, and their reading skills can be honed with menu scatter and space races. Writing skills are employed in learning, spelling, tests, and space races to type responses. It also tries to provide feedback to students as they complete activities within a set of terms. Zhou (2016) found that using Quizlet in a CFL classroom significantly improved student motivation and vocabulary learning through Quizlet-based activities. Therefore, this research attempts to integrate Quizlet into Chinese language teaching classrooms and evaluate its influence on the score of foreign students’ listening, speaking, reading, and writing, as well as their engagement. The two research questions are as follows:

• 1. To what extent does Quizlet technology improve the score of foreign students in Chinese listening, speaking, reading, and writing?

• 2. How is student engagement in Quizlet-based teaching activities for learning Chinese?

Literature review

MALL, Quizlet application, and L2 teaching

The availability of multiple learning apps has made MALL an important topic of research and a method of choice (Barjesteh et al. 2022). MALL for language teaching has been studied extensively. The majority of these studies use English as their primary target language. Johnston and Marsh (2014) asserted that integrating information literacy into the curriculum through the use of iPads and iBooks promoted teamwork and interaction in the classroom and that the programs’ “hands-on” nature and the interactivity provided by the apps aid in raising student engagement in the content. Another study conducted by Albahiri et al. (2020) indicated that a key factor in improving students’ competency was the use of YouTube for studying English-spoken conversation. Hao et al. (2018) find that using mobile apps enhances learning in both independent and collaborative environments. Scholars in the field also speculate that language learners have a positive attitude toward apps-based learning (Cardenas-Robledo and Pena-Ayala, 2018; Law and Geng, 2018; Legault et al. 2019). They list a number of advantages for MALL, including the applications’ cost-effectiveness, accessibility, and pervasiveness; their user-friendliness and ability to adapt to the needs of language learners; and their portability and adaptability (Barjesteh et al. 2022).

As analyzed above, Quizlet seamlessly integrates the advantages of MALL applications, including their portability and adaptability, and has been assessed in many different L2 research studies. Dizon (2016) discovered that Quizlet was an effective method for learning L2 vocabulary in the classroom. According to Sanosi (2018), an ideal environment for active learning might be created with Quizlet inside as well as outside of the classroom. The live learning mode in the classroom, in particular, offered students who were required to share information and trade responses with other groups of students in a competitive spirit, a beneficial collaborative experience. According to Anjaniputra and Salsabila (2018), boosts students’ tenacity and involvement in the language learning process. According to Çinar and Arı (2019), students who utilized Quizlet for vocabulary learning learned more than those who did not, and they suggested using Quizlet as a permanent, student-centered teaching strategy in foreign language learning. The four main elements of language learning that Quizlet supports are listening, speaking, reading, and writing. Hikmah (2019) claimed that using Quizlet could boost students’ confidence in their language-learning ability and decrease their levels of boredom.

Technology in CFL

With the growing popularity of CFL throughout the world, mobile-assisted CFL learning has gained traction (Zhou, 2020). Research on the subject of mobile CFL learning has previously concentrated on how the usage of mobile apps and devices influences the acquisition of a variety of language skills, as well as how students’ attitudes, strategies, and motivations concerning learning foreign languages change as a result (Luo and Yang, 2016; Jiang and Li, 2018; Jin, 2018). Sumo and Bah (2021) focused on using creative pedagogy methodologies, such as utilizing media, interpreting advertisements from movies and newspapers, and applying techniques like sand blots in international Chinese language classrooms, which proved to improve students’ ability to absorb L2. A prototype Virtual Experiential Language Learning Environment (VELLE) was employed in the study by Yang et al. (2022) to provide immersive learning experiences and help CFL students strengthen their communication techniques. The results demonstrated that the VELLE and productive failure designs had an impact on how participants used different sorts of communication tactics. According to students, speaking in a virtual reality (VR) setting enhanced their engagement (Yang et al. 2022). Another research discovered that the use of scaffolding materials might improve the autonomy of foreign students learning Chinese online (Chen, 2021). The results of Li and Tong’s (2020) research showed that compared to the pictorial-verbal coding strategy, the pictographic-verbal coding approach helped learners acquire and recall more Chinese words, and students who were taught using this method showed greater learning motivation. However, CFL learners and technology are still not as widely investigated as other languages. Therefore, this study attempts to find out the impact of Quizlet technology on foreign students’ performance and experiences in Chinese language education.

Technology and skills of listening, speaking, reading, and writing

Language is an integral component of communication and helps people interact with others effectively. It may involve speaking, writing, reading, and listening in various combinations in different settings or on different occasions (Powers, 2010). Language skills are abilities that allow you to interact with people and convey your thoughts coherently (Husain, 2015). These abilities, including reading, writing, listening, and speaking, provide structure and significance to the information you want to express to the audience. It is essential to emphasize that the four language skills collectively constitute overall proficiency in one language (Canale and Swain, 1980; Beckman, 1990). The argument posited for the integration of all four language skills, as opposed to selective testing, is premised on the overarching concept of communicative proficiency, which holds paramount importance in various academic and professional domains, aligning with the preferences of users of tests such as the TOEFL, TOPIK, and HSK.

Technology and skills of listening

Listening skills are the active process of receiving and responding to both spoken and unspoken signals. DeVito (2019) describes listening as the efficient process of taking in information, processing it, and responding appropriately to what one has heard. Listening is the practice of paying attention to and attempting to make sense of what we hear (Fussalam et al. 2019). Furthermore, Vandergrift (2002) emphasizes that it is a complicated and dynamic process of matching what one listens to with what one already knows. Listening skills can be defined as the ability to understand, interpret, and analyze the speaker’s message while actively connecting it to existing knowledge. According to Walker (2014), the examination of listening skills entails considering linguistic features, cultural influences, and psychological processes, with assessment methods encompassing analysis of comprehension accuracy, retention, and application in various contexts.

Listening skills are critical in L2 since receiving linguistic information is the cornerstone of learning a language (Gilakjani, 2016). When students receive adequate understandable information, they are able to absorb the material, according to Hamouda (2013). In Newton’s (2016) work, the importance of skilled listening extends beyond mere segmentation of the speech stream. He draws upon Vandergrift’s (2007) observation, emphasizing that acquiring proficiency in listening within L2 necessitates adept coordination of metacognitive and cognitive strategies. Kutlu and Aslanoğlu (2009) did a study to establish the elements that affect young students’ L2 listening skills, and they discovered that five factors including “the number of books at home” had a substantial impact on listening performance. Fussalam et al. (2019) investigated the connections between viewing movies and listening skills. They believed that using movies as teaching and learning material might help students grasp the language in realistic circumstances and improve their L2 listening skills. Etemadfar et al. (2020) investigated the impact of flipped classrooms on the listening comprehension of Iranian English learners. The results suggested that teaching through flipped classrooms resulted in higher listening performance than regular courses. Furthermore, due to the accessible resources, participants in the flipped classes are very motivated to learn. Hilmatunisa et al. (2018) found that the use of podcasts had a beneficial effect on teaching L2 listening skills in narrative texts. Similar findings were also found in the studies conducted by Harsa et al. (2020), who suggested that students who used two instructional media, audio-visual and audio, for the pre-test and post-test exhibited improved L2 listening scores.

Technology and skills of speaking

Speaking is an important oral communication skill (Husain, 2015). Like the other abilities, it includes more than just word pronunciation and is initially more difficult than it appears. Many L2 learners regard speaking ability as the measure of knowing a language. It can be defined as the ability to effectively convey information and interact with others through the use of appropriate vocabulary, grammar, and pronunciation, while also demonstrating an understanding of the social and cultural context in which communication takes place (Kürüm, 2016; Bashir et al. 2019). The assessments typically evaluate a speaker’s ability to articulate ideas clearly, use appropriate vocabulary and grammar, maintain coherent discourse, and demonstrate sensitivity to social and cultural norms in communication. Additionally, automated speech recognition technology can be employed to provide objective measures of pronunciation accuracy and fluency (Kürüm, 2016).

According to Bashir et al. (2019), technology-based speaking instruction also helps with the recollection of lexical phrases and improves the memory of phonemic and morphological distinctions in L2. If students subvocalize while learning new words and phrases, even computer programs that only display electronic flashcards can be beneficial (Teixeira, 2015). While in-person instruction primarily assists students in identifying gaps in their L2 speaking, computer-assisted pronunciation training has garnered some interest from the academic community as well. O’Brien (2006), and Gorjian et al. (2013) all emphasize the importance of linking pronunciation teaching with L2 intonation training, as does patterning. Practical implementation of this type remains limited to date. Many learning management systems now provide the ability to make video postings using a Flash plugin, that promotes L2 speaking practice (Blake, 2016). Synchronous video-conferencing techniques are essential for promoting L2 speaking practice because they enable real-time text, image, and video interchange between students. Students can progressively develop improved discourse transitions and increased fluency in this way, both of which are crucial elements of speaking.

Technology and skills of reading

Reading refers to the cognitive abilities they use while interacting with texts (Liu, 2010). Gedik and Akyol (2022) define reading as the process of interpreting written and pictorial materials. The reports produced by The National Reading Panel (NRP) in 2000 indicated that phonemic awareness, phonics, reading fluency, and reading comprehension, however, are all identified by the panel as being particularly important to reading development (Connors-Tadros, 2014). Taxonomies of reading skills often categorize the component skills and processes as ones that occur higher or lower in language processing. For example, word recognition skills are considered lower-level processing skills. In contrast, inference-making is considered a higher-level processing skill because it aids the construction of the meaning-based representation of the text (Cain et al. 2004). Within this resource framework, slow or inaccurate word reading is supposed to affect reading by using too much processing capacity. In accordance with this theory, word reading is the best predictor of reading skills level in the early years (Juel et al. 1986).

Many researchers have been interested in reading research and the effectiveness of technology in L2 reading. Ahmad and Khoo (2019) conclude that there is a need for educators to shift toward utilizing interactive multimedia as a teaching tool to improve reading skills among underachievers. The result of Van and Louw’s research (2008) indicated the improvement of young learners’ reading skills through technology-assisted reading programs. Yanguas (2009) confirmed the efficacy of combined glosses, encompassing text and visual elements, in improving both vocabulary recognition and reading comprehension. Chun (2006) further demonstrated that incorporating visual aids like pictures or videos alongside translations of unfamiliar words enhances vocabulary retention among L2 learners. These findings highlight the significant role of technology instruction in promoting L2 lexical growth and reading comprehension. Furthermore, Grabe (2004) and Chun (2006) stressed the significance of word recognition fluency in improving general reading skills, which is assisted through multimedia. Arispe (2012) also emphasizes the role that background information, L2 language ability, and first language reading levels have in developing fluent reading skills. To summarize, technology has the potential to augment L2 learners by offering multimodal learning experiences that enhance vocabulary retention, reading comprehension, and overall reading skills.

Technology and skills of writing

Writing skills are described as the ability to freely express oneself (Iftanti, 2016). According to the text-oriented approach, writing is the ability to write “contextually” proper forms of language following established patterns at the sentence or discourse level (Yi, 2009). It is predicated on the deliberate and suitable use of language, with its capacity for communication and structural correctness (Fareed et al. 2016). Writing is a cognitive activity that calls for memory, thinking, and linguistic command in order to express ideas clearly, according to Kellogg (2001). Meanwhile, writing skills are described as the capacity to begin and generate ideas and then apply particular revising and editing methods to bring them to maturity in a certain context (Yi, 2009). According to Breland (1983), writing skills have been assessed using two methods: direct measurement and indirect measurement. Direct measures are those in which a sample of an examinee’s writing is gathered under controlled settings and then assessed by one or more judges, generally, instructors experienced in making writing competence assessments. Indirect measures are estimates of likely writing proficiency based on observations of certain types of writing expertise. He mainly analyzes direct measurement of writing skills and believes that it contributes, but strategies to increase dependability and reduce costs must be developed (Breland, 1983). Through the literature review, many studies have employed indirect measuring methods. Dwiyanti and Suwastini (2021) summarized evaluating students’ writing skills in five primary topic areas: word completion, sentence formation, and syntax, comprehension, tenses and grammar, and handwriting. Javed et al. (2013) used an accomplishment test with different items to measure students’ writing sub-skills such as word completion, sentence formation comprehension, grammar, and handwriting.

Yundayani et al. (2019) emphasized the relevance of using visual media to assist students in enhancing their writing abilities and supported Canva’s beneficial influence on students’ writing skills. To start, corpus tools and concordances might be particularly useful for L2 writers when it comes to mobile techniques and tools (Yoon, 2008). In terms of online writing, the Internet helps with writing improvement through blogs, wikis, shared documents, electronic discussion forums, and a variety of writing tools found in today’s learning management systems (Blake, 2016). Oskoz and Elola (2014) and Kessler et al. (2012) have praised the benefits of integrating social media into a multimodal, staged writing process.

People often employ a mix of these skills at once when communicating. When we refer to someone as “speaking” a language with fluency, we often mean that they are proficient in all four areas. However, the L2 four skills that learners acquire might be out of balance in most situations. For instance, a student may be good at reading but deficient in listening, writing, or speaking. According to research, speaking skills are different from those of reading, listening, and writing (Powers, 2010; Sawaki et al. 2009). Consequently, a proficient speaker may not always be a proficient reader, writer, or listener. Language skills in all four categories frequently connect to one another, contributing uniquely to an individual’s overall communicative proficiency. Such correlations, however, are insufficient to let the assessment of one ability take the place of another (Galaczi, 2018). Furthermore, equitable assessment practices entail affording test takers the opportunity to demonstrate their skills through various means, including different tests and question formats (Powers, 2010). We cannot extrapolate performance in one skill, such as speaking, from that of another, such as listening, nor can we substitute tests of linguistic knowledge including grammar and vocabulary for actual language uses.

Therefore, the theoretical framework of this research investigates the influence of Quizlet-based teaching activities on the improvement of listening, speaking, reading, and writing skills among foreign L2 Chinese students.

Student engagement

Student engagement is a key link in linking the provision of written feedback to learning outcomes (Han and Hyland, 2015). According to Robinson (2012), student engagement is broadly defined as actively participating in listening to individual and group viewpoints from students on issues related to their experiences in higher education. The benefits of Quizlet for vocabulary learning were examined by Anjaniputra and Salsabila (2018), who found that using Quizlet increased student engagement with all classroom activities. Zhou’s (2016) study indicated that using Quizlet is valuable and helpful in improving students’ sense of engagement. More studies show that the use of Quizlet encourages students to become more actively involved in every class activity. Although some students claim that using Quizlet is a perfect command, they end up using it even voluntarily in and out of class (Vargas, 2011; Barr, 2016).

Student engagement is a multifaceted construct that can be interpreted in a multitude of ways. Some prefer a model that includes three different dimensions: behavioral engagement and emotional engagement, and cognitive engagement (Fredricks et al. 2004; Bowyer and Chambers, 2017; Yu and Gao, 2022). While Fredricks et al. (2004) and Bowyer and Chambers’ (2017) framework form the basis of the current study, it was originally proposed for education in general and blended learning rather than for second language acquisition in particular. The most well-articulated definition of student engagement in second language acquisition thus far is probably presented in Ellis’s (2010) componential framework. This framework introduces three perspectives for evaluating student engagement: the cognitive perspective, the behavioral perspective, and the attitudinal perspective. Within this framework, cognitive engagement is described as how students focus on the framework they receive. On the other hand, behavioral engagement concerns students’ responses, such as uptake or revisions prompted by the framework. Lastly, attitudinal engagement encompasses students’ emotional responses and attitudes towards the framework. In the study of Han and Hyland (2015), the multidimensional framework of student engagement has been modified from Ellis’s (2010) concept of engagement to be applied to written corrective feedback on L2 learners. Zhang (2017) saw the emotional (almost emotive), behavioral, and cognitive components of student engagement with computer-generated feedback. Zheng and Yu (2018) used a qualitative method to look into the three aspects of student engagement. The focus was on understanding how Chinese EFL lower-proficiency students engaged affectively, behaviorally, and cognitively with their teacher in the context of writing.

Building on the preceding analysis, this study employed a well-established framework widely acknowledged in second language acquisition. The investigation thoroughly explores student engagement with Quizlet-based activities across three dimensions: behavioral engagement, emotional engagement, and cognitive engagement.

Methodology

Research design

Mixed methods research has grown in popularity in the social sciences as a whole and is now seen as a valid, independent research design (Creswell, 2002). It can be described as “the collection or analysis of both quantitative and qualitative data in a single study in which the data are integrated at one or more stages in the research process, are prioritized, and are collected concurrently or sequentially” (Creswell et al. 2003). Researchers can enhance their findings in ways that are not possible with a single type of data when they incorporate both quantitative and qualitative data into their study (Hanson et al. 2005). According to Firestone (1987), quantitative research persuades readers by putting less emphasis on the reader’s own judgment and more on the application of accepted practices, which produces results that can be applied to entire populations. Yin (2004) argues that qualitative research overcomes the abstraction inherent in quantitative studies and allows for generalization to theory by means of comprehensive description and strategic comparison across examples.

Therefore, this study designed a mixed methods approach. Two groups of students were selected to participate in performance tests, which analyzed improvements in Chinese listening, speaking, reading, and writing over an 8-week period. At the conclusion of the study, an open-ended questionnaire on student engagement was administered to all participants from the experimental group. Qualitative interviews, as suggested by Udo (2006), were conducted to identify any unobserved data pertinent to the study’s research questions. The questions were distributed via email to potential participants, who were invited to join the study voluntarily. The initial email provided details about the research objectives, assured confidentiality of responses and included contact information for further inquiries. Upon expressing their willingness to participate, respondents were then emailed the open-ended questionnaire as an attachment. Zohrabi (2013) asserts that open-ended questions are crucial as they lead to a higher degree of discovery and ensure that responders’ answers more accurately reflect their intentions. Please refer to Fig. 1 for the research framework.

Fig. 1

Research framework

Treatment

The treatment technology adopted in this study was the Quizlet application. To ensure the accuracy of the research, the instructor and course content, which were the control variables in this study, remained the same for the two groups. The distinction was that when the teacher was in the control group, they used textbooks and blackboards for traditional classroom instruction without any treatment. When in the experimental group, the teacher integrated Quizlet technology presenting new knowledge and creating corresponding teaching activities. A 90-min detailed lesson plan outline is shown as follows to further illustrate the treatment of the research (Table 1).

Table 1 An outline of the lesson plan in the experimental group (90 min)

Population and sample

The population of the study was a total of 60 international freshmen enrolled in the Chinese language program at an ordinary undergraduate university in Liaoning Province, located in the northeast of China. Since the Chinese language classes in this university usually adopt small-group teaching, the total population of this study was 60 people. These students, having successfully passed the university’s Chinese proficiency entrance exam and gaining admission as first-year Chinese language majors, are equally proficient in Chinese. They have all studied in the same environment for six months, and with a vocabulary of 300–400 words. They all had access to the Quizlet app, which was another requirement for participation in this study.

Two Linguistics Podesva and Sharma (2014) believe that the most reliable way to learn about a specific group of people’s language use will be to gather linguistic information from every individual in the population, which, in the context of the social sciences, refers to all members of the population. Although the cost of taking the entire population as a sample makes it difficult for big populations, a census, as in this case, is appealing for small populations. A census sampling method was adopted in this study based on the above analysis. The census method refers to a statistical approach in which data are collected for each element or unit of the population (Singh and Masuku, 2014). Since every aspect of the universe is considered in this form of study, the results are more precise and dependable, and it is simpler to come to firm conclusions.

By using census sampling, all the students were selected as the research sample in this study to ensure credibility and accuracy. One class, consisting of 26 students, was randomly selected as a control group and the other class, consisting of 34 students, was as an experimental group. They came from Russia, Kazakhstan, Kyrgyzstan, Belarus, Turkey, Poland, and South Korea, aged between 18 and 24 years old.

Instruments

This study utilized the Hanyu Shuiping Kaoshi (the New HSK), a standardized test developed to measure the level of non-native speakers of standard Chinese, as the instrument to test the hypotheses of quantitative research. Of all the examinations for CFL, the HSK has the most test takers and is issued and administered by the Office of Chinese Language Council International under the Higher Education Department, Ministry of Education, People’s Republic of China (Su and Shin, 2015). HSK tests serve two purposes. The first, which has been successfully accomplished, is to support Chinese language instruction both domestically and abroad. According to Peng et al. (2021), the second goal of the HSK is to give test takers a helpful reference for their skills in the Chinese language for making a variety of decisions. According to Hanban (2014), test results can not only be used to assess students’ levels and evaluate students’ learning progress but can also be used as a reference for admission or class placement and teaching differentiation of students by educational institutions or projects; Chinese language program evaluation purpose (e.g., how effective a program is in training Chinese language learners). Su and Shin (2015) also point out that it is positioned as a general Chinese proficiency test and can be used as a reference standard for academic and career decision-making procedures. Therefore, these studies prove that HSK can be used as a measurement tool to test students’ Chinese language skills.

The new HSK is divided into three sections: listening, reading, and writing while speaking is evaluated in an optional and independent test (HSKK) (Hanban, 2010). Although language skills in all four categories frequently connect to one another, such correlations, however, are insufficient to let the assessment of one ability take the place of another (Galaczi, 2018). We cannot extrapolate performance in one skill, such as speaking, from that of another, such as listening. Given that the questions in the four sections were unrelated, featuring distinct content, question types, and evaluation criteria, this study finds it viable to employ the scores from each section as representative indicators of students’ reading, speaking, listening, and writing. While acknowledging language proficiency as a nuanced and intricate linguistic system, which is more applicable to daily use, academic assessments of individual language skills require a deliberate transition to a singular scoring approach. This inclination is supported by the accessibility and quantifiable attributes, making it more fitting for precise and comprehensive evaluations within an academic context.

There are a total of six levels in the HSK, starting with level 1 and level 2 for beginners, then levels 3 and 4, and finally levels 5 and 6 for advanced. According to the entrance exam results and previous studying time of the participants in this study, their vocabulary was around 500–600, which was suitable for taking the HSK level 3 test. In the level 3 test, the listening section consists of different types of questions that test listening ability, from simply judging whether a given picture matches a word heard, to complex skills such as conversations and short text comprehension. The first part of HSKK asks test-takers to repeat or retell what is presented to them. In addition, it also involves describing two pictures and answering two questions presented in pinyin. Similarly, the reading section includes different types of questions that test reading skills; from simply matching a printed word to a picture to complex cloze, sentence reordering, and paragraph comprehension. Writing is tested only from level 3 onwards and consists of two parts, requiring candidates to write a complete sentence or to rearrange the order of characters, or fill in missing characters in the sentence using pictures given (Table 2).

Table 2 Corresponding levels of the New HSK and CEFR (Hanban, 2010)

The open-ended questions were developed based on the theory of student engagement in school proposed by Fredricks et al. (2004). The data from three elements were collected: behavioral engagement (three questions), emotional engagement (three questions), and cognitive engagement (three questions) (Appendix 1).

Validity and reliability

China’s Ministry of Education oversees Hanban, an organization that administers the HSK exam. To guarantee the quality of the New HSK, testing research has influenced test development from the very beginning. Despite the new HSK exam being established as an industry standard, there remain scholars critically examining its efficacy. Su and Shin (2015) evaluated the new HSK test based on Bachman and Palmer’s (1996) model of test usefulness and Kunnan’s (2014) model of test fairness. Peng et al. (2021) adopt Bachman and Palmer’s (1996) Assessment Use Argument to review and appraise the current version of HSK. According to Su and Shin (2015), these protocols support the New HSK’s promotion as the globally recognized assessment of Chinese language ability.

The validity of open-ended questions was evaluated using the Content Validity Index (CVI) technique. All of the questionnaire items obtained a score of 1 in agreement that they were appropriate, according to the results of three experts’ evaluations. Besides, open-ended questions were utilized in qualitative research. So, there was no attempt made to evaluate the reliability.

Data collection

Informed consent was obtained from all participants involved in the study. All procedures involving human participants have been conducted in accordance with the Declaration of Helsinki. Participants were provided with detailed information about the nature, purpose, and potential risks of the study, and their voluntary agreement to participate was documented. Besides, as a protection for the participants, all personally identifiable information (such as name and ID number) in the experiment was anonymized. The confidentiality and welfare of the participants were prioritized throughout the research process.

The experiment went on for 2 months in total. During this time, two groups of 60 international students attended eight Chinese classes every week at the university, consisting of two listening classes, two speaking, two reading classes, and two writing classes. Each class lasted for 90 minutes. In other words, the weekly learning time for the four skills was three hours, respectively. The research design was divided into three stages: preparation, treatment, and data collection. Quantitative data was collected through pre-test and post-tests. They were taken under proctored examination conditions with all of the groups taking the same test at the same time. All participants were required to finish the tests on time and take them seriously. The statistical method for quantitative data was an independent samples t-test, using Jamovi software version 2.3.26 (Table 3).

Table 3 Overall research design

Data collection for this qualitative research was conducted using an open-ended questionnaire administered via email. To obtain the proper number of samples indicated in the sample size, the questionnaire on student engagement in Chinese listening, reading, and writing classes was distributed to all participants from the experimental group. An initial email was sent to potential students, inviting them to take part in the study voluntarily. The email explained the purpose of the research, assured confidentiality of responses, and provided contact information for any inquiries. Upon expressing their willingness to participate, respondents were then emailed the open-ended questionnaire as an attachment. The questionnaire consisted of nine open-ended questions that encouraged students to provide detailed and unrestrained responses. These questions were designed to elicit rich qualitative data, allowing participants to share their unique viewpoints and reflections on student engagement. This procedure was administered after the post-test in week eight. Qualitative data were analyzed through thematic analysis using NVivo 14 software.

Results

Quantitative data analysis

For descriptive information, differences in Chinese listening skills were observed between the two groups. The control group’s mean listening score increased from 74.5 to 78.6, with a 4.1-point improvement, while the experimental group improved from 75.1 to 82.9, showing a more significant 7.8-point increase. In terms of speaking scores, the control group demonstrated a mean pre-test score of 79.3 and a mean post-test score of 81.8, whereas the experimental group displayed a mean pre-test score of 77.0 and a mean post-test score of 81.6. Both groups experienced mean reading score increases, with the experimental group showing a greater improvement of 7.0 points compared to the control group’s 5.3 points. In the writing skills test, the experimental group’s mean score increased by 7.4 points, surpassing the control group’s 4.5-point improvement, indicating greater progress (Tables 4 and 5).

Table 4 Descriptive statistics of the scores of listening, speaking, reading, and writing in the control group

Table 5 Descriptive statistics of the scores of listening, speaking, reading, and writing in the experimental group

The independent samples t-test was calculated to compare students’ score improvement between the control group and the experimental group. In the listening, speaking, and reading hypothesis testing section, the t-test was statistically significant, with p < 0.05. Thus, the null hypotheses were rejected. The results indicated that the score improvement of students’ listening, speaking, and reading was different between the two groups. While in the writing scores testing section, the t-test did not show significance, with p > 0.05. The result indicated that the score improvement of students’ writing was not different between the two groups (Table 6).

Table 6 Summary of hypothesis testing and results

Qualitative data analysis

NVivo 14 software was used to conduct a thematic analysis of the qualitative data from three dimensions: behavioral engagement, emotional engagement, and cognitive engagement (Ellis, 2010; Han and Hyland, 2016; Zheng and Yu, 2018). The results are as follows.

For behavioral engagement, students frequently mentioned that they became actively involved and contributed during Quizlet-based Chinese teaching activities by asking questions, initiating language games, sharing ideas about Chinese culture and literature, and participating in discussions and group activities. To stay focused and attentive, the students provided many strategies. They indicated creating a distraction-free learning environment, setting study goals, note-taking, and actively engaging in quizzes and discussions. When discussing Quizlet activities, students described many enjoyable examples, such as vocabulary race, picture description, sentence scramble game, and role-playing session. These activities made the learning process enjoyable, motivating participants and strengthening their vocabulary in Chinese. The findings were relevant to Laricchia (2013) who assumed that “engagement is all about being genuinely curious and actively exploring whatever has caught your interest”.

When interviewed students answered questions about how they feel when engaged in Quizlet-based Chinese learning activities, the most frequently mentioned words were the following: exciting, enjoyable, fulfillment, accomplishment, rewarding, motivated, curiosity, enthusiasm, and so on. When discussing how to overcome the challenges, students shared their experiences including exploring new topics, switching between study modes, and searching for collaborative learning partners. Speaking of overall enjoyment, students noted that Quizlet featured a range of engaging and interactive activities, such as flashcards, matching games, and quizzes, which made the learning process entertaining and engaging. As some participants claimed:

“Having the ability to track my work and notice my improvements offers a sense of accomplishment and encourages me to continue learning Chinese with enthusiasm...”

“Quizlet has become an indispensable tool in my Chinese learning journey.”

“The platform’s collection of user-created content allows me to explore various aspects of the language and culture, keeping my interest piqued and my motivation high.”

...

In the cognitive engagement section, students from the experimental group were asked about their strategies for connecting Quizlet’s information to their existing knowledge of the Chinese language, cognitive strategies, and the utilization of Quizlet to enhance cognitive engagement and comprehension of Chinese materials. The results illustrated that diverse cognitive engagement strategies were adopted by students during Quizlet-based Chinese learning activities. The capacity to integrate new material with current knowledge, utilize appropriate cognitive strategies, and actively interact with Quizlet’s study modes contributed to an enhanced learning experience and promoted student cognitive engagement.

Discussions

Answers to research questions

This study confirms the significant contribution of Quizlet technology to improving the listening, speaking, and reading skills of foreign students. The result well proves that Quizlet is an effective tool for L2 learning as mentioned in the previous research and can improve students’ language learning results (Dizon, 2016; Sanosi, 2018; Çinar and Arı, 2019; Zhou, 2016). However, it partially confirms the research of Hikmah (2019), that Quizlet can be used to improve one’s language skills in the areas of listening and reading. However, the difference is that the improvement in writing skills was not proven in this study. The independent t-test did not show statistical significance, which means this technology did not have a statistically significant impact on the improvement of writing skills compared to traditional teaching methods.

The result indicated that teaching Chinese writing skills using Quizlet did not show a significant impact. This finding aligns with previous research on writing, particularly regarding the use of various writing techniques and strategies by L2 learners. As highlighted by Blake (2016), while the internet offers a plethora of writing tools and platforms, not all tools have a significant impact on writing skills. Similarly, Yundayani et al. (2019) emphasized the potential effects of media-assisted learning, though, in the study, the effectiveness of Quizlet was not evident. However, studies such as Fareed et al. (2016) and Kellogg (2001) underscore the importance of writing activities in cognition and linguistic proficiency. While the study did not validate the direct influence of Quizlet on Chinese writing skills, it still offers learners a diverse learning method and tool. This resonates with the multimodal writing approach proposed by Oskoz and Elola (2014) and Kessler et al. (2012), suggesting that a combination of various tools and strategies may be more conducive to enhancing writing skills in language learning.

To complement the quantitative research on student engagement in Quizlet-based teaching activities for learning Chinese, this study conducted an in-depth qualitative analysis. Students actively engaged in Quizlet activities, demonstrating strong behavioral, emotional, and cognitive involvement through participation, genuine interest, and strategic learning approaches. This result confirms Anjaniputra and Salsabila’s (2018) proposition that Quizlet can increase students’ persistence and engagement in the language learning process. Similar findings were found in Zhou’s (2016) study, where using Quizlet in Chinese as a second language classroom significantly improved students’ learning experience and sense of engagement. This study is based on the multi-dimensional perspective of student engagement and integrates the well-known framework in the field of second language acquisition (Ellis, 2010; Han & Hyland, 2015; Zheng & Yu, 2018), providing an overall and contextualized description of student engagement. In combination with previous studies, it appears that increased engagement not only enhances participation but also further improves the outcomes of Chinese language acquisition.

Implications for practice

It would be beneficial to the CFL teachers if they had enough abilities and awareness of the technology that could be used for teaching and learning purposes. Instructors should be educated on how to efficiently integrate Quizlet into their teaching methodologies, maximizing its potential to optimize students’ language learning results. For students who still struggle with conventional techniques for learning Chinese as a foreign language, the study reveals the benefits of utilizing Quizlet as a supplemental learning support. They may make use of Quizlet’s flashcards and study modes to connect with the vocabulary, enhance listening skills, and reinforce their reading comprehension. Meanwhile, it is suggested that students integrate Quizlet into their particular study routines, providing self-paced learning and tailored practice. The findings also provide useful insights for future researchers in the field of Chinese language teaching. Examining the long-term impact of Quizlet integration on Chinese language skills and exploring the ideal ways to combine Quizlet with other instructional approaches could contribute to the continued development of mobile-assisted language teaching techniques.

Conclusions and future research

By combining performance tests and an open-ended questionnaire, this study gained comprehensive insights into Quizlet’s effectiveness in teaching Chinese to foreign learners. Results indicated its notable impact on enhancing listening, speaking, and reading skills, though its influence on writing skills was less significant. Moreover, Quizlet emerged as a potent tool, positively influencing student engagement in participating in Chinese teaching activities.

Given the limited influence of the Quizlet application on the demonstrated writing skills in this study, future research could focus on designing interventions that specifically target individual components of writing, such as vocabulary acquisition, grammar accuracy, sentence structure, or coherence, instead of broad interventions targeting overall writing. Exploring strategies to supplement Quizlet with targeted writing activities may yield more discernible effects on specific aspects of writing skills. Additionally, examining combined intervention strategies that integrate multiple writing tools and techniques could amplify the impact on writing skills by combining the strengths of different approaches and uncovering synergistic effects that individual tools may not achieve alone.

Appendix 1

Open-ended questions on student engagement in Quizlet-based Chinese teaching activities

Behavioral engagement:

• 1. How do you actively participate and contribute during Quizlet-based Chinese teaching activities?

• 2. What strategies or techniques do you use to stay focused and attentive during Quizlet sessions for Chinese learning?

• 3. Please share an example of a Quizlet activity you participated in and how it enhanced your engagement with Chinese learning.

Emotional engagement:

• 4. How do you feel when you are engaged in Quizlet-based Chinese learning activities?

• 5. How does Quizlet contribute to your overall enjoyment of learning Chinese?

• 6. Describe a time when you faced challenges during Quizlet-based Chinese learning activities and how you managed to stay motivated and engaged.

Cognitive engagement:

• 7. How do you connect the information presented in Quizlet to your existing knowledge of the Chinese language?

• 8. What cognitive strategies or approaches do you use to enhance your understanding of Chinese language concepts through Quizlet?

• 9. How do you use Quizlet’s flashcards or other study modes to enhance your cognitive engagement and comprehension of Chinese materials?