Instructional practices and students’ reading performance: a comparative study of 10 top performing regions in PISA 2018

Abstract

This comparative study investigated the associations between instructional practices and students’ reading performance among 10 top performing regions that participated in the Program for International Student Assessment (PISA) 2018. A nationally representative sample consisting of 80,016 15-year-old students from 5 Asian regions (B-S-J-Z [China], Singapore, Macao, Hong Kong, and Korea) and 5 Western regions (Estonia, Canada, Finland, Ireland, and Poland) were included. A secondary analysis of PISA survey and assessment data was conducted. T test and ANOVA analyses revealed systematic differences in instructional practices of the 10 regions. B-S-J-Z (China) had significantly higher levels of teacher support, teacher-directed instruction, and teacher stimulation than the other sample regions. Asian regions tended to have higher levels of teacher support, teacher-directed instruction, teacher feedback, adaptive instruction, and teacher enthusiasm compared with Western regions, although variations were also found within Asian regions or within Western regions. Hierarchical linear regression (HLR) analyses indicated that reading performance was positively predicted by teacher support, adaptive instruction, teacher stimulation, and teacher enthusiasm, but negatively predicted by teacher-directed instruction and teacher feedback. This study sheds light on the effective instructional practices for optimizing students’ reading performance across different cultural contexts.

Introduction

As a crucial competency for living and learning, reading is one core subject assessed in the Program for International Student Assessment (PISA). The outstanding performance of the top performing regions has always attracted researchers to study the factors underpinning their academic success (e.g., Lau & Ho, 2015; Qian & Lau, 2022). Instructional practice is among one of the key factors contributing to high levels of reading performance (Karaman, 2022; Nurmi et al., 2013; OECD, 2019a). While a plethora of research has investigated the associations between instructional practices and reading performance, relatively little attention is given to the role of cultural context in shaping this relationship (Haw & King, 2022). This comparative study sought to delve into this issue further by examining PISA data.

PISA is a triennial international large-scale assessment test conducted by the Organization for Economic Cooperation and Development (OECD). PISA evaluates the effectiveness of education systems worldwide by measuring the knowledge and cognitive skills of 15-year-old students from OECD member and partner countries (OECD, 2019a, 2019b). The latest PISA assessment (i.e., PISA 2018) focuses primarily on reading, supplemented with mathematics and science. Besides assessing students’ performance in these subjects, PISA 2018 surveys contextual factors that may impact students’ reading ability (OECD, 2019a). Of relevance to this study, several variables in the PISA questionnaire specifically target students’ perceived reading instructional practices, which can then be related to their performance (Koyuncu & Fırat, 2020; OECD, 2019b; Rojas-Torres et al., 2021). Drawing on data from PISA 2018, this study compared 10 top performing regions in terms of their instructional practices, and examined the associations between instructional practices and reading performance through a cross-cultural lens.

Reading literacy: PISA 2018 reading performance assessment framework

According to OECD (2019b), reading literacy is defined as ‘understanding, using, evaluating, reflecting on, and engaging with texts in order to achieve one’s goals, to develop one’s knowledge and potential, and to participate in society’ (p. 28). The skill of reading is not just important for living in the contemporary society, it is also important for learning and demonstrating skills in other academic subjects (e.g., Nortvedt et al., 2016; OECD, 2019a; Yang et al., 2018). Measures of reading literacy are provided by international test assessments such as PISA.

Aligned with the reading assessment framework of previous cycles, PISA 2018 measures students’ reading performance based on three categories of cognitive processes—locating info (PVRCLI), understanding (PVRCUN), and evaluating and reflecting (PVRCER). PVRCLI is about getting and retrieving information in a piece of writing, as well as finding and choosing the relevant text. PVRCUN involves comprehending the literal meaning of the text and making inferences. PVRCER refers to judging the quality and reliability of the text, as well as pondering its content and structure (see reading assessment subscales; OECD, 2019a, 2019b).

Recognizing the growing importance of digital literacy, PISA 2018 was expanded to incorporate digital reading as a crucial component in the assessment (OECD, 2019a). Reflecting this changing nature of reading literacy, PISA 2018 also assesses students’ ability to read and understand different text structures such as single-source texts (PVRTSN) and the ability to read multiple-source texts, static and dynamic texts, continuous and non-continuous texts, and mixed texts (PVRTML) on different reading situations (personal reading; educational reading; occupational reading). The assessment focuses on the dynamic reading skills which enable learners to cope with the demands of reading in the digital era (Hu & Wang, 2022; Yang et al., 2018).

Understanding the factors that promote success in reading is crucial. According to OECD (2019a), a number of factors could facilitate the development of reading literacy. These include individual factors such as individual reading practices and motivation, as well as school factors such as teaching practices and classroom support for reading growth and engagement. Among these factors, high-quality instructional practice is of great importance (Bozkurt, 2022; Karaman, 2022; Qian & Lau, 2022). High-quality instructional practice builds foundational reading skills, and promotes reading motivation and engagement (Haw & King, 2022; Nurmi et al., 2013). With the increasingly digital world, it is timely for research to inform how instructional practices may be adjusted to align with this broader conceptualization of literacy—digital literacy (Hu & Wang, 2022).

Indices of reading instructional practice in PISA 2018

Instructional practice refers to the things that happen in the classroom (Depaepe & König, 2018). It is a domain-specific construct studied with a wide range of dimensions and theoretical frameworks (Francisco & Celon, 2020; Guthrie et al., 2012). In PISA assessment, instructional practice refers to a broad range of teaching and learning activities. Drawing on the Teaching, Monitoring, Questioning Techniques framework (TMQT), PISA 2018 formulated a model of the practices of reading instruction encompassing three dimensions—cognitive activation, classroom management and structure, and classroom supportive climate (Gu & Lau, 2023; OECD, 2019a).

Indices of reading instructional practice were included in PISA 2018 to suggest their importance on reading literacy. The index teachers’ stimulation of reading engagement (STIMREAD), derived from the dimension of cognitive activation, refers to the strategies teachers use to foster students’ motivation and engagement in reading. Teacher-directed instruction (DIRINS), based on the dimension of classroom management and structure, refers to the teaching approach in which teachers plan, structure, and deliver the lesson. Teacher academic support (TEACHSUP), teacher feedback (PERFEED), adaptive instruction (ADAPTIVITY), and teacher enthusiasm (TEACHINT) are derived from the dimension of classroom supportive climate. TEACHSUP refers to the support and guidance provided by teachers for facilitating students’ learning in reading. PERFEED refers to the extent teachers give comments to students regarding their reading performance. ADAPTIVITY refers to the adjustments teachers make in their instruction for meeting students’ learning needs. TEACHINT refers to the passion and excitement that teachers demonstrate in teaching.

Most of the abovementioned indices aligned with that of the previous cycle with reading as the focus (i.e., PISA 2009). For instance, TEACHSUP, DIRINS, PERFEED, and TEACHINT are completely identifiable with those in PISA 2009. However, one index has been slightly revised (i.e., STIMREAD) and one has been newly added (i.e., ADAPTIVITY). These indices were designed to provide a comprehensive assessment regarding the quality of reading instruction (Hu & Wang, 2022). It should be noted that these indices of reading instructional practices are applicable to all classes in general; PISA assessment adopts a generic view of literacy, thus the general skills of reading are assumed to be taught in many different subjects (OECD, 2019a).

Instructional practices and reading performance: a review of PISA studies

Reading literacy is a main goal of education. It is well-established that effective reading instruction is essential for students to become proficient readers and to develop strong literacy skills (Nurmi et al., 2013). Teaching practices encompassing explicit and systematic instruction in the key skills of reading significantly impact on students’ language learning outcomes such as achievement, motivation, and engagement (e.g., Lau & Ho, 2015; Guthrie et al., 2012).

Such a relationship has also been observed in studies that utilized PISA data (e.g., PISA 2009, 2018). The studies concerned have identified key instructional practices associated with reading performance from various theoretical perspectives. For instance, Haw et al. (2021), through the lens of self-determination theory, found that need supportive teaching was positively correlated with reading achievement. Meng et al. (2017), adopting Stronge’s teaching dimensions, found that teacher feedback and class discussion were related to reading performance. Additionally, Qian and Lau (2022), using PISA reading instruction dimensions, found that classroom disciplinary climate, adaptive instruction, and stimulation of reading engagement were associated with reading performance. Although empirical evidence is not lacking, the extant literature has yielded inconsistent findings. While some studies have shown that teacher feedback is associated with reading performance (Gu & Lau, 2023; Meng et al., 2017), other studies have found either a negative or insignificant contribution (Bozkurt, 2022; Qian & Lau, 2022). Similarly, the link between teacher-directed instruction and reading literacy is positive in some studies (e.g., Karaman, 2022), but negative in the others (e.g., Hu & Wang, 2022; Ma et al., 2022; Qian & Lau, 2022). There is little consensus on what constitutes effective instruction for successful reading outcomes.

The inconsistent findings in the literature can be attributed to factors such as differences in participants’ proficiency levels and the contexts in which they are situated in. Most of these studies involve participants with varying proficiency levels. It is worth noting that instructional practices can function more or less effectively depending on students’ abilities and prior knowledge (Lau & Lam, 2017; Medina & McGregor, 2019). Also, these studies either focused on a single region, or many regions with diverse cultural backgrounds. However, the effectiveness of instructional practice is greatly influenced by the unique characteristics and environments inherent in each educational system (Yang et al., 2020). Comparative studies involving students with similar levels of achievement from different cultural regions are, therefore, needed to clarify the associations between instructional practices and reading performance.

Instructional practices and reading performance in PISA studies: potential influences of cultural context

Contextual factors such as culture play an important role in shaping the effectiveness of instructional practices (e.g., Gu & Lau, 2023; Ma et al., 2022). Different educational systems emphasize different instructional practices due to varying cultural and pedagogical beliefs (Yang et al., 2020). East-Asian education is often characterized by high levels of teacher-centered instruction, whereas Western education is typically more student-centered (Gan et al., 2023). As revealed in PISA studies, Chinese students reported a more disciplinary climate and lower levels of teacher stimulation and scaffolding than their Western counterparts (Lau & Ho, 2015; Ho & Lau, 2018). Due to the differences in the preferences for instructional practices across cultural contexts, the effectiveness of a particular instructional practice may also differ across cultures. For example, cross-cultural comparative studies have revealed that the association between teachers’ reading stimulation and performance was significant in the Chinese context but not in the USA (Meng et al., 2017). These findings suggest that an instructional practice which is effective in one cultural context may be ineffective in another due to cultural differences. As a result, a cross-cultural investigation into the associations between instructional practices and performance is warranted.

Indeed, cross-cultural comparative research on the impact of instructional practice on reading performance is of growing interest. For example, Ma et al. (2022) examined the relations between perceived teacher feedback and reading achievement across individualistic and collectivistic cultures. They found that the indirect effect of teacher feedback on reading achievement via reading self-concept was stronger in attentive or harmonious classroom climate, particularly in Asian regions such as B-S-J-Z (China) and Singapore. Similarly, Gu and Lau (2023) investigated the associations between reading instruction and reading performance in B-S-J-Z (China), Hong Kong, and Chinese Taipei. It was found that disciplinary climate and teacher support positively predicted reading performance for all the three Chinese regions. However, the link between reading instruction (teacher’s stimulation of reading engagement and adaptive instruction) and reading performance was significant in B-S-J-Z (China) and Hong Kong, but not in Chinese Taipei.

Although these studies have identified notable cross-cultural and intra-cultural similarities and differences in the effectiveness of reading instruction on reading performance, there remains a gap regarding the similarities and differences in the effective instructional practices that promote high reading performance across different regions (i.e., top performing regions in PISA 2018). Using PISA data as a comprehensive basis for cross-cultural comparison, we may better understand how cultural contexts shape the effectiveness of instructional practice for fostering success in reading.

A cross-cultural comparative study of instructional practices and reading performance in PISA studies: theoretical framework

The influence of instructional practices on students’ learning outcomes can be understood from the sociocultural constructivist perspective. Sociocultural theory (SCT; Vygotsky, 1986) posits that both the environment and culture play an influencing role in individual’s learning. Learning occurs within the context of sociocultural interactions that involve both the learner and teacher (Mascolo, 2009). From this perspective, reading is considered as a sociocultural and cognitive process (Tracey & Morrow, 2006; Li et al., 2022). When the learner acquires cultural knowledge in the text and connect it to his or her personal interests and experience, meaning in reading is constructed (Van Oers, 2009). Effective instruction requires the teacher to direct the learners, help them facilitate understanding (e.g., through teacher-directed instruction), provide them with scaffolding (e.g., through teacher feedback), and engage them (e.g., through stimulation of reading engagement) as they construct meaning for themselves (Guthrie et al., 2012; Mascolo, 2009). Previous studies have applied SCT in studying instructional practices and students’ learning outcomes. For instance, Van Rijk et al. (2017) adopted SCT to examine the characteristics of learning environment that promotes students’ reading comprehension. In Li et al.’s (2022) study, the effectiveness of strategy-based reading instruction program developed from SCT was investigated in relation to reading performance of language learners from diverse cultures. SCT may shed light on the most effective instructional practices for reading performance in international assessments (e.g., PISA) for different cultural contexts.

Summary

Reading is the focus of assessment in PISA 2018. Many studies have utilized PISA data to examine the factors contributing to students’ reading performance, with instruction practices being one crucial factor. However, inconclusive findings have been observed regarding some indices of instructional practice that contribute to reading performance. It is unclear as to what instructional practices lead to high reading performance. Also, limited attention has been given to the confounding influence of cultural context on this relationship. As such, additional comparative research is needed to ascertain the extent to which instructional practice foster high reading performance across different regions, particularly the top performing ones. The insights gleamed may inform policies and practices aimed at enhancing reading performance with the development of more culturally relevant instructional practices.

The present study

Therefore, the purpose of this study was to examine the associations between instructional practices and reading performance in PISA 2018, and whether such associations vary across different regions. For the analysis, we intentionally chose 10 regions to focus on, namely Beijing-Shanghai-Jiangsu-Zhejiang (B-S-J-Z [China]), Singapore, Macao, Hong Kong, Estonia, Canada, Finland, Ireland, Korea, and Poland. These regions were the top performers in PISA 2018. By studying these explementary examples, we may provide robust evidence on the specific instructional practices that relate to high reading performance. Given that this selection includes both Asian and Western regions, a cross-cultural comparison is made possible. Unlike comparative studies that typically use China to represent Asian culture and America to represent Western culture, this selection includes more diverse regions. In addition to revealing the best practices pertaining to each region, we may also unveil important variations within each cultural group (e.g., Asian regions) that have previously been overlooked. There are two research questions that this study aims to address: (1) What are the levels of instructional practices and reading performances of the 10 top performing regions in PISA 2018? (2) How are the instructional practices of these 10 regions related to reading performances in PISA 2018?

Method

The data source and sample

This study utilized questionnaire and assessment data from PISA 2018. The dataset for secondary analysis was downloaded from PISA database, which can be assessed on OECD website (http://www.oecd.org/pisa/data/2018database). The 2018 data was selected for analysis as it is the most recent version with reading being the main assessment domain. Approximately 600,000 students aged 15 from 79 regions participated in PISA 2018. Student aged 15 who are in the seventh grade or above were recruited randomly using a two-stage stratified sampling method by which schools are sampled first, and subsequently the classes within the sampled schools are chosen (see PISA technical report for the details; OECD, 2019c). Given the aims of this study, a total of 80,016 students with a mean age of 15 years old from 2963 schools within the 10 top performing regions were selected. Specifically, 12,058 (15.1%) were from B-S-J-Z (China), 6676 (8.3%) were from Singapore, 3775 (4.7%) were from Macao, 6037 (7.6%) were from Hong Kong, 5316 (6.6%) were from Estonia, 22,653 (28.3%) were from Canada, 5649 (7.1%) were from Finland, 5577 (7.0%) were from Ireland, 6650 (8.3%) were from Korea, and 5625 (7.0%) were from Poland. Among all these students, 40,591 were male (50.7%) and 39,425 were female (49.3%). As of their cultural background, 35,196 were from Asian regions (43.9%), whereas 44,820 were from Western regions (56.1%). The demographic information for the sample regions was shown in Table 1.

Table 1 Demographic information of the 10 top performing regions in PISA 2018

Measures

Reading performance

PISA 2018 reading test used questionnaire and cognitive items to measure students’ reading literacy—the ability to understand, use and reflect on written and digital texts (i.e., the three cognitive processes). In the 1-h computer-based test, students were given 3 sets of reading tasks and responded to multiple choice and open-ended questions. The test items correspond to students’ abilities due to an adaptive approach to assessment. In such an approach, the difficulty level of tasks is determined based on students’ performance in prior stages (OECD, 2019a).

The overall reading performance (PVREAD) is represented by 10 plausible values (PVs), with the results standardized with a mean of 500 and SD of 100 using the Rasch method. PISA also provided 10 PVs for each assessment subscale (PVRCLI, PVRCUN, PVRCER, PVRTSN, and PVRTML). The 10 PVs are generated from posteriori distribution by combining Item response theory (IRT) scaling of items and a regression model with questionnaire data. Compared with an overall composite score, the 10 PVs depict a more accurate estimation of students’ possible ability range. PISA recommends researchers to use all these 10 PVs for secondary data analysis (see technical report; OECD, 2019c). As in most PISA studies (e.g., Ma et al., 2021; Meng et al., 2017), students’ mean reading scores in 10 plausible values (i.e., variables PV1READ to PV10READ) were used as the outcome variable of this study.

Instructional practices

The PISA 2018 student questionnaire used 6 sets of indices (a total of 22 items) to measure students’ perception of reading instructional practices in their classes in general (OECD, 2019a, 2019c). The indices included TEACHSUP, DIRINS, PERFEED, ADAPTIVITY, STIMREAD, and TEACHINT. TEACHSUP was assessed based on 4 items measuring the frequency students perceived their teachers provided support in learning. DIRINS was assessed based on 4 items regarding how often students perceived their teachers provide direct instruction when teaching reading. PERFEED was based on 5 items that assess how often students perceived their teachers to provide feedback. ADAPTIVITY was assessed with 3 items on the extent to which students perceived their teachers to be adaptive in teaching. STIMREAD was assessed with 4 items about how often students perceived their teachers to provide stimulation. TEACHINT was based on 4 items measuring the degree to which students perceived their teachers to be enthusiastic with teaching. Students responded to all these 22 items with a 4-point Likert scale from 1 (never or hardly ever/never or almost never/strongly agree) to 4 (every lesson/all lessons/every lesson or almost every lesson/strongly disagree). For the complete items and response options, see Appendix.

All the above indices were scaled using weighted warm likelihood estimate to have a mean of 0 and a standard deviation of 1. Positive values indicate having more positive attitudes of instructional practices perceived than the average OECD student (OECD, 2019c). In our analyses, the weighted likelihood estimates (WLE) scores were used for all reading instructional variables (Khorramdel et al., 2020).

Demographic variables

Student-level demographics (gender and socioeconomic status) were included as control variables given their confounding effects on students’ reading performance in PISA 2018. Previous studies have provided evidence of gender differences in reading achievement with girls consistently outperforming boys (e.g., Khorramdel et al., 2020; Ma et al., 2022). Another important variable to consider is socioeconomic status. Socioeconomically advantaged students tend to demonstrate greater reading motivation and success than their disadvantaged peers (e.g., Koyuncu & Fırat, 2020; OECD, 2019a; Qian & Lau, 2022). In the current study, students’ socioeconomic status was represented by the economic and sociocultural status (ESCS) index. This index was based on the weighted mean of three indicators (parental occupation, parental educational level, and home possessions) scaled with a mean of 0 and a standard deviation of 1. Gender (GENDER) was coded as a dichotic variable (1 = female, 2 = male).

Data analysis

All data were analyzed using SPSS 20.0. The subjects with missing data were excluded. Descriptive statistics and zero-order correlations were calculated for all variables in each of the sample regions. To examine the levels of reading performance of different regions (RQ1), we calculated their mean reading and subscales scores (PVREAD, PVRCLI, PVRCUN, PVRCER, PVRTSN, and PVRTML). Within each region, the scores of different reading subscales were then compared to explore its respective strengths in reading. To examine differences in the levels of instructional practices across the 10 regions (RQ1), one-way ANOVA analyses were performed with the 10 regions as independent variables and the 6 instructional practices as dependent variables (an exception was for Canada with only data from ADAPTIVITY and STIMREAD). Follow-up post-hoc tests (multiple comparison of means) were conducted using Bonferroni method for revealing cross-regional differences. To further examine whether the differences are influenced by the cultural context, independent t tests were conducted for all regions within two cultural groupings: Asian and Western regions. The 6 instructional practices were used as the dependent variables, with the two cultural groupings as the independent variables. To test the associations between instructional practices and reading performance (RQ2), hierarchical linear regression (HLR) analyses were performed for all region samples, as well as each individual region sample. The outcome variable was students’ reading performance (PVREAD) and the independent variables were the 6 indices of instructional practices (TEACHSUP, DIRINS, PERFEED, ADAPTIVITY, STIMREAD, and TEACHINT). In step 1, demographics variables were entered as covariates. In step 2, the variables of instructional practices were entered. In every step, the fit indices and R² change of the model were assessed. The regression analyses were conducted for all the 10 reading PVs, incorporating the sampling weights. For each predictor, the mean of all the 10 regression coefficients obtained was considered (Rojas-Torres et al., 2021). The level of statistical significance was set at p < 0.05.

Results

Regional differences in reading performance

Table 2 shows the means and standard deviations of reading performance for all the 10 top performing regions in PISA 2018. The overall reading score (PVREAD) and subscale scores measuring reading performance in different cognitive processes (PVRCLI, PVRCUN, and PVRCER) and text structures (PVRTSN and PVRTML) were presented. Among the 10 regions, B-S-J-Z (China) achieved the highest scores in all subscales. In terms of its performance, B-S-J-Z (China) demonstrated higher performance in PVRCER compared to other subscales. Canada and some Asian regions including Singapore, Macao, and Hong Kong also exhibited better performance in PVRCER. Korea had a higher score in PVRTML while Finland and Ireland had higher scores in PVRCLI. Estonia and Poland showed consistent performance across all subscales.

Table 2 Means and standard deviations of the 10 top performing regions on overall reading performance and scores of reading assessment subscales in PISA 2018

Regional differences in instructional practices

Table 3 presents the means and standard deviations for each instructional practice. One-way ANOVA revealed significant cross-regional differences for TEACHSUP (F[8, 56690] = 375.295, p < .001), DIRINS (F[8, 56651] = 553.743, p < .001), PERFEED (F[8, 56418] = 321.272, p < .001), STIMREAD (F[9, 77017] = 465.02, p < .001), and TEACHINT (F[8, 56554] = 447.47, p < .001). B-S-J-Z (China) had significantly higher levels of TEACHSUP, DIRINS, and STIMREAD compared to the other sample regions. Singapore had a significantly higher level of PERFEED than the other regions. In contrast, Finland had a lower level of STIMREAD whereas Poland had a significantly lower level of TEACHINT than the others.

Table 3 Means and standard deviations of the 10 top performing regions on perceived instructional practices in PISA 2018 (N = 80,016)

The results of post-hoc testing with Bonferroni method revealed some differences within Asian or Western regions. For TEACHSUP, Macao and Hong Kong had significantly lower scores compared to other Asian regions, such as B-S-J-Z (China), Singapore, and Korea. Similarly, Finland and Ireland had higher TEACHSUP scores than their Western counterparts, such as Estonia and Poland. Although DIRINS was found to be high in Asian regions such as B-S-J-Z (China) and Korea, scores below the OECD average were found for Singapore, Macao, and Hong Kong. As for STIMREAD, Estonia and Finland had lower scores than the other Western regions, such as Canada, Ireland, and Poland.

Further to these, independent t tests showed that cultural context had a significant effect on TEACHSUP (t[56697] = 26.031, p < .001), DIRINS (t[56658] = 42.675, p < .001), PERFEED (t[54625] = 25.636, p < .001), ADAPTIVITY (t[76860] = 28.306, p < .001), and TEACHINT (t[56561] = 41.927, p < .001), but not for STIMREAD (p > .05). The Asian sample regions showed significantly higher levels of TEACHSUP, DIRINS, and TEACHINT compared to both the Western regions and the OECD average. The Western regions had levels of these variables that were below the OECD average. On the other side, the Asian regions displayed significantly higher levels of PERFEED and ADAPTIVITY than Western regions, despite both regions having scores higher than the OECD average. STIMREAD was above the OECD average for both Asian and Western regions.

Instructional practices and students’ reading performance

Prior to the regression analyses, the correlations among all variables were calculated for all sample regions. As shown in Table 4, the demographic variables GENDER and ESCS were found to have a significant negative and positive correlation, respectively, with overall reading performance. In the subsequent analysis, GENDER and ESCS were controlled to partial out their influences on the outcome variable. In all sample regions, TEACHSUP, ADAPTIVITY, STIMREAD, and TEACHINT were significantly and positively correlated with reading performance in all sample regions. The correlation between PERFEED and reading performance was either negative or insignificant in some Asian and Western regions such as Macao, Korea, Estonia, Finland, and Poland. On the other hand, DIRINS showed a negative correlation with reading in both Asian and Westerns regions. However, in Macao and Korea, it was positively associated with reading performance. **p < 0.001, *p < 0.05

Table 4 Zero-order correlations among all variables for the 10 top performing regions in PISA 2018 (N = 80,016)

HLR models shows the contribution of instructional practices to reading performance for all the 10 regions (Table 5). The first model with covariates was significant (F [2, 55762] = 2816.42, p < .001). Both GENDER and ESCS were associated with reading (β =  − .12, p < .001 and β = .07, p < .001). The covariates accounted for 9.2% variance in reading performance. As the instructional practices were entered in step 2, the model was also significant, F(8, 55756) = 1021.42, p < .001. All the paths toward reading performance were significant. Results indicated a significant positive effect of TEACHSUP, ADAPTIVITY, STIMREAD, and TEACHINT on reading performance. Notably, STIMREAD had the strongest effect on reading performance (β = .14, p < .001), followed by ADAPTIVITY (β = .07, p < .001), TEACHINT (β = .06, p < .001), and TEACHSUP (β = .06, p < .001). In contrast, the effect of DIRINS (β =  − .13, p < .001) and PERFEED (β =  − .09, p < .001) on reading performance was negative. All the variables accounted for 12.8% of the variance explained in reading performance. The second model with instructional practices included showed significant improvement from the first one (ΔF[6, 6] = 1795, p < .001, ΔR² = .36), indicating an additional 3.6% of variance explained in reading performance.

Table 5 Hierarchical linear regression results for instructional practice variables predicting reading performance for the 10 top performing regions in PISA 2018 (The Entire Sample; N = 80,016)

Regional differences in instructional practices and students’ reading performance

The HLR models were significant for both Asian regions (Table 6) and Western regions (Table 7). As illustrated in the tables, the predictive effects of all these instructional practices were generally consistent with TEACHSUP, ADAPTVITY, STIMREAD, and TEACHINT as positive predictors, and DIRINS and PERFEED as negative predictors. STIMREAD was the strongest predictor of performance in Asian regions such as B-S-J-Z (China [β = .18, p < .001]), Singapore (β = .09, p < .001), Macao (β = .14, p < .001), and Hong Kong (β = .10, p < .001), and in Western regions including Ireland (β = .12, p < .001) and Poland (β = .09, p < .001). TEACHINT was the strongest predictor in Korea (β = .13, p < .001). ADAPTIVITY was the strongest predictor in Western regions such as Estonia (β = .10, p < .001), Canada (β = .05, p < .001), and Finland (β = .11, p < .001). Despite this, a few variations on the impact of these instructional practices across the 10 regions were found: PERFEED was positively associated with reading performance (β = .01, p < .001) in Singapore; TEACHSUP was negatively associated with reading performance (β =  − .00, p < .001) in Ireland; ADAPTIVITY was negatively linked with reading performance (β =  − .05, p < .001) in Macao. After controlling for the covariates, all the instructional variables accounted for a substantial amount of variance in reading performance for all regions: B-S-J-Z (China [4.2%]), Singapore (3.4%), Macao (2.1%), Hong Kong (1.9%), Estonia (4.2%), Canada (0.5%), Finland (4.3%), Ireland (4.5%), Korea (4.9%), and Poland (2.9%).

Table 6 Hierarchical linear regression results for instructional practice variables predicting reading performance for the 10 top performing regions in PISA 2018 (The Asian Regions; N = 35,196)

Table 7 Hierarchical linear regression results for instructional practice variables predicting reading performance for the 10 top performing regions in PISA 2018 (The Western Regions; N = 44,820)

Discussion

Reading performance

In terms of reading performance, B-S-J-Z (China) obtained both the highest overall reading score (i.e., PVREAD) and subscale scores measuring reading performance on different reading cognitive processes (i.e., PVRCLI, PVRCUN and PVRCER) as well as text structure subscale scores (i.e., PVRTSN and PVRTML) in PISA 2018, suggesting that B-S-J-Z (China) has been able to maintain a similar level of performance in reading as seen in previous cycles of PISA (OECD, 2012; Zheng et al., 2022). The three other Asian regions (i.e., Singapore, Macao, and Hong Kong) also scored higher on both the overall reading performance and subscale scores compared with Western regions. These results were consistent with the general perception of the superiority of students from the Confucian-heritage cultures over Western students in academic achievement documented in cross-national studies of achievement (e.g., Li, 2002; Cai & Zhu, 2017; Rao et al., 2000; Lau & Lam, 2017). For example, Cai and Zhu (2017) found that Chinese adolescents scored higher than their Finnish counterparts on the reading literacy variable in the PISA 2009 reading programme. Lau and Lam (2017) also reported that science performance of students from Asian regions (i.e., Singapore, China, Hong Kong, Chinese Taipei, Macao, Japan, Korea, and Vietnam) ranked at the top of the 2015 PISA league table. Such superiority in large-scale international academic testing among students from the Confucian-heritage learning cultures in Asian regions can be interpreted with reference to the social and cultural context of education where there is considerable pressure for students to achieve academically. From a cultural perspective, Confucian ideas emphasize that effort and perseverance are more important to scholastic success than ability (Rao & Chan, 2010; Wang, 2013). Students therefore exert effort in academic work because they believe that success is the result of hard work (Gan, 2009; Gan et al., 2023; Hau & Ho, 2010; Weiner, 1986).

Another explanation that was often advanced to account for ‘Confucian heritage culture’ students’ superiority over Western students in cross-national studies of achievement is that parents in Asian regions typically have high expectations for their children’s academic success, which in turn may lead to students feeling the pressure to do well in academic studies (Ho et al., 2007). On the other hand, students may feel ashamed when they are judged negatively by their parents due to inadequate learning (Li, 2002). This emotion of shame-guilt may urge them to work harder to pursue better learning achievement.

Finally, due to a competitive learning environment in some Asian regions, since their early academic training students are inculcated a belief that high grades on exams largely determine admission into higher education or for getting a job as well as for their social lives in general (Lau et al., 2008). This tends to result in highly efficient adoption of adoption of a performance-approach goal in learning among the students, which has recently been shown to both benefit students in achieving high grades in conventional assessments and lead to a deep cognitive engagement (Liem et al., 2008). Liem et al. (2008) emphasize that in a highly competitive learning environment where meritocracy system can regulate and determine one’s advancement in life, attempting to outperform others in academic studies through pursuing a performance-approach goal cannot be regarded as a maladaptive behavior.

Variations in instructional practices

In terms of instructional practices, the results showed significant differences in the instructional practices investigated (i.e., teacher support, teacher-directed instruction, perceived feedback, adaptive instruction, teacher stimulation, and teacher enthusiasm) across the ten regions. The indices of the three of the six instructional practices variables (i.e., teacher support, teacher-directed instruction, and teacher stimulation) reported by B-S-J-Z (China) were significantly higher compared to the other regions. A higher value on the index of each of these teacher behaviors means higher academic support from a teacher, more frequent teacher-directed instruction, and more frequent teacher use of engagement strategies respectively among students in B-S-J-Z (China). It could be that due to the influence of a long tradition of Confucian philosophy, Chinese students tend to expect the teacher to possess knowledge, act as a role model to them, and be responsible for the assessment of their learning (Chuang, 2012), which eventually lead to teachers feeling the pressure to exert much effort in order to do well in teaching.

When the sample regions involved in this study were classified as two broad cultural groups (i.e., Asian and Western regions), the impact of the cultural context on students’ perceived instructional practices was still considerably visible as the indices of teacher support, teacher-directed instruction, teacher enthusiasm, teacher feedback, and adaptive instruction were significantly higher for the Asian regions than for the Western regions. As the sample regions could therefore generally be placed on an Asian-Western continuum, the results lend support for the role of the broad cultural context in shaping instructional practices, providing empirical evidence for the generalizations that are often made about teaching and learning between Asian and Western cultures (Littlewood, 2001). In other words, cultural context apparently did affect self-reports of the level of perceived teacher behavior. The generally consistently higher indices of instructional practices across Asian countries could be due to sociocultural congruencies that are based on Confucian tradition and emphasis on effort, responsibility, and achievement orientation (Martin et al., 2014). For example, schools and national education systems in Asian regions tend to be conducted along competitive, iteratively assessed, didactic instruction lines (Martin et al., 2014). In some Asian regions such as China, exam-oriented education remains entrenched in the psyche and behavior of educational stakeholders (Tan and Chua, 2015). Koh and Luke (2009) also reported that students’ test scores continue to be the major indicator of teachers’ job performance and school effectiveness in Singapore. Tan and Chua (2015) observed that confronted with the demand to produce high exam scores, didactic teaching usually dominates in many schools in China. Similarly, a competitive and pragmatic learning culture prevails in Singapore where teachers and schools have had great success in defining and assigning, producing and rewarding factual and basic knowledge, and where civic and media, parental and student consciousness is often influenced and mediated by a competitive focus on examination results and their consequences for students’ lives.

Instructional practices and students’ reading performance

In this study, the results revealed that even when GENDER and ESCS were controlled, four (i.e., teacher support, adaptive instruction, teacher stimulation, and teacher enthusiasm) of the six instructional practices investigated were significantly and positively correlated with reading performance in all sample regions. Interestingly, perceived feedback and teacher-directed instruction were found to be mostly negatively or insignificantly correlated with reading performance in either Asian or Western regions. HLR modeling analyses further confirmed significant positive predictive effects of teacher support, adaptive instruction, teacher stimulation, and teacher enthusiasm on reading performance, with teacher stimulation being the strongest predictor, followed by adaptive instruction, teacher enthusiasm, and teacher support. HLR modeling analyses also confirmed the generally negative effects of teacher-directed instruction and perceived feedback on reading performance. These patterns of positive or negative relationships between instructional practices and reading performance was still visible when Asian regions and Western regions were considered separately. These findings therefore enable us to conclude that students of a particular culture or cultures might benefit or suffer more from certain instructional practices (Haw & King, 2022; Meng et al., 2017).

While the associations of the instructional practices with reading performance are highly consistent across Asian and Western regions, there were some exceptions where the association of the instructional practices with reading performance scores did vary within Asian or Western regions. For example, adaptive instruction showed a negative linkage with reading performance in Macao; teacher feedback was found to be positively correlated with reading performance in Singapore; teacher support was found to be negatively correlated with reading performance in Ireland. The reason for such exceptions could be due to different ways in which schools and national education systems implement their processes (Martin et al., 2014). Future studies can adopt qualitative methods such as school observations and interviews to uncover the nuances of school or national cultures that might lead to the variances within individual regions.

The prevalence of the negative associations of teacher-directed instruction and teacher feedback with reading performance within either Asian or Western regions is particularly worth noting. Teacher-directed instruction was one of the three instructional practice variables reported to be significantly more common either in China or in Asian regions that also performed better in reading than other Western regions. The lack of a positive relationship between teacher-directed instruction and reading performance in this study, however, gains cross-validation from other evidence (e.g., Lau & Lam, 2017). It could be that over-relying on teacher-directed instruction strategies caused lower performance, and that academically weaker students tended to receive more teacher-directed instruction than their academically stronger counterparts (Medina & McGregor, 2019). Researchers and educators also view feedback as one of the powerful influences on learning and achievement (Hattie & Timperley, 2007). However, somewhat similarly to the negative impact of teacher-directed instruction on performance documented in this study, it was likely that students who reported they received constant feedback tended to struggle in class the most (Medina & McGregor, 2019; OECD, 2016). The generally negative impact of teacher feedback on performance documented in this study might also suggest that it is the quality of feedback rather than the quantity that matters most.

Implications

This study contributes to the discussion of the influence of social and cultural contexts on students’ learning and teachers’ instructional practices in the literature. The study provides empirical support to such socio-cultural theorizing. The significant mean-level differences in both students’ reading performance and their perceived instructional practices across Asian and Western regions suggested that that context did play a powerful role in shaping students’ learning performance and teachers’ teaching behavior. These differences muse be recognized when attempting to transfer educational policy or theoretical constructs from one culture to another, designing instructional interventions to enhance students’ learning motivation, and initiating pre‐service teacher education, in‐service training, and curriculum reform. For example, pre-service language teacher education courses can have teacher candidates read and analyze empirical studies of the role of different instructional practices in students’ learning achievement so as to develop an awareness of differential effectiveness of these instructional practices in aiding student learning. Undoubtedly, incorporation of such a component into the pre-service teacher education programs empowers teacher candidates to be aware of the utility of optimal instructional practices that will lead to the most desirable outcomes of student learning in a particular context.

Furthermore, this study enriched our understanding of the patterns of relations between instructional practices and students’ reading performance across different cultural contexts. Very interestingly, the study confirmed that both the positive and negative relations of some instructional practices to reading performance were consistent across Asian and Western regions, which may shed light on universality of instructional practices for optimizing students’ reading performance despite the differences in cultural contexts. For example, within both Asian and Western regions, teacher-directed instruction and teacher feedback were found to negatively predict students’ reading performance. This result alerts us to the fact that something is taught or a teacher provides feedback on error does not mean that students will acquire it or adopt the correct form immediately or consistently (Lightbown & Spada, 2011). As Lightbown and Spada (2011) put it, the teacher’s principal role in student learning is to provide a supportive environment in which students are stimulated, engaged in activities that are appropriate to their learning needs and style, and where students can experience success in learning.

Conclusion

This study contributes to the field by (1) employing the latest PISA data (PISA 2018 database) to analyze the reading instructional practices of the 10 top-performing regions; (2) examining the associations of reading instructional practices with students’ reading performance; (3) revealing the general existence of a continuum with one end of the Asian regions and the other end of the Western regions in relation to instructional practices and student reading performance; (4) suggesting that students of a particular culture or other cultures might benefit or suffer from certain instructional practices. Therefore, on the one hand, cultural clusters could be identified in the prevalence of certain instructional practices such as teacher support, teacher-directed instruction, teacher feedback, adaptive instruction, and teacher enthusiasm; on the other hand, the prevalence in the positive associations of teacher support, adaptive instruction, teacher stimulation, and teacher enthusiasm with reading performance, along with the prevalence in the negative associations of DIRINS and PERFEED with reading performance, was considerably consistent within either Asian or Western regions. Our findings thus shed informative light on socio-cultural theorizing relevant to the role of context in shaping instructional practices and learning achievement, and they assist understanding of effects attributable to national context and effects attributable to cultural ethnicity.

Although this study filled the research gap by presenting a holistic picture of the associations of instructional practices with student reading performance based on the latest PISA data, a few limitations need to be acknowledged. First, as only the 10 top performing regions in PISA 2018 were included in this study, some Asian regions such as Taiwan and Japan or some Western regions such as America and Britain were not included for analysis. Future studies should include a larger sample of regions to achieve a better generalizability of the findings. Second, while the teacher instructional practice factors included in the present study performed generally well in predicting student reading performance, the influences of student factors such as reading enjoyment, reading self-efficacy, and reading engagement on reading performance were not examined in the present study. Future research should thus consider how these student factors may predict reading performance. Third, since PISA is a cross-sectional study, the associations of instructional practices with reading performance examined in this study were only correlational. It is thus not possible to establish the causal relations among the various variables that were explored in this study. Future research should continue to examine the specific mechanisms by which instructional practices impact on students’ reading performance. Fourth, the measures of teacher instructional practices were largely based on student self-reports, which means that the findings might not accurately reflect the reality. Research in the future should consider longitudinal and mixed-methods designs in order to better illustrate the influences of teacher practices, student and environmental factors on student reading performance.

Appendix

Indices of instructional practice in PISA 2018 (OECD, 2019a, 2019c)

Variables	Items	Response Options
TEACHSUP	The teacher shows an interest in every student’s learning; The teacher gives extra help when students need it; The teacher helps students with their learning; The teacher continues teaching until the students understand	4-point scale: 1 = never or hardly ever, 2 = some lessons, 3 = most lessons, 4 = every lesson
DIRINS	The teacher sets clear goals for our learning; The teacher asks questions to check whether we have understood what was taught; At the beginning of a lesson, the teacher presents a short summary of the previous lesson; The teacher tells us what we have to learn	4-point scale: 1 = never or hardly ever, 2 = some lessons, 3 = most lessons, 4 = every lesson
PERFEED	The teacher gives me feedback on my strengths in this subject; The teacher tells me in which areas I can still improve; The teacher tells me how I can improve my performance	4-point scale: 1 = never or hardly ever, 2 = some lessons, 3 = most lessons, 4 = all lessons
ADAPTIVITY	The teacher adapts the lesson to my class’s needs and knowledge; The teacher provides individual help when a student has difficulties understanding a topic or task; The teacher changes the structure of the lesson on a topic that most students find difficult to understand	4-point scale: 1 = never or almost never, 2 = some lessons, 3 = many lessons, 4 = every lesson or almost every lesson
STIMREAD	The teacher encourages students to express their opinion about a text; The teacher helps students relate the stories they read to their lives; The teacher shows how the information in texts builds on what they already know; The teacher poses questions that motivate students to participate actively	4-point scale: 1 = never or hardly ever, 2 = some lessons, 3 = most lessons, 4 = all lessons
TEACHINT	It was clear to me that the teacher liked teaching us; The enthusiasm of the teacher inspired me; It was clear that the teacher likes to deal with the topic of the lesson; The teacher showed enjoyment in teaching	4-point scale: 1 = strongly agree, 2 = agree, 3 = disagree, 4 = strongly disagree

1. TEACHSUP Teacher academic support, DIRINS Teacher-directed instruction, PERFEED Teacher feedback, ADAPTIVITY Adaptive instruction, STIMREAD Teachers’ stimulation of reading engagement, TEACHINT Teacher enthusiasm