Washback of assessment on English teaching-learning practice at secondary schools

Abstract

This paper presents the findings of a study which aimed at exploring the washback of the Secondary School Certificate (SSC) English examinations on English teaching and learning practice at secondary schools in Bangladesh. For this purpose, a qualitative research (employing interviews with English teachers, FGDs with students and classroom observations protocol) was conducted in 12 secondary schools. The results from data analysis indicate that the SSC English tests had strong negative washback on English teaching-learning practice at secondary schools resulting from a lack of a congruence between the objectives of the curriculum and the format of the tests, teachers’ lack of adequate understanding about the curriculum and present assessment system, respondents’ negative attitudes towards the test, and pressure from the schools and the parents to ensure good grades in the final examination. Consequently, teachers and learners adopted such teaching and learning approaches which are highly examination-oriented. The study puts forward some suggestions on how positive washback could be generated. The findings of the study have implications for the decision makers, designers of curricula, assessment and testing, teachers, and teacher trainers involved in the Bangladeshi educational system and elsewhere.

Introduction

English language teaching has always been a challenging area in Bangladesh (Rahman & Pandian, 2018). The quality of English teaching and learning in this part of the world has constantly been criticized. In order to improve the quality of English teaching and learning, several attempts have been made to reform the curriculum. One such change in English curriculum was done in 1996 when GTM was replaced with CLT approach (National Curriculum & Textbook Board [NCTB], 2012; Nuby, Rashid, & Hasan, 2019; Nur & Islam, 2018; Rahman & Pandian, 2018; Rahman, Pandian, & Kaur, 2018; Sultana, 2018) with the hope that it would increase learners’ proficiency by raising the standard of English teaching and learning at educational institutes of Bangladesh (Hamid & Baldauf, 2008; Rahman, 2012; Roshid, 2009). But unfortunately, regardless of various attempts of curriculum reforms, several recent studies (Al Amin & Greenwood, 2018; Hoque, 2016; Maniruzzaman & Hoque, 2010; Rahman et al., 2019) confirm that there is a very insignificant change in teaching and learning process in the English classrooms. Teachers do not show positive attitudes towards CLT (Rahman et al., 2019; Selim & Mahboob, 2001), and our students are always in the move to pass in the test (Rasul, 2014).

Among other reasons, the lack of a concordance between the high-stakes public examinations such as SSC, HSC (Higher Secondary Certificate), the syllabus, their (teachers) notions about the process of learning, and the teaching method (Quader, 2001; Rahman et al., 2019) stands in the way of creating a teaching-learning environment which can foster students’ learning. High-stakes public examinations (e.g., IELTS, TOEFL, SSC, HSC examinations) are commonly attributed to influence language teaching and learning in the classroom and beyond (Barnes, 2016; Hayes & Read, 2004; Hoque, 2011; Sultana, 2019). This phenomenon (the effect of high-stakes test on teaching and learning) is popularly termed as “washback”.

Research gap

Although it is a widely held notion that washback (i.e., positive or negative effects of a test on teaching and learning) exists, we should not consider it an automatic effect of any test or examination (Spratt, 2005). Hence, studies suggest that each test or exam (especially high-stake ones) demands a tailor-made study on boarding multiple stakeholders (such as teachers and students) to detect its washback effect (Cheng, Andrews, & Yu, 2011; Fulcher & Davidson, 2007; Spratt, 2005). Besides, there is a lack of sufficient evidence to confirm its existence and especially the nature of its effect in the context of the current study (Nuby et al., 2019; Nur & Islam, 2018; Rahman et al., 2018; Sultana, 2019). Literature indicates that there are certain shortcomings in the field of language testing in Bangladesh with regard to washback: paucity of empirical studies and lack of students’ perspectives in the washback studies (Ali & Hamid, 2020; Nuby et al., 2019; Rahman et al., 2019, 2018; Sultana, 2019). Moreover, previous researchers of the current context suggested future researchers to evaluate the contents of the tests while studying washback (see Rahman et al., 2019; Sultana, 2018). Through this present washback study, we made an effort to contribute to these three areas.

Theoretical framework

Understandably, if classroom teaching refers to the instructional and pedagogical synchronization between students and teachers, which creates learning through interaction, testing and assessment is an influential tool that examines what the students are learning in the classroom (Tayeb, Aziz, & Ismail, 2014). On the other hand, since learning is acknowledged as the ultimate target of teaching, and testing as a dominant tool to measure the attainment of that target, the existence of tests’ effect on teaching and learning process cannot be denied (Saif, 1999). This effect of language testing on language teaching and learning (e.g., Brown, 2004; Cheng, 2005; Cheng & Curtis, 2004; Fulcher & Davidson, 2007; McNamara, 2000) is termed as washback (e.g., Alderson & Wall, 1993; Bailey, 1996) or backwash (e.g., Hughes, 1993). Furthermore, this effect can be observed in the classroom referring to micro-level of language teaching and learning (Cheng, Sun, & Ma, 2015; McNamara, 2000; Wall, 2005; Watanabe, 2004).

Washback of testing is commonly observed as either positive or negative. It has to do with fostering or hindering the achievement of curricular objectives. When teaching the intended curriculum becomes the same as teaching to a particular test, it is said to have positive washback effect of that particular test (Taylor, 2005). On the contrary, when there is incongruence between the curricular objectives and the focus of the test, which encourages teachers and learners to narrowing the curriculum in favor of the test format, it is said to have negative washback effect of that particular test (Brown, 2004).

According to Alderson and Wall (1993), washback compels teachers and students to do what “they would not necessarily otherwise do” (p. 117). Not only this, this effect of tests is not limited to classrooms only, rather goes beyond the language classroom influencing the school, the community, and the entire educational system. This effect is termed as test impact (Andrews, 2004; McNamara, 2000; Odo, 2012). This happens when tests have wider effects on language teaching and learning in a high-stakes situation (e.g., in our case, the SSC examination)—when the test is used to stratify students and schools or even the national education systems.

A high-stake testing indicates those tests the results of which affect the students directly since these results are the basis of making important decisions about the students, in particular, and the schools as a whole (Cheng, 2005; Dong, 2020; Qi, 2005). It creates pressure on the teachers to ensure that students score high in the test and consequently, encouraging some teachers to skip the contents and tasks that are not included in the examination and narrow the curriculum, and thus, teachers grow negative attitudes towards standardized high-stakes testing (Smith, 1991). High-stakes test typically leads teachers to teaching the test and teaching to the test. The former phenomenon encourages gathering ideas of what test questions will be and accordingly teaching the required answers to the learners. Such practice cannot encourage learning; rather, it is thoroughly dishonest and hence can be termed as cheating. On the other hand, the latter phenomenon encourages locating the concept that will be assessed and teaching these concepts to learners (Sukyadi & Mardiani, 2011).

Alderson and Wall (1993) developed 15 washback hypotheses according to what is influenced by a test: attitudes, teaching, learning, content, degree, sequence, rate, depth, and also the proportion of teachers or learners. Following Hughes’s trichotomy model (Hughes, 1993 cited in Bailey, 1996), any washback study can also be carried out by focusing on participants (learners, teachers, material and content developers, publishers), process (participants’ actions towards learning, for example, syllabus design, the development of materials, and teaching methods), and products (what is learned and the quality of learning). 3Ps, the abbreviation, coined by Saville (2009) denotes these 3 main washback factors (participants, processes, and products).

Studies on washback

Since the publication of the phenomenal work of Alderson and Wall (1993), a large number of washback studies were conducted around the world, although only a few washback studies (Al Amin & Greenwood, 2018; Ali & Hamid, 2020; Hoque, 2011, 2016; Maniruzzaman, 2012; Maniruzzaman & Hoque, 2010; Sultana, 2019) conducted in different education levels (SSC and HSC) were found in Bangladesh. These washback studies conducted in Bangladesh and elsewhere explored various aspects related to washback of high-stakes tests. For example, the study of Orafi and Borg (2009) explored that Libyan secondary English teachers struggled to execute their instructional practice as suggested by the curriculum and concluded that the reason behind this failure was the gap between the orientation of the curriculum and that of the examination system. On the contrary, in light of other studies (Chen, 2002; Hoque, 2011; Orafi & Borg, 2009), it can be said that negative washback of the examination also resulted from teachers’ lack of understanding and awareness of curricular goals. Chen (2002) explored that teachers’ lack of sufficient knowledge of and understanding about the objectives of the prescribed curriculum encouraged them to be test-oriented.

Amengual-Pizarro (2009) and Stecher, Chun, and Barron (2004) found that the teaching methods and approaches of the teachers were influenced by the test. These studies explored that teachers changed their teaching methods and approaches to suit the requirements of the test. The findings received from previous studies (Hoque, 2011; Onaiba, 2013; Qi, 2005; Rahman et al., 2019; Sultana, 2019) indicate that in the test-oriented classrooms, English teachers taught their students whatever they liked to teach, and English teachers’ preference and choice of chapters and topics got priority in the classroom teaching and learning. Other washback studies (e.g., Cheng, 2005; Dong, 2018, 2020; Maniruzzaman & Hoque, 2010; Qi, 2005) also explored a strong negative washback of the high-stakes tests on the teachers’ selection and teaching of contents. Teachers were found to skip and ignore some of the lessons of the textbook that were less likely to be set in the test (Abu-Alhija, 2007; Qi, 2005; Rind, Mari, & A., 2019). In the study of Watanabe (2000, p. 44), ignoring the curriculum, teachers used different kinds of self-made materials to suit the targeted test. Moreover, the high-stakes tests not only dictated English teaching and learning, but also took the form of curriculum itself, hidden curriculum (Booher-Jennings, 2008; Minarechová, 2012) for parts of the academic year. Similarly, in several other studies (e.g., Al Amin & Greenwood, 2018; Han, Dai, & Yang, 2004; Hayes & Read, 2004; Hoque, 2016; Rahman et al., 2019), both the teachers and students were found to depend highly on commercially produced notes and guidebooks.

The findings of a study conducted by Tsagari (2006) indicated that the feelings, perceptions, and attitudes of students, and their motivation towards language learning were influenced by the test. Xie (2015) studied students’ attitudes towards and perceptions of the two changes brought in College English Test (CET) and their effects on their time management, their test preparation approaches, and their test performance. The findings of the study display that students possessed positive attitudes towards the CET associated with more engagement in learning activities as well as test preparation. Cheng, Andrews, and Yu (2010) explored a direct connection between the perceptions of students towards test-related learning activities and those of their English levels.

Previous studies (e.g., Abu-Alhija, 2007; Cheng, 2005) pointed out that tests also affected teachers’ feelings, perceptions, and attitudes towards their behavior and classroom practice in a way that they enhance teachers’ stress and lower their morale, or else, inspire them to work tougher and embrace innovative techniques and methods to meet the intended objectives of the test. While investigating test effect on respondents’ perceptions and teaching material design, Tsagari (2009) found that teachers felt anxious, embarrassed, and stressed while they were trying to complete all of the materials prescribed in the syllabus. Moreover, teachers are predominantly pressured by the school authority, their students, and parents to customize their teaching methodologies to suit the test (Hoque, 2011; Onaiba, 2013). Consequently, teachers experience “a tension between pedagogical and ethical decisions (Spratt, 2005; p. 24)”.

Watanabe (2004) highlights the importance of context factors in mediating the process of washback. He divided these factors into two categories: micro-context factors (e.g., classroom or school setting) and macro-context factors (e.g., the society). Studies explored that pressure on teachers from external sources (e.g., parents, society) compelled them to bring about instructional differences, particularly when professional success of teachers was measured by students’ scores leading to either rewards or sanctions for the teachers (Abu-Alhija, 2007; Al Amin & Greenwood, 2018; Cheng et al., 2010).

Context of the present study

The study was conducted in a secondary school context in Bangladesh (one of the South Asian countries) with focus on English teaching-learning and testing. English, in Bangladesh, is taught as a mandatory subject from Grades 1 to 10 (at secondary level) and Grades 11–12 (at higher secondary level) (Ali & Hamid, 2020; Rahman, 2012). When students finish Grade 10, they appear at a school-completion national examination, namely, the SSC examination (Ali & Hamid, 2020). Their further education depends on the success in this examination (Al Amin & Greenwood, 2018). Along with other subjects, they have to sit for two mandatory English tests. These are the following: English 1st Paper and English 2nd Paper. They appear at these two papers on two separate days (Ali & Hamid, 2020). The present research studied washback with regard to these two tests. Table 2 demonstrates the tasks, and the allocation of marks for the English 1st Paper and English 2nd Paper tests.

Objective of the study

The objective of the study is to explore the washback effect of SSC English examinations on English teaching and learning practice at secondary schools in Bangladesh. It explores the three main factors (participants, processes, and products) of washback.

Methods

This empirical study was qualitative in its nature and was primarily based on primary data. The qualitative research methodology was adopted because it can better illustrate washback phenomena than quantitative methods (Watanabe, 2004). Since washback of test is unpredictable and contextual (Tsagari & Cheng, 2017), a comprehensive understating of the phenomenon can be gained through conducting research in multiple settings. Hence, data was collected from 12 secondary schools. To conduct the study, the schools were selected following the convenient sampling procedure considering easy access to the schools.

Participants

The participants of the study were English teachers and students of the sample schools. Twelve teachers of English were interviewed, having one from each school. Twelve FGDs were conducted in 12 schools. Each FGD comprised 9 students. Hence, a total of 108 (12×9) students participated in the study. The teachers and students were selected following the purposive sampling technique. This technique was applied for choosing individuals or groups deliberately not only based on our research questions, but also based on information available about these individuals or groups (Patton, 2015). The participating teachers and students were selected considering their potential for generating data which could reveal teachers’ and students’ perceptions in general.

The selected teachers were teaching English at class/grade 10 in the sample schools. Therefore, they could deliver the required information corresponding to the objectives of the study. All of them (except T1, T2, T5, T10) were examiners of English of the SSC examination. The following table reflects the teachers’ characteristics in terms of their gender, age, experience of teaching, educational qualification, trainings, and settings (Table 1).

Table 1 Profile of the teachers

On the other hand, the student participants for FGDs were also learning English at class/grade 10 (the SSC level). They were candidates for the upcoming SSC examination. While selecting the student participants, the researchers also followed Patton’s (2015) “maximum variation sampling” technique, i.e., in each of the focused groups, there was a combination of students (the front-benchers, the mediocre students, and the back-benchers), so that voices of all categories of students could be heard and elicited data could represent all categories of students. Students’ merit list was used to select these three merit levels.

Instruments and data collection procedure

Focus group discussion (FGD) agenda, interview schedule, and lesson observation protocol were used for collecting primary data. Besides these, relevant documents (such as the National English Curriculum, the textbook, commercially produced books, and past test papers of SSC English examination) also provided valuable insights into the study.

We used semi-structured interview schedule for collecting data from the English teachers, FGD agenda for guiding FGDs with students. Moreover, we developed a classroom observation protocol to observe English lessons. We (the first, second, and third authors of this paper) collected the data in two rounds with the two groups of participants spanning over a period of 3 months. In the first round, FGDs with the students were conducted while in the second round lesson observations and teacher interviews were conducted.

The objectives of the teacher interviews were to explore teachers’ perceptions of the alignment of national English curriculum, their teaching practice, and SSC English examination, and how these perceptions contributed to the washback of SSC English tests. Q1 to Q3 of teacher interview elicited academic and professional information of the teachers, and Q4 to Q10 focused on how they perceived the test washback of SSC English. Q11 to Q12 were the concluding questions which explored their overall impressions about the washback of the SSC English tests on English teaching and learning. On the other hand, the objective of conducting FGDs was to understand how students perceived washback in their learning of English, their test preparation, and language proficiency. Q1 and Q2 of FGD agenda were starters of the discussion, Q3 to Q6 were main agenda which focused on the objectives of the study, and Q7 was the concluding agenda to have a summary of the conversation.

Interviews and FGDs were audio recorded with prior permission from the participants. These were carried out in Bangla because teachers and students preferred to converse in Bangla. These were transcribed verbatim and then translated into English. In order to confirm the reliability of the data, these translations were verified by one of our colleagues who teach English at a university in Bangladesh.

One English lesson of each of the twelve teachers was observed. The objective of lesson observation was to explore how the inter-relations between curriculum, textbook, testing, teaching, and learning were reflected in the classroom setting. Moreover, classroom observation data validated the claims teachers made in interviews since what teachers informed or claimed might differ from what happened in the classroom (Alderson & Wall, 1993; Watanabe, 2004). Each English lesson was around 35 min. A lesson observation protocol was used to capture the activities and interactions happening in the classroom. While observing lessons, we took notes on the protocol. This field note observational protocol produced a written account of what we, the observers, saw, heard, and experienced while collecting the data (Bogdan & Biklen, 1998). The guiding principles for this field note observational protocol were as follows: (a) how objectives of the English curriculum were aligned with the objectives of the textbook and objectives of teachers’ classroom instructions; (b) how the test tasks were aligned with the activities of the English textbook and activities in the classroom; (c) how activities in the classroom were facilitating students’ language learning and their preparation for the SSC examination; and (d) what effects the SSC examination had on English teaching and learning in the classroom.

Data analysis

In order to analyze the qualitative data of a study, most of the researchers prefer to use a combination of approaches (Green & Thorogood, 2004). We followed the same path. A transcript-based analysis approach (for teacher interviews) and a tape-based analysis approach (for FGDs) (Onwuegbuzie, Dickinson, & Leech, 2009) were used to deal with the interview and FGD data. Each of the interviews and FGDs was transcribed word by word.

To analyze the lesson observation data, we utilized “template analysis” technique, which comprises the development of a coding template. We developed a coding template which summarized the main themes we pointed out and classified them in a useful and meaningful way. Hierarchical coding was emphasized, using broad themes, incorporating subsequently narrower and more specific ones.

At the very outset, the data generated from documents, interview and FGD transcripts, and lesson observations underwent the process of selecting, focusing, simplifying, abstracting, and transforming the data which is termed as “data reduction” or “data condensation” (Miles & Huberman, 1994); p. 10, i.e., the entire data went through the process of organization and reconfiguration. Thus, the data was organized in such a manner that is not only manageable but also comprehensible pertaining to the phenomena being discussed. At this phase, writing summaries of the relevant parts of the documents; transcribing each interviews and FGDs; coding interview, FGD, and lesson observation data; and developing themes were done. Document analysis offered valuable insights into the objectives of English language teaching and learning, English test format, test tasks, value of test results, and their compatibility to the objectives of the curriculum, since they were vital in understanding and interpreting washback of SSC English examination. Hence, the focus of the summaries of the relevant parts of the documents was on the aims and objectives of language teaching and learning, and the description of how SSC English tests had to be conducted as outlined by the national English curriculum.

The analysis of interview and FGD data commenced after the transcription and familiarization with the data from the first interview and FGD. As we went through interview, FGD, and lesson observation data, we write memos to note down the emergent themes.

For each of the transcripts, lesson observation data, and documentary data, individual coding was done manually applying a word processing program. When coding was complete, the evidence for each code was reviewed. Recoding was also done once any coding discrepancies were identified. Majority of the time, we agreed on the preliminary coding, and for the remaining cases, we reached an agreement through discussion based on the review of preliminary data. Other members of the team who took part in the collection of data then provided feedback on results. Thus, data were coded before these were being classified into themes, and eventually connections were sought across the themes. Hence, thematic analysis (Braun & Clarke, 2006) was applied to identify, analyze, organize, describe, and report themes found derived from the data set. Besides, following an inductive approach, the researchers allowed themes to derive from the data instead of having preconceived notions about washback of SSC English examination.

Validity of the findings

Several approaches were utilized to ensure the validity of the findings of our study. While analysing the data, we compared emerging themes constantly, by navigating back and forth between the data and the objectives of the study so that we could make sure that the findings of the study were rooted in the data (Miles & Huberman, 1994). Secondly, we applied both method triangulation and data triangulation (Denzin, 1978). Data triangulation was achieved by having different sets of data cross-checked. On the other hand, method triangulation was achieved by scrutinizing the data received from the interviews and FGDs against the relevant documents. Additionally, the trustworthiness and credibility of the findings was achieved by member checking (Lincoln & Guba, 1985). We shared the interview transcripts with the interviewees to validate the transcriptions. Nine of them confirmed that their views were duly replicated in the transcriptions. Rest of them, however, ask for minor corrections.

Major findings

The study found that washback, more specifically, significant negative washback of SSC English tests on English teaching-learning, exists. Negative washback was found on such facets as teaching-learning styles, teaching content, teaching methods and approaches, and classroom practices. These negative washback effects on English teaching-learning practice result from the stakes of the tests, respondents’ attitudes towards the test, lack of a congruence between the objectives of the curriculum and the format of the tests, teachers’ lack of comprehensive understanding about the curriculum, pressure from the guardians of the students to ensure good grades in the final/SSC examination, etc.

SSC English tests as high-stakes tests

The findings of the study reveal that both teachers and learners of the current study consider SSC English tests as high-stakes tests. Findings received from interviews and FGDs confirm that the scores of these tests are used to evaluate students and assess schools, while simultaneously used by parents and schools to evaluate teachers. These considerations negatively influence (i.e., negative washback) English teaching-learning practice at secondary schools by encouraging both teachers and learners to be test-oriented. During interviews, one of the teachers (T10) commented,

Students are literally spending almost half of the year learning SSC test-taking strategies. At schools in the classrooms, they are being taught the test-taking strategies through tutorials and then practicing mock tests. Day in and day out the SSC English test is in the vanguard of learning, spearheading all English language teaching and learning activities at school in grade IX and X. The pressure is paramount since the result of this public examination is the benchmark of sanction or reward for schools and teachers.

Incongruence between curricular objectives and test formats

The study has explored incongruence between curricular objectives and SSC English test formats, which also caused negative washback. All students in Bangladesh have to appear at two high-stake English tests (papers 1 and 2) for passing their secondary education level. An insight into these tests has been gained through an analysis of the official documents of the secondary level English language curriculum (including the format of the tests of the SSC examination) available at the website of the National Curriculum and Textbook Board (NCTB) of Bangladesh. It also contains the general theoretical framework of English language teaching and learning in the secondary schools of Bangladesh. The National Curriculum (2012), which was developed by the NCTB, states two main objectives of English teaching (at class 9–10). These are (i) helping learners gain communicative competence in all four language skills, i.e., listening, speaking, reading, writing, and (ii) helping them use the gained competence for effective communication in real life situations (NCTB, 2012).

But the SSC English tests can neither fulfil the curricular objectives, nor does it have construct validity. An analysis of the test formats and test items of the two high-stakes English tests (papers 1 and 2) shows that these compulsory tests that all the students have to take comprise only linguistic competence along with its 3 sub-competencies (textual competence, grammatical competence, and textual coherence), though the communicative competence must include both linguistic competence and pragmatic competence.

Table 2 shows that English tests are predominantly tests of grammatical competence and reading and writing skills. Out of the 27 items (shown in Table 2) of these tests, 13 test exclusively grammatical competence: knowledge of discrete vocabulary items or grammar structure. The other 14 items test textual competence or textual coherence and writing skills. In order to attempt the cloze tests, candidates need to know the right verb/word form or the correct preposition in a sentence. In some cases, they also need to understand relationships between adjacent clauses or sentences, asking them to choose the right conjunction or connector.

Table 2 English test item types (NCTB, 2012)

Other activities, like completing story and dialogue writing, which are potentially suitable for developing other competencies like speaking, cannot be exploited, and they may turn out to be rote memorization of written texts (or dialogues) because practicing dialogue has to be in the spoken form. This dialogue writing task is not authentic in that the test takers have to write a dialogue on a given topic. Dialogue is not something that can be meant for writing. This can never be a requirement for using the language in real life. The “text comprehension” items comprise simplified texts (short sentences with simplified structures and common vocabulary). These items consist of questions/items which test operational understanding (the general idea and the intention of the given text, inferences that can be gained from the given text). These also include discrete vocabulary items and paraphrasing of phrases.

Thus, the test items in the SSC English tests range from those that assess the stock of vocabulary and mastery of basic grammar to those which ask for general understanding of short and simplified texts, along with making inferences of specific passages and writing different test items. Following this format of the test to the point, teachers were found to teach. One of the interviewed teachers (T4) commented,

Because of the exam’s demands, and students’ expectations, I myself and many teachers, I know, prefer to ignore the sections that are less frequently used in the final exam, such as some items of writing, listening and speaking tasks, telling students these are not important. Meanwhile, we focus on other aspects such as grammar, vocabulary and reading texts.

The tests have considerable focus on forms. In some cases, answering a test item without fully comprehending the meaning or the context of the text is also possible. Although some of the test items require comprehension beyond sentence level, the analysis of the test items explores that the larger discourse is never longer than one paragraph of non-authentic language. Moreover, the test items do not test pragmatic competence of the students. Furthermore, there is no provision of testing two (speaking and listening) skills of language.

Teachers’ understanding and awareness of the curriculum

The teacher interview schedule started with a question which elicited teachers’ understanding about national English curriculum (for class 9 and 10) and its objectives. Of the 12 English teachers, three teachers informed that they did not read/see the English curriculum at all, while five teachers lack clear idea about it, although what they shared about it and its objectives are in line with what has been written in the curriculum (see National Curriculum-English Class IX-X, p.34-69). The following is one of the examples of their opinions:

I do not have clear idea about the present English Curriculum. But I know the question patterns and the way questions are set in the present CLT based syllabus. This communicative curriculum expects that our students will be able to develop reading, writing and speaking skills (T8).

The rest (four) of the English teachers, however, had a clear and comprehensive understanding about the curriculum and its objectives, and they supported it. One of them (T5) said,

I think the goals and objectives of the English Language curriculum for the students are to make them to learn English not only for passing the exams but also for using it in real life situation.

This lack of proper understanding and awareness of the objectives of curriculum is the consequence of negative washback of the examination. One of the reasons of such unawareness and negligence towards the curriculum, as reported by the interviewed teachers, is the mismatch between curricular objectives and test objectives, which correlates with our test analysis presented above.

Teachers’ lack of assessment literacy

Teachers are always at the forefront as designers, developers, and users of language assessments. Hence, they have to be familiar with and proficient in the effective use of language assessment. As far as the teachers’ understanding about the present assessment system is concerned, out of the twelve teachers, only three teachers have sufficient idea about it. The rest of them only follows the school syllabus and question formats of SSC English examinations. This study has explored that teachers’ lack of language assessment literacy accelerated negative washback in the teaching and learning of English and hindered students’ learning. Since they lack language assessment literacy, they simply follow the question formats of SSC English examinations. They were found to fail to utilize alternative and continuous assessment techniques to bring diversity in assessment. Hence, students’ learning remains confined to those areas only (i.e., linguistic competence, textual competence, and grammatical competence) which are covered by the question items/formats. One of the interviewed teachers (T10) commented: “I normally develop exam question papers matching the SSC final exams. I use them to assess my students in monthly tests.” During the FGDs, students also raised the same point that the test items of their class tests and terminal examination resembled previous years’ SSC exam papers.

Negative attitude towards the tests

The study found that teachers possessed overall negative attitude towards the SSC English tests, which encouraged them to teach to the test. Teachers’ attitudes towards and perceptions of tests and assessment process have the power to positively or negatively influence several aspects of teaching and learning. In the current study, although teachers had mixed reactions and attitudes, they were mostly negative. The findings of the study explore that the respondents’ perceptions towards the test are greatly incompatible with the intended or positive washback of the test. The study found that out of the twelve teachers, only two consider the present assessment process appropriate for evaluating the learning outcomes and achieving the objectives of the English curriculum assuming its implementation in the ideal form.

This is appropriate in the sense that in the terminal examination students are assessed in reading and writing; and in the school based formative assessment system, listening and speaking can be tested. I mean the combination of formative assessment and terminal examination is appropriate for evaluating the students’ learning outcome (T12).

Another teacher, however, had a mixed feeling about the current assessment system. For him, the current assessment system is more appropriate than the immediate past ones. He believed that because of the current assessment system his English classes were more effective than ever before. His students did not hesitate to communicate with him and could learn English with fun, which was favorable to achieving the learning outcomes. He, however, recommended including a compulsory monthly test in the classroom for the betterment of the students. Majority of them (nine out of twelve), on the contrary, believe the present assessment process is not appropriate for evaluating the learning outcomes and achieving the objectives of the English curriculum. They believed that there was a mismatch between current classroom practice of communicative language teaching-learning and assessment system. They opined that the existing assessment system focused on reading and writing only, ignoring the other two skills (i.e., listening and speaking). They believed that students learnt only for passing the examination, ignoring the real purpose of language learning. One of the teachers commented,

I think that present assessment system isn’t appropriate for evaluating students’ learning outcome of English language. In the current system, students learn only for passing examination, not for communicative language learning. It does not assess all skills. It gives emphasis on writing. There is a mismatch between current classroom practice of communicative language learning and assessment system (T7).

This is why, in the interview, some of them commented that students’ performance in the final examination, i.e., assessing them through marks can hardly reflect their actual development of different language skills. On the other hand, during FGDs, some of the students also conveyed the same massage. FGD data also explored that similar to their teachers, learners did not hold positive attitude towards the tests. During FGDs, some of the students commented that SSC English tests were not comprehensive enough to demonstrate their actual language proficiency level because these tests only measured their mastery of grammar, reading, and writing ability.

Hence, one of the teachers suggested that some marks could be allocated to test students’ speaking skill because according to his belief, his students did not practice speaking since speaking was not a part in the SSC English examinations and not included in the total marks of the said examinations. These findings indicate that learners learnt to the tests. Such phenomena demonstrate negative washback of SSC English tests on English teaching-learning practice at secondary schools.

From the study of the official documents of NCTB and the interviews with the teachers, it is understood that although in the test format of high-stakes English test of SSC examination there is no mark allocation and provision of testing speaking and listening competency of examinees, students at school are supposed to practice and take speaking and listening competency test. The data from teachers’ interviews and national English curriculum confirm that 20 marks (out of 100 in English 1st Paper test) are allocated for testing students’ competency in speaking (10 marks) and listening (10 marks) (Table 3). National English curriculum shows the skill-wise distribution of 100 marks in English 1st Paper test at school in the following way.

Table 3 Distribution of marks for English 1st Paper (NCTB, 2012)

The curriculum mentions that listening and speaking skills of the students will be assessed through continuous assessment, and these (10+10 = 20) marks will be added to the marks of final examination to ascertain their final grade (NCTB, 2012). Teachers, however, informed that since listening and speaking skills were not assessed in the SSC examination, and there was no allocation of marks for these, they neither engaged their students in practicing these skills in the classroom nor arranged any test. Rather, as parts of continuous assessment, model/mock tests replicating the SSC English examination formats were arranged. Here again, an evidence of strong negative washback is spotted, where teachers teach to the test.

Bearing the above perceptions of the current English curriculum and assessment system, different teachers considered different issues during the presentation of subject contents in the classroom. Data received from teacher interviews and classroom observations reveal these. Most of them informed that they tried to identify whether students understand the lesson or not. One of them (T4) said, “I identify the weak and silent students who do not talk in the classroom. I try to involve those weak students in a group”. Another of them told that he put emphasis on how much benefit could be gained from the lesson and what students learnt from the lesson. Negative washback, however, reaches its highest limit when a teacher (T12) had to utter:

Our government stresses on increasing the pass rate. For that reason, I keep in mind that the poor students can pass the exam and advanced students can speak in English. So, I apply various techniques. For example, I give them letter writing task following the education board questions (i.e. the format of previous SSC examination papers).

Teachers’ advice and its influence on students

Besides, this study’s triangulated findings from teacher interviews, FGDs, and classroom observations also identified teachers’ advice as a facilitating factor in creating washback on their students’ language learning. Moreover, such advice is more often pertinent to learners’ preparation for the tests than their development of language skill, and hence, creates negative washback. During FGDs, students informed that their teachers’ preference and choice of chapters and topics got priority in the classroom teaching and learning. They also informed that their English teachers hardly involved them to practice learning and speaking English, instead they were taught the strategies to answer to the English questions to ensure high scores and good grades in the final examination. The interviewed teachers largely asserted that they were under pressure from the head teachers, the students, and the parents of the students to teach to the test so that students could obtain good grades in the final examination. The study, however, also explored that not only the teachers faced pressure and stress, students also experienced the same. During FGDs, students informed that they suffered from stress and anxiety because of the SSC English tests. During the interviews, one of the teachers commented,

My students are usually worried about how to pass the SSC examination. Consequently, I become worried. I also feel a lot of pressure on me. Therefore, I usually engage my students on drills of past SSC examination papers to familiarize them with the SSC test format. For example, once I am done with a chapter, I come up with some previous years’ SSC examination questions. This was very useful and effective because I observed my students could perform well in the exam, and the school authority and parents and guardians were pleased with my students’ result (T4).

Hence, teachers taught their students the examination taking strategies to prepare themselves for the final/SSC examination. Accordingly, our study explored that teachers’ professional practice, and instruction for the SSC English tests formed the basis for learners’ learning style and preparation for the tests.

Thus, this study reports how teachers and students, the participants (one of the three main factors [participants, processes, and products] of Hughes’s trichotomy model of washback [1993] and the 3Ps of Saville (2009)), were negatively influenced by the SSC English tests.

Washback processes

Processes, according to Hughes (1993) in Bailey, 1996), refers to syllabus design, development of materials, changes in teaching-learning methodology, and strategies of assessment and testing among others. Just as the teachers stated in the interviews, the syllabus of the course was question format/item driven. So, they did not teach every chapter or usually chose to skip some of the activities and lessons of the textbook that were not or less expected to be set in the SSC examination. During the lesson observations, it was found that apart from reading exercises, most lessons dealt with discrete points, like adjectives, right form of words, vocabulary, and guided writing. More extensive writing tasks such as describing graphs/charts, composition writing, and completing stories were not dealt with in the classrooms, rather were assigned as homework before the topic was actually developed. The analysis of the classroom observation data showed that most of the activities conducted during the time of observations were directed towards the development of reading skill and grammatical competence (gap filling exercises, grammar exercises, and pronunciation practice). Beyond these were cloze exercises and guided writing tasks. Other activities, such as oral presentations and dialogue dramatizations, which are potentially suitable for developing other competencies, were not exploited. In other cases, the presentations consisted in reading aloud texts that contained several mistakes. There were evidences of the other students’ not following or understanding the presentations or dialogues. During one of the lesson observation periods, the observer (1st author) himself found it difficult to understand them. The students were most of the time practicing rules in artificial exercises that focused on form, with predictable responses. Feedback on dialogues was very little, usually very general.

During FGDs with students, it was revealed that teachers mostly concentrated on test-oriented activities and ignored tasks and activities which were directly related to passing the SSC examination. Positive attitude towards tests encourages learners to spend more time on learning than on preparing for the test, but similar to their teachers, students also did not bear favorable perception and attitude towards the tests.

Interview and FGD data explored that both the teachers and learners considered question papers of previous years’ SSC English tests important for the preparation of the upcoming SSC English test. Most of the interviewed teachers and students were of the opinion that practicing these question papers of previous years’ SSC English tests helped students got familiarized with the format and contents of the said tests and perform better in the examination.

Document analysis revealed that the publishers, who produced the commercial exam materials, collected all the English question papers mostly from the famous schools of the country just after the test examination (mock examination held at all schools of the country that is held 2 or 3 months before the actual SSC examination and that resembles the SSC examination) was over and the question papers of several past years’ SSC English test. Then, they published them in a book commonly named Test Papers. This book was found to be very popular with both the teachers and the students.

Washback product

In a washback study, the products commonly refer to what is learned. In this case, the focus is supposed to be the development of communicative competence. However, the answer to what is learned can be derived from the activities done by students and how they are assessed in the classroom. Triangulated data from the three data instruments give us the following pictures about the activities in which students were engaged in the classrooms and in-class assessment done by the teachers.

As we can see from the analysis (Table 4), students seem to be working exclusively towards the development of linguistic and grammatical competence rather than communicative competence. They learned two skills: reading skill and writing skill as they are the ones set in the SSC examination papers.

Table 4 Classroom activities and assessment practice

Discussion

In line with the findings and opinions of local researchers (Roshid, 2009; Sultana, 2018, 2019), the external SSC English tests were found to be considered high-stakes tests in our study. When external tests receive growing importance in measuring performances of students, teachers, and schools (Cheng, 2005; Dong, 2020; Qi, 2005), and teachers are under the pressure of parents’, students’, and school’s expectation (Loumbourdi, 2016; Papakammenou, 2016), a risk remains that teachers, rather than contemplating on what they are teaching and being entirely mindful of the extent to which their professional practice (i.e., teaching) is impacted by external tests, begin to teach to the test (Ali & Hamid, 2020) because they believe they are obliged to ensure good grades for the students and reputation of the academic institutions (Hardy, 2015; Vlachos, 2019). Based on the findings of the current study, it may be the case with reading, writing, and grammar and vocabulary subtests. This case might be understood as positive washback if we only look at the findings from the point of view of reading and writing skills since these skills received more time, importance, and instruction. But, we cannot do this because learning a language is not confined to these two skills only. Such focus on the part of both teachers and students rather produce counterproductive or negative washback effects on English teaching and learning.

This type of examination misaligned with the objectives of the curriculum generating potential negative washback. Green (2007) demonstrated that test design is intensely associated with positive or negative washback. He advocated that in order to exert positive washback, the design of the test should be closer to the focal construct. The more a test embodies target skills, the more positive would be washback (Johnson & Shaw, 2018). When the objectives of any language test do not reciprocate the curricular objectives, it creates negative washback on language teaching and learning (Andrews, 2004; Brown, 2004). The document analysis in our study disclosed misalignment between the test format of the SSC English tests and the objectives of the curriculum, which was also shown by Sultana (2018). The SSC English tests can neither fulfil the curricular objectives, nor does it have construct validity since it fails to measure what it is supposed to measure (Das, Shaheen, Shrestha, Rahman, & Khan, 2014; Sultana, 2018) as it has been found. The test items of the SSC English tests fail to assess practical competence of the learners. Moreover, negative washback is also observed resulting from an absence of assessing speaking and listening skills (Sultana, 2018) without which learners will be unable to use the language in everyday life where speaking and listening skills are more required that reading and writing skills (Al Amin & Greenwood, 2018; Hoque, 2011, 2016; Rahman et al., 2019; Sultana, 2019).

On the other hand, teachers are among the factors that can mediate the washback of a test. Their classroom practice; perceptions about the curriculum and examination system; their nature, purposes, relevance, and appropriacy in the context, etc. play significant roles in creating positive or negative washback effect (Spratt, 2005). These aspects were explored during the interviews with the English teachers. The study found teachers’ lack of proper understanding and awareness of the objectives of curriculum augmented negative washback of the examination, which is also claimed by a number of previous washback studies. This finding of the current study confirms the findings of a similar study by Hoque (2011). On the other hand, in her study, Chen (2002) found that teachers preferred to teach to the test when they lacked sufficient idea and knowledge of curriculum goals. The mismatch between curricular objectives and test objectives was reported to be behind this lack. The study by Orafi and Borg (2009) explored similar findings, i.e., mismatch between curricular objectives and test objectives created unawareness and negligence of teachers towards the curriculum.

Moreover, our study found that teachers lack literacy and training in language assessment, which is considered “professional suicide” in the academic arena (Popham, 2004) ; p. 82. Because of this lack of assessment literacy, teachers failed to understand the philosophy of the curriculum and language teaching, and students’ learning remains confined to those test items only. Their failure to utilize assessment effectively encouraged students indirectly to ignore speaking and listening skills. Besides, exclusion of test items and marks for these skills in the said examinations discouraged them in this regard. These findings indicate that learners learn to the tests and support the view that if certain contents are not included in the test format, these get ignored or not underscored (Dong, 2018, 2020; Qi, 2005). Thus, negative washback of tests was generated, a finding that supports a previous study conducted by Sultana (2019).

Furthermore, in this study, teachers’ negative attitudes towards the tests also impacted several facets of language teaching and learning, such as their preparation for the class, administration practices, test quality, the meaning in test scores, and use of test information and scores (Urdan & Paris, 1994). Teachers’ attitudes to the SSC English tests, however, appeared to be pretty different from other similar studies. Amengual-Pizarro (2009), Wall (2005), and Watanabe (2000), for example, emphasized that the participant teachers had mixed yet positive attitudes to the introduced test. In the present study, however, teachers had mostly negative attitudes. Students expressed the same reaction to the SSC tests. They did not show positive attitude to the tests. Resultantly, teachers as well as students confirmed that the scores of SSC English examination could not be considered an appropriate indicator of English language proficiency of the students. Thus, this finding matches with the finding of a doctoral study conducted by Onaiba (2013) who concluded that “students may gain high marks in the exam but their real levels of language lag behind, and they are unable to express themselves in real-life situations, or even produce any type of writing (2013, p. 265).” Such experience-based opinions of the interviewed teachers resonate the findings of previous washback studies conducted by Maniruzzaman (2012), Xie (2013), and Dong (2018, 2020) who found that spending more time in test preparation failed to achieve curricular objectives and learning outcomes.

One of the most important findings of the study was that teachers taught to the test, which was explored in several pioneer studies (Alderson & Wall, 1993; Cheng, 2005; Qi, 2005). In line with previous literature (Allen, 2016; Sato, 2019; Shih, 2007; Xie & Andrews, 2012; Zhan & Andrews, 2014), this study also explored that teachers’ advice has significant role in producing washback on learners’ learning. Such advice is in the form of guidelines for the students demonstrating the test-taking strategies. Teachers narrowed the contents of the syllabus and skipped chapters considered less important for the tests. These findings correlate with the findings of the studies conducted by Allen (2016), Qi (2005), Rind et al. (2019), and Sato (2019), Shih (2007), and Xie and Andrews (2012). Moreover, the washback of these tests on the teaching methods of the teachers is negative as the teachers were found to employ whatever methods they deem perfect for assisting their students to prepare for the tests and score high grades (Alderson & Wall, 1993; Sultana, 2018). Furthermore, similar to the findings of the study by Zhan and Wan (2014), our study explored that teachers’ professional practice and instruction for the SSC English tests formed the basis for learners’ learning style and preparation for the tests. These test preparation and test-focused learning enhanced candidates’ test scores, the findings that match with previous studies though conducted in foreign contexts (e.g., Dong, 2020; Farnsworth, 2013; Holcombe, Jennings, & Koretz, 2013). For test preparation and test-focused teaching-learning, both teachers and students depended heavily on commercially produced exam materials, such as guide books, notebooks, and model/mock test books. The findings also have similarities with the study of Han et al. (2004) in China. Their study also found that for their preparation of CET, the candidates are seriously reliant on test-related materials produced commercially. Besides, these findings also correspond with the findings of the studies conducted in the similar context by Al Amin and Greenwood (2018), Green (2007), Hoque (2011), Rahman et al. (2019), Shih (2007), and Zhan and Andrews (2014). They also found that the candidates depended highly on commercially produced test-focused notes and guidebooks. Thus, these phenomena clearly lead us to conclude that the SSC English tests have negative washback on English teaching, learning, and assessment practice at secondary schools of Bangladesh.

Conclusion and recommendations

Obtaining a good grade in the SSC examination is considered the keystone of the students’ academic future in Bangladesh. Consequently, it has a substantial impact on the students, the schools, and the society as a whole. Despite spending a substantial amount of money and making several efforts by parents, educational institutes, and education ministry to assist students to cope up with the challenges of the SSC examination, its washback effect is unavoidable because of high-stakes it embodies for the stakeholders (especially the learners and teachers). A test can be reliable and a good one, but it can hardly influence students’ learning and develop their competency in the effective and meaningful terms if the constructs of the test reflect the trail of real things only which is the case with the SSC English tests.

The current study explores clear indications of negative washback effects of the SSC English examination on the elements and aspects of English language teaching and learning process in the secondary schools of Bangladesh, in particular, its effect on what (the contents) and how (the methods) the English teachers teach, and its effect on what and how (means, ways, and styles) the students learn. English teachers devote majority of their class time to coaching their students for the final examination, rather than teaching a NCTB English curriculum that could assist their students to develop their language competence. In these circumstances, it is the policy makers who can play a decisive role in determining whether to continue English teachers and learners to the high-stakes public examination or to encourage them to concentrate on real language teaching and learning.

The study carries certain implications for the decision makers; designers of curricula, assessment, and testing; teachers; and teacher trainers involved in the Bangladeshi educational system and elsewhere. The findings of the study lead to some suggestions on how the SSC English test can be improved. For example, massive revamping of the content, format, and test-items of the SSC English test is required. It should include test items covering all four language skills (namely, listening, speaking, reading, and writing skills) equally, avoid repetition of reading texts and other test-items including writing topics, etc.

The study, however, also explored that the SSC English test was not the sole reason behind the negative washback. Teacher characteristics, student characteristics, and contextual factors also play a significant role in this regard. It has found that washback of the SSC English tests is a complex phenomenon which include multiple stakeholders. Hence, ensuring effective and positive interplays among the components and elements of the current English education system will not be a simple and straight forward task. Required reform should have its beginning from the decision-making level ensuring inclusion and assimilation of the voices of all the stakeholders and available and required resources and swim through intervening level and finally end fruitfully with the implementing level. It is hoped that the findings and recommendations of the study will help the concerned authority to work effectively for future reforms.

Limitations and future research

Although the current study offers some significant insights into the washback of SSC English examinations, it has certain limitations. Since it is a qualitative study, the sample size of the study is limited. Hence, it is hard to term these samples as representative of all the schools of Bangladesh. However, the homogeneity and centralization of the school system in Bangladesh (Hossain & Tavakoli, 2008) specify that even if the findings are specific to the particular research contexts, other schools are undeniably expected to be facing the same phenomena.

Notwithstanding its limitations, this study is one of very few studies that have reported empirical evidence of washback of SSC English examinations on English teaching and learning practice at secondary schools in Bangladesh. The findings of this study indicate that the washback of SSC English examinations is analogous in its complexity to the washback of high-stakes tests explored in previous large-scale studies conducted outside Bangladesh and hence, creates an extent of research study worthy of further investigation. Further large-scale studies employing multi methods research design with a greater number of respondents including parents, and administrators are required to be conducted in the secondary schools of Bangladesh so that a more insightful understanding of the washback of SSC English examinations can be gained.