Designing and validating an assessment agency questionnaire for EFL teachers: an ecological perspective

Abstract

Teacher agency is a pivotal element of professionalism and second/foreign language (L2) education. However, its role in L2 assessment has remained under-researched. Part of this negligence is due to the absence of a validated questionnaire to measure the construct and its underlying components. To address this gap, drawing on the ecological perspective, the present study developed a scale on teacher ecological assessment agency (TEAA) by gleaning data from 539 Iranian EFL teachers. The results of exploratory and confirmatory factor analysis (EFA, CFA) culminated in 5 components and 32 items in the scale. The components were labelled “Iterational”, “Projective”, “Teachers’ Practical-Evaluative View”, “Schools’ Practical-Evaluative Effect”, and “Professional/Democratic Community’s Practical-Evaluative Effect”. Furthermore, the convergent validity and reliability of the scale were statistically approved (p > .05). The study presents some implications for EFL teachers, teacher trainers, and assessment policy-makers in that they can understand and highlight the pivotal role of teacher agency in L2 assessment practices.

Introduction

Teacher agency is a crucial element of teaching and teacher education given its strong influence on instructional quality, decision-making, and learning outcomes (Ahmad & Shah, 2022; Ashton, 2022; Kusters et al., 2023). In the teaching profession, including second language (L2) education, teachers should act out agency in their professional practices to overcome challenges induced by education reforms and changes in pedagogical trends (Gao et al., 2018; Sari, 2021). Moreover, agentic practices are posited to shape L2 teachers’ career conditions and professionalism (Biesta et al., 2015; Meihami, 2023; Polatcan et al., 2023). In simple terms, teacher agency pertains to how teachers respond to changes in their educational setting through their personal capacities and the environment (Pedaste & Leijen, 2020; Wang & Zhang, 2021). The term agency has been interpreted differently in previous years, resulting in multiple theorizations, conceptualizations, and confusions as a consequence (Farmasari, 2021). Previous studies in agency have taken different stances. In their seminal work, Emirbayer and Mische (1998) introduced a three-dimensional agency and defined it as a social engagement process, informed by the past (the iterational dimension), oriented toward the future (the practical-evaluative dimension), and performed in the present (the projective dimension). These dimensions contribute to agency achievement in different ways while interacting with each other (Wang et al., 2021). Biesta and Tedder (2007) considered agency not as something innate, but something that can be achieved. They called for more attention on the context. Drawing on Biesta and Tedder (2007), Priestley et al. (2015) took an ecological approach to agency. They argued that agency does not necessarily result from personal attributes or variability in social action; instead, it is contingent upon teachers’ engagement in their context (McNeil, 2018). This view of agency verifies teachers’ potential to bring about change in their professional environment and considers agency achievement as depending on how teachers strategically pursue their goals through their context (Farmasari, 2021).

This ecological model of agency considers teacher agency as an emergent construct that is time-oriented and highly context-sensitive (Ghamoushi et al., 2022; Sahragard & Rasti, 2017). Moreover, educational systems and structures could influence teachers’ agentic practices including those related to assessment (e.g., Luxia, 2007). In EFL contexts, teachers usually have passive agency and lack of freedom in making educational decisions, especially those concerning assessment plans and practices, being obliged to follow particular educational policies (Rezagah, 2022). This passiveness hampers teachers’ performance, knowledge, and practices in the class (Vu, 2020). Therefore, teachers should have a voice in their teaching practice and employ their contexts in taking actions and making decisions (Ghamoushi et al., 2022). Given its pivotal role in L2 education, teacher agency has been a subject of concern in some recent studies, which considered it as a construct in tandem with teachers’ professional development and educational quality (Lai et al., 2016), teacher reflection (Reichenberg, 2022), teacher identity (Connolly et al., 2018; Nguyen & Ngo, 2023), curriculum reform (Jiang et al., 2022; Poulton, 2020), and writing skills (Jang, 2022).

Although researching teacher agency is maturing, most of the existing studies are narrowed to how it is enacted across contexts and forms of education (Brod et al., 2023; Guerrero & Camargo-Abello, 2023; Nguyen & Bui, 2016; Paloniemi et al., 2023). It appears that the current body of knowledge regarding this construct is inadequate in EFL/ESL contexts (Vu, 2020). Furthermore, teacher agency has seemingly won much more research attention in Western countries, where individuality and activism are valued (Sahragard & Rasti, 2017). Nevertheless, in Asian countries, how English language teachers sense and practice the construct of agency has received scant scholarship (Nguyen & Bui, 2016; Vu, 2020). Another gap concerns the dearth of research on teacher agency in relation to L2 assessment (also called assessment agency). Only few studies have focused on this issue, which again examined teachers’ views and practices in light of washback effect (Ali & Hamid, 2023) and educational changes that embraced assessment reform (Rezagah, 2022; Willis et al., 2019). One of the reasons behind L2 researchers’ negligence of the concept of teacher assessment agency (TAA) might be the absence of a validated scale to measure it. Moreover, their inadequate knowledge of the ecological perspective of teacher agency, especially in relation to assessment, might have thwarted researching this fascinating construct. To address these research gaps, this study aimed at developing and validating a questionnaire for assessing EFL teachers’ assessment agency through the ecological lens. The study could assist EFL/ESL researchers and practitioners by providing them with an instrument that possesses psychometric properties to measure teachers’ ecological assessment agency as an overlooked domain.

Literature review

Teacher agency: definitions, approaches, and dimensions

The concept of teacher agency has be given numerous definitions in the literature (Kusters et al., 2023). Although reaching an agreement has been tough for scholars, they generally acknowledge that agency is a “socio-culturally mediated capacity to act” (Ahearn, 2001, p. 112), which is an indispensable part of teachers’ professional development that empowers them to take action and make changes (Ghamoushi et al., 2022; Leijen et al., 2021). In simple terms, agency pertains to one’s capacity to critically respond to problematic circumstances (Biesta & Tedder, 2006). Teacher agency is a concept referring to teachers’ ability to make and conduct pedagogical changes and apply their discreetness and judgment in aligning their practice and curriculum with their students’ needs (Hemi et al., 2021; Priestley et al., 2015). The construct is not separated from one’s broader sociopolitical and ideological context, and this feature has made it a complicated and situated practice (Ali & Hamid, 2023). Agency echoes teachers’ ability to act decisively and productively to guide their own as well as others’ professional development (Calvert, 2016). The commonality among various definitions of agency is that teachers’ agentic practices incur changes in their work environment (Kusters et al., 2023). It can contribute to educational policies and practices in different forms of education by giving teachers a genuine voice (Guerrero & Camargo-Abello, 2023; Polatcan et al., 2023).

Behind such a multiplicity of definitions, there exist three general approaches to agency in the literature (Goller & Paloniemi, 2017). In the first approach, agency is regarded as an individual capacity (Bandura, 2001). This perspective perceives agency as an individual’s ability to exercise control over life quality and the nature. In the second approach, agency is intertwined with action. That is to say, agency is practiced when teachers make decisions and take stances that affect their professional identity and teaching practice (Lai et al., 2016). However, these variegated conceptions of teacher agency capitalized on teachers’ capacities to make sound decisions, take responsible actions, and be innovative (Orland-Barak, 2017) are too individualistic and negligent of the role of context and culture (Biesta & Tedder, 2007). In a sharp contrast to previous approaches that regarded teacher agency as a personal trait, the ecological approach (Priestley et al., 2015) perceived teacher agency as an ecologically emerging phenomenon, which is influenced by socio-cultural and contextual particularities (Karimpour et al., 2022; Naraian & Schlessinger, 2018; Zhang & Wright, 2017). From this theoretical lens, teacher agency is no longer an individual characteristic, rather something achievable through the interaction of personal capacity, resources, affordances, and constraints in the socio-cultural context (Ashton, 2022; Priestley et al., 2015). In light of this conceptualization, it can be extrapolated that agency is attained through teachers’ everyday practice and decisions, which are influenced by the present context, their past experiences, and future goals (Chaaban et al., 2021; Ghamoushi et al., 2022). A missing part in all the three approaches to teacher agency is the role of teacher psychology, free will, and emotional factors. The first approach was too individualistic, the second one was action-oriented without considering context/culture, and the third one overestimated the impact of ecological systems on establishing agency without approving the fact that people can still have agency even in the absence of strong and positive ecological systems.

After the crystallization of the definitions and theoretical underpinnings of teacher agency, researchers turned their attention to the dimensionalization of agency, especially teachers’ ecological agency. In a seminal study, for instance, Priestley et al. (2015) perceived teacher agency as a time-contingent and situated achievement, resulting from the interplay among the three pivotal dimensions of ecological teacher agency (Fig. 1).

Fig. 1

Ecological Model of Teacher Agency (Priestley et al., 2015)

The iterational dimension portrays teachers’ past experiences, accumulated knowledge and skills, their capacity, beliefs, and values (Muhonen et al., 2021). Professional engagements, like their classroom teaching practice, interactions with colleagues, and developing an innovative and questioning mindset could form teacher agency (Priestley et al., 2015). The projective dimension concerns teachers’ professional ambitions and goals, which are rooted in the iterational dimension that allows taking actions based on the actor’s possible future path. The practical-evaluative dimension distinguishes between three contextual elements (i.e., cultural, structural and material aspects), that provide the conditions and affordances to reach teacher agency. Cultural resources relate to teachers’ thinking and understanding of the conditions, as well as their own thinking or dialogue with oneself and their interactions or dialogue with others in a specific context. In addition, structural resources concern teachers’ relationships with others that facilitate or inhibit agency. Finally, the material resources include the available resources and physical environment (Priestley et al., 2015).

In summation, the ecological approach to teacher agency, as the most comprehensive approach, could contributed to a rigorous understanding of the implications of the policies in a particular context and how such policies could influence teacher actions and decision-making. Therefore, examining the concept of teacher agency in light of this approach could potentially offer a means to apprehend or even reverse the past misleading regulation of teachers’ work (Priestley et al., 2015). However, most of the current studies using this perspective have focused on the pedagogical side of L2 education, while ecological agency, as explained below, may play a part in assessment, as well.

Teacher assessment agency

It is a fallacy to assume that teacher agency is exclusive to teachers’ pedagogical decision, behaviors, and practices without having a role in assessment. In many contexts, decisions about classroom assessment, academic goals, and whether the teacher needs to focus only on test-taking strategies instead of pure language learning form the core of EFL/ESL teachers’ assessment agency (Liyanage et al., 2014; Estaji & Ghiasvand, 2019; Willis et al., 2019). Similarly, decisions regarding the mixing of formative and summative assessment in L2 education allow teachers to perceive and work on performance and progress-based language learning tests/tasks (Wang, 2007). Nevertheless, in some cases, some teachers may vigorously resist assessment reform and enact only those parts, which are easy and compatible with their assessment beliefs and stick to their present assessment practices (Willis & Klenowski, 2018). Such a demand for assessment change/reform empowers teachers, as agents, to turn assessment policies into actions and align mandated assessments with the curriculum and classroom practices (Poulton, 2020). Likewise, the way teachers perceive and cope with the washback effect of language tests represents their agency (Ali & Hamid, 2023). In particular, in exam-oriented communities, where the teachers are grappling with the demands of high-stakes tests, which underscore teaching and testing discrete skills rather than communicative uses of language, teachers possess less agency (Ali & Hamid, 2023; Liyanage et al., 2014; Estaji & Ghiasvand, 2019, 2021). The consequences of students’ test scores have constrained teachers’ agency regarding assessment decisions and practices (Fang & Warschauer, 2004).

However, with the introduction and rise of alternative assessments, EFL teachers could manifest more agency during assessment practices (Poulton, 2020; Verberg et al., 2016; Wang et al., 2023). For example, in formative assessment practices, teachers are vastly involved in the assessment process and the interchange of ideas with students (Verberg et al., 2013). Moreover, in negotiated assessment, teachers share the assessment responsibility and control with their learners by negotiating the pre-figured objectives of the assessment (Gosling, 2000). This shared practice shows teachers’ agency and equal power relations with learners, too. Finally, given the importance of students’ ultimate learning even during assessment, a great attention has been paid to Learning-oriented Assessment (LOA) practices to inform teachers and students of the learning process, challenges, and progression constantly (Banitalebi & Ghiasvnad, 2023; Derakhshan & Ghiasvand, 2022; Willis et al., 2019). This assessment technique requires teachers to exercise agency in choosing their actions, enact them, and promote students’ learning and agency, too (Willis et al., 2019). Despite some progress in the area of assessment agency, the literature is mainly confined to the presence or absence of teacher agency in relation to assessment reforms and high-stakes tests. Yet, the ecology of assessment agency has remained under-explored in EFL settings.

Related studies

Owing to the significant and facilitative role of teacher agency in many areas of education, recent studies have focused on the conceptualizations, representations, and influences of this construct on general education, early childhood education, special education, and online education. For example, in a correlational study in the eastern, central, and western Anatolia, Polatcan et al. (2023) investigated the mediating role of teacher self-efficacy in 349 school teachers’ leadership and agency. The results showed a positive correlation among the variables. Additionally, in Colombia, Guerrero and Camargo-Abello (2023) took an ethnographic perspective to examine the implementation of agency by five teachers from five schools. The results showed that early childhood teachers negotiate and recreate agentic practices in relation to school peculiarities and children’s needs. Moreover, in the context of special education, Paloniemi et al. (2023) reported a positive relationship between 238 Finnish teachers’ agency and collaboration with students with special needs. Like other areas, with the spread of the COVID-19 pandemic, teacher agency research gained scholarly attention, as well (Brod et al., 2023). In this regard, Teruya (2023) explored the role of teacher agency in teachers’ pedagogy during the pandemic. Using an autoethnographic practitioner inquiry, the researcher argued that the divergent pedagogical discourse of online education modifies teachers’ agentic roles, responsibilities, and identities.

The multifarious contributions of teachers’ agentic practices to their professionalism, identity construction, and competency to change curriculum have also caught the attention of educational researchers. They shifted their attention from students’ agency to teachers’ agency to allow teachers to deploy repertoires, make compelling decisions, and frame their future course of actions (Priestley et al. 2015). Previous research endorses the vital role of agency in the transformation of teachers’ teaching practice, leadership, self-efficacy, and overall educational development (Lai et al., 2016; Polatcan et al., 2023). Moreover, teacher agency has been claimed to be an imperative factor in teacher professionalism and sustainable educational change (Kusters et al., 2023). In an empirical study in China, Lai et al. (2016) demonstrated the impact of teachers’ agency on their professional learning. They found multiple factors including power relations, social suggestions, imposed identity, teachers’ professional status, and their social roles and positioning efficacious in developing teacher agency. In addition, Imants and Van der Wal (2019) developed a model of teacher agency in professional development and school reform in the Netherlands. The model proved as a promising tool to study multilevel complexity considering theoretical insights, and research results of school reform and professional development in light of agency. Moreover, Durrant (2019) considered teacher agency as a cardinal aspect of professional development and school improvement. It aimed to provide practical ideas and strategies to be used to inform and evaluate practice and policy.

With regard to the relationship between teacher agency and identity, Connolly et al. (2018) used a socio-cultural approach to inspect beginning teachers’ self-views of their professional identity and found that beginning teachers’ incorporation of competing professional identities could enhance their sense of professional agency. Furthermore, Karimpour et al. (2022) reported the experiences that hinder language teachers’ agency, using a narrative inquiry methodology. They found how agency could negatively influence language teacher’s identity construction when their subjectivity conflicts with power relations, constraining teachers’ agentive action. In addition, some recent studies took a macro-level perspective and argued that investment in teacher agency exponentially could lead to teachers’ curriculum change competency in academic contexts (e.g., Jenkins, 2020; Poulton, 2020).

Another strand of research has concentrated on the triggering effect of reflection and reflective practices on teachers’ agency. In this regard, Kramer (2018) argued that reflectivity leads to transformative agency, while Reichenberg (2022) found reflection conducive to teachers’ agency in literacy coaching contexts. In sum, the current literature on teacher agency is dominated by studies form educational systems and modes other than L2 education, which has unique features meriting due scholarly attentions. Furthermore, studies on agency in language education has been mostly examined in relation to students (Ashton, 2022) and the exploration of teachers’ agency has sporadically emerged in this field (Ashton, 2021; Canagarajah, 2007; Kayi-Aydar, 2019). Likewise, there is a dearth of research on teachers’ agency in assessment contexts (Ali & Hamid, 2023). To shed light on this domain, Willis et al. (2019) focused on the changes made by the new curriculum policy in Australia to see how the conditions mediated teacher agency during assessment reform. It was indicated that some conditions influenced teacher agency development when they aimed to design assessment. Additionally, in Iran, Rezagah (2022) examined EFL teachers’ language assessment literacy in the light of the educational system of the country. He made a reference to teachers’ lack of agency in assessment practices. The findings depicted teachers’ concern for their agency and discussed the educational system’s need for shifting its policy towards offering more freedom for the teachers in assessment and selection of classroom materials.

The role and state of teacher agency in exam-oriented communities and systems, which force teachers to align with the demands of high-stakes tests has also been examined by L2 researchers in the past decade (Estaji & Ghiasvand, 2021; Liyanage et al., 2014). For instance, in their recent study in Bangladesh, Ali and Hamid (2023) scrutinized the interaction between teacher agency and washback effect in rural areas. Using classroom observations and interviews with two teachers, the researchers explained how teachers’ enact their agency in assessment domains, where they are under strong political pressure to increase students’ test scores. Moreover, a set of studies have highlighted teachers’ assessment agency and enactment in alternative assessment techniques, like formative assessment (Poulton, 2020; Verberg et al., 2013, 2016), LOA (Banitalebi & Ghiasvand, 2023; Willis et al., 2019), and negotiated assessment (Gosling, 2000). Additionally, Simpson (2017) reported on the impact of literature-based assessment on teachers’ professional agency negotiation. Despite these studies, a glimpse at the literature reveals that researching EFL teachers’ assessment agency from the ecological perspective is in its baby stage and requires empirical evidence to mature. The literature is limited to teachers’ assessment agency in relation to high-stakes exams and curriculum reforms, yet its ecology in L2 contexts has remained uncharted, to date. As one of the first attempts in this line of thinking, the present study intended to provide a validated scale for measuring EFL teachers’ ecological assessment agency as lack of a measure and awareness of the underlying components of this novel construct might have been the reasons for overlooking this research strand, to date. The study was guided by the following question:

• 1) What are the underlying components and psychometric properties of the Teacher Assessment Agency (TEAA) questionnaire?

Method

Context and participants

The data collection of the study took place in two phases. In the first phase, 211 Iranian EFL teachers were non-randomly chosen to answer the 41-items questionnaire using convenience sampling that demands available and eager participants. After excluding unengaged responses, this number reduced to 186. In the second phase, the moderated version of the questionnaire was answered by 395 other respondents. The screening of the answers in this phase also resulted in exclusion of 51 unengaged participants. Overall, the gleaned data reported here belongs to a sample of 539 EFL teachers with different backgrounds. Majority of the participants (N = 376) were students/graduates of Teaching English as a Foreign Language (TEFL). Their age ranged from 20 to 49 (M = 25.85). The demographic features of the respondents are presented in Table 1.

Table 1 Demographic features of the participants

It is essential to mention that, to observe the basic ethics of conducting research, the researchers explained the goal of the study to the participants and they were given permission to withdraw from study at any time without requesting further explanations of the reasons. Moreover, their privacy and identity were ensured to remain secret throughout the study and after its termination. The respondents also gave their consent to the researchers and voluntarily attended the study. Finally, the participants were assured that the data would be completely destroyed after the publication of the study.

Instrument

Teacher Ecological Assessment Agency (TEAA) Questionnaire

The present study designed a new questionnaire to measure EFL teachers’ perceptions of TEAA, which was then validated against the data gathered from 539 Iranian EFL teachers (Additional file 1). The newly developed questionnaire comprised two sections; the first one asked the participants about their demographic information (e.g., age, gender, experience, major etc.). However, the second section, which was the main part of the scale, was associated with EFL teachers’ perceptions about ecological assessment agency. More specifically, 41 items constituted this section, which were disseminated across three sub-categories in line with the possible components of assessment agency (i.e., Iterational, 12 items; Projective, 12 items, and Practical-Evaluative, 17 items). The items followed a five-point Likert scale in which the respondents’ answers would range from “1” (strongly disagree) to “5” (strongly agree). The researchers also examined the internal consistency of the TEAA questionnaire through Cronbach’s alpha. The obtained results illustrated a high reliability index (r = 0.91). “I draw on my past personal and professional experiences to forecast possible challenges in assessing L2 students” is a sample item from the questionnaire.

Data collection procedure

In an effort to design a rigorous and valid questionnaire to measure EFL teachers’ assessment agency from an ecological perspective, the researchers took different steps. First, they read some resources considering the standard stages to go through in developing a questionnaire in L2 education (e.g., Dörnyei, 2014; Dörnyei & Dewaele, 2022). Then they moved to the initial item generation phase, where they benefited from both deductive and inductive approaches. The former draws on the available literature and review previous studies and scales to write items (Hinkin, 1995), while the latter uses experts’ views and experiences regarding the construct via interviews (Cheng, 2017). To be more precise, the researchers initially interviewed eight language testing and assessment experts majoring in Applied Linguistics with sufficient experience. They were asked about the definitions, dimensions, and representations of assessment agency in EFL contexts. The interviews shed some light on the underlying components of TEAA. However, this was not enough for producing items of the questionnaire. Therefore, an extensive literature review was carried out by the researchers to inspect the current conceptualizations and frameworks of teacher agency, in general, and in relation to assessment, in particular. The mix of these two steps led to the first version of the TEAA questionnaire in which the dimensionality of the construct was observed. After that, the layout and the number of the items beneath each component were determined.

Next, a booklet questionnaire was generated based on an initial pool of items, which encompassed 48 questions under three categories. Then, six experts were invited to check the content validity of the items in this draft. The panel of experts were assistant and associate professors of applied linguistics in Tehran, who rated each item on a scale from 1 to 5 considering clarity, relevance, and language appropriateness. To do that, the experts were given a qualitative comment box in front of each item based on which the items were either edited or removed. Additionally, the researchers consulted with the panel regarding the revision and removal of some items with ceiling effects and confusing wordings that could produce misleading data. Upon the termination of the experts’ analysis, 7 items were suggested to be excluded from the scale leaving it with 41 items. This final version was then piloted on 30 EFL teachers to safeguard its quality and rigor (Creswell, 2003). Finally, an online version of the questionnaire was designed and distributed among the participants given the convenience and appealing nature of technology-based tools among modern L2 teachers. The online scale began with an image related to assessment to make the layout engaging for the respondents. Then the items were presented through a formal font size and color. In addition to a thanking note, the researchers designed a space at the end of the questionnaire’s link, in the Google form, to gather the participants’ comment on the items. After making minor revisions in the grammar and word-choice of some items, the 41-items scale was finalized. The scale was then filled in by 539 Iranian EFL teachers after 4 months from January 1st to June 2nd 2022. Before that, the goal of the study was explained to the target sample and the researchers made sure they had no conflict of interest with the participants. Although the respondents were free to submit their responses, the researchers kept reminding them to do so by sending emails and private messages in social media applications. The completion time of the questionnaire ranged from 20 to 25 min, which is consistent with Dörnyei and Taguchi’s (2010) proposition that completion time of questionnaires should remain under 30 min. Further, it is noteworthy that the researchers gave some gift cards and research methodology textbooks to some respondents by draws to compensate for their precious time. Lastly, different statistical analyses were employed to validate the TEAA questionnaire for EFL teachers.

Data analysis

The analysis of the data was done in two phases. First, the answers of 186 participants were analyzed through an exploratory factor analysis (EFA) using IBM SPSS (version 24) to capture the underlying pattern. After reducing the items unto factors and omitting the problematic items, a confirmatory factor analysis was run on the answers of the second group (N = 353), who had answered the moderated questionnaire (34-items). The CFA was run using IBM AMOS (version 24) and psychometric features, including measures of reliability and convergent/divergent validity, were obtained.

Results

Initially, data was screened for unengaged responses. As a result, 22 cases from the initial sample (N = 221) and 46 cases from the second sample (N = 395) with constant/increasing/decreasing patterns were spotted and excluded. Moreover, through examination of the standard deviation of the answers for each respondent, 8 more cases (3 from the initial sample and 5 from the second one) with low standard deviations (below 0.5) were excluded as they were considered unengaged respondents. This left our final sample with 539 cases.

To capture the underlying pattern, EFA was run on the responses of the initial sample. Regarding the adequacy of sample size, Larson-Hall (2013) recommends somewhere from 150 to 300 participants. In addition, we inspected the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy which showed the acceptable value of 0.809. Moreover, the result of Bartlett’s Test of sphericity (χ² (820) = 3219.66, p = 0.000) was statistically significant at p < 0.05, meaning the null hypothesis that the factors in the matrix are not independent of each other (an identity matrix) could be rejected.

Initially, the EFA with principle component analysis (PCA) resulted in the extraction of 11 factors with eigenvalues above 1. However, after running parallel analysis, we decided to reduce the number of factors to 5 fixed factors. The inspection of scree plot acknowledged our decision as there was a clear break after the fifth factor. Moreover, as the results of correlation matrix showed significant correlations among the factors, Promax rotation, which is an oblique rotation robust to strong correlations among the factors, was chosen. Table 2 presents the total variance explained by these five factors.

Table 2 EFA total variances explained

The loadings of items onto the five factors are presented in Table 3.

Table 3 EFA: rotated pattern matrix

In Table 3, loadings below 0.45 are suppressed to clearly present the extracted pattern. As it is evident, six items (Q01, Q13, Q15, Q17, Q26, and Q38) failed to have standardized loadings above 0.45 to any of the factors. These items, according to Kline (2016) endanger the convergent validity of a construct. Moreover, one item (Q24) was misloaded to factor 1 while it was expected that this item would group with items in factor 2. This item was also excluded. Comparing the content of the remaining items with the literature, we used Priestley et al.’s (2015) labels for factors 1 and 2, calling them Iterational and Projective, respectively. Factors 3 to 5 are parts of Priestley et al.’s (2015) third factor, i.e., practical-evaluative. However, as our results indicated, they needed to be considered as separate factors at this stage. We named factor 3 Teachers’ Practical-Evaluative View (TPEE); factor 4 was named Schools’ Practical-Evaluative Effect (SPEE); and the final factor was named Professional/Democratic Community’s Practical-Evaluative Effect (PDCPEE).

After obtaining the pattern, the shortened questionnaire with 34 items was filled out with the second group and CFA was run on the collected data. Overall, 353 gleaned data were analyzed in this phase. As Kline (2016) recommends at least 20 observations for each parameter in the model, the sample seemed large enough to run the analysis. Our initial analysis showed that there were strong correlations among SPEE, TPEE, and PDCPEE. Therefore, we decided to use a second-order model for these factors and put them all under a shared component suggested originally by Priestley et al. (2015), i.e., practical-evaluative (PE). Table 4 presents the unstandardized and standardized estimates obtained from the CFA.

Table 4 CFA unstandardized and standardized estimates

As reported in Table 4, all items had significant unstandardized loadings and none of the standardized loadings were low (below 0.45). While this was a harbinger of convergent validity, the inspection of average variance extracted (AVE) showed that the Projective factor falls short (estimated AVE = 0.392). To compensate for this shortage, two items (Q18 and Q23) with lowest loadings in this category were excluded. This increased AVE to 407. Then, the modifications suggested by the software that would cause parameter changes above 15 were taken into account, providing that they did not contradict with the literature. The final CFA model is presented in Fig. 2.

Fig. 2

The Final CFA Model

According to Hu and Bentler (1999), in order for the model to have a goodness of fit, a number of criteria have to be met. These criteria are minimum discrepancy function by degree of freedom (CMIN/df), root mean square error of approximation (RMSEA), comparative fit index (CFI), Tucker-Lewis index (TLI), parsimonious normed fit index (PNFI), and standardized root mean square residual (SRMR). The threshold for each criterion alongside the estimated values obtained from the data, are reported in Table 5.

Table 5 Goodness of fit measures

As reported in Table 5 all obtained values represented acceptable to excellent goodness of fit indices. Having made sure of the model fit, the reliability and discriminant validity of the model were examined (Table 6).

Table 6 Composite reliability and discriminant validity

As reported in Table 6, all of the variables had composite reliability (CR) values above 0.7, which reveals acceptable reliability. For all components in the model, the maximum shared variance (MSV) was below the AVE, which is an indication of convergent validity. Moreover, the square root of AVE (the bold values in the table) was above inter-correlations of the factors, indicating discriminant validity, according to Fornell and Larcker (1981).

Discussion

This study aimed to design and validate a questionnaire regarding EFL teachers’ ecological assessment agency. The results of SEM analysis illustrated 32 items and five factors in the final questionnaire. The extracted components of TEAA resonate with Priestley et al.’s (2015) model of teacher ecological agency, which was the underpinning theory of this study. However, the third component of this model (i.e., practical-evaluative) is broken into three components in the present study. The extraction of “iterational” and “projective” dimension is also in line with Ghamoushi et al. (2022), who ran a validation study on assessing EFL teachers’ ecological agency in Iran and found these components as vital parts of the construct of teacher agency. Regarding the “iterational” component, prior studies endorse the role of previous experiences in one’s actions and agency (e.g., Muhonen et al., 2021; Wang et al., 2021). This can be ascribed to nature of teacher assessment agency, which is the outcome of teachers’ previous experiences, trial and errors, and agentic pedagogies. Additionally, it can be argued that the participants’ prior experiences and professional practices related to assessment might have formed an agency system in them. This interpretation is also on a par with the situated essence of teacher agency that foregrounds the role of context, culture, and personal experiences in one’s perception and enactment of agentic practices. Another explanation for extracting this component might be the participants’ familiarity with the ecology of teacher agency in L2 education, especially assessment in Iran. Their university education or pre-service training could be the trigger of this awareness.

As for the second component (i.e., projective), the results are in compliance with Hemi et al. (2021), who pinpointed that teachers’ concerns for goal-setting and goal-accomplishment are the keys to their positive investments that foster professional development, quality, and agency. An explanation for this may be the participants’ assessment knowledge and beliefs regarding the importance of “goal perception” in teachers’ practices. Moreover, the EFL teachers’ aspirations for reconfiguring their future assessment actions might have caused them to highlight the future-oriented aspect of TEAA. For the participants, TEAA seems to be governed by both past experiences and future desires. This is again commensurate with the nested and contextual conceptualization of teacher agency, which underscores teachers’ knowledge construction through integrating past experiences into their actual practices or simply connecting theory to practice (McNeil, 2018). It seems that the participants of this study had been assessment literate in such a way that they viewed assessment practices and the associated agentic practices as formative, ongoing, and future-oriented rather than one-time and transitory practices. This futuristic outlook could be the drive behind their emphasis on future goal achievements in assessment. These inferences lend further support to Kusters et al.’s (2023) idea that agency leads to sustainable changes in teachers’ current and future practices. With this view of agency, the participants may have capitalized on the future side of assessment agency in L2 contexts.

The third component (i.e., teachers’ practical-evaluative view) targets their assessment-related decisions and evaluations that reflect the socio-cultural context, where they work. This dimension is in tune with the pragmatist approach to agency, which considers it the result of an interplay of several personal abilities and contextual conditions. Moreover, previous research highlights the critical impact of teachers’ socio-cultural views on their agency and agentic behaviors (e.g., Ashton, 2022; Biesta et al., 2015; Ghamoushi et al., 2022; Wang & Zhang, 2021). This finding can be attributed to Iranian EFL teachers’ professional perceptions of L2 assessment, as a collective practice affected by macro-structures, policies, and organizational discourses. This claim resonates with Karimpour et al.’s (2022) study, which argued that social structures and cultures strongly influence EFL teachers’ professional agency and agentic enterprise. Another possible explanation for this finding could be the participants’ high knowledge of the social dimensions of language assessment, which consider assessment a socio-political, contextual, and ideological tool. Their understanding that teachers’ classroom practices (i.e., pedagogical and assessment) are the outcomes of an interaction among micro, meso, and macro structures might be the reason behind this result.

Concerning the fourth component (i.e., schools’ practical-evaluative effect), the results echo those of Chaaban et al. (2021) and Priestley et al. (2015), who maintained that institutional structures, policies, and relationships may facilitate or even impede teachers’ agency and professional performance. This contention is valid because assessment is by no means an indifferent practice, and institutional policies and forces direct the way an EFL teacher conducts assessment practices in the class. Again, this idea is consistent with the ecological perspective of teacher agency, which argues for a situated, interactional, and collective nature of the construct. It considers teacher agency to be formed in-situ (Pedaste & Leijen, 2020). A justification for this finding might be the participants’ knowledge and understanding of the impact of top-down assessment cultures/systems on their agency, identity, and practices when it comes to assessment. Given their possible experience in such systems, the participants may have endorsed this component as a key to their ecological assessment agency.

The final extracted component (i.e., professional/democratic community’s practical-evaluative effect) highlights the significance of a positive, friendly, democratic, and professional context in which a teacher can enact his/her assessment agency. This finding concurs with Priestley et al. (2015) and Naraian and Schlessinger (2018), who argued that teacher agency establishment and enactment are determined by the quality of one’s interactions and relationships with others at workplaces. Probably because of their assessment literacy and professional experiences, the participants of this study considered teacher agency as a contextualized practice that is fostered by collegial relationships and interactions. Another reason might be Iranian EFL teachers’ shift of focus from seeing L2 assessment as a solitary act to a collaborative and joint practice in which teachers, colleagues, and students have a role to play. When the community of practice (COP) and the overall assessment culture in which an EFL teacher is working is positive, democratic, and professional, he/she becomes a professional agent in assessment and regards others as sources of learning and professional progress. This preference for collaboration and emotionality to enact agenctic practices is also substantiated by recent studies (e.g., Paloniemi et al., 2023; Sari, 2021). The participants’ content knowledge and field experiences could explain such a joint and situated view of assessment agency.

To conclude, although there are some validated questionnaire on teacher agency in the literature (e.g., Ghamoushi et al., 2022; Leijen et al., 2021), they made no contribution to L2 assessment. This is where the novelty of the present study resides. It expands teacher ecological agency to assessment territories in EFL/ESL contexts and adds two more components to previous models and inventories considering teacher agency. Despite these insights, care should be taken to extrapolate the results of this study, as the proposed components of TEAA might not be reproducible on wider scales in different educational contexts.

Conclusion and implications

Like other areas of L2 education, assessment is a complex and multi-faceted practice requiring teachers to be active agents, who take initiatives in choosing how to assess their students’ academic performance. However, in many EFL/ESL contexts still L2 assessment is governed by top-down policies leaving no room for teacher agency despite the fact that agency is the heart of teacher identity and professionalism (Nguyen & Ngo, 2023). To move the current knowledge regarding EFL teacher agency from “pedagogy” to “assessment”, this study designed an inventory to measure the ecological assessment agency of a cohort of Iranian EFL teachers. The main conclusion drawn from the findings is that TEAA is a multi-dimensional construct, which is influenced by several personal, contextual, and socio-cultural factors. It is an essential quality of L2 teachers in the neoliberal era of teacher education that highlights agentic teacher practices. In light of this study, EFL teachers may identify the prominence of teacher agency in assessment practices and make principled decisions that represent who they are as active agents of their profession rather than passive, static subjects. This quality also improves teachers’ performance in that they realize where they can and cannot exert agency to influence students’ achievement. Teacher professional development programs may reflect on the findings and offer EFL teachers instruction and feedback regarding their assessment performance, which can guide their professionalism and agency. Such an improved performance may also increase the quality of L2 education and the public trust in educational systems.

Moreover, human resources committees may consider assessment agency qualities as another factor in recruiting EFL teachers in their institutes. This makes teachers accountable for their assessment decisions and practices ensuring that they are adhering to the assessment principles set by the system. Teacher trainers may also use the designed questionnaire and measure the level of agency in teachers, then design special courses to form and enhance their current assessment agency level. Additionally, L2 scholars may use this study as a starting point to explore the construct of TEAA in more depth in the future. Finally, at the macro-level, language assessment policy-makers may find this study beneficial in that they can echo the extracted dimensions of TEAA into their policies and decisions and provide proper infrastructures for EFL/ESL teachers to manifest more agency in their assessment practices in the class.

Concerning the limitations of the study, it is noteworthy that the sample size could be larger to have more findings that are generalizable. Additionally, the items were design based on a single model of teacher ecological agency (i.e., Priestley et al., 2015), while other perspectives and approaches could be used, as well. The data was collected from one context; hence, cross-cultural studies can be done to replicate the study to see if the same psychometric properties are obtained. Finally, future studies can use qualitative instruments like focus groups, discussion panels, classroom observations, and reflective journals to have a deeper image of TEAA.