Skip to main content

Reliability assessment and construct validation of translation competence questionnaire (TCQ) in Iran

Abstract

The present study investigates the construct validation of Translation Competence Questionnaire (TCQ) in the context of Iran. For this purpose, one hundred ninety seven EFL learners, whose major was translation studies at Tabaran, Institute of Higher Education, Mashhad, Iran, were asked to participate in the study and fill in the questionnaire and give it back to the researcher within a week. First, internal consistency of the instrument was assessed using Cronbach’s coefficient alpha reliability. Then, confirmatory factor analyses (CFAs) were conducted using LISREL 8.80. The internal consistency analysis of the TCQ utilizing Cronbach’s coefficient alpha reached acceptable alpha(s). The results of the CFA as assessed by the CFI/NNFI, RMSEA and chi-x 2, reached acceptable fits. The factor structures were defined and the factors were clearly distinguishable from one another. Based on the findings, the researchers concluded that Translation Competence Questionnaire by Mariana Orozco and Amparo Hurtado Albir, modified and re-developed by Alavi and Ghaemi, has undoubtedly strong psychometric characteristics and good constructs validity in the context of Iran.

Background

Over the recent years, translation and interpretation studies have attracted a great deal of attention; this appears to be affected by the temporal and special needs and demands of academic and industrial circles. The advances and developments in science and technology and the need to exchange the newly-developed information, knowledge and technology in native speakers’ countries highlight a deep demand to train professional translators.

One of the requirements of training professional translators is to look more deeply into the translation professionalism. One of the most critical factors contributing to professionalism in translation is the development of translation competence, which has been called differently by different scholars, as Transfer Competence (Nord, 1992), Translational Competence (Toury, 1995), Translation Performance (Wilss, 1989), and even Translation Skill (Lowe, 1987).

Borsch (1986), Gerloff (1987), Seguinot (1991) and Lorscher (1991) examined translation competence acquisition. PACTE group, i.e. Process of Acquisition of Translation Competence and Evaluation, (2002) presents the translation competence model that is the basis for designing the hypotheses of an empirical-experimental study of translation competence. Their research is the first stage in a larger project to investigate the process of translation competence acquisition. They describe theoretical framework and the first models that were designed in 1998; along with the modification introduced in 1998 translation competence model were developed as a result of the first exploratory studies.

After the translation competence model has been established, the researchers, i.e. Orozco and Albir, tried to design and develop the instruments which could accurately measure it. In 2002, Orozco and Albir developed the instruments for measuring the process of acquiring translation competence in written translation. Translation competence and its process of acquisition were described, and then three measuring instruments especially developed to measure translation competence acquisition were presented: (I) to measure notions about translation, (II) to measure students’ behavior when faced with translation problems, and (III) to measure errors. Pilot studies were carried out for three years to test, improve and validate the measuring instruments.

Multi-dimensionality of translation competence questionnaire

Orozco (2001) has identified three basic characteristics that are shared by all the indicators of translation competence acquisition. First, all the translation competence acquisition indicators should affect the whole process of translation, not just some of the stages. Second, they should be observable and measurable. Finally, all of them should give an indirect view of translation strategies, which is not directly observed but is essential to translation competence (Orozco, 2001).

Orozco and Albir (2002) have chosen three indicators to make translation competence acquisition operational. These indicators are as follows:

  1. 1.

    Behavior when encountered with a translation problem

  2. 2.

    Behavior related to translation errors

  3. 3.

    General Notions about translation

As for any kind of instrument, there should be theoretical foundations on which the instruments have been developed. The definitions on which the instruments of measuring translation competence acquisition are developed are as follow:

According to Nord (1992), p. 132, a translation problem “is an objective problem which every translator has to solve during a particular translation task”. Orozco (2001) believed that all translation problems share three features that make them reliable indicators of progress in acquiring translation competence. First of all, they should appear at any stage of translation process. Second, it is observable, and lastly students will have the ability to use translation strategies, which is one of the elements of translation competence, to solve their problems (Orozco, 2001).

Nord (1997) states that the birthplace and source of a translation error is mostly a translation problem, which has not been solved or has been solved inappropriately. As can be seen, this element also shares the same three qualities as translation problems, that is, it can be observed, it can happen at any stage of translation, and it is indicator of a student’s use of translation strategies.

At last, general notions of translation depicts the students’ whole process of translation, because it depends on the students’ ideas and attitudes towards translation, and shows that they have an aim for a particular translation task, which determines the students’ solution of translation problems throughout the translation process (Orozco, 2001).

Nord (1997) puts out that the lack of knowledge of translation notions causes translation errors. Thus, translation notions also share the same three qualities mentioned above.

Overall, three instruments, which have been developed by Orozco (2001), to assess the Translation Competence Acquisition, are as follows:

  1. 1.

    Translation Notions Instruments

  2. 2.

    Translation problems Instruments

  3. 3.

    Translation Errors Instruments

Göpferich (2009) carried out a process research into the development of translation competence. They gave a general idea of the methods employed in process-oriented investigations of translation competence and its development and described their advantages and shortcomings. Special emphasis was placed on the contrastive evaluation of methods, on longitudinal studies, as well as on the documentation and diffusion of process data. The design of one longitudinal study, TransComp, which investigated the development of translation competence in 12 students of translation over a period of three years and compared it to that of 10 professional translators with more than 10 years of experience, were introduced.

Moreover, asset management systems were suggested to make translation process data accessible to the scientific community.

PACTE group, in 2005, investigated the TCA from conceptual and methodological viewpoints. This group, which consists of eight researchers and one principle investigator, used empirical - experimental research methods to investigate translation competence and how it is acquired. They presented the design of Translation Competence research project. The first results of the pilot test were related to decision-taking in the translation process and the interrelation between the use of internal and external support. The results pointed to the differences in the processes followed by professional translators and other language professionals.

Renad Abbadi, in 2010, worked on translation competence and quality in Jordan from an empirical corpus-based perspective. He presented and discussed some results from a corpus based study focusing on professional translators’ errors and the underlying problems resulting in them, in view of the concept of translation competence.

In 2003, Wolfgang Lorscher, carried out a project to analyze psycholinguistically translation performance as contained in a corpus of orally produced translations from German into English and vice versa. This was done in order to reconstruct translation strategies. The study addressed the question of what steps can be taken to favor the development from a subject’s basic ability to mediate towards translation competence? The researcher, i.e. Wolfgang Lorscher, believed that it was indispensable to face foreign language learners and in particular teachers with the drawbacks of and the horrible translation errors often caused by the use of solely sign-oriented procedures. Subjects should be sensitized to the dearth and inadequacies of those translations which were principally produced by an exchange of signs. As a result, target-language texts were produced which were neither equivalent in sense to the respective source-language texts nor grammatically or stylistically acceptable texts by themselves. Researcher believed that this was not caused by lack of competence. Finally, Lorscher concluded that making the foreign language learners and teachers aware of these deficits and making them continuously check on the sense of the texts they produce in their translations may be a first step towards sense- oriented translating.

The last but not the least study carried out on TCA was done by Ressurrecció, Piorno, and Izquierdo, who investigated the acquisition of translation competence through textual genre. They explored the relation between genre and translation competence, on the one hand, and the communicative and textual sub-competence, on the other. They drew the conclusion that there seemed to be a certain amount of convergence between translation competence and the theory of text genres. In this regard, they suggested that text genre could be a very useful educational aid when it came to planning and carrying out the teaching of specialized translation; this was particularly true in the case of developing communicative and textual competence, but could be extended to other translation competencies.

Purpose of the study

Translation competence is defined differently by different scholars. For example, Bell (1991) defines translation competence as the knowledge and skills the translators should have in order to do translation. But, according to Orozco (2001), the most comprehensive definition is the one given by PACTE research group (2000), p. 36, which defines translation competence as “the underlying system of knowledge and skills needed to be able to translate”. Based on this definition, Orozco and Albir (2002) developed TC questionnaire.

The present study, as its first aim, re-develops and modifies the TC questionnaire developed by Orozco and Albir (2002) and, as its utmost goal, assesses the validity and utility of the multi-dimensional Translation Competence Questionnaire by Mariana Orozco and Amparo Hurtado Albir (2002) in the Iranian sample. This will open a gate way for other credible studies in the field of translation studies, role of translation competence in translator training programs and teachers’ understanding of the influence of the translation competence factors on translation abilities of translation trainees.

Hypotheses

Ho 1: The Translation Competence Questionnaire does not show good internal consistency.

Ho 2: The instrument does not show good fit indices as measure by the Confirmatory factor analysis.

Method

Participants

One hundred ninety seven EFL learners, who are majoring Translation Studies in Tabaran, Institute of Higher Education, Mashhad, Iran, were asked to participated in the study and fill in the questionnaire and give it back to the researcher within a week. All participants were considered to be intermediate and upper-intermediate based on the completion of their previous semesters and the guidelines offered by the institute.

Instruments

The measure used for the study was Translation Competence Questionnaire (TCQ) by Mariana Orozco and Amparo Hurtado Albir (2002). TCQ is a learner rated assessment of the extent to which each learner is competent in translation. Orozco (2001) has identified three basic characteristics that are shared by all the indicators of translation competence acquisition. First, all the translation competence acquisition indicators should affect the whole process of translation, not just some of the stages. Second, they should be observable and measurable. Finally, all of them should give an indirect view of translation strategies, which is not directly observed but is essential to translation competence (Orozco, 2001).

Translation Competence Acquisition Questionnaire which includes three sub-instruments as follows:

  1. 1.

    Translation Error Instrument (TEI)

  2. 2.

    Translation Problem Instrument (TPI)

  3. 3.

    Translation Notions Instrument (TNI)

Orozco (2001) suggests that the above instruments are the measuring instruments of TCA.

The first measuring instrument, i.e. TEI, measures errors, the second, i.e. TPI, measures behaviors of translators when faced with translation problems, and the third, i.e. TNI, measures the knowledge about translation. All three instruments were unified in a single questionnaire called TC Questionnaire and it was administered twice, once at the beginning of the treatment and once the treatment was completed.

Translation notions instrument (TNI)

TNI is a multidimensional questionnaire as it covers seven factors within the “abstract” notion of what translation is (Orozco and Albir, 2002). Seven factors like notions about translation, notions about translation problems, the translation units, translation equivalence, translation functions, translation competence and translation strategies are included in the questionnaire (Orozco, 2000). Based on the findings of Orozco and Albir (2000), this questionnaire measures two main constructs of Knowledge about translation, as measured by items 1, 2, 3, 4, 5, 6, 7, 12, 13, and 14, of the TC questionnaire and Strategic sub-competence, which is measured by items 15, 16, 17, 40, 50, 51, 52, 53, 54, 55, and 56.

Translation problems instrument (TPI)

TPI questionnaire consists of two parts. The first part includes a task translating a text, and the second one a TPI questionnaire. In the text students are supposed to translate four translation problems, namely: pragmatic, extra-linguistic, transfer and linguistics. Orozco and Albir (2002) maintained that these four types of translation problems are chosen based on the rationale that in order to solve them the translator needs to mobilize all the components of translation competence (Orozco and Albir, 2002). After students translated the text, they were asked to answer the TPI questionnaire.

The evaluator read the translated text together with the TPI questionnaire. Therefore, the translation of each student was checked to see whether each problem had been solved or not. This questionnaire measures two main constructs of Bilingualism and Instrument sub-competences. Bilingual sub-competence was measured through items 21, 22, 23, 24, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 43, 44, and 45. Also, Instrumental sub-competence was measured through items 8, 9, 10, 11, 18, 19, 20, 28, 41, and 42 of TC questionnaire.

Translation error instrument (TEI)

TEI was aimed to measure two constructs of Extra-linguistics and Psycho-physiological components. The former was measured through items 25, 26, 44, 49, 56, and 57 and the latter through items 27, 46, 54, 55 and 57.

Since the questionnaire is re-developed in the context of Iran, its Cronbach’s Alpha reliability and internal consistency were measured. Also, the construct validity of the TC questionnaire was calculated using LISREL 8.8. The results of construct validation are as follow:

In line with the work of Batinic, Wolff and Haupt (2007), the goodness of fit statistics was obtained emphasizing the Root mean square error of approximation (RMSEA), the comparative fix index (CFI), Non-normed fit index (NNFI) and incremental fit index (IFI). The X 2 test was also used to test the fitness of the model. RMSEA values less than .05 indicate good fit and values as high as .08 represent acceptable errors of approximation (Bentler, 1990). The CFI/NNFI and IFI differ along a 0 to 1 continuum in which values greater than .90 and .95 are considered to show an acceptable and outstanding fit of the data (Bentler, 1990). However, an index of .90 and above is considered as acceptable fit (Harrington, 2009).

TC questionnaire, used in the present study, consists of three main sections namely: Translation Notions Instrument (TNI), Translation Problems Instrument (TPI) and Translation Error Instrument (TEI). According to Orozco and Albir (2002), the first part of the questionnaire, i.e. TNI, including 21 items, covers two main latent variables of notions about translation and notions about translation competence. The second part comprises a text to be translated and TPI including 28 items, all of which should be measured based on the translated text. The items of TPI represent four latent variables, i.e. four translation problems namely: pragmatic, transfer, linguistic, and extra-linguistic problems. The last part, i.e. TEI, consists of 11 items, in which two constructs of Extra-linguistics and Psycho-physiological components, were measured.

Internal consistency

The Cronbach’s alpha coefficient estimated for the instrument and subscales was acceptable and reached the target reliability of at least .70 (Garson, 2005;Lewicki & Hill,2006).

Confirmatory factor analysis

The CFA model estimated the ability of the three factors to clarify the relationship among the 60 items (item 21 consists of 4 sub-items). This model with 60 items positing two TC factors provided a good fit to the data (CFI = .94, NNFI = .93, IFI = .92, RMSEA = .07 and x 2 =972.09, df =569, p = 0.0). The solution was entirely proper and the factor structure was well definite with all factor loading being positive and significant and were larger than .30 (See Table 1). The correlations among the three factors were small to moderate (rs = .91), indicating that the factors were clearly distinguishable from one another (see Table 2). In sum, there was support for the three factor model in Iranian sample based on the finding of; (a) a reasonable model fit (i.e. CFI, NNFI, IFI = .93), (b) good factor loading for the model (.43 and above for each item loading on the respective factor), and (c) reasonably low correlation among the three factors (<.7).

Table 1 The Cronbach’s alpha(s) Coefficient for TCQ
Table 2 Summary of the Item - factor loading

Modeling approach

The most significant basis on which SEM is based is the correlation matrix and/or covariance matrix. So, for the purpose of this study, first, the data obtained from the questionnaires of TNI and TPI were analyzed using SPSS software and then the correlation matrix which was obtained from the SPSS was imported to and run through LISREL software.

Since the Chi – Square equals 13, the p-value is larger than 0.05 and RMSEA is less than 0.05, we conclude that the model is fit. The Goodness of Fit Index (GFI) equals 0.91, Adjusted Goodness of Fit Index (AGFI) equals 0.76 and Parsimony Goodness of Fit Index (PGFI) equals 0.35. These findings also confirm that the data fits the model.

The values which are written on each arrow are demonstrated in the Estimated Mood, and they are not interpretable. In all SEM models run in LISREL software, the values of Estimated Mood are not interpretable because there is no principle to which one can compare these values. In order to make the values interpretable, we should change the mood from Estimated Mood to T-Value Mood. Having changed the mood to T-Value mood, we see that all the values written on the arrows of the above model changed and are higher than 1.96 (1.96 is a predetermined principle value to which all the values are to be compared). As a result, we can conclude that there is a meaningful relationship between the observed variables (TNI, TPI and TEI) and their latent variables, i.e. Instrument Sub-Competence, Bilingualism Sub-Competence, Extra-linguistics, Psycho-Physiological, Knowledge about Translation, and Strategic sub-competence.

Now, we should assess the relationship between TNI, TPI and TEI to see whether both sections of TCQ are in the same line with the purpose of TCQ. As illustrated in Figure 1, the relationship among TNI, TPI and TEI equals 2.75 and based on the SEM literature, since this figure is larger than 1.96, we come to the conclusion that there is a statistically meaningful relationship among TEI, TNI and TPI.

Figure 1
figure 1

Relationships between latent and observed variables. Chi-Square=13, P-Value=0.16, RMSEA=0.035.

As seen in Table 1 the internal consistency analysis of the TCQ utilizing Cronbach’s coefficient alpha reached acceptable alpha(s). The results of the CFA as assessed by the CFI/NNFI, RMSEA and chi-x 2, reached acceptable fits. Table 2 showed that the factor structures were defined and Table 3 showed that the factors were clearly distinguishable from one another. The factor structures of TCQ suggest that this instrument is thoroughly qualified to be used in the context of Iran and on Iranian Samples.

Table 3 Sub-factor correlation of the three factors of TCQ

Discussion

As seen in Table 1 the internal consistency analysis of the TCQ utilizing Cronbach’s coefficient alpha reached acceptable alpha(s) which rejected the first hypothesis. The results of the CFA as assessed by the CFI/NNFI, RMSEA and chi-x 2, reached acceptable fits. Table 1 showed that the factor structures were defined and Table 2 showed that the factors were clearly distinguishable from one another; this also rejected the second hypothesis. The factor structures of TCQ suggest that this instrument is thoroughly qualified to be used in the context of Iran and on Iranian Samples. It is now feasible for all the researchers to carry out their research in the context of Iran with the newly validated questionnaire.

Conclusion

Empirical research in translation studies, in particular TC, appeared in 1980s. At first, the scholars concentrated on the development of a model for TC, but recently they are more focused on the components and factors affecting it. Orozco and Albir (2002) proposed that the period of time exposure to translation training methodologies may have some influence on TC, but they never carried out research into it. Most of the empirical researches in translation studies, as Orozco and Albir, Orozco and Albir 2002 suggest, are concerned with problems not only from the scientific point of view, but also from the theoretical perspective because none of them explores translation competence as a whole, except some of its aspects (Orozco and Albir, 2002). It also appears that no study targeted the reliability assessment and construct validation of TCQ in an Iranian context.

The present study aimed at investigating the construct validly of Translation Competence Questionnaire in the context of Iran. Based on the findings, the researchers concluded that Translation Competence Questionnaire by Orozco and Albir has undoubtedly strong psychometric characteristics and good constructs validity in the context of Iran.

References

  • Abbadi R: Translation competence and quality in Jordan: An empirical study based on parallel corpus data. Jordan: Paper presented at the Second Jordan International Conference on Translation, Petra University; 2010.Abstract retrieved from http://uop.edu.jo/download/JICOT2/Abstracts2JICOT1.pdf

    Google Scholar 

  • Batini B, Wolff H, Haupt CM: Construction and factorial structure of a short version of the trendsetting questionnaire (TDs-K): a cross validation using multi group confirmatory factor analyses. Eur J Psychol Assess 2007, 24(2):88-94.

    Article  Google Scholar 

  • Bentler PM: Comparative index in structural models. Psychol Bull 1990, 107: 238-246.

    Article  Google Scholar 

  • Borsch S: Introspective methods in research in interlingual and intercultural communication. In Discourse and cognition in translation and second language acquisition studies. Edited by: House J, Blum-Kulka S. Tubingen: Narr; 1986:195-210.

    Google Scholar 

  • Garson GD: Structural equation modeling. 2005. Retrieved from http://www2.chass.ncsu.edu/garson/pa765/structur.htm

    Google Scholar 

  • Gerloff P: Identifying the unit of analysis in translation: Some uses of think-aloud protocol data. In Introspection in second language research. Edited by: Faerch C, Kasper G. Clevedon: Multilingual Matters; 1987:135-158.

    Google Scholar 

  • Göpferich S: Towards a model of translation competence and its acquisition: the longitudinal study 'TransComp'. Behind the Mind: Methods, Models and Results in Translation Process Research 2009. Hrsg. Susanne Göpferich, Arnt Lykke Jakobsen und Inger M. Mees. (Copenhagen Studies in Language 37). Kopenhagen: Samdundslitteratur, 11-37.

    Google Scholar 

  • Harrington D: Confirmatory factor analysis. New York: Oxford University Press; 2009.

    Google Scholar 

  • Lewicki P, Hill T: Statistics: Method and applications. 2006. Retrieved from http://www.statsoft.com/textbook/stathome.html

    Google Scholar 

  • Lörscher W: Translation performance, translation process, and translation strategies. Tübingen: Narr; 1991.

    Google Scholar 

  • Lörscher W: Models of the translation process: claim and reality. Target 2003, 1(1):43-68.

    Article  Google Scholar 

  • Lowe P: Revising the ACTFL/ETS scales for a new purpose: rating skill in translating. In American translators association series. Edited by: Rose MG. New York: Suny Binghamton Press; 1987:53-61.

    Google Scholar 

  • Nord C: Text analysis in translator training. In Translating as a purposeful activity. Functionalist approaches explained. Edited by: Dollerup C, Lindegaard A. Amsterdam: John Benjamins; 1992:39-48.

    Google Scholar 

  • Nord C: Translating as a purposeful activity. Functionalist Approaches Explained. Manchester: St. Jerome Publishing; 1997.

    Google Scholar 

  • Orozco M: Instrumentos de medida de la adquisición de la competencia traductora: Construcción y validación [Measuring instruments of acquisition of translation competence: Construction and validation]. Bellaterra (Barcelona): Universitat Autònoma de Barcelona; 2000.Unpublished doctoral dissertation

    Google Scholar 

  • Orozco M: Revisión de investigaciones empíricas en traducción escrita [Review of empirical research in written translation]. Barcelona: Gedisa; 2001:63-85.

    Google Scholar 

  • Orozco M, Albir AH: Measuring translation competence acquisition. J des traducteurs/Meta Transl J 2002 Retrieved from http://id.erudit.org/iderudit/008022ar

    Google Scholar 

  • PACTE: Acquiring translation competence: hypotheses and methodological problems in a research project. In Investigating Translation. Edited by: Presas R. Amsterdam: John Benjamins; 2000:99-106.

    Chapter  Google Scholar 

  • PACTE: Exploratory tests in a study of translation competence. Interpret Transl 2002, 4(2):41-62.

    Google Scholar 

  • Séguinot C: A study of student translation strategies. In Empirical research in translation and intercultural studies. Edited by: Tirkkonen-Condit S. Tübingen: Gunter Narr; 1991:156-174.

    Google Scholar 

  • Toury G: Descriptive Translation studies–and beyond. Amsterdam: John Benjamins; 1995.

    Book  Google Scholar 

  • Wilss W: Towards a multi-facet concept of translation behavior. Target 1989, 34: 129-149.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hamed Ghaemi.

Additional information

Competing interests

The authors declare that they have no competing interest.

Authors’ contributions

Both authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Alavi, S.M., Ghaemi, H. Reliability assessment and construct validation of translation competence questionnaire (TCQ) in Iran. Language Testing in Asia 3, 18 (2013). https://doi.org/10.1186/2229-0443-3-18

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/2229-0443-3-18

Keywords