University faculty members’ perspectives on English language demands in content courses and a reform of university entrance examinations in Japan: a needs analysis

Abstract

Background

This small-scale needs analysis study examined content course faculty members’ perspectives on English language demands in their courses and their perceptions of the introduction of four-skill English language assessment for university entrance examinations currently being planned as part of a major English education reform in Japan.

Methods

This author conducted one-on-one semi-structured interviews with six faculty members in mathematics and earth sciences at a private university in Tokyo to examine (1) the use of L1 and L2 as well as language use activities involving reading, listening, speaking, and writing in English in content courses they were teaching or had taught previously and (2) their perceptions of four-skill English language assessment for student admission.

Results

A qualitative analysis of interview results showed various similarities between the two disciplines, concerning the perceived importance of reading ability for satisfactory completion of content courses as opposed to minimal involvement of language use tasks requiring listening, speaking and writing at the undergraduate-levelcourse work, followed by an expansion of advanced academic English demands at the graduate level encompassing all four skills. Moreover, the participants generally supported the proposal to introduce four-skill English language assessment to university entrance examinations in the country.

Conclusion

Despite the fairly imbalanced representation of the four skills in the content courses taught by the participants, their views were generally consistent with the proposed direction for the on-going reform of English language assessment for university admission in Japan.

Background

Currently Japan has been undergoing a major government-initiated reform of English language instruction and assessment from the primary school to university levels. With meeting the needs of the rapidly globalizing international community being the ultimate goal, this reform aims to boost the overall English language ability of school-based English as a foreign language (EFL) for learners in the country and to promote the balanced development of their English language ability across the four competency skills (reading, listening, speaking, and writing). Results of two large-scale national English language assessments recently administered to upper secondary school seniors at randomly-selected public schools nationwide (approximately 70,000 students in 2013 and 90,000 students in 2014) by the Ministry of Education, Culture, Sports, Science, and Technology (henceforth, MEXT, 2015, 2016a) confirmed the urgency to proceed in this direction. Even after at least 6 years of formal EFL instruction, the reading and listening ability levels of a majority of the national samples were found to be in the high A1 and low A2 levels on the Common European Framework of Reference (CEFR; Council of Europe, 2001), while their speaking and writing ability levels were even lower, with more than 80% of the students classified at the A1 level. These results came as a shock to those involved in English language education in the country, making it appear difficult to achieve the target English language proficiency level for high school graduates that MEXT (2013) had defined as part of the current reform: the B1 to B2 levels on the CEFR.

Recognizing the generally low English language ability levels of Japanese EFL learners for decades, MEXT plans to implement a revised course of study for EFL instruction in primary and secondary schools in 2020. Another key initiative along this line is a drastic reform of the university entrance examination system in an attempt to enhance the articulation of EFL instruction between upper secondary schools and universities (MEXT, 2016b). Traditionally, English components of university entrance examinations practiced in the country—both the National Center Test administered nationwide once a year by the National Center for University Entrance Examinations (NCUEE) and those developed and administered by individual universities locally—have focused primarily on the assessment of the receptive skills. (see Kuramoto and Koizumi, 2016, for further details about the university entrance examination system in Japan). To promote a more balanced development of English communication skills of EFL learners in the country, however, panels of experts organized by the Japanese government (Education Rebuilding Implementation Council, 2013) as well as MEXT (2011, 2013, 2014a) made a series of recommendations to start assessing all four skills in university entrance examinations and to promote the adoption of commercial English language tests for this purpose. Accordingly, MEXT (2016b) presented two tentative scenarios to transform the National Center Test to a four-skill assessment: (1) to discontinue the English section of the National Center Test and switch entirely to commercial four-skill English language assessments in 2020 and (2) to make both the English section of the National Center Test and commercial four-skill English language assessment available at least until 2023, so that individual universities can select which ever option suits their needs. The decision as to which option to adopt is being finalized based on public comments collected in May and June, 2017 (MEXT, 2017).

The top-down reform described above is expected to impact a large number of individuals and institutions across the country. According to the NCUEE website (http://www.dnc.ac.jp/data/shiken_jouhou/h29/index.html), in January 2017 approximately 580,000 university applicants took the National Center Test, the scores of which were required to apply for 848 public and private universities and junior colleges in Japan. Individual universities are expected to determine whether they use either the National Center Test, commercial four-skill assessments, English language tests developed and administered by universities locally, or any combinations of those tests for screening candidates applying for their own degree programs. Accordingly, revising the National Center Test would necessitate individual universities to re-examine their existing admission policies and requirements.

In order for individual universities to specify optimal admission requirements in the above context, a thorough examination of two key issues is in order. One is to clearly identify the target language use domain (the TLU domain; Bachman and Palmer, 2010) for an admissions test. One function that an admissions test might play is to assess students’ achievement in secondary school EFL. This could be achieved by employing the National Center Test, which is based on the course of study for upper secondary school EFL instruction published by MEXT. Another important function of an admissions test would be to screen students who have sufficiently high English language ability to meet English language demands at the university. Defining the TLU domain at universities in Japan is not straightforward, however, for two reasons. First, content courses at universities in Japan have typically been conducted in Japanese, although reading materials written in English might be adopted, which is a similar situation to such countries as Israel (Inbar-Lourie and Donitsa-Schmidt, 2013) and Sweden (Salö, 2010, cited by Kuteeva and Airey, 2014). However, little empirical research examining the practice of English language use in such content courses in the Japanese context is currently available. Second, what complicates the matter further is that the TLU domain itself is changing partly because of the promotion of English medium instruction (EMI) at the university level by the government (Education Rebuilding Implementation Council, 2013). As is the case in other countries worldwide (Dearden, 2015), the number of universities that offer EMI courses at the undergraduate and graduate levels is on the rise, accounting for approximately 30% of 776 national, public, and private universities surveyed in 2013 (MEXT, 2014b).¹

A second key issue, which is related to the first, is that needs and values of key stakeholder groups should be reflected adequately in the TLU domain definition. The importance of incorporating stakeholder perspectives into the test design has been suggested in assessment validity and use frameworks such as Messick’s (1989) unitary concept of test validity that entails social consequences of assessment use as well as more recent argument-based approaches to language test validation and use (e.g., Bachman and Palmer, 2010; Chapelle, Enright, and Jamieson, 2008). One way to address this issue is to conduct a needs analysis. In the context of defining language use tasks in the TLU domain for a job-specific performance assessment development, McNamara (1996) states that such investigations may take four different forms: consultation with expert informants, literature search, job analysis and workplace observation, and collection and examination of texts (spoken and written) from the workplace. Such investigations have been employed in the development and validation of English for academic purposes (EAP) assessments as well. As an illustrative example, among various studies conducted to define the TLU domain for designing the TOEFL iBT® test was a large-scale job analysis survey of university students and faculty members conducted by Rosenfeld, Leung, and Oltman (2001). Rosenfeld et al. identified frequent and important language use tasks across the four skills in undergraduate- and graduate-level courses offered at universities in North America, which confirmed that the design of prototype assessment tasks that a panel of experts had developed was consistent with the perspectives of two key stakeholder groups for the test: university students and faculty members. The results in turn offered backing for the domain definition, a key building block in the validity argument for the TOEFL iBT test (Chapelle et al., 2008).

Similar types of investigations that enhance the TLU domain definition would be required for the university entrance examination context in Japan. However, relevant empirical results available so far are limited. Green (2014) recently conducted a large-scale questionnaire survey of 3868 students and 423 teachers at upper secondary schools and 19 faculty members at a private university in Japan as part of a longitudinal investigation into the impact of introducing to the country a new standardized commercial EAP test for university admissions, the Test of English for Academic Purposes (TEAP; Eiken Foundation of Japan). Green’s study yielded important findings regarding the current situation of EFL education at the upper secondary school level as well as the stakeholder groups’ perceptions of changes the introduction of the TEAP test might bring to the context. For instance, Green identified the stakeholder groups’ general support for a four-skill assessment in university entrance examinations, the strong belief among the survey respondents in the impact of introducing the four-skill assessment on EFL instruction in Japan, and competing trends between younger upper secondary school teachers’ support for assessing four skills as opposed to older teachers’ endorsement of traditional assessments focusing on reading.

While Green’s (2014) study provides support for the general direction of the introduction of the assessment of four skills to university entrance examinations in Japan, the study alone does not offer sufficient guidance for individual universities to define their admission requirements for two reasons. First, a large-scale survey study such as Green’s does not allow a contextualized analysis of stakeholder needs. Thus, such a study should be supplemented with more in-depth, qualitative investigations into the views of stakeholder groups conducted at individual universities locally. Second, because Green’s study focused mainly on upper secondary school teachers and students, detailed analyses of university faculty members’ views were not conducted. An elaborate analysis of university faculty members, particularly those involved in content courses, would be essential because content course faculty members constitute a legitimate stakeholder group for university entrance examinations. They teach students who are admitted based on university entrance examinations. While students usually take EFL courses to fulfill their foreign language requirements once admitted, oftentimes content course faculty members are also tasked to provide students with support for developing literacy in their respective disciplines, both in L1 (Japanese) and L2 (English). Despite this, only certain stakeholder groups (e.g., students, EFL practitioners and researchers, policy-makers, and test publishers) have been involved in the discussions on the current reform so far, while the perspectives of content course faculty members in disciplines other than those related to English language education are severely underrepresented.

With the above as the background, this paper reports on results of a small-scale needs analysis interview study with six content course faculty members at a private university in Japan. This study was conducted as a preliminary investigation that could inform the design and development of a larger, quantitative needs analysis survey of English language demands across different disciplines. This study focused specifically on two academic disciplines: mathematics and earth sciences. A primary reason for selecting these two disciplines was the current underrepresentation of views of content course faculty members in mathematics and sciences, in particular, despite their active involvement in various international research activities. Furthermore, English language demands in two disciplines, rather than those only in one, were examined in an attempt to represent perspectives of faculty members in multiple disciplines. This study addressed the following research questions:

• 1. How can English language demands in content courses be characterized in terms of the four competency skills (reading, listening, speaking, and writing)?

• 2. What are content course faculty members’ perceptions of the introduction of four-skill English language assessment for university entrance examinations?

Methods

Study site and participants

The study was conducted at a selective private university with a large student enrollment in Tokyo, which has been funded by MEXT as a Top Global University since 2014 (https://www.jsps.go.jp/english/e-tgu/index.html). With the 10-year funding, each Top Global University, including the study site, has been conducting various activities to promote the globalization of the university, such as increasing the numbers of Japanese students attending study-abroad programs, international faculty members, and academic courses taught in English (EMI courses). The faculty in which this study was conducted was not an EMI faculty. Thus, course instructors were allowed to use Japanese, English, or both as the medium of instruction at their will. A non-EMI faculty was selected as the study site because, despite the recent increase of EMI courses in universities in Japan, as mentioned above, a majority of universities in the country offer non-EMI content courses. Thus, taking this approach would allow this study to offer insights that are applicable to a fairly large number of universities currently offering the traditional style of content instruction.

A total of six full-time faculty members teaching content courses at the university participated in this study. The participants were specialists in mathematics and earth sciences (three participants in each discipline) in their 30s through 60s, with 2 to 34 years of previous experience of teaching university-level content courses. All participants were native speakers of Japanese. The faculty members reported on content courses they taught in two different undergraduate degree programs within the same faculty (with the enrollment of approximately 330 undergraduate students in mathematics and 190 in earth sciences) as well as their experience of teaching graduate-level courses and supervising graduate students in their home faculty and/or two others offering graduate degree programs involving the two disciplines. In terms of the entrance examination, both undergraduate degree programs employed the same English language test developed locally, which was designed to assess reading comprehension only.

Procedure

This author conducted one-on-one semi-structured interviews with the participants in Japanese at faculty offices on campus. All interviews were audio-recorded. Each interview lasted from 29 to 75 min, covering the main issues below:

• 1. A brief overview of various content courses participants were teaching or had taught previously at the time of their study participation;

• 2. The use of L1 and L2 in a selected content course as an illustrative example with a special focus on reading materials, class discussion, and course assignments;

• 3. The participant’s perception of four-skill English assessment for student admission.

For the second point above, the participant was encouraged to select the course involving English introduced earliest in their degree programs among courses taught by him/her. This is because English language demands in such a course are more directly relevant to the definition of the TLU domain that could serve as the basis for conceptualizing university entrance examinations than those in courses students take later in their degree programs.

Analysis

The author transcribed the audio-recorded interview protocols. The analysis of the protocols was qualitative and exploratory. First, the transcribed interview protocols were divided into segments, each of which represented an utterance concerning a single topic. Then, this author categorized each segment into the six broad themes below:

• Description of content courses taught by the faculty member (e.g., format, level, theme, and schedule): for all courses taught by the faculty member and the focal course described in detail

• Language use activities involving reading in English (e.g., reading assignments)

• Language use activities involving listening/speaking in English (e.g., classroom discourse)

• Language use activities involving writing in English (e.g., term papers)

• University entrance examinations

• Other (e.g., English language curriculum at the university, general issues about how to study English)

Then, for each theme, the content of all applicable segments was examined in an exploratory manner to identify subthemes that emerged from the interview discussion. For language use activities involving reading in English, for instance, the identified subthemes included input for reading assignments (content/amount of reading and availability/quality of the text), expected output of reading assignments, the effects of disciplinary knowledge on reading comprehension, the effects of language of reading on course achievement, and instructor support. The present author translated all excerpts and quotes from the interviews below from Japanese to English.

Results

An overview of the use of the four competency skills in instruction

Table 1 summarizes the use of English in the courses in mathematics and earth sciences that the individual participants chose to describe as illustrative examples in the interviews. For example, “UG3-4” for the course taught by participant 1 indicates that it was offered to students in the third and fourth years of the undergraduate degree program. Among the six courses described, one was an elective field study course open to undergraduate students of all levels, while the other five were required seminar courses offered at upper division undergraduate or graduate level². This reflects the fact that, according to the participants, few lecture/discussion classes targeting beginning undergraduate students in their disciplines involved English.

Table 1 The use of English in academic courses discussed by participants

The right-hand columns in Table 1 show the involvement of reading, listening, speaking, and writing in English in the six content courses. As can be seen in the table, reading in English was required in all courses, while the involvement of the other three skills (listening, speaking, and writing) was minimal. Exceptions were participant 4’s earth science course involving occasional, brief writing exercises (to be discussed later) and participant 6’s field study course, where students visited geological sites overseas and attended introductory lectures delivered in English by guest speakers on site. Participant 6 noted, however, that the required level of the listening ability to understand orally presented lectures in the field study course was not necessarily high because the Japanese course instructor concurrently provided explanations of the lecture content in Japanese, as required.

Reading

Reading in English was involved in all six mathematics and earth science courses in Table 1. Five participants adopted textbooks (participants 1, 2, 3, 5, and 6) as the main course reading material. Among them, participant 6 also employed supplemental instructor-made handouts to introduce basic terms and concepts in both English and Japanese. The only course that employed journal articles was participant 4’s earth science course, where each student selected a journal article for presentation in class according to his/her topic of interest.

Rationales for adopting texts written in English

Reasons mentioned by the participants for adopting texts written in English primarily concerned three issues. One was the high quality of English materials, which was described by two participants in earth sciences (participants 5 and 6) in terms of the abundance of high-quality graphics as well as the extensiveness, comprehensiveness, and logical presentation of the information. A second reason concerned the limited availability of suitable materials written in Japanese. Two participants in mathematics (participants 2 and 3) and one in earth sciences (participant 4) mentioned that, because many academic materials in mathematics and sciences are published in English, textbooks written in Japanese were limited in number, particularly beyond introductory levels. Participant 2 (mathematics) also raised the concern of the time lag between the publication of the English original and its Japanese translation. A third reason, raised by two participants in mathematics (participants 1 and 2) and one in earth sciences (participant 5), was to get students accustomed to reading in English because many advanced materials are available only in English, as noted above.

The effects of language of reading on students’ level of understanding

Two participants explicitly commented on various issues concerning this point. While participant 4 (earth sciences) had the perception that students’ understanding was “slightly lower” when reading texts in English than in Japanese, participant 2 (mathematics) felt that the language did not affect students’ level of understanding of the content. He reasoned that this might be partly due to the sufficiency of his students’ English reading skills for grasping text meaning.

One point on which all participants in both disciplines agreed was the difficulty of adopting English texts as the sole reading material soon after students start to take undergraduate courses in their disciplines due to the lack of students’ disciplinary knowledge and academic reading skills. Accordingly, they reported taking various measures to fill the gap. A strategy shared between two participants in mathematics (participants 1 and 2) was to select introductory textbooks for which published Japanese translation was available. Participant 1 emphasized its importance “for students to concentrate on mathematics” without demotivating them to learn the content. This participant stated further that students get accustomed to the discourse structure of a typical textbook or research paper section unique to mathematics (definition→example→remark→claim) around the fourth year of the undergraduate program with repeated reading practice and the accumulation of sufficient disciplinary knowledge.

Meanwhile, the three earth science faculty members described various other strategies to support students’ reading in English. Participant 5 suggested that, given the minimal disciplinary knowledge in-coming students typically have, “the appropriate timing” for introducing reading materials in English is the third year of the undergraduate program, when students have accumulated basic disciplinary knowledge through various introductory-level courses offered in Japanese. According to the participants, various types of support are provided to students during the first 2 years of the undergraduate program so as to ease their transition from Japanese to English in subsequent years. Such strategies included: distributing copies of the course textbook written in English used for third-year seminar courses to all first-year students, so they could start appreciating tables, charts, and pictures in the book (participants 5 and 6); teaching in Japanese the topics covered in the third-year course textbook in the second year (participant 5); providing instructor-made handouts for presenting technical terms and concepts both in English and Japanese along with their explanations in Japanese (participant 6); making introductory books on earth sciences for the general audience available in the laboratory (participant 5); and encouraging students to use bilingual dictionaries of earth sciences (participants 4 and 5).

Reading assignments

Weekly reading assignments in the mathematics and earth science courses focused on relatively short sections of the course textbooks or a journal article, where a two to five page reading of a textbook section or a journal article was typical. Many of the texts assigned to students in those courses involved formulae (both disciplines) as well as empirical data presented in tables and charts (earth sciences). Participant 1 reported that the reading materials were so dense that four to five pages would be the maximum that could be covered weekly. In all the five seminar courses, different sections of the texts were assigned to different students or student pairs for presentation in Japanese in class. The student or the student pair in charge was also responsible for explaining math problems included in the specific section (participants 1 and 2; mathematics) and for verifying the calculation of statistical analysis results when applicable (participant 4; earth sciences). Exactly how to conduct the presentation on the reading assignment varied across courses. In all mathematics courses the student or student pair in charge was required to explain key points of the specific section orally, using the whiteboard. In contrast, in the two seminar courses in earth sciences, the student prepared a handout containing a Japanese translation (participant 5) or a summary of the key points (participants 4). In the case of the field study course taught by participant 6, students worked in groups before leaving for the field trip to review various sections of the course textbook and discussed the content in Japanese to verify their understanding of the text while referring to the instructor-made handout on basic terms and concepts on geology mentioned above and the instructor’s brief oral explanations of them in Japanese. Generally speaking, the primary purpose of the reading activities in mathematics and earth sciences was to ensure students’ precise understanding of key concepts and technical terms. Students were instructed to attend to all details of the assigned text closely, looking up all unfamiliar words and phrases in a dictionary, where reading accuracy was emphasized over reading fluency. Throughout the interviews, the participants focused primarily on precise understanding of specific information in the text as opposed to grasping main ideas of a long stretch of discourse or synthesizing ideas within a text or across different texts. In terms of mathematics, this may be partly explained by participant 1’s comment that typically texts in mathematics are segmented into short sections with subheadings.

Listening and speaking

As can be seen in Table 1, in all mathematics and earth science courses, lectures, presentations, and discussions were conducted in Japanese, involving no speaking or listening in English. The exception was participant 6’s field study course in earth sciences mentioned above that involved listening to introductory lectures in English on site. As described above, however, discussions to prepare for the field trip led by the participant were conducted in Japanese.

All participants in both disciplines commented on the inefficiency of using English for conducting classes, where two primary issues emerged as specific reasons. One concerned the high cognitive load associated with aural/oral processing of information in English within the limited class time. Participant 1 (mathematics) specifically noted the lack of fluency in speaking and listening in English on the part of the instructor as well as students. According to him, L1 was the better medium of class discussion because it would allow them to keep up with the speed of their thinking.

A second issue explicitly mentioned by three participants (one in mathematics and two in earth sciences) emphasized the advantage of using L1 for engaging in reasoning, which allows deep processing of substantive content. This view is illustrated by the comment, “We should appreciate the value of Japanese more,” made by participant 3 in mathematics. This participant reported engaging in mathematical reasoning in Japanese all the time, despite his extensive previous experience of researching and teaching overseas. Interestingly enough, when asked for their rationales to support this view, participants 5 and 6 (earth sciences) referred to the educational context of Japan, where its citizens have received basic education and training in various fields in L1 since the establishment of the modern school system in the Meiji Era.³ Around the latter half of the nineteenth century, when the country was opened to other countries for trade, a variety of texts imported from overseas were translated into Japanese for learning about Western technologies and cultures. Both participants asserted that great achievements that Japan has made in technology and sciences since then are attributable to the existence of such translated texts, which allowed deep processing of the content in L1. The excerpt from the interview of an earth science participant below illustrates this point:

Even if there is a trend to promote content instruction in English, we should not stop using Japanese in class. Ever since information in many science areas was imported for the modernization of the country, many technical terms have been translated into Japanese... I believe that Japanese will continue to think in Japanese, so we should be able to discuss technical issues in Japanese... Although English dominates our professional discussion, it doesn’t necessarily mean that we don’t have to explain anything in Japanese. We need the ability to communicate the content in Japanese. We need the ability to do so both in English and Japanese. (participant 6)

Writing

As can be seen in Table 1, writing activities in English were involved in none of the courses except for one in earth sciences. Participants’ comments revealed that the involvement of academic writing in English was limited in the mathematics and earth science courses because grades for these courses were based primarily on class discussions and presentations (participants 1, 4, and 5) or reports written in Japanese (participants 2, 3, and 6).

The only course that involved writing in English was participant 4’s undergraduate seminar in earth sciences. While reading journal articles was the primary activity in this course, the last 10 min of the class were occasionally spent for sentence-level translation exercises from Japanese to English with a special focus on sentence constructions required for writing in the discipline. Participant 4 reported preparing slides for this writing exercise, which included topics such as parallel structures (e.g., respectively), noun phrases containing multiple adjectives modifying the noun (e.g., fine red particles, well-preserved fossil), reporting verbs (e.g., show, indicate, demonstrate), and voice (passive vs. active). For each topic, the first slide presented a Japanese translation of a few sentences participant 4 had sampled from journal articles in the discipline. This was followed by a second slide containing model answers, which was presented in class once students completed translating the original sentence into English.

Gaps in academic English language demands at the undergraduate and graduate levels

It is worth noting that, despite the exclusive focus on reading as opposed to the minimal involvement of listening, speaking, and writing in English at the undergraduate level, the great increase of academic English language demands at the graduate school level was pointed out by all participants, as illustrated by participant 1’s (mathematics) comment that “it is a challenge that English language demands change at the graduate school…to involve all four skills.” On one hand, one participant each from both disciplines (participants 1 and 5) explicitly commented on the demands of academic listening and speaking skills for professional communication, particularly for students to deliver oral presentations and handling questions and discussions in English at academic conferences. In addition, participants 2 and 3 (mathematics) and participant 6 (earth sciences) reported the importance of basic listening and speaking skills for interaction with international visitors to their laboratories on various topics, both academic and non-academic.

On the other hand, five participants (all except participant 2 in mathematics) commented on a sharp increase in the needs for academic writing as well. At the master’s level, some students start writing either abstracts of or the entire master’s theses in English, while they also start preparing poster presentations for conferences and journal manuscripts (participants 3, 5, and 6). To illustrate the difficulties students face at this stage, participant 3 in mathematics stated that students “get into trouble as soon as they start submitting manuscripts to international journals.” All participants’ comments suggested the great amount of time and efforts they put into advising graduate students on writing academic papers in English individually by providing comments on student paper drafts. Participant 4 (whose occasional use of sentence-level translation exercises from Japanese to English is described above) reported that this activity was an attempt not only to help students build skills on a discipline-specific style of writing but also to reduce the instructor’s workload for eventually editing students’ manuscripts. Yet, participants (1 and 3) in mathematics noted that writing could still be taught to some extent because of the fixed discourse organization comprising short sentences specific to the discipline and the availability of well-written published papers that could serve as models for students.

Content course instructors’ perceptions of assessing four skills in entrance examinations

At the end of their interviews, the participants were asked for their opinions about assessing the four skills as part of university entrance examinations. The answers of two participants each in the two disciplines (participants 2, 3, 4, and 5) indicated their positive attitudes toward this direction. To support this view, participants 2 and 5 commented on the importance of English language ability for conducting research later in the program. At the same time, however, participants 2, 3, and 4 noted the critical importance of reading among the four skills, commenting that assessing only reading might suffice for now. Meanwhile, participant 1 (mathematics) explicitly expressed concerns about assessing all four skills because “reading should definitely be emphasized” in university entrance examinations due to the critical importance of reading as the basis for coursework and research. This participant also expressed his fear of deemphasizing reading as a result of assessing all four skills because accepting students with “mediocre or lower ability levels across the board” would make content instruction ineffective. Participant 6’s (earth science) view was more nuanced. While explicitly endorsing the assessment of multiple skills (reading, listening, and writing), this participant felt that only reading and writing could be assessed due to the concern of practicality.

Discussion and conclusions

This small-scale interview study conducted at a private university in Japan examined English language demands in content courses in mathematics and earth sciences and course instructors’ perspectives on the introduction of four-skill English language assessments to university entrance examinations in Japan. Various commonalities were observed in terms of English language demands between the two disciplines. A notable similarity was the strong emphasis on academic reading ability required to complete course reading assignments as opposed to a minimal representation of listening, speaking, and writing skills at the undergraduate level. On many grounds, the participants in both disciplines shared their rationales for the ways in which reading materials in English were assigned to students, and L1 was used as the primary vehicle for class discussions. Furthermore, in both disciplines, a sharp increase of English language demands was reported at the graduate level, which required academic English skills encompassing all four skills not only for earning course credits but also for writing theses and journal manuscripts as well as delivering oral presentations and participating in discussions at academic conferences. Finally, the participants were generally favorable about the introduction of four-skill assessments into university entrance examinations due to advantages of high English language ability on course achievement and conducting research in a long run, although the importance of reading was emphasized.

Two issues concerning the English language demands in the four disciplines are worth noting. First, the delayed introduction of reading materials written in English (in the third year of the undergraduate programs) due to the critical importance of building topical knowledge in L1 reported by the study participants may be attributable to the nature of the knowledge-construction process in sciences described as a vertical discourse (Bernstein, 1999; Kuteeva and Airey, 2014), where knowledge is hierarchically organized and accumulated for the construction of general theory shared across contexts. While earth sciences and mathematics seem to share some features of knowledge construction practices with other disciplines in the sciences, Bernstein and Kuteeva and Airey also note the uniqueness of the mathematics discourse featured by its distinct symbolic codes and reasoning style (e.g., mathematical proofs) rather than empirical data analyses. These salient features of mathematics discourse may partly explain the reason why two mathematics participants in this study explicitly stated that students learn to read textbooks and write papers in English through familiarizing students with conventions of the written discourse in the discipline to some extent.

Second, the heavy emphasis on reading as opposed to listening, speaking, and writing in mathematics and earth sciences seems to be explained by Hino’s (2016) point that “English used to be perceived in Japan as a tool for absorbing advanced Western technologies and cultures” (p. 28). Coupled with the perceived effectiveness of using L1 for lectures, discussions, and reasoning explained in relation to the modernization of the country by some participants, the content faculty members’ perspectives appear to be shaped, in a way, by traditional values of Japanese history and culture. Yet, an intriguing point is another perspective of the study participants, whose comments suggested their commitment to training students as functional members of the professional communities in their disciplines, both domestic and international. This was particularly evident in their comments on the importance of developing discipline-specific literacy in both L1 and L2 concerning listening and speaking for academic communication as well as the various challenges they face in advising students on academic writing at the graduate level.

To summarize the study results, first of all, despite the seeming incongruence between the imbalanced representation of the four skills in the content courses as opposed to the direction for revising university entrance examinations suggested by MEXT, it was of interest that a majority of the study participants were generally positive about the adoption of four-skill assessment, consistent with the views of high school students and teachers and university faculty members in Green’s (2014) study. As noted by one anonymous reviewer of this article, it would be a challenge to design and implement a “democratic test” that reflects views of different stakeholder groups. This is particularly the case when stakeholder perspectives are incongruent with one another (Akiyama, 2004; Green, 2014). In the case of this study site, a potential conflict between the two stakeholder groups’ viewpoints (those of MEXT and content course faculty members in mathematics and earth sciences) does not seem to be a major issue of concern. However, caution should be exercised in interpreting the study results because this study is based on a small sample and does not cover perspectives of other important stakeholder groups (e.g., students and school administrators at the university).

Second, although the scope is limited, results of this study offered further insights into how such an assessment should look in the specific university context. As an attempt to explicate the TLU domain for entrance examinations at the study site, Fig. 1 graphically summarizes the English language demands in mathematics and earth sciences reported by the study participants in relation to general English that students learn through secondary school and university EFL instruction. The two concentric circles to the right represent the English language demands at the undergraduate level (the inner circle; areas C and D) and those at the graduate level (the outer circle; areas B and E), while the circle to the left depicts the instructional domain (Bachman and Palmer, 2010) corresponding to general English language ability that students develop through secondary school and university EFL instruction (areas A, B, and C). The study results show that English language demands at the undergraduate level seem to be represented mostly by language use tasks involving reading, which comprises basic reading skills that the faculty members specified as important for precise understanding of text meaning (area C) and academic reading tasks for completing course reading assignments (area D). Meanwhile, English language demands at the graduate level are extended to more advanced reading activities as well as to those involving listening, speaking, and writing. The academic listening/speaking and writing skills required for conference presentations and writing research papers in these two disciplines correspond to area E, and non-academic, interpersonal communication skills for interaction with international scholars to area B. While the language use tasks employed in the secondary school and university EFL curriculum may not fully overlap with the English language demands at the undergraduate and graduate levels (area A), students are expected to learn general English language ability that lead to the academic English ability required in these disciplines through EFL instruction (areas B and C).

Fig. 1

The target language use (TLU) domain for mathematics and earth sciences at the study site

In defining specific TLU tasks that a university entrance examination should target in the case of the study site, it would be of primary importance to represent the portions of areas A, B, and C in Fig. 1 attributable to the secondary school EFL curriculum. This is because, as noted in the introduction section, an entrance examination may be designed to play the dual functions of assessing achievement of secondary school EFL (areas A, B, and C) and basic language ability required for further studies at the university (areas B and C). Furthermore, the degree of coverage of areas B, and C in the university entrance exam should be determined in relation to the extent to which the university EFL instruction can bridge the gap between secondary school EFL and the English language demands in university content courses. In contrast, including D and E in the entrance examination would not be feasible because they require the use of English assuming disciplinary knowledge.

Key results of this study could further explicate what TLU tasks areas B and C may entail as well. On one hand, Area C may focus, for instance, primarily on the degree to which students can execute careful reading operations as defined in Khalifa and Weir’s (2009) cognitive processing model of reading comprehension for both local and global text comprehension featured by “slow, careful, linear, incremental reading for comprehension” (p. 46). This type of reading could be tested based on introductory academic texts or those written for general audiences due to their immediate needs at the undergraduate level.  Khalifa and Weir’s expeditious reading operations (skimming, scanning, search reading) for understanding the gist and specific details of interest might also be included to some extent because they might become important as the amount of reading increases and the purpose of reading diverse toward the graduate level work. Accurately understanding the text meaning in relation to information presented graphically (e.g., tables and charts) may be another important TLU task to consider as well. As for area B, while there does not seem to be an immediate need for advanced academic listening, speaking, or writing skills soon after starting undergraduate studies, including basic oral/aural interaction and writing tasks for general purposes might be considered. This is because the ability to execute such tasks would smooth out the reported difficulties associated with the transition from the undergraduate level to the graduate level, enhancing students’ skills for interacting with international researchers and writing for specific, academic audiences in the future. On the other hand, to what extent and how the potential language use activities in areas B and C above are actually included in the entrance exam should also be determined by considering the nature of EFL instruction provided as undergraduate foreign language requirements. In the case of the study site, various EFL courses for cultivating language ability in all four skills are offered. However, the small number of EFL courses that individual students are required to take and the primary emphasis on general English as opposed to academic English in the courses may partly explain the difficulties in developing discipline-specific English language ability reported by the study participants. Therefore, it may be reasonable to consider representing all the above candidate tasks in the entrance exam so as to promote secondary school students’ awareness of important TLU tasks in the disciplines in which they intend to specialize, while re-examining the role of the university EFL curriculum and foreign language requirements in the degree programs. Doing so might contribute to bringing about positive consequences on the EFL and content instruction at the study site.

Given that the evaluation of the feasibility of this top-down reform in Japan requires reality checks by stakeholders who act as de facto agents of change locally, needs analysis investigations of various types should ideally be conducted by individual universities and degree programs from multiple perspectives. As noted by McNamara (1996), a needs analysis like the present study could be supplemented with an analysis of spoken and written texts involved in content courses, for instance. Another viable approach would be to examine competing perspectives of different stakeholder groups, as Akiyama (2004) did in his mixed-methods study on the introduction of speaking assessment to an upper secondary school entrance examination in Japan. Such active involvement of stakeholders in discussions related to this reform would facilitate the implementation of an English assessment that serves stakeholders’ decision-making well, allowing meaningful test score interpretation and use in a specific local context.