Problems With Section Two ITP TOEFL Test

This study was designed to investigate (1) the difficulties faced by EFL university students with section two of the ITP, and (2) whether part A or part B was more difficult for them and why. A number of 26 students from two different universities, Syiah Kuala University and the State Islamic University Ar-Raniry were the samples for the test. The data was obtained from a multiple choice questionnaire test consisting of 46 questions, each with 4 answers to choose from. The results showed that inversions (12%), subject-verb agreements (10%), adverb clause connectors (7%), passives (6%), reduced adjective clauses (5%), parallel structures (5%) and use of verbs (5%) were the most difficult questions for the students. Furthermore, they felt that part B was more difficult than part A, as finding an error in a sentence was harder than completing a sentence from a multiple choice. Furthermore, the length of questions in part A did not affect the amount of time the students spent to complete part A and did not cause them to panic. Also, unfamiliar words in part A were not regarded as a problem by the students. Hence, TOEFL teachers and trainers are highly encouraged to pay more attention to doing study exercises for the seven topics with the highest percentages above in part A and also to more practice for part B.


INTRODUCTION 1.1 Background
The Test of English as a Foreign Language (TOEFL) is a test administered to measure the English proficiency of non-native speakers of English.As Phillips (2001, p. xiii) asserts that "the TOEFL test is a test to measure the level of English ability of use to improve their ITP score.As regards the theoretical contribution, this study is expected to enrich the theories of the problems faced by students in the structure and written expression part of the ITP test so that it can be used as a reference for further studies conducted by other researchers.

Research Questions
This study was carried out to answer the following two research questions: 1.What are the difficulties faced by EFL university students in section 2 of the ITP test? 2. Which part of section two in the ITP is considered more difficult by these students, part A or part B? Why?

Research Objectives
Based on the formulation of the two research questions above, it can be understood that the objectives of this study were to find out what kind of difficulties were faced by Acehnese EFL university students in the structure and written section of the ITP TOEFL test and to figure out which part of section two was more difficult for these EFL university students, part A or part B together with the reasons why these participants found one part more difficult than the other.

Brief History of TOEFL
The Test of English as a Foreign Language (henceforth, TOEFL) is a test which is standardized to measure the English proficiency of non-native speakers of English.This test was first started by an American Council for the testing of English as a foreign language in early 1962 (Sulistyo, 2009).Since its establishment, TOEFL has developed several versions due to shifts in the views of the nature of language.
The first TOEFL was the Paper and Pencil Based TOEFL (PBT).It was developed on the basis of the belief that language comprises two layers: the layer of form and that of meaning (Sulistyo, 2009).In this version of TOEFL, good mastery of English grammar was ultimately important.It can be seen from the format of the test.The PBT had three sub-sections: (1) the listening section, (2) the structure and written expression section, and (3) the reading comprehension section.Another two tests, commonly tested separately, were the Test of Written English (TWE) and the Test of Spoken English (TSE) (Sulistyo, 2009).In the listening section, there were three parts: part A is the comprehension of fragmented sentences, part B is the comprehension of dialogues, and part C is the comprehension of texts larger than dialogs/monologs.In part A, accuracy of common grammatical points were tested (Sulistyo, 2009).In section two, structure and written expression, it was crystal clear that test takers must have an adequate knowledge of English grammar without which they would be sure to get a low score in this section.From the explanation above, we can understand that the PBT still put great emphasis on the form of language.
However, a need to assess the test takers' proficiency in using English communicatively found there were shortcomings in the PBT and the next version of TOEFL, the computer based TOEFL (CBT) was developed.In the early part of its development, however, the CBT also still included sub-sections similar to the PBT: listening, structure and written expression, and reading comprehension as well as TWE (Test of Written English) and TSE (Test of Spoken English) were both tested separately.The major difference was that while the PBT was paper based, the CBT was computer based.In the later development of the CBT, more emphasis on communication skills was prioritized.It can be seen from the tested sections which include macro skills: listening, speaking, reading and writing.In the most recent development of the CBT, the concept of language as a means of communication was viewed as highly important (TOEFL internet based test, 2007, as cited in Sulistyo, 2009).
The development of the CBT gave birth to the new version of TOEFL, so called internet based TOEFL (IBT) in 2005.Linguistically, the rationale behind IBT and CBT is the same, namely to test the students' ability to use language communicatively and functionally.The difference is that in the IBT academic settings and themes are more emphasized (Sulistyo, 2009).To summarise, the creation of the IBT has the purpose to assess the ability of the test takers' to use English for speaking, orally.

Institutional Testing Program (ITP)
Aside from the three versions of TOEFL elaborated above, there is one other version called ITP, which has become the focus of this study.Basically, ITP includes the same sub-tests as does the PBT explained above, but in the ITP, there are no TWE and TSE.In line with this, Sharpe (2004) reckons that ITP and PBT are quite similar in terms of their length, format, and level of difficulty.
In ITP there are three sections: listening, structure and written expression, and reading comprehension.The following subheadings deal with the explanation of these three sections.However, The examples of conversations, dialogue, reading passages, and questions are not given here so to have a clear look at what all these look like go to Pyle and Page (2002); Philip (2001); TOEFL ITP assessment series (2013).

Section One: Listening Section
The purpose of the listening section is to assess the ability to comprehend spoken English (Pyle & Page, 2002).There are three parts to the listening section: Part A, with short dialogues containing 30 questions, Part B with longer dialogues consisting of 8 questions, and part C, a lecture with 12 questions.In all parts of these sections, the conversations and dialogues are not played a second time so that test takers need to listen to the recordings very carefully and attentively.Then, they choose one out of four possible answers for each of the 50 questions.The time allotted for the listening section is 30 to 35 minutes (Pyle & Page, 2002).

Section Two: Structure and Written Expression
The structure and written expression section was developed to test the ability to identify appropriate language for standard written English (TOEFL ITP assessment series, 2013).This section entails two parts: Part A, Structure, and Part B, Written Expressions.In Part A, there are 15 incomplete sentences; beneath each sentence are four words or phrases, marked (A), (B), (C), and (D).Test takers are to choose the one word or phrase that most perfectly completes the sentence.On the answer sheet, find the number of the question and fill in the corresponding blank space with the letter for the answer they have chosen (Pyle & Page, 2002).Subsequently follows Part B, which is the written expressions, with 25 questions which each having four underlined words or phrases.The four underlined parts of the sentence are marked (A), (B), (C), and (D).Test takers are to choose the one underlined word(s) or phrase that must be changed so that the sentence will be correct.This section usually lasts for 25 minutes.

Section Three: Reading Comprehension
The last section is reading comprehension whose purpose is to measure the ability to read and comprehend short passages which uses the same topics and styles as most universities and colleges (TOEFL ITP assessment series, 2013).There are usually four to five long passages, each of which commonly has about eight to twelve questions.Test takers are asked to answer these questions which are usually related to main ideas of the whole passage, vocabulary, detailed information, inferences and reference items.The time given for this section is 50 minutes.

Previous Related Studies
It is true that the ITP TOEFL test is still a serious challenge for students as scientifically proven by previous studies (Mahmud, 2014;Hambali, 2008).In his study, Mahmud (2014) employed semi-structured interviews and classroom observations to investigate the problems faced by students in the TOEFL tests and the nature of their difficulties.Not surprisingly, he found that the students confessed that the TOEFL test was very difficult for them to pass due to a low level of basic skills, less practice, less motivation, and individual differences amongst students such as age and family status.
Another study was carried out by Hambali (2008).Through participant observations at the Graduate School at Sriwijaya University during the TOEFL preparation class held in their language laboratory, he found that many students had problems in TOEFL, such as with identifying verbs, identifying the correct word order of subject and verb in a noun clause, and identifying words with derivations and their functions.He further went on to say that the hardest problems mainly occurred in section two of the ITP TOEFL test.

Scope of Study
This study focused only on investigating the problems that students had in section two of the ITP TOEFL, dealing with structure and written expression and the reasons why the students found one of the parts in section two of the ITP TOEFL (part A or part B) to be more difficult than the other; as a result, problems with listening and reading comprehension were not considered.The location of this research study was in the English language Department at Syiah Kuala University.

The Theoretical Framework
The theoretical framework for this study was based on the grammar topics commonly tested in ITP TOEFL taken from Phillips (2001) as presented above.The topics with high frequency of errors were categorized as the most serious problems faced by English learners in section two of the ITP TOEFL.

The Choice of Respondents
In a bid to ensure the validity of the data, the samples chosen were 26 English students from Syiah Kuala University and from the National Islamic University Ar-Raniry who had taken the actual TOEFL test so that they could give reasons, based on their experiences, why they considered one part of section two in the ITP TOEFL test to be harder than the other part.To choose the sample, the researcher employed a random sampling procedure in which the students were chosen randomly.The reason for choosing this procedure was that by using random sampling, the result can be generalized for the entire population (Creswell, 2003).

Test
The test items in this study were adopted from Phillips (2001) with some modifications.The writer used several ways to collect the data for the study.First, the researcher gave a test to the participants consisting of the topics most often tested in the actual TOEFL test in the form of multiple choice questions like the TOEFL tests.The test included 23 topics as presented above, each with two questions (the test is set out in Appendix 1).Therefore, the total number of questions was 46 questions that had to be answered within 50 minutes.This test was used to answer the first question of this study.

Questionnaire
The questionnaire was designed based on the findings of Abboud and Hussein (2011) with modifications to suit the study objectives.It had six questions pertaining to the reasons why each student found one of the parts in section two of the ITP TOEFL test (part A or part B) to be more difficult than the other.Accordingly, this questionnaire provided answers for the second question of this study.

Data Analysis
The data were analyzed through several stages.For the first instrument, the test, the researcher began by checking the answers given by the students.After that, he classified the correct and incorrect answers.Next, he counted the percentage of errors in each topic by using a formula given by Sudijono (2010): In which: P = percentage of errors in each topic f = the number of wrong answers in each topic N = total number of all wrong answers The topics with the highest percentage of errors became the most serious problems for the students.In addition, the data analysis for the questionnaire was conducted by classifying the proportions of the samples choosing "yes" and those choosing "no" for each item to ease the researcher in analysing and interpreting the data (Aridah, 2004).

Analysis of the Test
The results from the data analysis are shown in Figure 1   As shown in Figure 1, there were seven topics that had the highest percentage of errors.The higher the percentage of errors that a topic has, the harder it is for students.The first topic with the highest percentage was "inversion" amounting to 12%, followed by "subject-verb agreement" (10%).In addition, the topic "adverb clause connectors" (7%) was the topic with the third highest percentage of errors, whereas, "passive" is the fourth topic with 6% of the total errors.Meanwhile, the other three topics were "reduced adjective clauses", "parallel structures", and "use of verb" or "misuse of verbs", each of which received 5%, respectively.

Questionnaire Results
Table 1 shows the results from the questionnaire.From the results of the questionnaire in Table 1, when the samples were asked if part A or part B is more difficult, there were 12 samples answering "yes" and 14 "no".It can clearly be interpreted that part A is not more difficult than part B and thus, part B is more difficult than part A. In addition, question 2 revealed that more samples ( 19) answered "no" if part B is easier because finding an error is easier than completing the sentence.From this response, we can understand that finding an error in sentences is harder than just completing sentences, hence part B is harder.
It was also found that the number of samples choosing "yes" and that of samples answering "no" is similar when asked whether part A is more difficult because it consists of long sentences so that it takes time to read and answer it.It can be seen that for some samples, the length of the questions makes Part A more difficult than part B because they would spend much time reading it, whereas for the others, the length of questions in part A is not a matter.Pertaining to question 5, most of the samples (19) answered "no" that part A is more difficult than part B because they feel lazy to read long sentences.This response revealed, too, that the length of sentences did not affect the attitude toward part A, making it no harder than part B in section two of the ITP TOEFL.
Moreover, there were more samples (15) answering "no" than those choosing "yes" when asked question 6 regarding whether part A is harder than part B because its sentences contain a lot of unfamiliar words for them.It can be understood that the familiarity of words was not a source of difficulty to answer part A. Finally, question 6 showed that most of the samples (19) chose "no" when asked whether they quite often panic when they see long sentences like in Part A. Thus, we can understand that the lengthy sentences in part A did not make the samples panic and lose their concentration to finish part A.

DISCUSSION
The results of the test have strived to answer the first question of this study.As the test results suggests, students encountered serious problems in answering (1) inversion, (2) subject-verb agreement, (3) adverb clause connectors, (4) passive, (5) reduced adjective clause, parallel structure, and use of verbs.Inversion (12%) came as the first topic with the highest percentage of errors, followed by subject-verb agreement (10%), adverb clause connectors (7%), and passive (6%).However, the other topics of reduced adjective clause, parallel structure, and use of verb each had 5%, respectfully.
The findings from this study have both similarities and differences with prior studies.The similarity is that the result of this study was in line with the findings of some other research (Mahmud, 2014;Hambali, 2008) that Section B of the ITP test, Structure and Written Expression, was found to be difficult for the learners.Thus, it is very important for learners to enhance their ability and knowledge in structure and written expression by making some efforts, for example joining one or more TOEFL preparation classes (Hambali, 2008).
Furthermore, it is quite pivotal for students to understand that learning the TOEFL is not only to fulfill a requirement for academic purposes or the likes, but also to be regarded as self-improvement (Hambali, 2008).In addition, students also have to raise awareness toward the importance of budgeting much time to increase their TOEFL score, especially in part B. In relation to this, Brown (2000) contended students should be willing to spend their "personal time, effort, and attention" when learning a second language as these are deemed essential.
Interestingly, nevertheless, this study found something new, compared to the research carried out by Hambali (2008).In his study, he revealed that the topics that the students found problematic were: (1) identifying verbs, (2) identifying the correct word order of subject and verbs in noun clauses, (3) identifying words with derivations and functions.Meanwhile, the subjects of this study encountered hurdles in answering "inversions", "subject-verb agreement", "adverb clause connectors", "passive", "reduced adjective clauses", "parallel structures", and "use of verbs".
Pertaining to the finding from the questionnaire, we can see that in section two of ITP TOEFL, part B, written expression, was considered more difficult by most of the subjects than part A, the structure part.Again, interestingly, this finding contradicted with the finding of the study undertaken by Abboud and Hussein (2011) in which in their study, they found that the subjects considered part A to be harder than part B. Accordingly, the issue on whether part A or part B is more difficult for students is still debatable and may need further research.
The reason why the students in this study said Part B is harder than part A was that they said finding errors (part B) was harder than completing sentences (part A).On the contrary, the finding of Abboud and Husein (2011) proved that finding an error in part B in section two of TOEFL is easier than completing sentences in part A. Moreover, we can also see that the length of the sentences in part A in section two of ITP TOEFL does not make it more difficult than part B for the students in this study.Also, the students acknowledged that the length of the sentences in part A did not make them lazy to read the questions and/or cause them to panic.Also, unfamiliar words found in the questions of part A were not a problem for the students.Taking all these reasons into considerations, these students found that part B was more difficult than part A of section two in the ITP TOEFL test.

CONCLUSION
From the results of the test and the questionnaire, it can be concluded that of all the topics commonly included in section two of the ITP TOEFL test, there were seven topics which were the most difficult for the students.They were: "inversions", "subject-verb agreement", "adverb clause connectors", "passives", "reduced adjective clauses", "parallel structures", and "use of verbs".Other studies have found that students had troubles with other topics in section two of the ITP TOEFL test.However, at least the findings of this study have much in common with other studies that also found section two of the ITP TOEFL test is still a problem for test takers.Hence, it is strongly suggested that TOEFL teachers should pay more attention to this section, especially to the topics with the highest percentages of errors when teaching TOEFL to their students so that their students can answer these questions correctly in the future.
Furthermore, although the previous study revealed that part A proved to be harder than part B, this study indicated the opposite, that B seemed to be more problematic for the students.To summarize, teachers need to give more emphasis to teaching part B to their students to help them gain a high score in the ITP test.
Finally, the researcher fully realizes that this study still has some limitations.First, the population in this study was small and localized so that the findings from only a few students cannot be generalized to a much wider population.Furthermore, as stated in the discussion section, the findings of this study are contrary to the findings from other prior research.Finally, the number of questionnaire items used was small so that this study might only yield a few reasons for whether part A or part B is more challenging for students.Taking all these into consideration, the researcher strongly encourages other researchers to conduct further studies involving a larger number of subjects and using interviews and/or questionnaires with more items so that the findings of the next research can be more objective and applied to a wider context.Furthermore the researcher suggests doing research to see whether doing sample tests with larger number of times will make it much easier for students to get a high score in the ITP TOEFL test.In other words to what extent can more and more practices make perfect! below.

Figure 1 .
Figure 1.Seven topics with the highest frequency of errors.