Evaluating paper-based TOEFL preparation program using the Context, Input, Process, and Product (CIPP) model

Abdul Manan, Muhammad Arif Fadhilah, Kamarullah Kamarullah, Habiburrahim Habiburrahim

Abstract


An evaluation for the development of a paper-based test of English as a Foreign Language (TOEFL) for English Education Department students at one of the colleges in Langsa, Aceh, Indonesia, is necessary, considering their unsatisfactory scores even though the preparation program has been conducted. This research employed context, input, process, and product (CIPP) model developed by Stufflebeam upon the program. Methodologically, interviews, questionnaires, and observation were executed to five TOEFL lecturers and 34 students, while the data obtained were analyzed with the qualitative analysis method of Miles and Huberman. Based on the evaluation of context, the program had an appropriate background, goals, and objectives. Regarding the input, it still had problems related to the availability of resources and facilities. There were no standardized learning materials available, including the broken language laboratory. It affected the process since the lecturers could not maximize the facilities to create an effective learning environment. Eventually, it affected the product since most students could not reach an expected score. Most of the participants agreed that the program needed to be improved. Several improvements can be made to overcome these problems by providing required learning facilities, designing standardized learning materials, and clustering classes in accordance with students’ abilities.


Keywords


CIPP; language testing; paper-based TOEFL

Full Text:

PDF

References


Akmal, S., Risdaneva, R., Habiburrahim, H., & Sari, M. (2020). The English teachers’ challenges in TOEFL preparation for senior high school students. Journal on English as a Foreign Language, 10(1), 25-45.

Amurao, A. L. (2011). The types and nature of questions vis-à-vis students’ test-taking skills as significant indicators of second language examinees’ performance on the TOEFL-ITP Reading Comprehension sub-test. The International Journal of Research and Review, 6(2). 39-56.

Ananda, R. (2016). Problems with Section Two ITP TOEFL Test. Studies in English Language and Education, 3(1), 37-51.

Brown, H. D. (2001). Teaching by principles. Longman.

Chen, C.F. (2009). A case study in the evaluation of English training courses using a version of the CIPP Model as an evaluative tool [Unpublished doctoral thesis), Durham University.

Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (6th edition). Routledge.

ETS. (2019). About the TOEFL ITP assessment series. www.ets.org/toeflitp

Farkhan, M., Nurlia, V., & Haucsa, G. M. (2019). University students’ perception on TOEFL as a graduation requirement: A case in UIN Syarif Hidayatullah Jakarta. Elite Journal, 1(1), 51-64.

Farooqui, S. (2007). Developing speaking skills of adult learners in private universities in Bangladesh: Problems and solutions. Australian Journal of Adult Learning, 47(1), 94-110.

Flowerdew, J., & Miller, L. (2005). Second language listening: Theory and practice. Cambridge University Press.

Garcia, Y. P., Ramayan, A. J., Sepe, A. P., & Silor, A. C. (2014, March 4-5). Difficulties in reading comprehension and metacognitive strategies for technology livelihood education students [Paper Presentation]. Global Summit on Education, Kuala Lumpur, Malaysia.

Hakan, K., & Seval, F. (2011). CIPP evaluation model scale: Development, reliability and validity. Procedia-Social and Behavioral Sciences, 15, 592-599.

Kaneko, M. (2014). Is the vocabulary level of the reading section of the TOEFL internet-based test beyond the lexical level of Japanese senior high school students?. Vocabulary Learning and Instruction, 3(1), 44-50.

Karimnia, A., & Kay, E. (2015). An evaluation of the undergraduate TEFL program in Iran: A multi case study. International Journal of Instruction, VIII(2), 83-98.

Kasim, U. (2016, November 12-13). The implementation of TOEFL score as a requirement for script examination at Syiah Kuala University [Paper Presentation]. English Education International Conference, Banda Aceh, Indonesia.

Kokhan, K. (2012). Investigating the possibility of using TOEFL scores for university ESL decision-making: Placement trends and effect of time lag. Language Testing, 29(2), 291-308.

Liu, O. L. (2014). Investigating the relationship between test preparation and TOEFL iBT performance. ETS Research Report Series, 2014(2), 1-13.

Lodhi, M. A., Sahar, A. H., Qayyum, N., Iqbal, S., & Shareef, H. (2019). Relationship of school environment and English language learning at government schools. Public Administration Research, 8(1), 1-13.

Mahmud, M. (2014). The EFL students’ problems in answering the test of English as a foreign language (TOEFL): A study in Indonesian context. Theory and Practice in Language Studies, 4(12), 2581-2587.

Manan, A., & Azizah, A. (2016, November 12-13). Formative evaluation of the English structure of English department curriculum at teacher training faculty, the State Islamic University (UIN) Ar-Raniry [Paper Presentation]. English Education International Conference, Banda Aceh, Indonesia.

Manganello, M. (2011). Correlations in the new TOEFL era: an investigation of the statistical relationships between iBT scores, placement test performance, and academic success of international students at Iowa State University [Unpublished master’s thesis]. Iowa State University.

McGinnis, A. M. (2007). The role of attention in second language development [Unpublished master’s thesis]. University of Montana.

Miles, M. B. & Huberman H. M. (1992). Analisis data kualitatif: Buku sumber tentang metode-metode baru [Qualitative data analysis: A source book on new methods]. UI Press.

Ornstein, A. C., and Humkins F. P. (2004). Curriculum: Foundations, principles and issues. Prentice-Hall.

Pan, Y. C. (2015). Test impact: English certification exit requirements in Taiwan. TEFLIN Journal, 20(2), 119-139.

Salimi, E. A., & Farsi, M. (2016). Program evaluation of the English language proficiency program for foreign students a case study: University of the East, Manila Campus. English Language Teaching, 9(1), 12-21.

Samad, I. A., Jannah, M., & Fitriani, S. S. (2017). EFL students’ strategies dealing with common difficulties in TOEFL reading comprehension section. International Journal of Language Education, 1(1), 29-36.

Scheerend, J., Glas, C., & Thomas, S. M. (2003). Educational evaluation, assessment, and monitoring: A systematic approach. Swets and Zeitlinger Publishers.

Shih, Y. C., & Yuan, Y. P. (2019). Evaluating an English Elite Program in Taiwan using the CIPP Model. Journal of Asia TEFL, 16(1), 200-219.

Shiu, L. J. (2011). EFL learners’ perception of grammatical difficulty in relation to second language proficiency test [Unpublished doctoral dissertation] University of Toronto.

Silviyanti, T. M., Rahmadhani, R., & Samad, I. A. (2020). EFL students’ strategies in answering the Listening Section of the Longman TOEFL. Studies in English Language and Education, 7(1), 237-246.

Singh, H., Thamilvili, G., Zuraimi, N. A. B., & Veliapandian, Y. (2014). How to improve on the Malaysia University English Test (MUET). Asian Journal of Education and e-Learning, 2(06). 485-489.

Souriyavongsa, T., Rany, S., Abidin, M. J. Z., & Leong, L. M. (2013). Factors causes students low English language learning: A case study in the National University of Laos. International Journal of English Language Education, 1(1), 179-192.

Stufflebeam, D. L. (2000). The CIPP Model for evaluation. In D. L. Stufflebeam, G. F. Madaus, & T. Kellaghan (Eds.), Evaluation models: Evaluation in education and human services, (pp. 279-317). Springer.

Stufflebeam, D. L. (2003). The CIPP Model for evaluation. In T. Kellaghan, & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 31-62). Springer.

Stufflebeam, D. L., & Shinkfield, A. J. (1985). Systematic evaluation. Kluwe-Nijhoff.

Takeuchi, W. (2020). Guide to the 2020 Entrance Examination Department of Civil Engineering Graduate School of Engineering. The University of Tokyo.

Tunc, F. (2010). Evaluation of an English language teaching program at a public university using CIPP model [Unpublished master’s thesis]. Middle East Technical University.

Ulum, Ö. G. (2015). A needs analysis study for preparatory class ELT students. European Journal of English Language Teaching, 1(1), 14-29.

Wagener, D. (2006). Promoting independent learning skills using video on digital language laboratories. Computer Assisted Language Learning, 19(4-5), 279-286.

Walsh, M. B. G. (1995). A comparison of Instituto Tecnológico y de Estudios Superiores de Monterrey’s EFL exit examinations and the TOEFL [Unpublished master’s thesis). Instituto Tecnológico y de Estudios Superiores de Monterrey

Wang, Y. T. (2019). The impact of TOEFL on instructors’ course content and teaching methods. TESL-EJ, 23(3), 1-18.




DOI: https://doi.org/10.24815/siele.v7i2.16467

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Print ISSN: 2355-2794, Online ISSN: 2461-0275

This work is licensed under a Creative Commons Attribution 4.0 International License.


View Journal Stats