Evaluating the impact of AI debater on EFL debate competence: Perceptions of lecturers and learners

Ranta Butarbutar, Andi Haeriati Alimuddin, Seli Marlina Radja Leba

Abstract


Debate competence is essential in English as a Foreign Language (EFL) learning, yet many learners struggle with critical thinking, argumentation, and persuasive communication. While artificial intelligence (AI) offers promising support, evidence of its effectiveness in debate learning remains scarce. This study aims to evaluate the impact of AI Debater technology on the debate performance of EFL learners and explore the perceptions of its use among lecturers and learners. A quasi-experimental design was employed, involving 20 EFL learners and one lecturer, with data collected through observations and online interviews. Post-test results showed significant improvements in debate performance, with mean scores of 77.45 for content, 78.75 for arguments, and 79.35 for claims. Learners also reported positive perceptions: 85% rated the AI Debater as very useful, while 80% and 75% valued its role in developing claims and arguments, and 90% evaluated content positively; however, only 50% found it highly effective for collaboration. Complementing these findings, lecturers perceptions emphasized three areas: (1) reasons for choosing the AI Debater, such as accessibility, up-to-date information, suitability for 21st-century learners, and time efficiency, (2) its impact, including improved critical thinking, reasoning, negotiation, analytical skills, and persuasive communication, and (3) challenges, such as reliability issues, lack of creativity and empathy, and concerns about educational integrity. While AI Debater shows potential for enhancing debate skills, the absence of a control group, small sample size, and limited consideration of contextual factors constrain findings. Future research should address these gaps to deepen the understanding of AI in EFL debate learning.

Keywords


AI Debater; connectivism, 21st century skills; communicative proficiency; debate competence

Full Text:

PDF

References


Abulibdeh, A., Zaidan, E., & Abulibdeh, R. (2024). Navigating the confluence of artificial intelligence and education for sustainable development in the era of industry 4.0: Challenges, opportunities, and ethical dimensions. Journal of Cleaner Production, 140527. https://doi.org/10.1016/j.jclepro.2023.140527

An, X., Chai, C. S., Li, Y., Zhou, Y., & Yang, B. (2023). Modeling students’ perceptions of artificial intelligence assisted language learning. Computer Assisted Language Learning, 1-22. https://doi.org/10.1080/09588221.2023.2246519

Ardiningtyas, S. Y., Butarbutar, R., Weda, S., & Nur, S. (2023). Online scaffolding behavior for speaking EFL improvement: narrative inquiry issues. Interactive Learning Environments, 1-11. https://doi.org/10.1080/10494820.2023.2207608

Baldwin, A. J. (2024). An artificial intelligence language model improves readability of burns first aid information. Burns, 50(5), 1122-1127. https://doi.org/10.1016/j.burns.2024.03.005

Bar-Haim, R., Kantor, Y., Venezian, E., Katz, Y., & Slonim, N. (2021). Project debater APIs: Decomposing the AI grand challenge. Proceedings of the Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.emnlp-demo.31

Braun, V., Clarke, V., & Hayfield, N. (2022). ‘A starting point for your journey, not a map’: Nikki Hayfield in conversation with Virginia Braun and Victoria Clarke about thematic analysis. Qualitative research in psychology, 19(2), 424-445.

Bricker, B., & Justice, J. (2024). Human intelligence: justifying debate in the Age of AI. Argumentation and Advocacy, 1–19. https://doi.org/10.1080/10511431.2024.2386801

Buabbas, A. J., Miskin, B., Alnaqi, A. A., Ayed, A. K., Shehab, A. A., Syed-Abdul, S., & Uddin, M. (2023, May). Investigating students’ perceptions towards artificial intelligence in medical education. Healthcare, 11(9). https://doi.org/10.3390/healthcare11091298

Buku Pedoman National University Debating Championship (NUDC), 2024. Balai Pengembangan Talenta Indonesia Pusat Prestasi Nasional Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi.

Butarbutar, R. (2024). Collaborative Learning- Based Instruction in Teaching Speaking EFL: A Case Study in Indonesian Higher Education Context. (Doctoral Dissertation, Universitas Negeri Makassar).

Butarbutar, R., Ruing, F. H., Basri, N., Tuharea, V. U., & Leba, S. M. R. (2023). Unpacking Online Collaborative Learning in Teaching EFL Speaking: Insights from Three Rural Area Case Studies. The Qualitative Report, 28(12), 3379-3401. https://doi.org/10.46743/2160-3715/2023.6165

Butarbutar, R., Uspayanti, R., Manuhutu, N., & Palangngan, S. T. (2019). Analyzing of puzzle local culture-based in teaching English for young learners. IOP Conference Series: Earth and Environmental Science, 343(1), 012208. https://doi.org/10.1088/1755- 1315/343/1/012208

Cabral, S., Restrepo, D., Kanjee, Z., Wilson, P., Crowe, B., Abdulnour, R. E., & Rodman, A. (2024). Clinical reasoning of a generative artificial intelligence model compared with physicians. JAMA Internal Medicine, 184(5), 581-583. doi:10.1001/jamainternmed.2024.0295

Choukaier, D. (2024). Integrating AI In English Language Pedagogy: Innovations And Outcomes In Teaching English As Second/Foreign Language. Educational Administration: Theory and Practice, 30(5), 3311-3822. https://do.org/10.53555/kuey.v30i5.3538

Dilzhan, B. (2024). Teaching English and Artificial Intelligence: EFL Lecturers’ Perceptions and Use of ChatGPT. (Dissertation, SDU University).

Ferrer-Aracil, J., Giménez-Bertomeu, V. M., Aragonés-González, M., & Cortés-Florín, E. M. (2023). The Teaching–Learning of Community Social Work: Debating as an Instrument to Acquire Transversal Competences. Education Sciences, 13(7), 689.

Fjelland, R. (2020). Why general artificial intelligence will not be realized. Humanities and Social Sciences Communications, 7(1), 1-9. https://doi.org/10.1057/s41599-020-0494-4

Ghafar, Z. N., & Region-Iraq, K. (2024). The Effect of Classroom Debate on Students' Academic Achievement in Higher Education: an Overview. International Journal of Applied and Scientific Research, 2(1). https://doi.org/10.59890/ijasr.v2i1.1258

Govers, J., Velloso, E., Kostakos, V., & Goncalves, J. (2024, May). AI-Driven Mediation Strategies for Audience Depolarisation in Online Debates. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-18). https://doi.org/10.1145/3613904.3642322

Grájeda, A., Burgos, J., Córdova, P., & Sanjinés, A. (2024). Assessing student-perceived impact of using artificial intelligence tools: Construction of a synthetic index of application in higher education. Cogent Education, 11(1), 2287917. https://doi.org/10.1080/2331186X.2023.2287917

Guo, K., Zhong, Y., Li, D., & Chu, S. K. W. (2023). Investigating students’ engagement in chatbot-supported classroom debates. Interactive Learning Environments, 1–17. https://doi.org/10.1080/10494820.2023.2207181

Hasyim, N., Arismunandar, Butarbutar, R., Ramli, A. M., & Malik Nur, I. D. (2024). Mind mapping of teachers’ readiness for online teaching and learning: A reflective study of urban and suburban areas. Cogent Education, 11(1). https://doi.org/10.1080/2331186X.2023.2292864.

Huang, X., Zou, D., Cheng, G., Chen, X., & Xie, H. (2023). Trends, research issues and applications of artificial intelligence in language education. Educational Technology & Society, 26(1), 112-131.

Hwang, G. J., & Chien, S. Y. (2022). Definition, roles, and potential research issues of the metaverse in education: An artificial intelligence perspective. Computers and Education: Artificial Intelligence, 3, 100082. https://doi.org/10.1016/j.caeai.2022.100082

Jodoi, K. (2023). The Correlations Between Parliamentary Debate Participation, Communication Competence, Communication Apprehension, Argumentativeness, and Willingness to Communicate in a Japanese Context. Argumentation, 37(1), 91-118. https://doi.org/10.1007/s10503-022-09591-5

Korteling, J. H., van de Boer-Visschedijk, G. C., Blankendaal, R. A., Boonekamp, R. C., & Eikelboom, A. R. (2021). Human-versus artificial intelligence. Frontiers in artificial intelligence, 4, 622364. https://doi.org/10.3389/frai.2021.622364

Law, L. (2024). Application of generative artificial intelligence (GenAI) in language teaching and learning: A scoping literature review. Computers and Education Open, 100174. https://doi.org/10.1016/j.caeo.2024.100174

Leba, S. M. R., Butarbutar, R., & Werang, B. R. (2021). Exploring the English learning strategies of an Indigenous Papuan student of Indonesia. The Qualitative Report, 26(9), 2745-2768. https://doi.org/10.46743/2160-3715/2021.4881

Lee, Y. J., Davis, R. O., & Ryu, J. (2024). Korean in-Service Lecturers’ Perceptions of Implementing Artificial Intelligence (AI) Education for Teaching in Schools and Their AI Lecturer Training Programs. International Journal of Information and Education Technology, ijiet, 14 (2), 214-219.

Lee, Y. J., Davis, R. O., & Lee, S. O. (2024). University students’ perceptions of artificial intelligence-based tools for English writing courses. Online Journal of Communication and Media Technologies, 14(1), e202412. https://doi.org/10.30935/ojcmt/14195

Lee, N. Y., Wang, Z., & Lim, B. (2024). The development of critical thinking: What university students have to say. Teaching in Higher Education, 29(1), 286-299. https://doi.org/10.1080/13562517.2021.1973412

Liang, J., Ye, R., Han, M., Lai, R., Zhang, X., Huang, X., & Wei, Z. (2024). Debatrix: Multi-dimensinal Debate Judge with Iterative Chronological Analysis Based on LLM. https://ui.adsabs.harvard.edu/link_gateway/2024arXiv240308010L/doi:10.48550/arXiv.2403.08010

Lin, Y., & Yu, Z. (2024). A bibliometric analysis of artificial intelligence chatbots in educational contexts. Interactive Technology and Smart Education, 21(2), 189-213. https://doi.org/10.1108/itse-12-2022-0165

Martínez-Plumed, F., Prudêncio, R. B., Martínez-Usó, A., & Hernández-Orallo, J. (2019). Item response theory in AI: Analysing machine learning classifiers at the instance level. Artificial intelligence, 271, 18-42.

Meniado, J. C. (2024). Human–Machine Collaboration in Language Education in the Age of Artificial Intelligence. RELC Journal, https://doi.org/10.1177/00336882241266629

Meschi, M., Shirahmadi, S., Amiri, M., & Ebrahimi-Siaghi, N. (2024). Debating: effective and satisfactory learning method in dentistry. BMC Medical Education, 24(1), 307. https://doi.org/10.1186/s12909-024-05286-5

Moura, A., & Carvalho, A. A. A. (2024). Teachers’ perceptions of the use of artificial intelligence in the classroom. In International Conference on Lifelong Education and Leadership for All (ICLEL 2023) (pp. 140-150). Atlantis Press. https://doi.org/10.2991/978-94-6463-380-1_13

Năstasă, A., Dumitra, T. C., & Grigorescu, A. (2024). Artificial intelligence and sustainable development during the pandemic: An overview of the scientific debates. Heliyon, 10. https://doi.org/10.1016/j.heliyon.2024.e30412

Nemorin, S., Vlachidis, A., Ayerakwa, H. M., & Andriotis, P. (2023). AI hyped? A horizon scan of discourse on artificial intelligence in education (AIED) and development. Learning, Media and Technology, 48(1), 38-51. https://doi.org/10.1080/17439884.2022.2095568

Ng, D. T. K., Su, J., Leung, J. K. L., & Chu, S. K. W. (2023). Artificial intelligence (AI) literacy education in secondary schools: a review. Interactive Learning Environments, 1-21. https://doi.org/10.1080/10494820.2023.2255228

Nguyen, A., Hong, Y., Dang, B., & Huang, X. (2024). Human-AI collaboration patterns in AI-assisted academic writing. Studies in Higher Education, 1-18. https://www.tandfonline.com/action/showCitFormats?doi=10.1080/03075079.2024.2323593

Ogunsanya, M. E., Wagner, J. L., Bennett, K., Medina, M. S., Starnes, S., & Planas, L. G. (2024). Debate as an Active Learning Pedagogy Among Pharmacy Students in a Public Health Course. American Journal of Pharmaceutical Education, 100724. https://doi.org/10.1016/j.ajpe.2024.100724

Pan, S., Hafez, B., Iskandar, A., & Ming, Z. (2024). Integrating constructivist principles in an adaptive hybrid learning system for developing social entrepreneurship education among college students. Learning and Motivation, 87. https://doi.org/10.1016/j.lmot.2024.102023

Ouyang, F., Xu, W., & Cukurova, M. (2023). An artificial intelligence-driven learning analytics method to examine the collaborative problem-solving process from the complex adaptive systems perspective. International Journal of Computer-Supported Collaborative Learning, 18(1), 39-66. https://doi.org/10.1007/s11412-023-09387-z

Rowe, H. A. (2024). Problem solving and intelligence. Taylor & Francis.

Siemens, G. (2016). Elearnspace. Connectivism: A learning theory for the digital age. Elearnspace. org, 14-16. https://doi.org/10.3109/0142159x.2016.1173661

Slonim, N., Bilu, Y., Alzate, C., Bar-Haim, R., Bogin, B., Bonin, F., & Aharonov, R. (2021). An autonomous debating system. Nature, 591(7850), 379-384. https://doi.org/10.1038/s41586-021-03215-w

Tai, T. Y., & Chen, H. H. J. (2024). Navigating elementary EFL speaking skills with generative AI chatbots: Exploring individual and paired interactions. Computers & Education, 105112. https://doi.org/10.1016/j.compedu.2024.105112

Tan, S. C., Lee, A. V. Y., & Lee, M. (2022). A systematic review of artificial intelligence techniques for collaborative learning over the past two decades. Computers and Education: Artificial Intelligence, 3, 100097. https://doi.org/10.1016/j.caeai.2022.100097

Tanprasert, T., Fels, S. S., Sinnamon, L., & Yoon, D. (2024, May). Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-24). https://doi.org/10.1145/3613904.3642513

Vaganova, O. I., Tsyganova, L. V., Chelnokova, E. A., Lapshova, A. V., & Turchina, L. A. (2020). Technology "Debate" as a tool for communicative competence development. Amazonia investiga, 9(27), 367-375. https://doi.org/10.34069/ai/2020.27.03.40

Walter, Y. (2024). Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21(1), 15. https://doi.org/10.1186/s41239-024-00448-3

Wang, L., Chen, X., Wang, C., Xu, L., Shadiev, R., & Li, Y. (2024). ChatGPT's capabilities in providing feedback on undergraduate students’ argumentation: A case study. Thinking Skills and Creativity, 51, 101440. https://doi.org/10.1016/j.tsc.2023.101440

Wilson, B. (2024). Debating in Teaching and Learning English: Theory and Practice for Pedagogy and Curriculum. Bloomsbury Publishing.

Yang, R. (2024). Exploring Learner Engagement with English Debates: A Case Study of Chinese University Students. Transactions on Social Science, Education and Humanities Research, 8, 72-80.

Yeh, H. C. (2024). The synergy of generative AI and inquiry-based learning: transforming the landscape of English teaching and learning. Interactive Learning Environments, 1-15.https://doi.org/10.1080/10494820.2024.2335491

Zhang, C., Hallbeck, M. S., Salehinejad, H., & Thiels, C. (2024). The integration of artificial intelligence in robotic surgery: A narrative review. Surgery. https://doi.org/10.1016/j.surg.2024.02.005

Zhang, C., Meng, Y., & Ma, X. (2024). Artificial intelligence in EFL speaking: Impact on enjoyment, anxiety, and willingness to communicate. System, 121, 103259. https://doi.org/10.1016/j.system.2024.103259

Zhang, X., Sun, J., & Deng, Y. (2023). Design and application of intelligent classroom for English language and literature based on artificial intelligence technology. Applied Artificial Intelligence, 37(1), 2216051. https://doi.org/10.1080/08839514.2023.2216051

Zhai, C., & Wibowo, S. (2023). A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students’ interactional competence in the university. Computers and Education: Artificial Intelligence, 4, 100134. https://doi.org/10.1016/j.caeai.2023.100134




DOI: https://doi.org/10.24815/siele.v12i3.40581

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Print ISSN: 2355-2794, Online ISSN: 2461-0275

This work is licensed under a Creative Commons Attribution 4.0 International License.


View Journal Stats