Synthesis research on the application of chatbots (artificial intelligence software) in English language education

Document Type : Original Article

Authors

1 Department of English Language Teaching, Farhangian University, Ardabil, Iran.

2 Department of Educational Sciences, Farhangian University, PO Box 889-14665, Tehran, Iran.

Abstract

Background and Objectives: The integration of chatbots in education has garnered attention, particularly in enhancing English language learning. Innovations like ChatGPT, a large language model, can perform various natural language processing tasks that support educational outcomes. This study investigates the role of chatbots in English language education through a qualitative synthesis of existing literature. Methods: We analyzed content from 15 selected articles from reputable global databases using a qualitative research design. The analysis followed the methodology proposed by Sandelowski and Barroso (2007), ensuring a comprehensive understanding of the topic. To evaluate the reliability of our findings, we applied Cohen's Kappa coefficient, resulting in a robust value of 0.88. Findings: The synthesis revealed that chatbots are not just tools but practical and effective tools for learning English as a foreign language. Our analysis highlights learners' experiences with chatbot-assisted language acquisition, identifying factors that influence outcomes and the advantages and disadvantages of this approach. Despite specific challenges, the results overwhelmingly indicate positive educational impacts, instilling optimism about the potential of chatbots in language education. Conclusion: As the educational landscape evolves, teacher training programs should incorporate artificial intelligence and innovative methodologies to foster digital literacy. The findings empower educators to guide learners in utilizing chatbots throughout various stages of second language acquisition, enhancing their educational experiences.

Keywords

Main Subjects

References

Abedi Jafari, Hassan; Taslimi, Mohammad Saeed; Faqihi, Abolhasan; and Sheikhzadeh, Mohammad. (2011). 'Thematic Analysis and Theme Network: A Simple and Effective Method for Explaining Patterns in Qualitative Data.' Strategic Management Thought (Management Thought). 5 (2). 151-198. https://doi.org/10.30497/smt.2011.163
Alemdag, E. (2023). The effect of chatbots on learning: a meta-analysis of empirical research. Journal of Research on Technology in Education, 1-23. https://doi.org/10.1080/15391523.2023.2255698
Alkamel, M. A. A., & Alwagieh, N. A. S. (2024). Utilizing an adaptable artificial intelligence writing tool (ChatGPT) to enhance academic writing skills among Yemeni university EFL students. Social Sciences & Humanities Open, 10, 101095. https://doi.org/10.1016/j.ssaho.2024.101095
Alnefaie, A. M., & Bagadood, N. H. (2021). Distance learning for students with intellectual disability during the emerging coronavirus pandemic: Opportunities and challenges from parents' perspectives. IJCSNS International Journal of Computer Science and Network Security, 21(12), 85-91. https://koreascience.kr/article/JAKO202108038386124.pdf
Alrajhi, A. S. (2024). Artificial intelligence pedagogical chatbots as L2 conversational agents. Cogent Education, 11(1), 2327789. https://doi.org/10.1080/2331186X.2024.2327789
Anderson, R. (2007). Thematic content analysis (TCA). Descriptive presentation of qualitative data, 4-1. https://rosemarieanderson.com/wp-content/uploads/2014/08/ThematicContentAnalysis.pdf
Annamalai, N., Ab Rashid, R., Hashmi, U. M., Mohamed, M., Alqaryouti, M. H., & Sadeq, A. E. (2023). Using chatbots for English language learning in higher education. Computers and Education: Artificial Intelligence, 5, 100153. https://doi.org/10.1016/j.caeai.2023.100153
Annamalai, N., Eltahir, M. E., Zyoud, S. H., Soundrarajan, D., Zakarneh, B., & Al Salhi, N. R. (2023). Exploring English language learning via Chabot: A case study from a self-determination theory perspective. Computers and Education: Artificial Intelligence, 5, 100148. https://doi.org/10.1016/j.caeai.2023.100148
Bailey, D. (2019). Chatbots as conversational agents in the context of language learning. Proceedings of the fourth industrial revolution and education, 32-41. https://www.researchgate.net/profile/Daniel-Bailey-12/publication/334988843_Chatbots_as_Conversational_Agents_in_the_Context_of_Language_Learning/links/60bb6646a6fdcc22eadabe20/Chatbots-as-Conversational-Agents-in-the-Context-of-Language-Learning.pdf
Barlette, Y., Jaouen, A., & Baillette, P. (2021). Bring Your Own Device (BYOD) as reversed IT adoption: Insights into managers’ coping strategies. International journal of information management, 56, 102212. https://doi.org/10.1016/j.ijinfomgt.2020.102212
Baskara, R. (2023). INVESTIGATING THE IMPACT OF CHATBOTS IN DIFFERENT LEARNING CONTEXTS ON STUDENT ENGAGEMENT AND CRITICAL THINKING. The Journal of English Teaching for Young and Adult Learners, 2(2), 51-61. https://doi.org/10.21137/jeeyal.2023.2.2.1
 
Belda-Medina, J., & Calvo-Ferrer, J. R. (2022). Using chatbots as AI conversational partners in language learning. Applied Sciences, 12(17), 8427. https://doi.org/10.3390/app12178427
Chukharev‐Hudilainen, E., & Ockey, G. J. (2021). The development and evaluation of interactional competence elicitor for oral language assessments. ETS Research Report Series, 2021(1), 1-20. https://doi.org/10.1002/ets2.12319
Derakhshan, A., & Ghiasvand, F. (2024). Is ChatGPT an evil or an angel for second language education and research? A phenomenographic study of research‐active EFL teachers’ perceptions. International Journal of Applied Linguistics. https://doi.org/10.1111/ijal.12561
Escalante, J., Pack, A., & Barrett, A. (2023). AI-generated feedback on writing: insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education, 20(1), 57. https://doi.org/10.1186/s41239-023-00425-2
Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in human Behavior, 93, 279-289. https://doi.org/10.1016/j.chb.2018.12.023
Fryer, L., Coniam, D., Carpenter, R., & Lăpușneanu, D. (2020). Bots for language learning now: Current and future directions. http://hdl.handle.net/10125/44719
Grudin, J., & Jacques, R. (2019, May). Chatbots, humbots, and the quest for artificial general intelligence. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1-11). https://doi.org/10.1145/3290605.3300439
Guo, K., Li, Y., Li, Y., & Chu, S. K. W. (2024). Understanding EFL students’ chatbot-assisted argumentative writing: An activity theory perspective. Education and Information Technologies, 29(1), 1-20. https://doi.org/10.1007/s10639-023-12230-5
Guo, K., Wang, J., & Chu, S. K. W. (2022). Using chatbots to scaffold EFL students’ argumentative writing. Assessing Writing, 54, 100666. https://doi.org/10.1016/j.asw.2022.100666
Guo, K., Zhong, Y., Li, D., & Chu, S. K. W. (2023). Effects of chatbot-assisted in-class debates on students’ argumentation skills and task motivation. Computers & Education, 203, 104862. https://doi.org/10.1016/j.compedu.2023.104862
Guo, K., Zhong, Y., Li, D., & Chu, S. K. W. (2023). Investigating students’ engagement in chatbot-supported classroom debates. Interactive Learning Environments, 1-17. https://doi.org/10.1080/10494820.2023.2207181
Hedges, L. V., & Cooper, H. (2009). Research synthesis as a scientific process. The handbook of research synthesis and meta-analysis, 1(3), 4-7
Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot‐supported language learning. Journal of Computer Assisted Learning, 38(1), 237-257. https://doi.org/10.1111/jcal.12610
Jeon, J. (2022). Exploring a self-directed interactive app for informal EFL learning: a self-determination theory perspective. Education and Information Technologies, 27(4), 5767-5787. https://doi.org/10.1007/s10639-021-10839-y
Jeon, J., & Lee, S. (2024). Can learners benefit from chatbots instead of humans? A systematic review of human-chatbot comparison research in language education. Education and Information Technologies, 1-32. https://doi.org/10.1007/s10639-024-12725-9
Jeon, J., Lee, S., & Choi, S. (2023). A systematic review of research on speech-recognition chatbots for language learning: Implications for future directions in the era of large language models. Interactive Learning Environments, 1-19. https://doi.org/10.1016/j.compedu.2023.104898
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., ... & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
Klimova, B., Pikhart, M., & Al-Obaydi, L. H. (2023). The Use of Persona in Foreign Language Learning Facilitated by Chatbots. https://doi.org/10.21203/rs.3.rs-3129096/v1
Kohnke, L., Zou, D., & Moorhouse, B. L. (2024). Technostress and English language teaching in the age of generative AI. Educational Technology & Society, 27(2), 306–320. https://www.jstor.org/stable/48766177
Li, K. C., Chang, M., & Wu, K. H. (2020). Developing a task-based dialogue system for English language learning. Education Sciences, 10(11), 306. https://doi.org/10.3390/educsci10110306
Liang, J. C., Hwang, G. J., Chen, M. R. A., & Darmawansah, D. (2023). Roles and research foci of artificial intelligence in language education: an integrated bibliographic analysis and systematic review approach. Interactive Learning Environments, 31(7), 4270-4296. https://doi.org/10.1080/10494820.2021.1958348
Liu, C. C., Chiu, C. W., Chang, C. H., & Lo, F. Y. (2024). Analysis of a chatbot as a dialogic reading facilitator: its influence on learning interest and learner interactions. Educational technology research and development, 1-29. https://doi.org/10.1007/s11423-024-10370-0
Liu, C. C., Liao, M. G., Chang, C. H., & Lin, H. M. (2022). An analysis of children’interaction with an AI chatbot and its impact on their interest in reading. Computers & Education, 189, 104576. https://doi.org/10.1016/j.compedu.2022.104576
Lund, B. D., Khan, D., & Yuvaraj, M. (2024). ChatGPT in medical libraries, possibilities and future directions: An integrative review. Health Information & Libraries Journal, 41(1), 4-15. https://doi.org/10.1111/hir.12518
Melián-González, S., Gutiérrez-Taño, D., & Bulchand-Gidumal, J. (2021). Predicting the intentions to use chatbots for travel and tourism. Current Issues in Tourism, 24(2), 192-210. https://doi.org/10.1080/13683500.2019.1706457
Meng, J., Rheu, M., Zhang, Y., Dai, Y., & Peng, W. (2023). Mediated social support for distress reduction: AI Chatbots vs. Human. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1), 1-25. https://doi.org/10.1145/3579505
Mohamed, A. M. (2024). Exploring the potential of an AI-based Chatbot (ChatGPT) in enhancing English as a Foreign Language (EFL) teaching: perceptions of EFL Faculty Members. Education and Information Technologies, 29(3), 3195-3217. https://doi.org/10.1007/s10639-023-11917-z
Niloy, A. C., Bari, M. A., Sultana, J., Chowdhury, R., Raisa, F. M., Islam, A., ... & Hossen, M. A. (2024). Why do students use ChatGPT? Answering through a triangulation approach. Computers and Education: Artificial Intelligence, 6, 100208. https://doi.org/10.1016/j.caeai.2024.100208
Ou, A. W., Stöhr, C., & Malmström, H. (2024). Academic communication with AI-powered language tools in higher education: From a post-humanist perspective. System, 121, 103225. https://doi.org/10.1016/j.system.2024.103225
Papadopoulos, T., Baltas, K. N., & Balta, M. E. (2020). The use of digital technologies by small and medium enterprises during COVID-19: Implications for theory and practice. International journal of information management, 55, 102192. https://doi.org/10.1016/j.ijinfomgt.2020.102192
Safitri, F., Hardini, T. I., Setiadi, R., & Mutiarsih, Y. (2021, November). Usability measurement: chatbot as a pedagogical support for learning French grammar. In Fifth International Conference on Language, Literature, Culture, and Education (ICOLLITE 2021) (pp. 272-276). Atlantis Press. https://doi.org/10.2991/assehr.k.211119.042
Sandelowski, M. and Barroso, J. (2007) Handbook for Synthesizing Qualitative Research. Springer Publishing Company, New York.
Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862
Songsiengchai, S., Sereerat, B. O., & Watananimitgul, W. (2023). Leveraging Artificial Intelligence (AI): Chat GPT for Effective English Language Learning among Thai Students. English Language Teaching, 16(11), 1-68. https://pdfs.semanticscholar.org/b07a/9572e15b0b2ea025fe1ef40280e2009ee1c6.pdf
Tai, T. Y., & Chen, H. H. J. (2023). The impact of Google Assistant on adolescent EFL learners’ willingness to communicate. Interactive Learning Environments, 31(3), 1485-1502. https://doi.org/10.1080/10494820.2020.1841801
Tai, T. Y., & Chen, H. H. J. (2024). Improving elementary EFL speaking skills with generative AI chatbots: Exploring individual and paired interactions. Computers & Education, 220, 105112. https://doi.org/10.1016/j.compedu.2024.105112
Timpe-Laughlin, V., Sydorenko, T., & Daurio, P. (2022). Using spoken dialogue technology for L2 speaking practice: What do teachers think? Computer Assisted Language Learning, 35(5-6), 1194-1217. https://doi.org/10.1080/09588221.2020.1774904
Vázquez-Cano, E., Mengual-Andrés, S., & López-Meneses, E. (2021). Chatbot to improve learning punctuation in Spanish and to enhance open and flexible learning environments. International Journal of Educational Technology in Higher Education, 18, 1-20. https://doi.org/10.1186/s41239-021-00269-8
Wang, J., Hwang, G. H., & Chang, C. Y. (2021). Directions of the 100 most cited chatbot-related human behavior research: A review of academic publications. Computers and Education: Artificial Intelligence, 2, 100023. https://doi.org/10.1016/j.caeai.2021.100023
Wang, Y., & Xue, L. (2024). Using AI-driven chatbots to foster Chinese EFL students’ academic engagement: An intervention study. Computers in Human Behavior, 108353. https://doi.org/10.1016/j.chb.2024.108353
Werdiningsih, I., Marzuki, & Rusdin, D. (2024). Balancing AI and authenticity: EFL students’ experiences with ChatGPT in academic writing. Cogent Arts & Humanities, 11(1), 2392388. https://doi.org/10.1080/23311983.2024.2392388
Woo, D. J., Wang, D., Guo, K., & Susanto, H. (2024). Teaching EFL students to write with ChatGPT: Students' motivation to learn, cognitive load, and satisfaction with the learning process. Education and Information Technologies, 1-28. https://doi.org/10.1007/s10639-024-12819-4
Wu, R., & Yu, Z. (2024). Do AI chatbots improve students learning outcomes? Evidence from a meta‐analysis. British Journal of Educational Technology, 55(1), 10-33. https://doi.org/10.1111/bjet.13334
Wu, X., & Li, R. (2024). Unraveling Effects of AI Chatbots on EFL Learners’ Language Skill Development: A Meta-analysis. The Asia-Pacific Education Researcher, 1-12. https://doi.org/10.1111/bjet.13334
Xiao, Y., Zhang, T., & He, J. (2024). A Review of Promises and Challenges of AI-Based Chatbots in Language Education through the Lens of Learner Emotions. Heliyon. https://doi.org/10.1016/j.heliyon.2024.e37238
Xu, L., Sanders, L., Li, K., & Chow, J. C. (2021). Chatbot for health care and oncology applications using artificial intelligence and machine learning: systematic review. JMIR cancer, 7(4), e27850. https://doi.org/10.2196/27850
Yang, H., Kim, H., Lee, J. H., & Shin, D. (2022). Implementation of an AI chatbot as an English conversation partner in EFL speaking classes. ReCALL, 34(3), 327-343. https://doi.org/10.1017/S0958344022000039
Yin, J., Goh, T. T., Yang, B., & Xiaobin, Y. (2021). Conversation technology with micro-learning: The impact of chatbot-based learning on students’ learning motivation and performance. Journal of Educational Computing Research, 59(1), 154-177. https://doi.org/10.1177/0735633120952067
Yin, Q., & Satar, M. (2020). English as a Foreign Language Learner Interactions with Chatbots: Negotiation for Meaning. International Online Journal of Education and Teaching, 7(2), 390-410 https://doi.org/10.1177/0735633120952067
Yuan, Y. (2023). An empirical study of the efficacy of AI chatbots for English as a foreign language learning in primary education. Interactive Learning Environments, 1-16. https://doi.org/10.1080/10494820.2023.2282112
Zebua, J. A. Z., & Katemba, C. V. (2024). Students' Perceptions of Using the OpenAI ChatGPT Application in Improving Writing Skills. Journal of Language and Literature Studies, 4(1), 110-123. https://doi.org/10.36312/jolls.v4i1.1805
Zhai, C., & Wibowo, S. (2022). A systematic review on cross-culture, humor and empathy dimensions in conversational chatbots: the case of second language acquisition. Heliyon, 8(12). https://doi.org/10.1016/j.heliyon.2022.e12056
Zhai, C., Wibowo, S., & Cowling, M. (2022, May). Work-in-Progress—Embedding cross-cultural humorous and empathetic functions to facilitate language acquisition. In 2022 8th International Conference of the Immersive Learning Research Network (iLRN) (pp. 1-4). IEEE. https://doi.org/10.23919/iLRN55037.2022.9815992
Zhang, S., Shan, C., Lee, J. S. Y., Che, S., & Kim, J. H. (2023). Effect of chatbot-assisted language learning: A meta-analysis. Education and Information Technologies, 28(11), 15223-15243. https://doi.org/10.1007/s10639-023-11805-6
Zheng, S. (2024). The effects of chatbot use on foreign language reading anxiety and reading performance among Chinese secondary school students. Computers and Education: Artificial Intelligence, 7, 100271. https://doi.org/10.1016/j.caeai.2024.100271
Zohery, M. (2023). ChatGPT in academic writing and publishing: A comprehensive guide. Artificial intelligence in academia, research and science: ChatGPT as a case study, 10-61.

Files

Share

How to cite

Statistics