Does Chatting with Chatbots Improve Language Learning Performance? A Meta-Analysis of Chatbot-Assisted Language Learning
Publication in refereed journal
Officially Accepted for Publication
CUHK Authors
Full Text
Digital Object Identifier (DOI) DOI for CUHK Users |
Altmetrics Information
.
Other information
AbstractGiven the importance of conversation practice in language learning, chatbots, especially ChatGPT, have attracted considerable attention for their ability to converse with learners using natural language. This review contributes to the literature by examining the currently unclear overall effect of using chatbots on language learning performance and comprehensively identifying important study characteristics that affect the overall effectiveness. We meta-analyzed 70 effect sizes from 28 studies, using robust variance estimation. The effects were assessed based on 18 study characteristics about learners, chatbots, learning objectives, context, communication/interaction, and methodological and pedagogical designs. Results indicated that using chatbots produced a positive overall effect on language learning performance (g = 0.484), compared to nonchatbot conditions. Moreover, four characteristics (i.e., educational level, language level, interface design, and interaction capability) affected the overall effectiveness. In an in-depth discussion on how the 18 characteristics are related to the effectiveness, future implications for practice and research are presented.
Acceptance Date14/06/2024
All Author(s) ListFeifei Wang, Alan C. K. Cheung, Amanda J. Neitzel, Ching Sing Chai
Journal nameReview of Educational Research
Year2024
PublisherSAGE Publications
Place of PublicationUK and US
ISSN0034-6543
LanguagesEnglish-United States
Keywordschatbots, language learning, meta-analysis, artificial intelligence, ChatGPT
