Automated versus peer assessment: Effects on learners’ English public speaking
Publication in refereed journal
Full Text
View Full-text in CUHK  |
Other information
AbstractThis quasi-experimental research investigates the employment of a formative assessment platform aided by
artificial intelligence in an English public speaking course. The platform integrates deep learning,
automatic speech recognition, and automatic writing evaluation. It provides automated assessment and
immediate feedback on speakers’ public speaking anxiety and their speaking and writing competence. Fiftytwo
English public speaking learners were randomly assigned to two groups. The control group (G1)
undertook self-, peer, and teacher assessment via the platform, while the experimental group (G2)
experienced self-, automated, and teacher assessment. The ANCOVA results revealed that students in G1
perceived significantly higher social engagement than those in G2, which indicates that social interaction
between learners during peer assessment cannot be substituted by automated assessment. The chi-square
analysis showed students’ different concerns regarding online formative assessment. While students in G1
showed concerns for peers’ qualifications and willingness to provide feedback, students in G2 suggested
generating more detailed automated feedback to improve self-learning. No significant differences were
found in learners’ English public speaking self-efficacy, engagement, or competence. This indicates that
automated assessment can serve as an effective strategy for formative assessment and that AI tools may
supplement peers as reliable learning companions in the foreseeable future.
artificial intelligence in an English public speaking course. The platform integrates deep learning,
automatic speech recognition, and automatic writing evaluation. It provides automated assessment and
immediate feedback on speakers’ public speaking anxiety and their speaking and writing competence. Fiftytwo
English public speaking learners were randomly assigned to two groups. The control group (G1)
undertook self-, peer, and teacher assessment via the platform, while the experimental group (G2)
experienced self-, automated, and teacher assessment. The ANCOVA results revealed that students in G1
perceived significantly higher social engagement than those in G2, which indicates that social interaction
between learners during peer assessment cannot be substituted by automated assessment. The chi-square
analysis showed students’ different concerns regarding online formative assessment. While students in G1
showed concerns for peers’ qualifications and willingness to provide feedback, students in G2 suggested
generating more detailed automated feedback to improve self-learning. No significant differences were
found in learners’ English public speaking self-efficacy, engagement, or competence. This indicates that
automated assessment can serve as an effective strategy for formative assessment and that AI tools may
supplement peers as reliable learning companions in the foreseeable future.
All Author(s) ListZheng C., Chen X., Zhang H., Chai C. S
Journal nameLanguage Learning & Technology
Year2024
Month6
Volume Number28
Issue Number2
PublisherUniversity of Hawaii
Place of PublicationUS, Hawaii
Pages210 - 228
ISSN1094-3501
LanguagesEnglish-United States
KeywordsAutomated Assessment, Peer Assessment, English Public Speaking, Learner Engagement, Self-efficacy
