An ASR-Free fluency scoring approach with self-supervised learning
Refereed conference paper presented and published in conference proceedings

Altmetrics Information

Other information
AbstractA typical fluency scoring system generally relies on an automatic speech recognition (ASR) system to obtain time stamps in input speech for the subsequent calculation of fluency-related features or directly modeling speech fluency with an end-to-end approach. This paper describes a novel ASR-free approach for automatic fluency assessment using self-supervised learning (SSL). Specifically, wav2vec2.0 is used to extract frame-level speech features, followed by K-means clustering to assign a pseudo label (cluster index) to each frame. A BLSTM-based model is trained to predict an utterance-level fluency score from frame-level SSL features and the corresponding cluster indexes. Neither speech transcription nor time stamp information is required in the proposed system. It is ASR-free and can potentially avoid the ASR errors effect in practice. Experimental results carried out on non-native English databases show that the proposed approach significantly improves the performance in the "open response" scenario as compared to previous methods and matches the recently reported performance in the "read aloud" scenario.
All Author(s) ListWei Liu, Kaiqi Fu, Xiaohai Tian, Shuju Shi, Wei Li, Zejun Ma, Tan Lee
Name of Conference2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Start Date of Conference04/06/2023
End Date of Conference10/06/2023
Place of ConferenceRhodes
Country/Region of ConferenceGreece
Proceedings TitleICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
LanguagesEnglish-United States
KeywordsFluency scoring, automatic speech recognition, non-native speech, self-supervised learning.

Last updated on 2024-29-01 at 10:30