User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systems
Refereed conference paper presented and published in conference proceedings
CUHK Authors
Full Text
Digital Object Identifier (DOI) DOI for CUHK Users |
Times Cited
Altmetrics Information
.
Other information
AbstractUser Satisfaction Estimation (USE) is an important yet challenging task in goal-oriented conversational systems. Whether the user is satisfied with the system largely depends on the fulfillment of the user’s needs, which can be implicitly reflected by users’ dialogue acts. However, existing studies often neglect the sequential transitions of dialogue act or rely heavily on annotated dialogue act labels when utilizing dialogue acts to facilitate USE. In this paper, we propose a novel framework, namely USDA, to incorporate the sequential dynamics of dialogue acts for predicting user satisfaction, by jointly learning User Satisfaction Estimation and Dialogue Act Recognition tasks. In specific, we first employ a Hierarchical Transformer to encode the whole dialogue context, with two task-adaptive pre-training strategies to be a second-phase in-domain pre-training for enhancing the dialogue modeling ability. In terms of the availability of dialogue act labels, we further develop two variants of USDA to capture the dialogue act information in either supervised or unsupervised manners. Finally, USDA leverages the sequential transitions of both content and act features in the dialogue to predict the user satisfaction. Experimental results on four benchmark goal-oriented dialogue datasets across different applications show that the proposed method substantially and consistently outperforms existing methods on USE, and validate the important role of dialogue act sequences in USE.
Acceptance Date13/01/2022
All Author(s) ListYang Deng, Wenxuan Zhang, Wai Lam, Hong Cheng, Helen Meng
Name of ConferenceThe Web Conference
Start Date of Conference25/04/2022
End Date of Conference29/04/2022
Place of ConferenceLyon
Country/Region of ConferenceFrance
Proceedings TitleProceedings of the ACM Web Conference 2022
Year2022
Month4
Day25
PublisherACM
Pages2998 - 3008
LanguagesEnglish-United States
KeywordsUser Satisfaction Estimation, Goal-oriented Conversational System, Dialogue Act Recognition
