QoE Estimation of WebRTC-based Audiovisual Conversations from Facial Expressions

Bingol, G;Porcu, S;Floris, A;Atzori, L

2022-01-01

---

Abstract

Online videoconferencing applications have become more popular and occupy a significant portion of consumer Internet traffic. Thus, it is crucial to assess the Quality of Experience (QoE) for these applications to effectively implementing high-quality services. Although the major approach to assess the QoE is the conduction of a subjective quality assessment, this method relies on user's feedback and it is not suitable for real-time management systems. For this reason, in this paper, we propose alternative QoE estimation models based on facial expression features extracted from the face of the users during audiovisual conversations with a WebRTC-based telemeeting service. We first conducted a subjective quality assessment to investigate the QoE of two-party audiovisual conversations impaired by poor network conditions. Then, we used the extracted facial expression features obtained by observing the test participants to train Machine Learning (ML) algorithms and a Fully Convolutional Network (FCN) with the aim to estimate the perceived QoE solely on the base of these features. Finally, we discuss and compare the QoE estimation performance of the proposed models in terms of accuracy, precision, recall, and F1-score metrics. In particular, the ML model (Support Vector Machine) and the FCN achieved the mean estimation accuracy of 0.78 and 0.70, respectively.