Experimental Results on Multi-modal Deepfake Detection

Concas S.;Gao J.;Cuccu C.;Orru' G.;Feng X.;Marcialis G. L.;Puglisi G.;Roli F.

2022-01-01

---

Abstract

Book cover International Conference on Image Analysis and Processing ICIAP 2022: Image Analysis and Processing – ICIAP 2022 pp 164–175Cite as Experimental Results on Multi-modal Deepfake Detection Sara Concas, Jie Gao, Carlo Cuccu, Giulia Orrù, Xiaoyi Feng, Gian Luca Marcialis, Giovanni Puglisi & Fabio Roli Conference paper First Online: 17 May 2022 1051 Accesses Part of the Lecture Notes in Computer Science book series (LNCS,volume 13232) Abstract The advantages of deepfakes in many applications are counterbalanced by their malicious use, for example, in reply-attacks against a biometric system, identification evasion, and people harassment, when they are widespread in social networks and chatting platforms (cyberbullying) as recently documented in newspapers. Due to its “arms-race” nature, deepfake detection systems are often trained on a certain class of deepfakes and showed their limits on never-seen-before classes. In order to shed some light on this problem, we explore the benefits of a multi-modal deepfake detection system. We adopted simple fusion rules, which showed their effectiveness in many applications, for example, biometric recognition, to exploit the complementary of different individual classifiers, and derive some possible guidelines for the designer.