Abstract

Deep learning algorithms are used to detect as well as create deepfake images and videos. Images and videos are often used as evidence in police investigations and courts to resolve legal cases since they are considered to be reliable sources. However, deepfake technology increases the development of fake videos, and this may lead to image or video evidence unreliable. This paper aims to qualitatively compare deep learning algorithms and frameworks. To detect real and fake images or videos, various detection algorithms have been proposed after deepfakes were introduced. The current deepfake detection algorithms detect the deepfakes by eye blinking, eye teach and facial texture, head poses, face warping artifacts, eye color, lip movements, audio speakers, reflections in the teeth, spatiotemporal features, and capsule forensics. Deepfake detection algorithms and deep learning frameworks are selected and compared. Deep learning frameworks with different performance and features such as TensorFlow, CNTK, Caffe, Torch, Chainer, and Theano are compared. This helps to use appropriate deep learning algorithms and frameworks for deepfake detection.    

Keywords

  • Detection Algorithms
  • Deepfake Detection
  • Deep Learning
  • Image Detection
  • Video Detection

References

  1. • G. Shao, "What ‘deepfakes’ are and how they may be dangerous," 13 October 2019. [Online]. Available: https://deepfakedetectionchallenge.ai/.
  2. • Simplilearn, "Deep Learning Algorithms You Should Know About," Simplilearn, 28 May 2020. [Online]. Available: https://www.simplilearn.com/deep-learning-algorithms-article. [Accessed 10 June 2020].
  3. • D. D. Zahid Akhtar, "A Comparative Evaluation of Local Feature Descriptors for DeepFakes Detection," 07 November 2019.
  4. • R. V.-R. J. F. A. M. J. O.-G. Ruben Tolosana, "DeepFakes and Beyond A Survey of Face Manipulation and Fake Detection," pp. 1-15, 2020.
  5. • J. A. Marwan Albahar, "Deepfakes Threats and Countermeasures Systematic Review," JTAIT, vol. 97, no. 22, pp. 3242-3250, 30 November 2019.
  6. • B. P. W. W. J. D. Xinsheng Xuan, "On the Generalization of Generative Adversarial Networks Image Forensics," pp. 1-8, 2019.
  7. • S. M. Pavel Korshunov, "Vulnerability assessment and detection of Deepfake videos," pp. 1-6, 2019.
  8. • M.-C. C. a. S. L. Yuezun Li, "In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking," pp. 1-7, 2018.
  9. • M. R. Z. G. Marissa Koopman, "Detection of Deepfake Video Manipulation," in Proceedings of the 20th Irish Machine Vision and Image Processing conference, Belfast, 2018.
  10. • Y.-X. Z. a. C.-Y. L. Chih-Chung Hsu, "Deep Fake Image Detection Based on Pairwise Learning," Applied science, pp. 1-14, 2020.
  11. • V. N. J. Y. I. E. Darius Afchar, "MesoNet a Compact Facial Video Forgery Detection Network," pp. 1-7, 2018.
  12. • F. F. J. Y. I. E. Huy H. Nguyen, "Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos," pp. 1-8, 2019.
  13. • I.-S. N. S.-H. K. Nhu-Tai Do, " Forensics Face Detection From GANs Using Convolutional Neural Network," pp. 1-5, 2019.
  14. • C. M. N. D. T. N. D. T. N. S. N. Thanh Thi Nguyen, "Deep Learning for Deepfakes Creation and Detection," pp. 1-16, 2019.
  15. • W. M. Wubet, "The Deepfake Challenges and Deepfake Video Detection," International Journal of Innovative Technology and Exploring Engineering (IJITEE), vol. 9, no. 6, pp. 1-8, 2020.
  16. • G. A.-B. R. A.-Q. M. A.-A. Ali Shatnawi, "A Comparative Study of Open Source Deep Learning Frameworks," pp. 1-7, 2018..
  17. • G.-Y. W. D. B. Yu (Emma) Wang, "Benchmarking TPU, GPU, and CPU Platforms for Deep Learning," pp. 1-19, 2019.
  18. • K. K. M. Y. H. S. u. Wenpeng Yin, "Comparative Study of CNN and RNN for Natural Language Processing," pp. 1-7, 2017.
  19. • R. H. B. P. N. B. C. C. F. Brian Dolhansky, "The Deepfake Detection Challenge (DFDC) Preview Dataset," Deepfake Detection Challenge, pp. 1-4, 2019.
  20. • L. O. Alvaro Figueira, "The current state of fake news: challenges and opportunities," in ICHSCIST, Barcelona, 2017.
  21. • W. D. Mei Wang, "Deep Face Recognition: A Survey," pp. 1-26, 2019.
  22. • J. D. Britt Paris, "Deepfakes and Cheap Fakes the Manipulation of Audio and Visual Evidence," Media Manipulation, Los Angeles, 2018.
  23. • M. V. R. S. Prabhat Kumar, "Detecting Face2Face Facial Reenactment in Videos," pp. 1-9, 2020.
  24. • R. D. C. L. V. C. R. J. T. M. Nießner, "FaceForensics++: Learning to Detect Manipulated Facial Images," pp. 1-14, 2019.
  25. • D. D. B. B. Zahid Akhtara, "Face Authenticity An Overview of Face Manipulation Generation, Detection and Recognition," in ICCIP-2019, 2019.
  26. • Z. R. E. C. L. E. V. O. P. K. M. H. Xinyi Ding, "Swapped Face Detection using Deep Learning and Subjective Assessment," pp. 1-8, 2019.
  27. • J. C. A. J. W. A. I. M. P. N. Ekraam Sabir, "Recurrent Convolutional Strategies for Face Manipulation Detection in Videos," pp. 80-87, 2019.
  28. • E. J. D. David Guera, "Deepfake video detection using recurrent neural networks," AVSS, pp. 1-6, 2018.
  29. • J. Y. I. E. Huy H. Nguyen, "Capsule-Forensics Using Capsule Networks to Detect Forged Images snd Videos," pp. 1-5, 2018.
  30. • [30] S. M. Pavel Korshunov, "DeepFakes A New Threat to Face Recognition, Assessment and Detection," pp. 1-5, 2018.
  31. • [31] L. Z. V. L. L. T. Ying Zhang, "Automated face swapping and its detection," ICSIP, pp. 15-19, 2017.
  32. • S. L. Yuezun Li, "Exposing DeepFake Videos By Detecting Face Warping Artifacts," New York,, 2019.
  33. • C. Z. Y. L. Weiguo Zhang, "A Novel Counterfeit Feature Extraction Technique for Exposing Face-Swap Images Based on Deep Learning and Error Level Analysis," pp. 1-17, 2020.
  34. • W. K. J.-S. P. Young-Joo Han, "Efficient Eye-Blinking Detection on Smartphones: A Hybrid Approach Based on Deep Learning," Mobile Information Systems, pp. 1-9, 2018.
  35. • X. Y. P. S. H. Q. S. L. Yuezun Li, "Celeb-DF (v2): A New Dataset for DeepFake Forensics," pp. 1-6, 2019.
  36. • Y. L. S. L. Xin Yang, "Exposing Deepfakes Using Inconsistent Head Poses," ICASSP, pp. 8261- 8265, 2019.
  37. • E. C. J. J. A. A. W. M. I. N. P. Sabir, "Recurrent convolutional strategies for face manipulation detection in videos," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 80-87, 2019.
  38. • D. C. L. V. C. R. J. T. M. N. Andreas Rössler, "FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces," Computer Vision and Pattern Recognition, pp. 1-21, 2018.
  39. • M. K. F.-J. P. J. K. Ricard Durall, "Unmasking DeepFakes with simple Features," pp. 1-8, 2019.
  40. • S. C. X. X. L. M. Q. H. H. L. Y. L. J. Z. X. L. Qianyu Guo, "An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms," pp. 1-13, 2019.
  41. • Y. W. W. W. W. C. S. S. Q. Z. Ling Liu, "Benchmarking Deep Learning Frameworks: Design Considerations, Metrics and Beyond," in IEEE 38th International Conference on Distributed Computing Systems, Georgia, 2018.
  42. • S. D. M. B. V. T. Á. L. G. I. H. P. M. L. H. Giang Nguyen, "Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey," Artificial Intelligence Review, pp. 77-124, 2019.
  43. • N. R. L. S. M. S. Soheil Bahrampour, "Comparative Study of Deep Learning Software Frameworks," pp. 1-8, 2016.
  44. • J. C. Aniruddha Parvat, "A Survey of Deep-learning Frameworks," International Conference on Inventive Systems and Control (ICISC-2017), pp. 1-7, 2017.
  45. • Y. Z. J. Q. James Fox, "Software Frameworks for Deep Learning at Scale," pp. 1-7, 2018.
  46. • S. S. Y. Y. H. T. Y. T. M. P. R. M.-L. S. S.-. Samira Pouyanfar, "A Survey on Deep Learning: Algorithms, Techniques, and Applications.," ACM Computing Surveys, vol. 51, no. 5, pp. 92:1-36, 2018.
  47. • R. A.-Q. M. A.-A. A. S. Ghadeer Al-Bdour, "A DETAILED COMPARATIVE STUDY OF OPEN SOURCE DEEP LEARNING FRAMEWORKS," A PREPRINT, pp. 1-25, 2020.
  48. • Pathmind, "Comparison of AI Frameworks," Pathmind Inc., 2019. [Online]. Available: https://pathmind.com/wiki/comparison-frameworks-dl4j-tensorflow-pytorch. [Accessed 17 June 2020].
  49. • Q. W. P. X. X. C. Shaohuai Shi, "Benchmarking State-of-the-Art Deep Learning Software Tools," pp. 1-14, 2017.