Towards More General Video-based Deepfake Detection through Facial Feature Guided Adaptation for Foundation Model
CoRR(2024)
Abstract
With the rise of deep learning, generative models have enabled the creation
of highly realistic synthetic images, presenting challenges due to their
potential misuse. While research in Deepfake detection has grown rapidly in
response, many detection methods struggle with unseen Deepfakes generated by
new synthesis techniques. To address this generalisation challenge, we propose
a novel Deepfake detection approach by adapting rich information encoded inside
the Foundation Models with rich information encoded inside, specifically using
the image encoder from CLIP which has demonstrated strong zero-shot capability
for downstream tasks. Inspired by the recent advances of parameter efficient
fine-tuning, we propose a novel side-network-based decoder to extract spatial
and temporal cues from the given video clip, with the promotion of the Facial
Component Guidance (FCG) to guidencourage the spatial feature to include
features of key facial parts for more robust and general Deepfake detection.
Through extensive cross-dataset evaluations, our approach exhibits superior
effectiveness in identifying unseen Deepfake samples, achieving notable
performance improvementsuccess even with limited training samples and
manipulation types. Our model secures an average performance enhancement of
0.9
methods, especiallytablishing a significant lead of achieving 4.4
on the challenging DFDC dataset.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined