Facial attribute-aware DeepFake detection through semi-supervised facial attribute labeling

Abstract

This study investigates the correlation between misclassifications in DeepFake detection and high-level facial attributes. A pre-trained frame-level classifier is used to distinguish manipulated from authentic video content, and its wrong predictions are analyzed in detail. To enrich the dataset, we automatically annotate each video with additional labels, including gender, hair color, hair length, ear visibility, and ethnicity, using a semi-supervised facial-attribute recognition pipeline. We extend this analysis with controlled training-time exclusions on FaceForensics++, keeping a unified test set to isolate generalization bias. Compared to the no-exclusion baseline (Accuracy = 0.806, AUC = 0.823), excluding samples with ears visible yields the largest degradation (Accuracy = 0.741, AUC = 0.763), while excluding non-visible ears has a milder effect (Accuracy = 0.813, AUC = 0.832). Hair length shows a moderate but consistent impact that interacts with ear visibility. We also explain the observed confusion-matrix asymmetry as a consequence of fixed score thresholds and video-level k-of-n aggregation. The results demonstrate that ear visibility is a critical factor for robust FAKE vs. REAL discrimination and motivate attribute-aware training, including targeted data curation, attribute-specific augmentation, and threshold calibration. The proposed framework provides actionable guidance for bias-aware training strategies and supports the development of more interpretable and operationally reliable DeepFake detection systems.