Wavelet-Enhanced CLIP with Noise Residual Learning for Generalizable Deepfake Detection
- 1School of Artificial Intelligence, Guangzhou University, Guangzhou 510006, China
Abstract
The rapid development of artificial intelligence-generated content (AIGC) methods has posed severe threats to security and privacy, as photorealistic face forgeries (deepfakes) can be easily generated by modern generative models. Since existing CLIP-based deepfake detection methods mainly focus on high-level semantic information, they may struggle to capture subtle tampering artifacts in the frequency domain, leading to unsatisfactory generalization ability. To address this issue, in this paper, a frequency-aware deepfake detection network, which is composed of a CLIP-based semantic branch and a residual-based noise branch, is proposed to jointly capture forgery traces in the spatial and frequency domains. In the semantic branch, a wavelet convolution module is incorporated with the image encoder of CLIP to enhance multi-frequency representations. To characterize statistical inconsistencies around forged boundaries, a constrained convolutional layer and a noise encoder are adopted to extract residual features, which are fused with semantic features through learnable weighted fusion. To further improve generalization ability, an image–text class prototype alignment strategy is proposed to align the fused visual features with text embeddings of pre-defined prompts corresponding to specific classes. Extensive experimental results on several benchmark datasets demonstrate that the proposed method achieves competitive performance against other state-of-the-art methods on cross-dataset evaluations.
Related
- Evaluating Xception with Multitask Self-Supervised Learning for AI-Generated Image and Deepfake Video Detection
- Dual-domain self-supervised feature alignment via spectral–spatial representation learning for deepfake anomaly detection
- Prior-Informed Generalizable Low-Rank Representation Learning via Regularized Adversarial Reweighting
- OUT-OF-DISTRIBUTION AI-GENERATED IMAGE DETECTION WITH ZERO-PHASE DECONVOLUTION FOR RESIDUAL WHITENING
- No-Harm Physics-Informed Inverse Learning with Residual-Calibrated Uncertainty
BibTeX
@article{rao2026waveletenhanced,
title = {{Wavelet-Enhanced CLIP with Noise Residual Learning for Generalizable Deepfake Detection}},
author = {Yuan Rao and Jianhui Zeng and Weixuan Tang and Kongyang Chen and Cheng Ran and Haozhe Zeng},
journal = {Mathematics},
year = {2026},
month = nov,
doi = {10.3390/math14214010},
url = {https://www.mdpi.com/2227-7390/14/21/4010},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Wavelet-Enhanced CLIP with Noise Residual Learning for Generalizable Deepfake Detection
AU - Yuan Rao
AU - Jianhui Zeng
AU - Weixuan Tang
AU - Kongyang Chen
AU - Cheng Ran
AU - Haozhe Zeng
PY - 2026
DA - 2026/11/03
JO - Mathematics
DO - 10.3390/math14214010
UR - https://www.mdpi.com/2227-7390/14/21/4010
AB - The rapid development of artificial intelligence-generated content (AIGC) methods has posed severe threats to security and privacy, as photorealistic face forgeries (deepfakes) can be easily generated by modern generative models. Since existing CLIP-based deepfake detection methods mainly focus on high-level semantic information, they may struggle to capture subtle tampering artifacts in the frequency domain, leading to unsatisfactory generalization ability. To address this issue, in this paper, a frequency-aware deepfake detection network, which is composed of a CLIP-based semantic branch and a residual-based noise branch, is proposed to jointly capture forgery traces in the spatial and frequency domains. In the semantic branch, a wavelet convolution module is incorporated with the image encoder of CLIP to enhance multi-frequency representations. To characterize statistical inconsistencies around forged boundaries, a constrained convolutional layer and a noise encoder are adopted to extract residual features, which are fused with semantic features through learnable weighted fusion. To further improve generalization ability, an image–text class prototype alignment strategy is proposed to align the fused visual features with text embeddings of pre-defined prompts corresponding to specific classes. Extensive experimental results on several benchmark datasets demonstrate that the proposed method achieves competitive performance against other state-of-the-art methods on cross-dataset evaluations.
SN - 2227-7390
ER -