LianJC/Face-ViT-MultiLabel

license: bsd-3-clause library_name: pytorch pipeline_tag: image-classification tags: - facial-forgery-detection - multi-label-classification - vit - deepfake - acl-2026

Face-ViT: Multi-Label Facial Forgery Region Classifier

📖 Model Description

This is the Face-ViT auxiliary perception module proposed in the ACL 2026 paper: "Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline".

Face-ViT is a multi-label classifier based on the ViT-H/14 architecture. It is specifically trained to recognize 21 different types of facial manipulations (e.g., eye modification, skin smoothing, mouth tampering). In the DFF framework, it provides fine-grained visual cues that guide the large language model to generate accurate forensic explanations.

🛠️ Model Details

• Architecture: ViT-H/14 with an additional CNN branch and max-pooling for multi-label support.
• Input Size: 224x224 RGB images.
• Number of Classes: 21 (Facial attributes/manipulation types).
• Training Objective: Joint loss including BCE, Focal, Dice, and Jaccard loss.

🚀 Links

• Official Code: Generating-Attribution-Reports
• Main Framework (DFF): LianJC/DFF-InstructBLIP-Detection
• Dataset (MMTT): LianJC/MMTT-Dataset

📜 Citation

If you find this model useful, please cite:

@inproceedings{lian2026generating,
  title={Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline},
  author={Lian, Jingchun and others},
  booktitle={Proceedings of ACL},
  year={2026},
  note={To appear}
}