SimSwap

An arbitrary face-swapping framework on images and videos with one single trained model!

4,442

876

4,442

283

View on GitHub

Top Related Projects

first-order-model

14,432

This repository contains the source code for the paper First Order Motion Model for Image Animation

Quick Overview

SimSwap is an open-source face swapping framework that allows for high-quality face replacement in images and videos. It utilizes advanced deep learning techniques to achieve realistic and seamless face swaps while preserving the target's expressions and head pose.

Pros

High-quality face swapping results with realistic preservation of expressions and head pose
Supports both image and video face swapping
Includes a user-friendly GUI for easy operation
Offers various models and options for different use cases and quality levels

Cons

Requires significant computational resources, especially for video processing
May have ethical concerns regarding potential misuse for deepfakes
Limited documentation for advanced customization and integration
Dependency on specific versions of libraries may cause compatibility issues

Code Examples

Loading the SimSwap model:

from simswap import SimSwap

model = SimSwap(model_path='path/to/simswap_model.pth')

Performing face swap on an image:

import cv2

source_img = cv2.imread('source_face.jpg')
target_img = cv2.imread('target_image.jpg')

result = model.swap_face(source_img, target_img)
cv2.imwrite('swapped_result.jpg', result)

Face swapping in a video:

import cv2

video = cv2.VideoCapture('input_video.mp4')
source_img = cv2.imread('source_face.jpg')

output = cv2.VideoWriter('output_video.mp4', cv2.VideoWriter_fourcc(*'mp4v'), 30, (width, height))

while video.isOpened():
    ret, frame = video.read()
    if not ret:
        break
    
    swapped_frame = model.swap_face(source_img, frame)
    output.write(swapped_frame)

video.release()
output.release()

Getting Started

Clone the repository:

git clone https://github.com/neuralchen/SimSwap.git
cd SimSwap

Install dependencies:
```
pip install -r requirements.txt
```
Download pre-trained models:
```
python get_models.py
```
Run the GUI application:
```
python main.py
```
For command-line usage, refer to the examples in the repository's README.md file.

Competitor Comparisons

first-order-model

14,432

This repository contains the source code for the paper First Order Motion Model for Image Animation

Pros of first-order-model

More versatile, capable of animating a wider range of objects beyond faces
Supports video-to-video animation, not just image-to-image
Generally produces smoother and more natural-looking animations

Cons of first-order-model

Requires more computational resources and processing time
May struggle with extreme pose changes or complex facial expressions
Less focused on face swapping specifically, which can result in lower quality for that particular task

Code Comparison

first-order-model:

source_image = imageio.imread(opt.source_image)
driving_video = imageio.mimread(opt.driving_video)
generator, kp_detector = load_checkpoints(config_path=opt.config, checkpoint_path=opt.checkpoint)

SimSwap:

source = cv2.imread(args.source_img)
target = cv2.imread(args.target_img)
model = SimSwap(args.config)
result = model.swap(source, target)

Both projects aim to manipulate facial images or videos, but they approach the task differently. first-order-model is more flexible and can handle various objects, while SimSwap is specifically designed for face swapping. The code snippets show that first-order-model uses separate generator and keypoint detector models, while SimSwap encapsulates its functionality in a single model class. first-order-model's approach allows for more complex animations, but SimSwap's focused design may lead to better results for face swapping tasks.

faceswap

51,149

Deepfakes Software For All

Pros of faceswap

More established project with a larger community and extensive documentation
Offers a wider range of face swapping models and techniques
Includes a graphical user interface for easier use by non-technical users

Cons of faceswap

Generally slower processing times compared to SimSwap
May require more manual tweaking to achieve optimal results
Higher system requirements for training and execution

Code comparison

SimSwap:

from simswap import SimSwap
simswap = SimSwap()
result = simswap.swap(source_img, target_img)

faceswap:

from lib.cli import FullHelpArgumentParser
from scripts.train import Train
from scripts.convert import Convert

parser = FullHelpArgumentParser()
Train(parser).process()
Convert(parser).process()

SimSwap focuses on a simpler, more streamlined API for face swapping, while faceswap offers a more comprehensive set of tools and options through its command-line interface. SimSwap's approach may be more accessible for quick implementations, whereas faceswap provides greater flexibility and control over the face swapping process.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

SimSwap: An Efficient Framework For High Fidelity Face Swapping

Proceedings of the 28th ACM International Conference on Multimedia

The official repository with Pytorch

Our method can realize arbitrary face swapping on images and videos with one single trained model.

We are recruiting full-time engineers. If you are interested, please send an email to my team. Please refer to the website for specific recruitment conditions: Requirements

Training and test code are now available!

We are working with our incoming paper SimSwap++, keeping expecting!

The high resolution version of SimSwap-HQ is supported!

Our paper can be downloaded from [Arxiv] [ACM DOI]

This project also received support from SocialBook.

Attention

This project is for technical and academic use only. Please do not apply it to illegal and unethical scenarios.

In the event of violation of the legal and ethical requirements of the user's country or region, this code repository is exempt from liability

Please do not ignore the content at the end of this README!

If you find this project useful, please star it. It is the greatest appreciation of our work.

Top News

2023-09-26: We fixed bugs in colab!

2023-04-25: We fixed the "AttributeError: 'SGD' object has no attribute 'defaults' now" bug. If you have already downloaded arcface_checkpoint.tar, please download it again. Also, you also need to update the scripts in ./models/.

2022-04-21: For resource limited users, we provide the cropped VGGFace2-224 dataset [Google Driver] VGGFace2-224 (10.8G) [Baidu Driver] [Password: lrod].

2022-04-20: Training scripts are now available. We highly recommend that you guys train the simswap model with our released high quality dataset VGGFace2-HQ.

2021-11-24: We have trained a beta version of SimSwap-HQ on VGGFace2-HQ and open sourced the checkpoint of this model (if you think the Simswap 512 is cool, please star our VGGFace2-HQ repo). Please donât forget to go to Preparation and Inference for image or video face swapping to check the latest set up.

2021-11-23: The google drive link of VGGFace2-HQ is released.

2021-11-17: We released a high resolution face dataset VGGFace2-HQ and the method to generate this dataset. This dataset is for research purpose.

2021-08-30: Docker has been supported, please refer here for details.

2021-08-17: We have updated the Preparation, The main change is that the gpu version of onnx is now installed by default, Now the time to process a video is greatly reduced.

2021-07-19: Obvious border abruptness has been resolved. We add the ability to using mask and upgrade the old algorithm for better visual effect, please go to Inference for image or video face swapping for details. Please donât forget to go to Preparation to check the latest set up. (Thanks for the help from @woctezuma and @instant-high)

The first open source high resolution dataset for face swapping!!!

High Resolution Dataset VGGFace2-HQ

Dependencies

python3.6+
pytorch1.5+
torchvision
opencv
pillow
numpy
imageio
moviepy
insightface
timm==0.5.4

Training

Preparation

The training script is slightly different from the original version, e.g., we replace the patch discriminator with the projected discriminator, which saves a lot of hardware overhead and achieves slightly better results.

In order to ensure the normal training, the batch size must be greater than 1.

Friendly reminder, due to the difference in training settings, the user-trained model will have subtle differences in visual effects from the pre-trained model we provide.

Train 224 models with VGGFace2 224*224 [Google Driver] VGGFace2-224 (10.8G) [Baidu Driver] [Password: lrod]

For faster convergence and better results, a large batch size (more than 16) is recommended!

We recommend training more than 400K iterations (batch size is 16), 600K~800K will be better, more iterations will not be recommended.

python train.py --name simswap224_test --batchSize 8  --gpu_ids 0 --dataset /path/to/VGGFace2HQ --Gdeep False

[Colab demo for training 224 model]

For faster convergence and better results, a large batch size (more than 16) is recommended!

Train 512 models with VGGFace2-HQ 512*512 VGGFace2-HQ.

python train.py --name simswap512_test  --batchSize 16  --gpu_ids 0 --dataset /path/to/VGGFace2HQ --Gdeep True

Inference with a pretrained SimSwap model

Preparation

Inference for image or video face swapping

Colab demo

Stronger feature

[Colab for switching specific faces in multi-face videos]

Image face swapping demo & Docker image on Replicate

Video

Results

Results1

Results2

High-quality videos can be found in the link below:

[Mama(video) 1080p]

[Google Drive link for video 1]

[Google Drive link for video 2]

[Google Drive link for video 3]

[Baidu Drive link for video] Password: b26n

[Online Video]

User case

If you have some interesting results after using our project and are willing to share, you can contact us by email or share directly on the issue. Later, we may make a separate section to show these results, which should be cool.

At the same time, if you have suggestions for our project, please feel free to ask questions in the issue, or contact us directly via email: email1, email2, email3. (All three can be contacted, just choose any one)

License

For academic and non-commercial use only.The whole project is under the CC-BY-NC 4.0 license. See LICENSE for additional details.

To cite our papers

@inproceedings{DBLP:conf/mm/ChenCNG20,
  author    = {Renwang Chen and
               Xuanhong Chen and
               Bingbing Ni and
               Yanhao Ge},
  title     = {SimSwap: An Efficient Framework For High Fidelity Face Swapping},
  booktitle = {{MM} '20: The 28th {ACM} International Conference on Multimedia},
  year      = {2020}
}

@Article{simswapplusplus,
    author  = {Xuanhong Chen and
              Bingbing Ni and
              Yutian Liu and
              Naiyuan Liu and
              Zhilin Zeng and
              Hang Wang},
    title   = {SimSwap++: Towards Faster and High-Quality Identity Swapping},
    journal = {{IEEE} Trans. Pattern Anal. Mach. Intell.},
    volume  = {46},
    number  = {1},
    pages   = {576--592},
    year    = {2024}
}