insightface

State-of-the-art 2D and 3D Face Analysis Project

22,782

5,341

22,782

1,117

View on GitHub

Top Related Projects

face_recognition

52,807

The world's simplest facial recognition api for Python and the command line

facenet-pytorch

4,427

Pretrained Pytorch face detection (MTCNN) and facial recognition (InceptionResnet) models

face-api.js

16,478

JavaScript API for face detection and face recognition in the browser and nodejs with tensorflow.js

deepface

11,722

A Lightweight Face Recognition and Facial Attribute Analysis (Age, Gender, Emotion and Race) Library for Python

Quick Overview

InsightFace is an open-source 2D and 3D face analysis toolbox based on PyTorch and MXNet. It provides state-of-the-art face recognition algorithms, including ArcFace, and supports various face-related tasks such as detection, alignment, recognition, and attribute analysis.

Pros

High-performance face recognition algorithms with state-of-the-art accuracy
Supports multiple deep learning frameworks (PyTorch and MXNet)
Comprehensive toolset for various face-related tasks
Active development and community support

Cons

Steep learning curve for beginners
Limited documentation for some advanced features
Dependency on specific versions of deep learning frameworks
Resource-intensive for large-scale applications

Code Examples

Face detection and alignment:

from insightface.app import FaceAnalysis
import cv2

app = FaceAnalysis(providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))

img = cv2.imread('sample.jpg')
faces = app.get(img)
for face in faces:
    bbox = face.bbox.astype(int)
    cv2.rectangle(img, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (0, 255, 0), 2)

cv2.imshow('result', img)
cv2.waitKey(0)

Face recognition:

from insightface.app import FaceAnalysis
import numpy as np

app = FaceAnalysis(providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))

img1 = cv2.imread('person1.jpg')
img2 = cv2.imread('person2.jpg')

face1 = app.get(img1)[0]
face2 = app.get(img2)[0]

similarity = np.dot(face1.normed_embedding, face2.normed_embedding)
print(f"Face similarity: {similarity}")

Face attribute analysis:

from insightface.app import FaceAnalysis

app = FaceAnalysis(providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))

img = cv2.imread('sample.jpg')
faces = app.get(img)

for face in faces:
    print(f"Age: {face.age}")
    print(f"Gender: {'Male' if face.gender == 1 else 'Female'}")
    print(f"Emotion: {face.emotion}")

Getting Started

Install InsightFace:

pip install insightface

Download pre-trained models:

from insightface.app import FaceAnalysis
app = FaceAnalysis()
app.prepare(ctx_id=0, det_size=(640, 640))

Use the library for face analysis tasks as shown in the code examples above.

Competitor Comparisons

faceswap

51,149

Deepfakes Software For All

Pros of Faceswap

More user-friendly interface and documentation for beginners
Broader range of face manipulation techniques beyond just swapping
Active community with frequent updates and contributions

Cons of Faceswap

Less focus on high-precision facial recognition and analysis
May require more computational resources for some operations
Primarily designed for entertainment purposes, less suitable for professional applications

Code Comparison

InsightFace example:

from insightface.app import FaceAnalysis
app = FaceAnalysis()
app.prepare(ctx_id=0, det_size=(640, 640))
faces = app.get(img)

Faceswap example:

from lib.cli import FullHelpArgumentParser
from plugins.train.model import Model
from plugins.extract.pipeline import Extractor
args = FullHelpArgumentParser().parse_args()
model = Model(args)
extractor = Extractor(args)

InsightFace focuses on efficient facial analysis and recognition, while Faceswap provides a more comprehensive toolkit for face manipulation and swapping. InsightFace's code is more streamlined for quick facial analysis, whereas Faceswap's code structure reflects its broader range of features and customization options. Both projects serve different primary purposes: InsightFace for facial recognition and analysis, and Faceswap for creative face manipulation and deepfake generation.

face_recognition

52,807

The world's simplest facial recognition api for Python and the command line

Pros of face_recognition

Easier to use with a high-level Python API
Better documentation and examples for beginners
Supports multiple face recognition tasks out-of-the-box

Cons of face_recognition

Less accurate on challenging datasets compared to insightface
Slower performance, especially for large-scale face recognition tasks
Limited customization options for advanced users

Code Comparison

face_recognition:

import face_recognition

image = face_recognition.load_image_file("image.jpg")
face_locations = face_recognition.face_locations(image)
face_encodings = face_recognition.face_encodings(image, face_locations)

insightface:

import insightface
from insightface.app import FaceAnalysis

app = FaceAnalysis()
app.prepare(ctx_id=0, det_size=(640, 640))
faces = app.get(img)

face_recognition provides a simpler API for basic face recognition tasks, while insightface offers more advanced features and better performance at the cost of increased complexity. insightface is generally preferred for large-scale or high-accuracy applications, while face_recognition is suitable for smaller projects or beginners in face recognition.

facenet-pytorch

4,427

Pretrained Pytorch face detection (MTCNN) and facial recognition (InceptionResnet) models

Pros of facenet-pytorch

Focused specifically on PyTorch implementation, making it more accessible for PyTorch users
Includes pre-trained models and easy-to-use inference pipelines
Lightweight and easy to integrate into existing PyTorch projects

Cons of facenet-pytorch

Limited to FaceNet architecture, while InsightFace offers multiple model architectures
Smaller community and less frequent updates compared to InsightFace
Fewer features and tools for face recognition tasks beyond embedding generation

Code Comparison

facenet-pytorch:

from facenet_pytorch import MTCNN, InceptionResnetV1
mtcnn = MTCNN()
resnet = InceptionResnetV1(pretrained='vggface2').eval()
img = Image.open('path/to/image.jpg')
img_cropped = mtcnn(img)
img_embedding = resnet(img_cropped.unsqueeze(0))

InsightFace:

from insightface.app import FaceAnalysis
app = FaceAnalysis()
app.prepare(ctx_id=0, det_size=(640, 640))
img = cv2.imread('path/to/image.jpg')
faces = app.get(img)
embeddings = [face.embedding for face in faces]

Both repositories provide face recognition capabilities, but InsightFace offers a more comprehensive suite of tools and models for various face-related tasks. facenet-pytorch is more focused and easier to use for PyTorch users specifically interested in FaceNet implementation.

facenet

13,708

Face recognition using Tensorflow

Pros of FaceNet

Implements the original FaceNet architecture as described in the paper
Provides pre-trained models and easy-to-use inference scripts
Supports both TensorFlow and PyTorch implementations

Cons of FaceNet

Less actively maintained compared to InsightFace
Limited support for recent advancements in face recognition techniques
Fewer pre-trained models and less flexibility in model architectures

Code Comparison

FaceNet:

embeddings = model.forward(images)
distances = distance.cdist(embeddings, embeddings, metric='euclidean')

InsightFace:

embeddings = model.get_feature(images)
similarities = np.dot(embeddings, embeddings.T)

InsightFace offers a more comprehensive and up-to-date face recognition toolkit, with support for various model architectures and training methods. It provides better performance on benchmark datasets and includes additional features like face detection and alignment.

FaceNet, while pioneering the concept of facial embeddings, has seen less active development. However, it remains a solid choice for those specifically interested in the original FaceNet architecture or seeking a simpler implementation for educational purposes.

Both repositories offer pre-trained models and inference scripts, but InsightFace generally provides more options and better performance in real-world scenarios.

face-api.js

16,478

JavaScript API for face detection and face recognition in the browser and nodejs with tensorflow.js

Pros of face-api.js

Browser-based implementation, allowing for client-side face recognition
Extensive documentation and examples for easy integration
Supports multiple face detection models (SSD MobileNet V1, Tiny Face Detector)

Cons of face-api.js

Limited to JavaScript environments, reducing flexibility for server-side applications
May have lower performance compared to native implementations
Smaller community and fewer updates compared to InsightFace

Code Comparison

face-api.js:

await faceapi.loadSsdMobilenetv1Model('/models')
const detections = await faceapi.detectAllFaces(input)

InsightFace:

model = insightface.app.FaceAnalysis()
model.prepare(ctx_id=-1, det_size=(640, 640))
faces = model.get(img)

Both libraries offer face detection capabilities, but InsightFace provides a more comprehensive set of features and is generally more performant for server-side applications. face-api.js excels in browser-based scenarios and offers easier integration for web developers. The choice between the two depends on the specific requirements of your project, such as deployment environment, performance needs, and desired features.

deepface

11,722

A Lightweight Face Recognition and Facial Attribute Analysis (Age, Gender, Emotion and Race) Library for Python

Pros of DeepFace

Easier to use with a more user-friendly API
Supports multiple deep learning frameworks (TensorFlow, Keras, PyTorch)
Includes additional features like emotion recognition and age/gender estimation

Cons of DeepFace

Generally slower performance compared to InsightFace
Less accurate in some face recognition tasks
Smaller community and fewer updates

Code Comparison

InsightFace:

from insightface.app import FaceAnalysis
app = FaceAnalysis()
app.prepare(ctx_id=0, det_size=(640, 640))
img = cv2.imread('sample.jpg')
faces = app.get(img)

DeepFace:

from deepface import DeepFace
img = 'sample.jpg'
result = DeepFace.analyze(img_path=img, 
                          actions=['age', 'gender', 'emotion'])

InsightFace focuses on efficient face detection and recognition, while DeepFace offers a broader range of facial analysis features. InsightFace generally provides better performance and accuracy for face recognition tasks, but DeepFace is more accessible for beginners and offers more diverse functionality out of the box. The choice between the two depends on specific project requirements and the user's level of expertise in working with facial recognition systems.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

InsightFace: 2D and 3D Face Analysis Project

InsightFace project is mainly maintained By Jia Guo and Jiankang Deng.

For all main contributors, please check contributing.

License

The code of InsightFace is released under the MIT License. There is no limitation for both academic and commercial usage.

The training data containing the annotation (and the models trained with these data) are available for non-commercial research purposes only.

Both manual-downloading models from our github repo and auto-downloading models with our python-library follow the above license policy(which is for non-commercial research purposes only).

Top News

2024-08-01 We have integrated our most advanced face-swapping models: inswapper_cyn and inswapper_dax, into the Picsi.Ai face-swapping service. These models outperform almost all similar commercial products and our open-source model inswapper_128. Please visit the Picsi.Ai website to use the service and get help.

2024-05-04 We have added InspireFace, which is a cross-platform face recognition SDK developed in C/C++, supporting multiple operating systems and various backends.

2023-04-01: We integrated our most advanced face-swapping models: inswapper_cyn and inswapper_dax and move the service to Discord bot, which also support editing on Midjourney generated images, see detail at web-demos/swapping_discord and our Picsi.Ai website.

2022-08-12: We achieved Rank-1st of Perspective Projection Based Monocular 3D Face Reconstruction Challenge of ECCV-2022 WCPA Workshop, paper and code.

2021-11-30: MFR-Ongoing challenge launched(same with IFRT), which is an extended version of iccv21-mfr.

2021-10-29: We achieved 1st place on the VISA track of NIST-FRVT 1:1 by using Partial FC (Xiang An, Jiankang Deng, Jia Guo).

ChangeLogs

2024-05-04 We have added InspireFace, which is a cross-platform face recognition SDK developed in C/C++, supporting multiple operating systems and various backends.

2024-04-17: Monocular Identity-Conditioned Facial Reflectance Reconstruction accepted by CVPR-2024.

2023-08-08: We released the implementation of Generalizing Gaze Estimation with Weak-Supervision from Synthetic Views at reconstruction/gaze.

2023-05-03: We have launched the ongoing version of wild face anti-spoofing challenge. See details here.

2023-02-13: We launch a large scale in the wild face anti-spoofing challenge on CVPR23 Workshop, see details at challenges/cvpr23-fas-wild.

2022-11-28: Single line code for facial identity swapping in our python packge ver 0.7, please check the example here.

2022-10-28: MFR-Ongoing website is refactored, please create issues if there's any bug.

2022-09-22: Now we have web-demos: face-localization, face-recognition, and face-swapping.

2022-08-12: We achieved Rank-1st of Perspective Projection Based Monocular 3D Face Reconstruction Challenge of ECCV-2022 WCPA Workshop, paper and code.

2022-03-30: Partial FC accepted by CVPR-2022.

2022-02-23: SCRFD accepted by ICLR-2022.

2021-11-30: MFR-Ongoing challenge launched(same with IFRT), which is an extended version of iccv21-mfr.

2021-10-29: We achieved 1st place on the VISA track of NIST-FRVT 1:1 by using Partial FC (Xiang An, Jiankang Deng, Jia Guo).

2021-10-11: Leaderboard of ICCV21 - Masked Face Recognition Challenge released. Video: Youtube, Bilibili.

2021-06-05: We launch a Masked Face Recognition Challenge & Workshop on ICCV 2021.

Introduction

InsightFace is an open source 2D&3D deep face analysis toolbox, mainly based on PyTorch and MXNet.

Please check our website for detail.

The master branch works with PyTorch 1.6+ and/or MXNet=1.6-1.8, with Python 3.x.

InsightFace efficiently implements a rich variety of state of the art algorithms of face recognition, face detection and face alignment, which optimized for both training and deployment.

Quick Start

Please start with our python-package, for testing detection, recognition and alignment models on input images.

ArcFace Video Demo

Please click the image to watch the Youtube video. For Bilibili users, click here.

Projects

The page on InsightFace website also describes all supported projects in InsightFace.

You may also interested in some challenges hold by InsightFace.

Face Recognition

Introduction

In this module, we provide training data, network settings and loss designs for deep face recognition.

The supported methods are as follows:

Commonly used network backbones are included in most of the methods, such as IResNet, MobilefaceNet, MobileNet, InceptionResNet_v2, DenseNet, etc..

Datasets

The training data includes, but not limited to the cleaned MS1M, VGG2 and CASIA-Webface datasets, which were already packed in MXNet binary format. Please dataset page for detail.

Evaluation

We provide standard IJB and Megaface evaluation pipelines in evaluation

Pretrained Models

Please check Model-Zoo for more pretrained models.

Third-party Re-implementation of ArcFace

TensorFlow: InsightFace_TF
TensorFlow: tf-insightface
TensorFlow:insightface
PyTorch: InsightFace_Pytorch
PyTorch: arcface-pytorch
Caffe: arcface-caffe
Caffe: CombinedMargin-caffe
Tensorflow: InsightFace-tensorflow
TensorRT: wang-xinyu/tensorrtx
TensorRT: InsightFace-REST
ONNXRuntime C++: ArcFace-ONNXRuntime
ONNXRuntime Go: arcface-go
MNN: ArcFace-MNN
TNN: ArcFace-TNN
NCNN: ArcFace-NCNN

Face Detection

Introduction

In this module, we provide training data with annotation, network settings and loss designs for face detection training, evaluation and inference.

The supported methods are as follows:

RetinaFace is a practical single-stage face detector which is accepted by CVPR 2020. We provide training code, training dataset, pretrained models and evaluation scripts.

SCRFD is an efficient high accuracy face detection approach which is initialy described in Arxiv. We provide an easy-to-use pipeline to train high efficiency face detectors with NAS supporting.

Face Alignment

Introduction

In this module, we provide datasets and training/inference pipelines for face alignment.

Supported methods:

SDUNets is a heatmap based method which accepted on BMVC.

SimpleRegression provides very lightweight facial landmark models with fast coordinate regression. The input of these models is loose cropped face image while the output is the direct landmark coordinates.

Citation

If you find InsightFace useful in your research, please consider to cite the following related papers:

@inproceedings{ren2023pbidr,
  title={Facial Geometric Detail Recovery via Implicit Representation},
  author={Ren, Xingyu and Lattas, Alexandros and Gecer, Baris and Deng, Jiankang and Ma, Chao and Yang, Xiaokang},
  booktitle={2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG)},  
  year={2023}
 }

@article{guo2021sample,
  title={Sample and Computation Redistribution for Efficient Face Detection},
  author={Guo, Jia and Deng, Jiankang and Lattas, Alexandros and Zafeiriou, Stefanos},
  journal={arXiv preprint arXiv:2105.04714},
  year={2021}
}

@inproceedings{gecer2021ostec,
  title={OSTeC: One-Shot Texture Completion},
  author={Gecer, Baris and Deng, Jiankang and Zafeiriou, Stefanos},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021}
}

@inproceedings{an_2022_pfc_cvpr,
  title={Killing Two Birds with One Stone: Efficient and Robust Training of Face Recognition CNNs by Partial FC},
  author={An, Xiang and Deng, Jiangkang and Guo, Jia and Feng, Ziyong and Zhu, Xuhan and Jing, Yang and Tongliang, Liu},
  booktitle={CVPR},
  year={2022}
}
@inproceedings{an_2021_pfc_iccvw,
  title={Partial FC: Training 10 Million Identities on a Single Machine},
  author={An, Xiang and Zhu, Xuhan and Gao, Yuan and Xiao, Yang and Zhao, Yongle and Feng, Ziyong and Wu, Lan and Qin, Bin and Zhang, Ming and Zhang, Debing and Fu, Ying},
  booktitle={ICCVW},
  year={2021},
}


@inproceedings{deng2020subcenter,
  title={Sub-center ArcFace: Boosting Face Recognition by Large-scale Noisy Web Faces},
  author={Deng, Jiankang and Guo, Jia and Liu, Tongliang and Gong, Mingming and Zafeiriou, Stefanos},
  booktitle={Proceedings of the IEEE Conference on European Conference on Computer Vision},
  year={2020}
}

@inproceedings{Deng2020CVPR,
title = {RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild},
author = {Deng, Jiankang and Guo, Jia and Ververas, Evangelos and Kotsia, Irene and Zafeiriou, Stefanos},
booktitle = {CVPR},
year = {2020}
}

@inproceedings{guo2018stacked,
  title={Stacked Dense U-Nets with Dual Transformers for Robust Face Alignment},
  author={Guo, Jia and Deng, Jiankang and Xue, Niannan and Zafeiriou, Stefanos},
  booktitle={BMVC},
  year={2018}
}

@article{deng2018menpo,
  title={The Menpo benchmark for multi-pose 2D and 3D facial landmark localisation and tracking},
  author={Deng, Jiankang and Roussos, Anastasios and Chrysos, Grigorios and Ververas, Evangelos and Kotsia, Irene and Shen, Jie and Zafeiriou, Stefanos},
  journal={IJCV},
  year={2018}
}

@inproceedings{deng2018arcface,
title={ArcFace: Additive Angular Margin Loss for Deep Face Recognition},
author={Deng, Jiankang and Guo, Jia and Niannan, Xue and Zafeiriou, Stefanos},
booktitle={CVPR},
year={2019}
}

Contributing

Main contributors:

Jia Guo, guojia[at]gmail.com
Jiankang Deng jiankangdeng[at]gmail.com
Xiang An anxiangsir[at]gmail.com
Jack Yu jackyu961127[at]gmail.com
Baris Gecer barisgecer[at]msn.com

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot