facenet-pytorch

Pretrained Pytorch face detection (MTCNN) and facial recognition (InceptionResnet) models

4,427

944

4,427

View on GitHub

Top Related Projects

face_recognition

53,131

The world's simplest facial recognition api for Python and the command line

insightface

22,782

State-of-the-art 2D and 3D Face Analysis Project

openface

15,090

Face recognition with deep neural networks.

face.evoLVe

3,414

🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

deepface

11,722

A Lightweight Face Recognition and Facial Attribute Analysis (Age, Gender, Emotion and Race) Library for Python

Quick Overview

FaceNet-PyTorch is an implementation of the FaceNet face recognition system using PyTorch. It provides pre-trained models and utilities for face detection, recognition, and clustering. The library is designed to be easy to use and integrate into existing PyTorch projects.

Pros

Easy to use and integrate with existing PyTorch projects
Provides pre-trained models for face detection and recognition
Includes utilities for face clustering and verification
Supports both CPU and GPU acceleration

Cons

Limited documentation and examples
Dependency on other libraries like torchvision and opencv-python
May require fine-tuning for specific use cases
Performance can vary depending on the quality and diversity of input images

Code Examples

Face detection:

from facenet_pytorch import MTCNN
import torch

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
mtcnn = MTCNN(keep_all=True, device=device)

# Detect faces in an image
boxes, _ = mtcnn.detect(image)

Face recognition:

from facenet_pytorch import InceptionResnetV1

resnet = InceptionResnetV1(pretrained='vggface2').eval()

# Get face embeddings
aligned = mtcnn(image)
embeddings = resnet(aligned).detach().cpu()

Face clustering:

from sklearn.cluster import DBSCAN
import numpy as np

# Cluster face embeddings
clustering = DBSCAN(metric='cosine', n_jobs=-1)
labels = clustering.fit_predict(embeddings.numpy())

Getting Started

To get started with FaceNet-PyTorch, follow these steps:

Install the library:

pip install facenet-pytorch

Import the necessary modules:

from facenet_pytorch import MTCNN, InceptionResnetV1
import torch
from PIL import Image

# Set the device
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

# Create face detection and recognition models
mtcnn = MTCNN(keep_all=True, device=device)
resnet = InceptionResnetV1(pretrained='vggface2').eval().to(device)

# Load an image and detect faces
image = Image.open('path/to/image.jpg')
boxes, _ = mtcnn.detect(image)

# Get face embeddings
aligned = mtcnn(image)
embeddings = resnet(aligned.to(device)).detach().cpu()

This code sets up the face detection and recognition models, loads an image, detects faces, and computes face embeddings. You can then use these embeddings for various tasks such as face verification or clustering.

Competitor Comparisons

facenet

13,708

Face recognition using Tensorflow

Pros of facenet

Implemented in TensorFlow, which offers extensive GPU support and optimization
Provides pre-trained models for immediate use
Includes comprehensive documentation and examples

Cons of facenet

Less actively maintained compared to facenet-pytorch
Requires TensorFlow, which can be more complex to set up and use
Limited compatibility with PyTorch ecosystems

Code Comparison

facenet (TensorFlow):

import tensorflow as tf
from facenet import facenet

# Load model
model = facenet.load_model('path/to/model')

# Perform face recognition
embeddings = facenet.get_embeddings(images, model)

facenet-pytorch (PyTorch):

from facenet_pytorch import InceptionResnetV1, MTCNN

# Load model
mtcnn = MTCNN()
resnet = InceptionResnetV1(pretrained='vggface2').eval()

# Perform face recognition
face = mtcnn(image)
embedding = resnet(face.unsqueeze(0))

Both repositories implement the FaceNet architecture for face recognition, but they differ in their underlying frameworks and ease of use. facenet-pytorch offers a more modern, PyTorch-based implementation with easier integration into PyTorch workflows, while facenet provides a stable TensorFlow implementation with extensive documentation.

face_recognition

53,131

The world's simplest facial recognition api for Python and the command line

Pros of face_recognition

Easier to use with a high-level API for face recognition tasks
Supports multiple backends (CPU and GPU)
Includes command-line tools for quick face recognition tasks

Cons of face_recognition

Less flexible for custom model architectures
May have slower performance for large-scale applications
Limited to pre-trained models without easy fine-tuning options

Code Comparison

face_recognition:

import face_recognition

image = face_recognition.load_image_file("image.jpg")
face_locations = face_recognition.face_locations(image)
face_encodings = face_recognition.face_encodings(image, face_locations)

facenet-pytorch:

from facenet_pytorch import MTCNN, InceptionResnetV1

mtcnn = MTCNN()
resnet = InceptionResnetV1(pretrained='vggface2').eval()

img = Image.open("image.jpg")
img_cropped = mtcnn(img)
img_embedding = resnet(img_cropped.unsqueeze(0))

The face_recognition library provides a more straightforward API for basic face recognition tasks, while facenet-pytorch offers more flexibility and control over the underlying models and processes. face_recognition is better suited for quick prototyping and small-scale applications, whereas facenet-pytorch may be more appropriate for advanced users and large-scale deployments requiring customization and fine-tuning.

insightface

22,782

State-of-the-art 2D and 3D Face Analysis Project

Pros of InsightFace

More comprehensive, offering a wider range of face recognition models and algorithms
Better performance on large-scale face recognition benchmarks
Supports multiple deep learning frameworks (PyTorch, MXNet, and TensorFlow)

Cons of InsightFace

Steeper learning curve due to its more complex architecture
Less focused on PyTorch specifically, which may be a drawback for PyTorch-centric projects

Code Comparison

InsightFace:

from insightface.app import FaceAnalysis
app = FaceAnalysis()
app.prepare(ctx_id=0, det_size=(640, 640))
img = cv2.imread('sample.jpg')
faces = app.get(img)

FaceNet-PyTorch:

from facenet_pytorch import MTCNN, InceptionResnetV1
mtcnn = MTCNN()
resnet = InceptionResnetV1(pretrained='vggface2').eval()
img = Image.open('sample.jpg')
img_cropped = mtcnn(img)
img_embedding = resnet(img_cropped.unsqueeze(0))

InsightFace provides a more integrated approach with its FaceAnalysis app, while FaceNet-PyTorch requires separate initialization of detection and recognition models. InsightFace's code is more concise for a complete face analysis pipeline, whereas FaceNet-PyTorch's code is more explicit in its steps.

openface

15,090

Face recognition with deep neural networks.

Pros of OpenFace

More comprehensive documentation and tutorials
Supports multiple face recognition tasks beyond embedding generation
Includes pre-trained models for various applications

Cons of OpenFace

Less active development and maintenance
Older codebase with potential compatibility issues
Limited support for recent deep learning frameworks

Code Comparison

OpenFace (Python):

import openface

align = openface.AlignDlib("models/dlib/shape_predictor_68_face_landmarks.dat")
net = openface.TorchNeuralNet("models/openface/nn4.small2.v1.t7", 96)
embedding = net.forward(aligned_face)

FaceNet-PyTorch:

from facenet_pytorch import MTCNN, InceptionResnetV1

mtcnn = MTCNN()
resnet = InceptionResnetV1(pretrained='vggface2').eval()
img_cropped = mtcnn(img)
embedding = resnet(img_cropped.unsqueeze(0))

Both repositories provide face recognition capabilities, but FaceNet-PyTorch offers a more modern implementation with PyTorch integration. OpenFace provides a broader range of features and documentation, while FaceNet-PyTorch focuses on efficient embedding generation using recent architectures. The code comparison shows that FaceNet-PyTorch has a more streamlined API for face detection and embedding generation.

face.evoLVe

3,414

🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

Pros of face.evoLVe

Offers a wider range of pre-trained models and architectures
Includes more comprehensive data preprocessing and augmentation techniques
Provides tools for model interpretation and visualization

Cons of face.evoLVe

Less focused on PyTorch integration compared to facenet-pytorch
May have a steeper learning curve due to its broader scope
Documentation could be more detailed for some advanced features

Code Comparison

facenet-pytorch:

from facenet_pytorch import MTCNN, InceptionResnetV1

mtcnn = MTCNN()
resnet = InceptionResnetV1(pretrained='vggface2').eval()

img = Image.open('path/to/image.jpg')
img_cropped = mtcnn(img)
img_embedding = resnet(img_cropped.unsqueeze(0))

face.evoLVe:

from face_evolve import get_model, get_transform

model = get_model('ir_se50')
transform = get_transform(img_size=112)

img = Image.open('path/to/image.jpg')
img_tensor = transform(img).unsqueeze(0)
img_embedding = model(img_tensor)

Both repositories provide efficient ways to perform face recognition tasks, but face.evoLVe offers more flexibility in model selection and data preprocessing at the cost of slightly more complex usage.

deepface

11,722

A Lightweight Face Recognition and Facial Attribute Analysis (Age, Gender, Emotion and Race) Library for Python

Pros of DeepFace

Supports multiple face recognition models (VGGFace, Facenet, OpenFace, DeepFace, DeepID, ArcFace, Dlib)
Includes additional features like age, gender, emotion, and race prediction
Provides a high-level API for easy integration and usage

Cons of DeepFace

Larger package size due to multiple model support and additional features
May have slower performance for simple face recognition tasks
Requires more dependencies to be installed

Code Comparison

DeepFace:

from deepface import DeepFace

result = DeepFace.verify("img1.jpg", "img2.jpg")
print("Is same person:", result["verified"])

Facenet-PyTorch:

from facenet_pytorch import MTCNN, InceptionResnetV1
import torch

mtcnn = MTCNN()
resnet = InceptionResnetV1(pretrained='vggface2').eval()
img = mtcnn(img)
embedding = resnet(img.unsqueeze(0))

DeepFace offers a more straightforward API for face verification, while Facenet-PyTorch provides lower-level access to the model components, allowing for more customization but requiring more code for basic tasks.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Face Recognition Using Pytorch

You can also read a translated version of this file in Chinese ç®ä½ä¸æç.

This is a repository for Inception Resnet (V1) models in pytorch, pretrained on VGGFace2 and CASIA-Webface.

Pytorch model weights were initialized using parameters ported from David Sandberg's tensorflow facenet repo.

Also included in this repo is an efficient pytorch implementation of MTCNN for face detection prior to inference. These models are also pretrained. To our knowledge, this is the fastest MTCNN implementation available.

Table of contents
Quick start
Pretrained models
Example notebooks
Running with docker
Use this repo in your own git project
Conversion of parameters from Tensorflow to Pytorch
References

Quick start

Install:

# With pip:
pip install facenet-pytorch

# or clone this repo, removing the '-' to allow python imports:
git clone https://github.com/timesler/facenet-pytorch.git facenet_pytorch

# or use a docker container (see https://github.com/timesler/docker-jupyter-dl-gpu):
docker run -it --rm timesler/jupyter-dl-gpu pip install facenet-pytorch && ipython

In python, import facenet-pytorch and instantiate models:

from facenet_pytorch import MTCNN, InceptionResnetV1

# If required, create a face detection pipeline using MTCNN:
mtcnn = MTCNN(image_size=<image_size>, margin=<margin>)

# Create an inception resnet (in eval mode):
resnet = InceptionResnetV1(pretrained='vggface2').eval()

Process an image:

from PIL import Image

img = Image.open(<image path>)

# Get cropped and prewhitened image tensor
img_cropped = mtcnn(img, save_path=<optional save path>)

# Calculate embedding (unsqueeze to add batch dimension)
img_embedding = resnet(img_cropped.unsqueeze(0))

# Or, if using for VGGFace2 classification
resnet.classify = True
img_probs = resnet(img_cropped.unsqueeze(0))

See help(MTCNN) and help(InceptionResnetV1) for usage and implementation details.

Pretrained models

See: models/inception_resnet_v1.py

The following models have been ported to pytorch (with links to download pytorch state_dict's):

Model name	LFW accuracy (as listed here)	Training dataset
20180408-102900 (111MB)	0.9905	CASIA-Webface
20180402-114759 (107MB)	0.9965	VGGFace2

There is no need to manually download the pretrained state_dict's; they are downloaded automatically on model instantiation and cached for future use in the torch cache. To use an Inception Resnet (V1) model for facial recognition/identification in pytorch, use:

from facenet_pytorch import InceptionResnetV1

# For a model pretrained on VGGFace2
model = InceptionResnetV1(pretrained='vggface2').eval()

# For a model pretrained on CASIA-Webface
model = InceptionResnetV1(pretrained='casia-webface').eval()

# For an untrained model with 100 classes
model = InceptionResnetV1(num_classes=100).eval()

# For an untrained 1001-class classifier
model = InceptionResnetV1(classify=True, num_classes=1001).eval()

Both pretrained models were trained on 160x160 px images, so will perform best if applied to images resized to this shape. For best results, images should also be cropped to the face using MTCNN (see below).

By default, the above models will return 512-dimensional embeddings of images. To enable classification instead, either pass classify=True to the model constructor, or you can set the object attribute afterwards with model.classify = True. For VGGFace2, the pretrained model will output logit vectors of length 8631, and for CASIA-Webface logit vectors of length 10575.

Example notebooks

Complete detection and recognition pipeline

Face recognition can be easily applied to raw images by first detecting faces using MTCNN before calculating embedding or probabilities using an Inception Resnet model. The example code at examples/infer.ipynb provides a complete example pipeline utilizing datasets, dataloaders, and optional GPU processing.

Face tracking in video streams

MTCNN can be used to build a face tracking system (using the MTCNN.detect() method). A full face tracking example can be found at examples/face_tracking.ipynb.

Finetuning pretrained models with new data

In most situations, the best way to implement face recognition is to use the pretrained models directly, with either a clustering algorithm or a simple distance metrics to determine the identity of a face. However, if finetuning is required (i.e., if you want to select identity based on the model's output logits), an example can be found at examples/finetune.ipynb.

Guide to MTCNN in facenet-pytorch

This guide demonstrates the functionality of the MTCNN module. Topics covered are:

Basic usage
Image normalization
Face margins
Multiple faces in a single image
Batched detection
Bounding boxes and facial landmarks
Saving face datasets

See the notebook on kaggle.

Performance comparison of face detection packages

This notebook demonstrates the use of three face detection packages:

facenet-pytorch
mtcnn
dlib

Each package is tested for its speed in detecting the faces in a set of 300 images (all frames from one video), with GPU support enabled. Performance is based on Kaggle's P100 notebook kernel. Results are summarized below.

Package	FPS (1080x1920)	FPS (720x1280)	FPS (540x960)
facenet-pytorch	12.97	20.32	25.50
facenet-pytorch (non-batched)	9.75	14.81	19.68
dlib	3.80	8.39	14.53
mtcnn	3.04	5.70	8.23

See the notebook on kaggle.

The FastMTCNN algorithm

This algorithm demonstrates how to achieve extremely efficient face detection specifically in videos, by taking advantage of similarities between adjacent frames.

See the notebook on kaggle.

Running with docker

The package and any of the example notebooks can be run with docker (or nvidia-docker) using:

docker run --rm -p 8888:8888
    -v ./facenet-pytorch:/home/jovyan timesler/jupyter-dl-gpu \
    -v <path to data>:/home/jovyan/data
    pip install facenet-pytorch && jupyter lab

Navigate to the examples/ directory and run any of the ipython notebooks.

See timesler/jupyter-dl-gpu for docker container details.

Use this repo in your own git project

To use this code in your own git repo, I recommend first adding this repo as a submodule. Note that the dash ('-') in the repo name should be removed when cloning as a submodule as it will break python when importing:

git submodule add https://github.com/timesler/facenet-pytorch.git facenet_pytorch

Alternatively, the code can be installed as a package using pip:

pip install facenet-pytorch

Conversion of parameters from Tensorflow to Pytorch

See: models/utils/tensorflow2pytorch.py

Note that this functionality is not needed to use the models in this repo, which depend only on the saved pytorch state_dict's.

Following instantiation of the pytorch model, each layer's weights were loaded from equivalent layers in the pretrained tensorflow models from davidsandberg/facenet.

The equivalence of the outputs from the original tensorflow models and the pytorch-ported models have been tested and are identical:

>>> compare_model_outputs(mdl, sess, torch.randn(5, 160, 160, 3).detach())

Passing test data through TF model

tensor([[-0.0142,  0.0615,  0.0057,  ...,  0.0497,  0.0375, -0.0838],
        [-0.0139,  0.0611,  0.0054,  ...,  0.0472,  0.0343, -0.0850],
        [-0.0238,  0.0619,  0.0124,  ...,  0.0598,  0.0334, -0.0852],
        [-0.0089,  0.0548,  0.0032,  ...,  0.0506,  0.0337, -0.0881],
        [-0.0173,  0.0630, -0.0042,  ...,  0.0487,  0.0295, -0.0791]])

Passing test data through PT model

tensor([[-0.0142,  0.0615,  0.0057,  ...,  0.0497,  0.0375, -0.0838],
        [-0.0139,  0.0611,  0.0054,  ...,  0.0472,  0.0343, -0.0850],
        [-0.0238,  0.0619,  0.0124,  ...,  0.0598,  0.0334, -0.0852],
        [-0.0089,  0.0548,  0.0032,  ...,  0.0506,  0.0337, -0.0881],
        [-0.0173,  0.0630, -0.0042,  ...,  0.0487,  0.0295, -0.0791]],
       grad_fn=<DivBackward0>)

Distance 1.2874517096861382e-06

In order to re-run the conversion of tensorflow parameters into the pytorch model, ensure you clone this repo with submodules, as the davidsandberg/facenet repo is included as a submodule and parts of it are required for the conversion.

References

David Sandberg's facenet repo: https://github.com/davidsandberg/facenet
F. Schroff, D. Kalenichenko, J. Philbin. FaceNet: A Unified Embedding for Face Recognition and Clustering, arXiv:1503.03832, 2015. PDF
Q. Cao, L. Shen, W. Xie, O. M. Parkhi, A. Zisserman. VGGFace2: A dataset for recognising face across pose and age, International Conference on Automatic Face and Gesture Recognition, 2018. PDF
D. Yi, Z. Lei, S. Liao and S. Z. Li. CASIAWebface: Learning Face Representation from Scratch, arXiv:1411.7923, 2014. PDF
K. Zhang, Z. Zhang, Z. Li and Y. Qiao. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks, IEEE Signal Processing Letters, 2016. PDF

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

Top Related Projects

Quick Overview

Pros

Cons

Code Examples

Getting Started

Competitor Comparisons

Pros of facenet

Cons of facenet

Code Comparison

Pros of face_recognition

Cons of face_recognition

Code Comparison

Pros of InsightFace

Cons of InsightFace

Code Comparison

Pros of OpenFace

Cons of OpenFace

Code Comparison

Pros of face.evoLVe

Cons of face.evoLVe

Code Comparison

Pros of DeepFace

Cons of DeepFace

Code Comparison

Convert designs to code with AI

README

Face Recognition Using Pytorch

Table of contents

Quick start

Pretrained models

Example notebooks

Complete detection and recognition pipeline

Face tracking in video streams

Finetuning pretrained models with new data

Guide to MTCNN in facenet-pytorch

Performance comparison of face detection packages

The FastMTCNN algorithm

Running with docker

Use this repo in your own git project

Conversion of parameters from Tensorflow to Pytorch

References

Top Related Projects

Convert designs to code with AI