libfacedetection
An open source library for face detection in images. The face detection speed can reach 1000FPS.
Top Related Projects
Retinaface get 80.99% in widerface hard val using mobilenet0.25.
The world's simplest facial recognition api for Python and the command line
Pretrained Pytorch face detection (MTCNN) and facial recognition (InceptionResnet) models
State-of-the-art 2D and 3D Face Analysis Project
Quick Overview
libfacedetection is an open-source library for face detection in images. It is written in C++ and uses CNN-based algorithms to detect faces quickly and accurately. The library is designed to be lightweight and efficient, making it suitable for various applications, including mobile and embedded systems.
Pros
- High performance and speed, capable of processing 1080p images in real-time on modern CPUs
- Cross-platform compatibility, supporting Windows, Linux, macOS, and Android
- Lightweight and easy to integrate into existing projects
- Provides both C++ and Python interfaces
Cons
- Limited to face detection only, does not include face recognition or other advanced features
- May have lower accuracy compared to some state-of-the-art deep learning models
- Documentation could be more comprehensive for advanced usage scenarios
Code Examples
- Basic face detection in C++:
#include "facedetectcnn.h"
// Load image data into 'rgb_image'
int * pResults = facedetect_cnn(rgb_image, width, height, step);
for(int i = 0; i < (pResults ? *pResults : 0); i++)
{
short * p = ((short*)(pResults+1))+142*i;
int x = p[0];
int y = p[1];
int w = p[2];
int h = p[3];
// Process detected face...
}
- Face detection with Python bindings:
import cv2
from libfacedetection import detect_faces
image = cv2.imread("input.jpg")
faces = detect_faces(image)
for face in faces:
x, y, w, h = face[:4]
cv2.rectangle(image, (x, y), (x+w, y+h), (0, 255, 0), 2)
cv2.imwrite("output.jpg", image)
- Using the library with OpenCV in C++:
#include <opencv2/opencv.hpp>
#include "facedetectcnn.h"
cv::Mat image = cv::imread("input.jpg");
cv::Mat rgb_image;
cv::cvtColor(image, rgb_image, cv::COLOR_BGR2RGB);
int * pResults = facedetect_cnn(rgb_image.data, rgb_image.cols, rgb_image.rows, rgb_image.step);
// Process results as in the first example
Getting Started
-
Clone the repository:
git clone https://github.com/ShiqiYu/libfacedetection.git
-
Build the library:
cd libfacedetection mkdir build && cd build cmake .. make
-
Include the header and link the library in your C++ project:
#include "facedetectcnn.h" // Link with libfacedetection.so or facedetection.lib
-
For Python, install the package:
pip install libfacedetection
Competitor Comparisons
Retinaface get 80.99% in widerface hard val using mobilenet0.25.
Pros of Pytorch_Retinaface
- Utilizes the RetinaFace architecture, which is known for its high accuracy in face detection
- Implemented in PyTorch, offering flexibility and ease of use for deep learning researchers
- Provides pre-trained models and supports both CPU and GPU inference
Cons of Pytorch_Retinaface
- Generally slower inference speed compared to libfacedetection
- Requires more computational resources due to its complex architecture
- May have a steeper learning curve for users less familiar with PyTorch
Code Comparison
Pytorch_Retinaface:
from retinaface import RetinaFace
detector = RetinaFace()
faces = detector.detect_faces("path/to/image.jpg")
libfacedetection:
#include "facedetectcnn.h"
Mat image = imread("path/to/image.jpg");
int * pResults = facedetect_cnn(pBuffer, image.data, image.cols, image.rows, (int)image.step);
The Pytorch_Retinaface code snippet demonstrates a more Python-friendly interface, while libfacedetection shows a C++ implementation, highlighting the difference in language and usage complexity between the two libraries.
The world's simplest facial recognition api for Python and the command line
Pros of face_recognition
- Higher-level API, easier to use for beginners
- Includes face recognition capabilities, not just detection
- Supports multiple image formats and input methods
Cons of face_recognition
- Slower performance compared to libfacedetection
- Larger library size and more dependencies
- Less suitable for embedded systems or mobile devices
Code Comparison
face_recognition:
import face_recognition
image = face_recognition.load_image_file("image.jpg")
face_locations = face_recognition.face_locations(image)
face_encodings = face_recognition.face_encodings(image, face_locations)
libfacedetection:
#include "facedetectcnn.h"
unsigned char * result_buffer = new unsigned char[0x20000];
unsigned char * rgb_image_data = new unsigned char[width*height*3];
int * pResults = facedetect_cnn(result_buffer, rgb_image_data, width, height, 3);
face_recognition offers a more straightforward API for face detection and recognition tasks, making it easier for developers to implement these features quickly. However, libfacedetection provides better performance and is more suitable for resource-constrained environments. The code comparison shows that face_recognition uses a higher-level Python interface, while libfacedetection requires more low-level C++ programming.
Pretrained Pytorch face detection (MTCNN) and facial recognition (InceptionResnet) models
Pros of facenet-pytorch
- Implements the FaceNet architecture, providing face recognition capabilities
- Built on PyTorch, offering flexibility and ease of use for deep learning tasks
- Includes pre-trained models for quick deployment
Cons of facenet-pytorch
- Focused primarily on face recognition, not optimized for face detection
- May have higher computational requirements due to the deep learning approach
- Less suitable for embedded or resource-constrained environments
Code Comparison
facenet-pytorch:
from facenet_pytorch import MTCNN, InceptionResnetV1
mtcnn = MTCNN()
resnet = InceptionResnetV1(pretrained='vggface2').eval()
# Detect face and get embedding
img_cropped = mtcnn(img)
img_embedding = resnet(img_cropped.unsqueeze(0))
libfacedetection:
#include "facedetectcnn.h"
unsigned char * result_buffer = new unsigned char[0x20000];
int * pResults = facedetect_cnn(result_buffer, (unsigned char*)(rgbImageData), width, height, stride);
for(int i = 0; i < (pResults ? *pResults : 0); i++)
{
short * p = ((short*)(pResults+1))+142*i;
int x = p[0];
int y = p[1];
int w = p[2];
int h = p[3];
// Process detected face
}
State-of-the-art 2D and 3D Face Analysis Project
Pros of insightface
- More comprehensive face analysis toolkit, including recognition, detection, and alignment
- Supports multiple deep learning frameworks (MXNet, PyTorch, TensorFlow)
- Provides pre-trained models for various tasks and datasets
Cons of insightface
- Higher complexity and steeper learning curve
- Requires more computational resources due to its extensive features
- Larger codebase and dependencies
Code Comparison
insightface:
import insightface
model = insightface.app.FaceAnalysis()
model.prepare(ctx_id=0, det_size=(640, 640))
img = insightface.utils.get_image('t1.jpg')
faces = model.get(img)
libfacedetection:
#include "facedetectcnn.h"
Mat image = imread("image.jpg");
int * pResults = facedetect_cnn(pBuffer, (unsigned char*)(image.ptr(0)), image.cols, image.rows, (int)image.step);
insightface offers a higher-level Python API with more features, while libfacedetection provides a lower-level C++ implementation focused specifically on face detection. insightface is more versatile but may be overkill for simple face detection tasks, where libfacedetection could be more efficient.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
libfacedetection
This is an open source library for CNN-based face detection in images. The CNN model has been converted to static variables in C source files. The source code does not depend on any other libraries. What you need is just a C++ compiler. You can compile the source code under Windows, Linux, ARM and any platform with a C++ compiler.
SIMD instructions are used to speed up the detection. You can enable AVX2 if you use Intel CPU or NEON for ARM.
The model files are provided in src/facedetectcnn-data.cpp
(C++ arrays) & the model (ONNX) from OpenCV Zoo. You can try our scripts (C++ & Python) in opencv_dnn/
with the ONNX model. View the network architecture here.
Please note that OpenCV DNN does not support the latest version of YuNet with dynamic input shape. Please ensure you have the exact same input shape as the one in the ONNX model to run latest YuNet with OpenCV DNN.
examples/detect-image.cpp and examples/detect-camera.cpp show how to use the library.
The library was trained by libfacedetection.train.
How to use the code
You can copy the files in directory src/ into your project, and compile them as the other files in your project. The source code is written in standard C/C++. It should be compiled at any platform which supports C/C++.
Some tips:
- Please add facedetection_export.h file in the position where you copy your facedetectcnn.h files, add #define FACEDETECTION_EXPORT to facedetection_export.h file. See: issues #222
- Please add -O3 to turn on optimizations when you compile the source code using g++.
- Please choose 'Maximize Speed/-O2' when you compile the source code using Microsoft Visual Studio.
- You can enable OpenMP to speedup. But the best solution is to call the detection function in different threads.
You can also compile the source code to a static or dynamic library, and then use it in your project.
CNN-based Face Detection on Intel CPU
Using AVX2 instructions
Method | Time | FPS | Time | FPS |
---|---|---|---|---|
X64 | X64 | X64 | X64 | |
Single-thread | Single-thread | Multi-thread | Multi-thread | |
cnn (CPU, 640x480) | 50.02ms | 19.99 | 6.55ms | 152.65 |
cnn (CPU, 320x240) | 13.09ms | 76.39 | 1.82ms | 550.54 |
cnn (CPU, 160x120) | 3.61ms | 277.37 | 0.57ms | 1745.13 |
cnn (CPU, 128x96) | 2.11ms | 474.60 | 0.33ms | 2994.23 |
Using AVX512 instructions
Method | Time | FPS | Time | FPS |
---|---|---|---|---|
X64 | X64 | X64 | X64 | |
Single-thread | Single-thread | Multi-thread | Multi-thread | |
cnn (CPU, 640x480) | 46.47ms | 21.52 | 6.39ms | 156.47 |
cnn (CPU, 320x240) | 12.10ms | 82.67 | 1.67ms | 599.31 |
cnn (CPU, 160x120) | 3.37ms | 296.47 | 0.46ms | 2155.80 |
cnn (CPU, 128x96) | 1.98ms | 504.72 | 0.31ms | 3198.63 |
- Minimal face size ~10x10
- Intel(R) Core(TM) i7-7820X CPU @ 3.60GHz
- Multi-thread in 16 threads and 16 processors.
CNN-based Face Detection on ARM Linux (Raspberry Pi 4 B)
Method | Time | FPS | Time | FPS |
---|---|---|---|---|
Single-thread | Single-thread | Multi-thread | Multi-thread | |
cnn (CPU, 640x480) | 404.63ms | 2.47 | 125.47ms | 7.97 |
cnn (CPU, 320x240) | 105.73ms | 9.46 | 32.98ms | 30.32 |
cnn (CPU, 160x120) | 26.05ms | 38.38 | 7.91ms | 126.49 |
cnn (CPU, 128x96) | 15.06ms | 66.38 | 4.50ms | 222.28 |
- Minimal face size ~10x10
- Raspberry Pi 4 B, Broadcom BCM2835, Cortex-A72 (ARMv8) 64-bit SoC @ 1.5GHz
- Multi-thread in 4 threads and 4 processors.
Performance on WIDER Face
Run on default settings: scales=[1.], confidence_threshold=0.02, floating point:
AP_easy=0.887, AP_medium=0.871, AP_hard=0.768
Author
- Shiqi Yu, shiqi.yu@gmail.com
Contributors
All contributors who contribute at GitHub.com are listed here.
The contributors who were not listed at GitHub.com:
- Jia Wu (å´ä½³)
- Dong Xu (å¾æ )
- Shengyin Wu (ä¼å£å¯ )
Acknowledgment
The work was partly supported by the Science Foundation of Shenzhen (Grant No. 20170504160426188).
Citation
The master thesis of Mr. Wei Wu. All details of the algorithm are in the thesis. The thesis can be downloaded at å´ä¼ç¡å£«æ¯ä¸è®ºæ
@thesis{wu2023thesisyunet,
author = {å´ä¼},
title = {é¢åè¾¹ç¼è®¾å¤çé«ç²¾åº¦æ¯«ç§çº§äººè¸æ£æµææ¯ç 究},
type = {ç¡å£«å¦ä½è®ºæ},
institution = {åæ¹ç§æ大å¦},
year = {2023},
}
The paper for the main idea of this repository https://link.springer.com/article/10.1007/s11633-023-1423-y.
@article{wu2023miryunet,
title = {YuNet: A Tiny Millisecond-level Face Detector},
author = {Wu, Wei and Peng, Hanyang and Yu, Shiqi},
journal = {Machine Intelligence Research},
pages = {1--10},
year = {2023},
doi = {10.1007/s11633-023-1423-y},
publisher = {Springer}
}
The survey paper on face detection to evaluate different methods. It can be open-accessed at https://ieeexplore.ieee.org/document/9580485
@article{feng2022face,
author = {Feng, Yuantao and Yu, Shiqi and Peng, Hanyang and Li, Yan-Ran and Zhang, Jianguo},
journal = {IEEE Transactions on Biometrics, Behavior, and Identity Science},
title = {Detect Faces Efficiently: A Survey and Evaluations},
year = {2022},
volume = {4},
number = {1},
pages = {1-18},
doi = {10.1109/TBIOM.2021.3120412}
}
The loss used in training is EIoU, a novel extended IoU. The paper can be open-accessed at https://ieeexplore.ieee.org/document/9429909.
@article{peng2021eiou,
author = {Peng, Hanyang and Yu, Shiqi},
journal = {IEEE Transactions on Image Processing},
title = {A Systematic IoU-Related Method: Beyond Simplified Regression for Better Localization},
year = {2021},
volume = {30},
pages = {5032-5044},
doi = {10.1109/TIP.2021.3077144}
}
Top Related Projects
Retinaface get 80.99% in widerface hard val using mobilenet0.25.
The world's simplest facial recognition api for Python and the command line
Pretrained Pytorch face detection (MTCNN) and facial recognition (InceptionResnet) models
State-of-the-art 2D and 3D Face Analysis Project
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot