Top Related Projects
GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration.
Real-ESRGAN aims at developing Practical Algorithms for General Image/Video Restoration.
Bringing Old Photo Back to Life (CVPR 2020 oral)
Official PyTorch Code and Models of "RePaint: Inpainting using Denoising Diffusion Probabilistic Models", CVPR 2022
[NeurIPS 2022] Towards Robust Blind Face Restoration with Codebook Lookup Transformer
Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
Quick Overview
GPEN (GAN Prior Embedded Network) is a deep learning-based face restoration method. It aims to restore low-quality facial images, addressing issues such as low resolution, blur, noise, and compression artifacts. GPEN leverages the power of Generative Adversarial Networks (GANs) to achieve high-quality face restoration results.
Pros
- Produces high-quality face restoration results
- Handles multiple degradation types simultaneously (e.g., blur, noise, low resolution)
- Offers pre-trained models for easy implementation
- Provides a user-friendly interface for both researchers and practitioners
Cons
- Requires significant computational resources for training and inference
- May struggle with extreme degradation cases or unusual facial features
- Limited to facial image restoration, not applicable to general image enhancement
- Dependency on specific versions of libraries may cause compatibility issues
Code Examples
- Loading a pre-trained GPEN model:
from gpen import GPEN
model = GPEN(model_path='pretrained/GPEN-BFR-512.pth',
size=512,
channel_multiplier=2,
narrow=1)
- Restoring a single image:
import cv2
import numpy as np
input_img = cv2.imread('low_quality_face.jpg')
restored_img = model.enhance(input_img, has_aligned=False)
cv2.imwrite('restored_face.jpg', restored_img)
- Batch processing multiple images:
import glob
input_files = glob.glob('input_folder/*.jpg')
for file in input_files:
img = cv2.imread(file)
restored = model.enhance(img, has_aligned=False)
cv2.imwrite(f'output_folder/{file.split("/")[-1]}', restored)
Getting Started
-
Clone the repository:
git clone https://github.com/yangxy/GPEN.git cd GPEN -
Install dependencies:
pip install -r requirements.txt -
Download pre-trained models from the provided links in the repository's README.
-
Use the following code to restore a face image:
from gpen import GPEN import cv2 model = GPEN(model_path='pretrained/GPEN-BFR-512.pth', size=512) img = cv2.imread('input_image.jpg') restored = model.enhance(img, has_aligned=False) cv2.imwrite('restored_image.jpg', restored)
Competitor Comparisons
GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration.
Pros of GFPGAN
- More advanced facial restoration capabilities, especially for severely degraded images
- Incorporates GAN-based upscaling for improved image quality
- Offers pre-trained models for different scenarios (e.g., anime faces, real-world faces)
Cons of GFPGAN
- Higher computational requirements due to more complex architecture
- May introduce artifacts in some cases, particularly with non-facial areas
- Less flexibility in terms of customization compared to GPEN
Code Comparison
GFPGAN:
from gfpgan import GFPGANer
restorer = GFPGANer(model_path='experiments/pretrained_models/GFPGANv1.3.pth', upscale=2)
restored_img, _ = restorer.enhance(img, has_aligned=False, only_center_face=False, paste_back=True)
GPEN:
from gpen import GPEN
model = GPEN(model_path='pretrained_models/GPEN-BFR-512.pth', size=512)
restored_img = model.enhance(img)
Both GFPGAN and GPEN are powerful tools for facial restoration, but GFPGAN tends to offer more advanced features at the cost of increased complexity. GPEN, while simpler, may be more suitable for users seeking a straightforward solution with lower computational requirements. The choice between the two depends on the specific use case and available resources.
Real-ESRGAN aims at developing Practical Algorithms for General Image/Video Restoration.
Pros of Real-ESRGAN
- More versatile, capable of enhancing various types of images beyond just faces
- Offers better general-purpose image upscaling and restoration
- Provides pre-trained models for different use cases
Cons of Real-ESRGAN
- May not perform as well on specific face restoration tasks
- Requires more computational resources for training and inference
- Less focused on facial details and features
Code Comparison
Real-ESRGAN:
from basicsr.archs.rrdbnet_arch import RRDBNet
from realesrgan import RealESRGANer
model = RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=23, num_grow_ch=32)
upsampler = RealESRGANer(scale=4, model_path='path/to/model', model=model)
GPEN:
from face_enhancement import FaceEnhancement
enhancer = FaceEnhancement(base_dir='./weights', size=512, model='GPEN-BFR-512')
enhanced_img = enhancer.process(img)
Both repositories focus on image enhancement, but Real-ESRGAN is more general-purpose, while GPEN specializes in face restoration. Real-ESRGAN offers broader applicability, while GPEN may provide better results for facial images. The code snippets demonstrate the different approaches, with Real-ESRGAN using a more complex architecture and GPEN offering a simpler interface for face enhancement.
Bringing Old Photo Back to Life (CVPR 2020 oral)
Pros of Bringing-Old-Photos-Back-to-Life
- Comprehensive approach: Handles multiple aspects of photo restoration (scratches, defects, colorization)
- Extensive documentation: Detailed instructions and explanations for usage and methodology
- Broader scope: Aims to restore entire images, not just faces
Cons of Bringing-Old-Photos-Back-to-Life
- More complex setup: Requires multiple dependencies and separate models
- Slower processing: May take longer to process images due to its comprehensive approach
- Less focused on facial details: May not provide as high-quality results for face restoration specifically
Code Comparison
GPEN (face enhancement focus):
from gpen import FaceEnhancement
enhancer = FaceEnhancement()
enhanced_face = enhancer.process(input_image)
Bringing-Old-Photos-Back-to-Life (full image restoration):
from restoration import Restoration
restorer = Restoration()
restored_image = restorer.restore(input_image, scratch_detection=True, colorization=True)
Both repositories offer valuable tools for image restoration, with GPEN focusing specifically on face enhancement and Bringing-Old-Photos-Back-to-Life providing a more comprehensive approach to full image restoration. The choice between them depends on the specific use case and desired level of control over the restoration process.
Official PyTorch Code and Models of "RePaint: Inpainting using Denoising Diffusion Probabilistic Models", CVPR 2022
Pros of RePaint
- Focuses on inpainting tasks, allowing for more targeted image restoration
- Utilizes a diffusion-based approach, potentially offering more natural-looking results
- Provides a more flexible framework for various image manipulation tasks
Cons of RePaint
- May require more computational resources due to the diffusion process
- Could have a steeper learning curve for users unfamiliar with diffusion models
- Might be less optimized for general face enhancement compared to GPEN
Code Comparison
GPEN (Face Enhancement):
import cv2
import torch
from gpen import GPEN
model = GPEN(model_path='GPEN-BFR-512.pth', device='cuda')
img = cv2.imread('input.jpg')
output = model.enhance(img, has_aligned=False)
RePaint (Inpainting):
from repaint import RePaint
model = RePaint(device='cuda')
img = Image.open('input.png')
mask = Image.open('mask.png')
output = model.inpaint(img, mask, num_samples=1)
Both repositories offer image enhancement capabilities, but they focus on different aspects. GPEN specializes in face enhancement, while RePaint provides a more general inpainting solution. The code examples demonstrate the different approaches, with GPEN directly enhancing faces and RePaint requiring a mask for inpainting tasks.
[NeurIPS 2022] Towards Robust Blind Face Restoration with Codebook Lookup Transformer
Pros of CodeFormer
- More versatile, handling both face restoration and color enhancement
- Offers a user-friendly web interface for easy interaction
- Provides more control over the balance between restoration and identity preservation
Cons of CodeFormer
- May require more computational resources due to its comprehensive approach
- Could have a steeper learning curve for users unfamiliar with advanced AI models
Code Comparison
CodeFormer:
from basicsr.utils import imwrite, img2tensor, tensor2img
from torchvision.transforms.functional import normalize
img = img2tensor(img / 255., bgr2rgb=True, float32=True)
normalize(img, (0.5, 0.5, 0.5), (0.5, 0.5, 0.5), inplace=True)
GPEN:
import cv2
import torch
img = cv2.imread(img_path, cv2.IMREAD_COLOR)
img = cv2.resize(img, (512, 512))
img = img.astype(np.float32) / 255.
img = torch.from_numpy(img).permute(2, 0, 1).unsqueeze(0).cuda()
Both repositories focus on face restoration, but CodeFormer offers a more comprehensive solution with additional features. GPEN, while potentially simpler to use, may not provide as much flexibility in terms of restoration quality and identity preservation. The code snippets show different approaches to image preprocessing, with CodeFormer using a more standardized normalization technique.
Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
Pros of IOPaint
- Offers a user-friendly web interface for image editing and inpainting
- Supports multiple AI models for inpainting, providing flexibility
- Includes features like object removal and image restoration
Cons of IOPaint
- May have a steeper learning curve for users unfamiliar with web-based tools
- Requires more setup and dependencies compared to GPEN
- Limited to inpainting and restoration tasks, while GPEN focuses on face enhancement
Code Comparison
IOPaint (Python):
@app.post("/inpaint")
async def inpaint(
image: UploadFile,
mask: UploadFile,
prompt: str = "",
negative_prompt: str = "",
num_inference_steps: int = 50,
guidance_scale: float = 7.5,
):
# Inpainting logic
GPEN (Python):
def process(self, img):
with torch.no_grad():
img = cv2.resize(img, (512, 512))
img = img.astype(np.float32) / 255.
img = torch.from_numpy(img).permute(2, 0, 1).unsqueeze(0).to(self.device)
output = self.model(img)
return output
Both repositories focus on image processing tasks, but IOPaint offers a broader range of features through its web interface, while GPEN specializes in face enhancement using a more straightforward approach.
Convert 
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
GAN Prior Embedded Network for Blind Face Restoration in the Wild
Paper | Supplementary | Demo | ModelScope
Tao Yang1, Peiran Ren1, Xuansong Xie1, Lei Zhang1,2
1DAMO Academy, Alibaba Group, Hangzhou, China
2Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China
Face Restoration
Selfie Restoration
Face Colorization
Face Inpainting
Conditional Image Synthesis (Seg2Face)
News
(2023-02-15) GPEN-BFR-1024 and GPEN-BFR-2048 are now publicly available. Please download them via [ModelScope2].
(2023-02-15) We provide online demos via [ModelScope1] and [ModelScope2].
(2022-05-16) Add x1 sr model. Add --tile_size to avoid OOM.
(2022-03-15) Add x4 sr model. Try --sr_scale.
(2022-03-09) Add GPEN-BFR-2048 for selfies. I have to take it down due to commercial issues. Sorry about that.
(2021-12-29) Add online demos 
(2021-12-16) Release a simplified training code of GPEN. It differs from our implementation in the paper, but could achieve comparable performance. We strongly recommend to change the degradation model.
(2021-12-09) Add face parsing to better paste restored faces back.
(2021-12-09) GPEN can run on CPU now by simply discarding --use_cuda.
(2021-12-01) GPEN can now work on a Windows machine without compiling cuda codes. Please check it out. Thanks to Animadversio. Alternatively, you can try GPEN-Windows. Many thanks to Cioscos.
(2021-10-22) GPEN can now work with SR methods. A SR model trained by myself is provided. Replace it with your own model if necessary.
(2021-10-11) The Colab demo for GPEN is available now 
.
Download models from Modelscope
- Install modelscope:
pip install "modelscope[cv]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
- Run the following codes:
import cv2
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from modelscope.outputs import OutputKeys
portrait_enhancement = pipeline(Tasks.image_portrait_enhancement, model='damo/cv_gpen_image-portrait-enhancement-hires')
result = portrait_enhancement('https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/marilyn_monroe_4.jpg')
cv2.imwrite('result.png', result[OutputKeys.OUTPUT_IMG])
It will automatically download the GPEN models. You can find the model in the local path ~/.cache/modelscope/hub/damo. Please note pytorch_model.pt, pytorch_model-2048.pt are respectively the 1024 and 2048 versions.
Usage
- Clone this repository:
git clone https://github.com/yangxy/GPEN.git
cd GPEN
-
Download RetinaFace model and our pre-trained model (not our best model due to commercial issues) and put them into
weights/.RetinaFace-R50 | ParseNet-latest | model_ir_se50 | GPEN-BFR-512 | GPEN-BFR-512-D | GPEN-BFR-256 | GPEN-BFR-256-D | GPEN-Colorization-1024 | GPEN-Inpainting-1024 | GPEN-Seg2face-512 | realesrnet_x1 | realesrnet_x2 | realesrnet_x4
-
Restore face images:
python demo.py --task FaceEnhancement --model GPEN-BFR-512 --in_size 512 --channel_multiplier 2 --narrow 1 --use_sr --sr_scale 4 --use_cuda --save_face --indir examples/imgs --outdir examples/outs-bfr
- Colorize faces:
python demo.py --task FaceColorization --model GPEN-Colorization-1024 --in_size 1024 --use_cuda --indir examples/grays --outdir examples/outs-colorization
- Complete faces:
python demo.py --task FaceInpainting --model GPEN-Inpainting-1024 --in_size 1024 --use_cuda --indir examples/ffhq-10 --outdir examples/outs-inpainting
- Synthesize faces:
python demo.py --task Segmentation2Face --model GPEN-Seg2face-512 --in_size 512 --use_cuda --indir examples/segs --outdir examples/outs-seg2face
- Train GPEN for BFR with 4 GPUs:
CUDA_VISIBLE_DEVICES='0,1,2,3' python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 train_simple.py --size 1024 --channel_multiplier 2 --narrow 1 --ckpt weights --sample results --batch 2 --path your_path_of_croped+aligned_hq_faces (e.g., FFHQ)
When testing your own model, set --key g_ema.
Please check out run.sh for more details.
Main idea
Citation
If our work is useful for your research, please consider citing:
@inproceedings{Yang2021GPEN,
title={GAN Prior Embedded Network for Blind Face Restoration in the Wild},
author={Tao Yang, Peiran Ren, Xuansong Xie, and Lei Zhang},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2021}
}
License
© Alibaba, 2021. For academic and non-commercial use only.
Acknowledgments
We borrow some codes from Pytorch_Retinaface, stylegan2-pytorch, Real-ESRGAN, and GFPGAN.
Contact
If you have any questions or suggestions about this paper, feel free to reach me at yangtao9009@gmail.com.
Top Related Projects
GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration.
Real-ESRGAN aims at developing Practical Algorithms for General Image/Video Restoration.
Bringing Old Photo Back to Life (CVPR 2020 oral)
Official PyTorch Code and Models of "RePaint: Inpainting using Denoising Diffusion Probabilistic Models", CVPR 2022
[NeurIPS 2022] Towards Robust Blind Face Restoration with Codebook Lookup Transformer
Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot