Paper | Supplementary | Demo

Hugging Face Spaces

Tao Yang ¹, Peiran Ren¹, Xuansong Xie¹, Lei Zhang ^1,2
1DAMO Academy, Alibaba Group, Hangzhou, China
²Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China

(2022年03月09日) Add GPEN-BFR-2048 for selfies.

(2021年12月29日) Add online demos Hugging Face Spaces. Many thanks to CJWBW and AK391.

(2021年12月16日) Release a simplified training code of GPEN. It differs from our implementation in the paper, but could achieve comparable performance. We strongly recommend to change the degradation model.

(2021年12月09日) Add face parsing to better paste restored faces back.

(2021年12月09日) GPEN can run on CPU now by simply discarding --use_cuda.

(2021年12月01日) GPEN can now work on a Windows machine without compiling cuda codes. Please check it out. Thanks to Animadversio. Alternatively, you can try GPEN-Windows. Many thanks to Cioscos.

(2021年10月22日) GPEN can now work with SR methods. A SR model trained by myself is provided. Replace it with your own model if necessary.

(2021年10月11日) The Colab demo for GPEN is available now google colab logo.

python pytorch cuda driver gcc

• Clone this repository:

git clone https://github.com/yangxy/GPEN.git
cd GPEN

• Download RetinaFace model and our pre-trained model (not our best model due to commercial issues) and put them into weights/.

  RetinaFace-R50 | ParseNet-latest | model_ir_se50 | GPEN-BFR-2048 | GPEN-BFR-512 | GPEN-BFR-512-D | GPEN-BFR-256 | GPEN-BFR-256-D | GPEN-Colorization-1024 | GPEN-Inpainting-1024 | GPEN-Seg2face-512 | rrdb_realesrnet_psnr

• Restore face images:

python demo.py --task FaceEnhancement --model GPEN-BFR-512 --in_size 512 --channel_multiplier 2 --narrow 1 --use_sr --use_cuda --save_face --indir examples/imgs --outdir examples/outs-bfr

• Colorize faces:

python demo.py --task FaceColorization --model GPEN-Colorization-1024 --in_size 1024 --use_cuda --indir examples/grays --outdir examples/outs-colorization

• Complete faces:

python demo.py --task FaceInpainting --model GPEN-Inpainting-1024 --in_size 1024 --use_cuda --indir examples/ffhq-10 --outdir examples/outs-inpainting

• Synthesize faces:

python demo.py --task Segmentation2Face --model GPEN-Seg2face-512 --in_size 512 --use_cuda --indir examples/segs --outdir examples/outs-seg2face

• Train GPEN for BFR with 4 GPUs:

CUDA_VISIBLE_DEVICES='0,1,2,3' python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 train_simple.py --size 1024 --channel_multiplier 2 --narrow 1 --ckpt weights --sample results --batch 2 --path your_path_of_croped+aligned_hq_faces (e.g., FFHQ)

When testing your own model, set --key g_ema.

Please check out run.sh for more details.

If our work is useful for your research, please consider citing:

@inproceedings{Yang2021GPEN,
 title={GAN Prior Embedded Network for Blind Face Restoration in the Wild},
 author={Tao Yang, Peiran Ren, Xuansong Xie, and Lei Zhang},
 booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
 year={2021}
}

© Alibaba, 2021. For academic and non-commercial use only.

We borrow some codes from Pytorch_Retinaface, stylegan2-pytorch, Real-ESRGAN, and GFPGAN.

If you have any questions or suggestions about this paper, feel free to reach me at yangtao9009@gmail.com.