drawing

a matting method that supports various forms of interaction

Run the following command to install required packages.

conda create -n mattepro python==3.10
conda activate mattepro

pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https://download.pytorch.org/whl/cu121

pip install -r requirements.txt

Install detectron2 please following its document, you can also run following command

python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'

drawing

Step 1.
Download the pretrained weights of MattePro and MEMatte, and place them in the weights directory.

weights/
├── MattePro.pth
└── MeMatte.pth

Step 2.

python demo_image_matting.py

Tips for Click Mode:
Click the left mouse button on foreground region
Click the roll mouse button on unknown region
Click the right mouse button on background region

Step 1.
Download the weight of BirefNet, and place them in the weights directory.

weights/
├── MattePro.pth
├── MeMatte.pth
└── BiRefNet-HRSOD_D-epoch_130.pth

Step 2.

python demo_video_matting.py --input-path 'videos/hair.mp4'

ps:This mode is under research and currently only supports salient video matting.

examples:

B-Box Matting Mode

python evaluate.py --testset AIM500

ps: If your GPU memory is limited, please set pachify to True.

MattePro is a two-stage workflow.

This stage can be trained within 24 hours using a single GPU(memory > 10GB).

Step 1: Download Dataset

Segmentation Dataset: DUTS, DIS5K

Matting Dataset: DIM, Distinctions-646, AM-2K, P3M-10k, Transparent-460

Background Images: COCO2017 (training images)

And then configure the path in in config.yml

Step 2: Pretrained SAM2 Checkpoint

Download checkpoint of SAM2 large, and place it in the pretrained directory

pretrained/
└── sam2.1_hiera_large.pt

Step 3:

python train.py

For the second stage, you can refer to MEMatte for training, or directly use our pre-trained MEMatte checkpoint. You can even replace MEMatte with any trimap-based matting model(e.g., ViTMatte or AEMatter, though this will require some minor necessary modifications to the code), and it still produces excellent results.

The code is released under the MIT License. It is a short, permissive software license. Basically, you can do whatever you want as long as you include the original copyright and license notice in any copy of the software/source.

Our project is developed based on SEMatte, MEMatte, SimpleClick, BiRefNet. Thanks for their wonderful work!
If you have any questions, feel free to contact us through zchenyi007@gmail.com and 23120298@bjtu.edu.cn