HandReader Advanced Techniques for Efficient Fingerspelling Recognition

We introduce three architecture each model possesses distinct advantages and achieves state-of-the-art results on the ChicagoFSWild and ChicagoFSWild+ datasets. For more information see our arxiv paper TBA.

Clone and install required python packages:

1. Create env for RGB and KP+RGB

   • create conda env with conda create -n RGB_KP python=3.9
   • install torch with pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu118
   • install turbojpeg with conda install conda-forge::pyturbojpeg=1.7.7
   • install other dependicies with pip install -r RGB_KP_reqs.txt

2. Create env for KP

   • create conda env with conda create -n KP python=3.9
   • install torch with pip install torch==2.3.1 torchvision==0.18.1 --index-url https://download.pytorch.org/whl/cu118
   • install turbojpeg with conda install conda-forge::pyturbojpeg=1.7.7
   • install other dependicies with pip install -r KP_reqs.txt

TBA

We provide models for each architecture trained on datasets ChicagoFSWild, ChicagoFSWild+, and Znaki, which can be downloaded from the link below.

python src/train.py --config-name <cgf_name.yaml>`

example to start training model for kp_rgb with dataset Znaki:

python src/train.py --config-name kp_rgb_Znaki.yaml`

python src/test_KP_RGB.py --config-name kp_rgb_Znaki.yaml

TBA

• Korotaev Pavel
• Surovtsev Petr
• Kapitanov Alexander
• Kvanchiani Karina
• Nagaev Alexander