|
| 1 | +## Environment |
| 2 | +- OS: MacOS El Capitan |
| 3 | +- Platform: Python 2.7.12 |
| 4 | +- Librarys: |
| 5 | + - OpenCV 2.4.13 |
| 6 | + - appscript |
| 7 | + |
| 8 | + |
| 9 | +## Demo Videos |
| 10 | +- Youtube: [Click here](https://youtu.be/CmBxUnp7XwM) |
| 11 | +- Youku: [Click here](http://v.youku.com/v_show/id_XMTc3MjI4MjQwOA==.html) |
| 12 | + |
| 13 | +## How to run it? |
| 14 | +- run it in python |
| 15 | +- press `'b'` to capture the background model (Remember to move your hand out of the blue rectangle) |
| 16 | +- press `'r'` to reset the backgroud model |
| 17 | +- press `'ESC'` to exit |
| 18 | + |
| 19 | +## Process |
| 20 | +#### Capture original image |
| 21 | + |
| 22 | +Capture video from camera and pick up a frame. |
| 23 | + |
| 24 | + |
| 25 | + |
| 26 | +#### Capture background model & Background subtraction |
| 27 | +Use background subtraction method called **Gaussian Mixture-based Background/Foreground Segmentation Algorithm** to subtract background. |
| 28 | + |
| 29 | +For more information about the method, check [Zivkovic2004](http://www.zoranz.net/Publications/zivkovic2004ICPR.pdf) |
| 30 | + |
| 31 | +Here I use the OpenCV's built-in function `BackgroundSubtractorMOG2` to subtract background. |
| 32 | + |
| 33 | +```python |
| 34 | +bgModel = cv2.BackgroundSubtractorMOG2(0, bgSubThreshold) |
| 35 | +``` |
| 36 | + |
| 37 | +Build a background subtractor model |
| 38 | + |
| 39 | + |
| 40 | + |
| 41 | +```python |
| 42 | +fgmask = bgModel.apply(frame) |
| 43 | +``` |
| 44 | +Apply the model to a frame |
| 45 | + |
| 46 | + |
| 47 | +```python |
| 48 | +res = cv2.bitwise_and(frame, frame, mask=fgmask) |
| 49 | +``` |
| 50 | + |
| 51 | +Get the foreground(hand) image |
| 52 | + |
| 53 | + |
| 54 | + |
| 55 | +#### Gaussian blur & Threshold |
| 56 | +```python |
| 57 | +gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) |
| 58 | +``` |
| 59 | +First convert the image to gray scale. |
| 60 | + |
| 61 | +```python |
| 62 | +blur = cv2.GaussianBlur(gray, (blurValue, blurValue), 0) |
| 63 | +``` |
| 64 | +By Gaussian blurring, we create smooth transition from one color to another and reduce the edge content. |
| 65 | + |
| 66 | + |
| 67 | + |
| 68 | +```python |
| 69 | +ret, thresh = cv2.threshold(blur, threshold, 255, cv2.THRESH_BINARY) |
| 70 | +``` |
| 71 | +We use thresholding to create binary images from grayscale images. |
| 72 | + |
| 73 | + |
| 74 | + |
| 75 | + |
| 76 | +#### Contour & Hull & Convexity |
| 77 | +We now need to find out the hand contour from the binary image we created before and detect fingers (or in other words, recognize gestures) |
| 78 | + |
| 79 | +```python |
| 80 | +contours, hierarchy = cv2.findContours(thresh1, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) |
| 81 | +``` |
| 82 | +This function will find all the contours from the binary image. We need to get the biggest contours (our hand) based on their area since we can assume that our hand will be the biggest contour in this situation. (it's obvious) |
| 83 | + |
| 84 | +After picking up our hand, we can create its hull and detect the defects by calling : |
| 85 | +```python |
| 86 | +hull = cv2.convexHull(res) |
| 87 | +defects = cv2.convexityDefects(res, hull) |
| 88 | +``` |
| 89 | + |
| 90 | + |
| 91 | + |
| 92 | + |
| 93 | +Now we have the number of fingers. How to use this information? It's based on your imagination... |
| 94 | + |
| 95 | +I add in a keyboard simulation package named **appscript** as interface to control Chrome's dinosaur game. |
| 96 | + |
| 97 | + |
| 98 | + |
| 99 | +---------------------- |
| 100 | +## References & Tutorials |
| 101 | + |
| 102 | +1. OpenCV documentation: |
| 103 | +http://docs.opencv.org/2.4.13/ |
| 104 | +2. Opencv python hand gesture recognition: |
| 105 | +http://creat-tabu.blogspot.com/2013/08/opencv-python-hand-gesture-recognition.html |
| 106 | +3. Mahaveerverma's hand gesture recognition project: |
| 107 | +[hand-gesture-recognition-opencv](https://github.com/mahaveerverma/hand-gesture-recognition-opencv) |
| 108 | + |
0 commit comments