Online robust action recognition based on a hierarchical model
- Original Article
- Published:
- Volume 30, pages 1021–1033, (2014)
- Cite this article
- Xinbo Jiang 1 ,
- Fan Zhong 1 ,
- Qunsheng Peng 2 &
- ...
- Xueying Qin 1,3
-
1783 Accesses
-
28 Citations
Abstract
Action recognition solely based on video data has known to be very sensitive to background activity, and also lacks the ability to discriminate complex 3D motion. With the development of commercial depth cameras, skeleton-based action recognition is becoming more and more popular. However, the skeleton-based approach is still very challenging because of the large variation in human actions and temporal dynamics. In this paper, we propose a hierarchical model for action recognition. To handle confusing motions, a motion-based grouping method is proposed, which can efficiently assign each video a group label, and then for each group, a pre-trained classifier is used for frame-labeling. Unlike previous methods, we adopt a bottom-up approach that first performs action recognition for each frame. The final action label is obtained by fusing the classification to its frames, with the effect of each frame being adaptively adjusted based on its local properties. To achieve online real-time performance and suppressing noise, bag-of-words is used to represent the classification features. The proposed method is evaluated using two challenge datasets captured by a Kinect. Experiments show that our method can robustly recognize actions in real-time.
This is a preview of subscription content, log in via an institution to check access.
Access this article
Subscribe and save
- Starting from 10 chapters or articles per month
- Access and download chapters and articles from more than 300k books and 2,500 journals
- Cancel anytime
Buy Now
Price includes VAT (Japan)
Instant access to the full article PDF.
Similar content being viewed by others
Complex Human Action Recognition Using a Hierarchical Feature Reduction and Deep Learning-Based Method
Explore related subjects
Discover the latest articles, books and news in related subjects, suggested using machine learning.Notes
In our implementation we adopt the standard implementation of KNN in OpenCV.
References
Matikainen, P., Hebert, M., Sukthankar, R.: Trajectons: action recognition through the motion analysis of tracked features. In: Workshop on Video-Oriented Object and Event Classification, ICCV 2009 (2009)
Wang, J., Liu, Z., Wu, Y., Yuan, J.: Mining actionlet ensemble for action recognition with depth cameras. In: CVPR’12, pp. 1290–1297 (2012)
Laptev, I.: On space-time interest points. Int. J. Comput. Vis. 64(2–3), 107–123 (2005)
Blank, M., Gorelick, L., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. In: The Tenth IEEE International Conference on Computer Vision (ICCV’05), pp. 1395–1402 (2005)
Kläser, A., Marszałek, M., Schmid, C.: A spatio-temporal descriptor based on 3d-gradients. In: British Machine Vision Conference, pp. 995–1004 (2008)
Willems, G., Tuytelaars, T., Gool, L.: An efficient dense and scale-invariant spatio-temporal interest point detector. In: Proceedings of the 10th European Conference on Computer Vision: Part II, ser. ECCV ’08, pp. 650–663. Springer, Berlin (2008)
Laptev, I., Marszałek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: Conference on Computer Vision and Pattern Recognition (2008)
Ke, Y., Sukthankar, R., Hebert, M.: Event detection in crowded videos. In: IEEE International Conference on Computer Vision (2007)
Shechtman, E., Irani, M.: Space-time behavior based correlation -or- how to tell if two underlying motion fields are similar without computing them? IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 29(11), 2045–2056 (2007)
Fathi, A., Mori, G.: Action recognition by learning mid-level motion features. In: CVPR (2008)
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, ser. CVPR ’11. Washington, DC, IEEE Computer Society, pp. 1297–1304 (2011)
Ellis, C., Masood, S.Z., Tappen, M.F., Laviola Jr, J.J., Sukthankar, R.: Exploring the trade-off between accuracy and observational latency in action recognition. Int. J. Comput. Vis. 101(3), 420–436 (Feb. 2013)
Fothergill, S., Mentis, H.M., Kohli, P., Nowozin, S.: Instructing people for training gestural interactive systems. In: Konstan, J.A., Chi, E.H., Höök, K. (eds.) ACM, pp. 1737–1746 (2012)
Yang, X., Zhang, C., Tian, Y.: Recognizing actions using depth motion maps-based histograms of oriented gradients. In: Proceedings of the 20th ACM international conference on Multimedia, ser. MM ’12, pp. 1057–1060. ACM, New York (2012)
Negin, F., Ozdemir, F., Akgul, C.B., Yuksel, K.A., Ercil, A.: A decision forest based feature selection framework for action recognition from rgb-depth cameras (2013)
Chatzis, S.P., Kosmopoulos, D.I., Doliotis, P.: A conditional random field-based model for joint sequence segmentation and classification. Pattern Recognit. 46(6), 1569–1578 (2013)
Acknowledgments
The authors gratefully acknowledge the anonymous reviewers for their comments to help us to improve our paper, and also thank Guofeng Wang for his enormous help in revising this paper. This work is supported by 973 program of China (No. 2009CB320802), NSF of China (Nos. U1035004, 61173070, 61202149), Key Projects in the National Science & Technology Pillar Program (No. 2013BAH39F00).
Rights and permissions
About this article
Cite this article
Jiang, X., Zhong, F., Peng, Q. et al. Online robust action recognition based on a hierarchical model. Vis Comput 30, 1021–1033 (2014). https://doi.org/10.1007/s00371-014-0923-8
Published:
Issue date:
DOI: https://doi.org/10.1007/s00371-014-0923-8
Share this article
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative