Journal of Intelligent & Robotic Systems

, Volume 87, Issue 3–4, pp 583–599 | Cite as

Egocentric-Vision based Hand Posture Control System for Reconnaissance Robots

  • Peng Ji
  • Aiguo Song
  • Pengwen Xiong
  • Ping Yi
  • Xiaonong Xu
  • Huijun Li
Article
First Online:
Received:
Accepted:

Abstract

To facilitate full-loaded commandos to control reconnaissance robots, in this paper, we propose a wearable hand posture control system based on egocentric-vision by imitating the sign language interaction way among commandos. Considering the characteristics of the egocentric-vision on the battlefield, such as complicated backgrounds, large ego-motions and extreme transitions in lighting, a new hand detector based on Binary Edge HOG Block (BEHB) features is proposed to extract articulated postures from the egocentric-vision. Different from many other methods that use skin color cues, our proposed hand detector adopts contour cues and part-based voting idea. This means that our algorithm can be used on the battlefield even in dark environment, because infrared cameras can be used to get contour images rather than skin color images. The experiment result shows that the proposed hand detector can get a better posture detection result on the NUS hand posture dataset II. To improve hand recognition accuracy, a deep ensemble hybrid classifier is proposed by combing hybrid CNN-SVM classifier and ensemble technique. Compared with other state-of-art algorithms, the proposed classifier yields a recognition accuracy of 97.72 % on the NUS hand posture dataset II. At last, to reduce misjudgments during consecutive posture switches, a vote filter is proposed and applied to the sequence of the recognition results. The scout experiment shows that our wearable hand posture control system is more suitable than traditional hand-held controllers for full-loaded commandos to control reconnaissance robots.

Keywords

Egocentric-vision Hand detection Hand posture recognition Convolutional Neural Network Reconnaissance robot 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Rautaray, S.S., Agrawal, A.: Vision based hand gesture recognition for human computer interaction: a survey. Artif. Intell. Rev. 43(1), 1–54 (2015)CrossRefGoogle Scholar
  2. 2.
    Li, C., Kitani, K.M.: Pixel-level hand detection in ego-centric videos. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (2013)Google Scholar
  3. 3.
    Betancourt, A., Lopez, M.M., Regazzoni, C.S., Rauterberg, M.: A sequential classifier for hand detection in the framework of egocentric vision. Paper presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2014)Google Scholar
  4. 4.
    Dardas, N.H., Georganas, N.D.: Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques. IEEE T Instrum. Meas. 60(11), 3592–3607 (2011)CrossRefGoogle Scholar
  5. 5.
    Kontschieder, P., Riemenschneider, H., Donoser, M., Bischof, H.: Discriminative Learning of Contour Fragments for Object Detection. Paper presented at the Proceedings Of The British Machine Vision Conference (2011)Google Scholar
  6. 6.
    Shotton, J., Blake, A., Cipolla, R.: Contour-based learning for object detection. Paper presented at the IEEE International Conference on Computer Vision (2005)Google Scholar
  7. 7.
    Haydar, J., Dalal, B., Hussainy, S., Khansa, L.E.: ASL fingerspelling translator glove. Int. J. Comput. Sci. Issues (IJCSI) 9(6) (2012)Google Scholar
  8. 8.
    Hsu, R.C., Jian, J.W., Lin, C.C., Lai, C.H., Liu, C.T.: Remotely controlling of mobile robots using gesture captured by the Kinect and recognized by machine learning method. Paper presented at the Proceedings of SPIE (2013)Google Scholar
  9. 9.
    Wang, C., Liu, Z., Chan, S.C.: Superpixel-based hand gesture recognition with kinect depth camera. IEEE T. Multimed. 17(1), 29–39 (2015)CrossRefGoogle Scholar
  10. 10.
    Yang, H.D., Park, A.Y., Lee, S.W.: Gesture spotting and recognition for human-robot interaction. IEEE T. Robot. 23(2), 256–270 (2007)CrossRefGoogle Scholar
  11. 11.
    Li, S.Z., Yu, B., Wu, W., Su, S.Z., Ji, R.R.: Feature learning based on SAE-PCA network for human gesture recognition in RGBD images. Neurocomputing 151(2), 565–573 (2015)CrossRefGoogle Scholar
  12. 12.
    Paulraj, M.P., Yaacob, S., Desa, H., Hema, C.R., Ab Majid, W.: Extraction of head and hand gesture features for recognition of sign language. Paper presented at the International Conference On Electronic Design (2008)Google Scholar
  13. 13.
    Pisharady, P.K., Vadakkepat, P., Loh, A.P.: Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds. INT J. Comput. Vis. 101(3SI), 403–419 (2013)CrossRefGoogle Scholar
  14. 14.
    Leibe, B., Leonardis, A., Schiele, B.: Robust object detection with interleaved categorization and segmentation. INT J. Comput. Vis. 77(1-3), 259–289 (2008)CrossRefGoogle Scholar
  15. 15.
    Opelt, A., Pinz, A., Zisserman, A.: A boundary-fragment-model for object detection. Paper presented at the 9th European Conference on Computer Vision (ECCV) (2006)Google Scholar
  16. 16.
    Yarlagadda, P., Monroy, A., Ommer, B.: Voting by grouping dependent parts. Paper presented at the 11th European Conference on Computer Vision(ECCV) (2010)Google Scholar
  17. 17.
    Schlecht, J., Ommer, B.: Contour-based object detection. Paper presented at the 22nd British Machine Vision Conference (2011)Google Scholar
  18. 18.
    Gu, C.H., Lim, J.J., Arbelaez, P., Malik, J.: Recognition using Regions Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (2009)Google Scholar
  19. 19.
    Mittal, A., Zisserman, A., Torr, P.: Hand detection using multiple proposals. Paper presented at the 22nd British Machine Vision Conference (2011)Google Scholar
  20. 20.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. Paper presented at the IEEE COMPUTER SOCIETY Conference On Computer Vision And Pattern Recognition (2005)Google Scholar
  21. 21.
    Tang, A., Lu, K., Wang, Y.F., Huang, J., Li, H.Q.: A real-time hand posture recognition system using deep neural networks. ACM T Intel. Syst. Tec. 6(212) (2015)Google Scholar
  22. 22.
    Niu, X.X., Suen, C.Y.: A novel hybrid CNN-SVM classifier for recognizing handwritten digits. Pattern Recogn. 45(4), 1318–1325 (2012)CrossRefGoogle Scholar
  23. 23.
    Huang, F.J., Lecun, Y.: Large-scale learning with SVM and convolutional for generic object categorization. Paper presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006)Google Scholar
  24. 24.
    Wozniak, M., Grana, M., Corchado, E.: A survey of multiple classifier systems as hybrid systems. Inform. Fusion 16(SI), 3–17 (2014)CrossRefGoogle Scholar
  25. 25.
    Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (2012)Google Scholar
  26. 26.
    Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Convolutional neural network committees for handwritten character classification. Paper presented at the 11th International Conference on Document Analysis and Recognition (ICDAR) (2011)Google Scholar
  27. 27.
    Maqsood, I., Khan, M.R., Abraham, A.: An ensemble of neural networks for weather forecasting. Neural Comput. Appl. 13(2), 112–122 (2004)CrossRefGoogle Scholar
  28. 28.
    Qian, K., Song, A.G., Bao, J.T., Zhang, H.T.: Small Teleoperated Robot for Nuclear Radiation and Chemical Leak Detection. Int. J Adv. Robot. Syst. 9(70) (2012)Google Scholar
  29. 29.
    Maire, M., Arbelaez, P., Fowlkes, C., Malik, J.: Using contours to detect and localize junctions in natural images. Paper presented at the IEEE COMPUTER Society Conference On Computer Vision And Pattern Recognition (2008)Google Scholar
  30. 30.
    Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE T Pattern Anal. 26(5), 530–549 (2004)CrossRefGoogle Scholar
  31. 31.
    Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM T Intel. Syst. Tec. 2(273SI) (2011)Google Scholar
  32. 32.
    Sancho-Asensio, A., Orriols-Puig, A., Golobardes, E.: Robust on-line neural learning classifier system for data stream classification tasks. Soft Comput. 18(8), 1441–1461 (2014)CrossRefGoogle Scholar
  33. 33.
    Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., Poggio, T.: Robust object recognition with cortex-like mechanisms. IEEE T Pattern Anal. 29(3), 411–426 (2007)CrossRefGoogle Scholar
  34. 34.
    Triesch, J., von der Malsburg, C.: A system for person-independent hand posture recognition against complex backgrounds. IEEE T Pattern Anal. 23(12), 1449–1453 (2001)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Peng Ji
    • 1
  • Aiguo Song
    • 1
  • Pengwen Xiong
    • 2
  • Ping Yi
    • 1
  • Xiaonong Xu
    • 1
  • Huijun Li
    • 1
  1. 1.School of Instrument Science and EngineeringSoutheast UniversityNanjingChina
  2. 2.Information Engineering SchoolNanchang UniversityNanchangChina

Personalised recommendations