Skip to main content
SpringerLink
Log in
Book cover

Proceedings of 15th International Conference on Electromechanics and Robotics "Zavalishin's Readings" pp 187–196Cite as

Hand Gestures Recognition Model for Augmented Reality Robotic Applications

  • Conference paper
  • First Online:

Part of the book series: Smart Innovation, Systems and Technologies ((SIST,volume 187))

Abstract

Augmented reality (AR) is a research promising field. Its main idea is to integrate and merge the virtual world with the real world. Augmented reality could improve or enhance our perception of the real world by integrating virtual objects. The existing hand gesture applications related to augmented reality can detect hand motion or track it in addition to the ability to construct a 3D model of tracked hand using markers and motion sensing devices like Kinect, LeapMotion, and AR/VR instruments. In this paper, a hand gesture recognition model based on deep convolutional neural network is proposed to be used in 3D virtual environments for robotic teleportation. This model is tested on HTC VIVE Pro AR/VR instruments using the VIVE eye and on a Kinect v2 to control an industrial manipulator in real time using only the hand movements in both online and offline control modes.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Ge, L., et al.: 3D hand shape and pose estimation from a single RGB image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10833–10842 (2019)

    Google Scholar 

  2. Newell, A., Yang, K., Deng J.: Stacked hourglass networks for human pose estimation. In: European Conference on Computer Vision, pp. 483–499 (2016)

    Google Scholar 

  3. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. Adv. Neural Inf. Process. Syst. 3844–3852 (2016)

    Google Scholar 

  4. Cai, Y., Ge, L., Cai, J., Yuan, J.: Weakly-supervised 3d hand pose estimation from monocular RGB images. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 666–682 (2018)

    Google Scholar 

  5. Murhij, Y., Serebrenny, V.: An application to simulate and control industrial robot in virtual reality environment integrated with IR stereo camera sensor. IFAC-PapersOnLine 52(25), 203–207 (2019). https://doi.org/10.1016/j.ifacol.2019.12.473

    Article  Google Scholar 

  6. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/cvpr.2016.90

  7. Verma, N., Boyer, E., Verbeek, J.: Feastnet: Feature steered graph convolutions for 3d shape analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2598–2606 (2018)

    Google Scholar 

  8. Fraunhofer-Anwendungszentrum Industrial Automation, smart factory-design space (2017)

    Google Scholar 

  9. Elia, V., Gnoni, M.G., Lanzilotto, A.: Evaluating the application of augmented reality devices in manufacturing from a process point of view: an AHP based model. Exp. Syst. Appl. 63, 187–197 (2016). https://doi.org/10.1016/j.eswa.2016.07.006

    Article  Google Scholar 

  10. Sacco, A.: Google glass takes flight at Boeing. 2016. http://www.cio.com/article/3095132/wearable-technology/google-glass-takes-flight-at-boeing.html. Accessed 26 Nov 2021

  11. Funk, M.: Augmented reality at the workplace: a context-aware assistive system using in-situ projection (2016)

    Google Scholar 

  12. Sand, O., Sebastian, B., Paelke, V., Carsten, R.: Virtual. Augmented Mixed Real. (2016). https://doi.org/10.1007/978-3-319-39907-2

    Article  Google Scholar 

  13. Yan, D., Hu, H.: Application of augmented reality and robotic technology in broadcasting: a survey. Robotics 6(3), 18 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Robotic Systems and Mechatronics Department, Bauman Moscow State Technical University, Moscow, Russia

    Youshaa Murhij & Vladimir Serebrenny

Authors
  1. Youshaa Murhij
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vladimir Serebrenny
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Youshaa Murhij .

Editor information

Editors and Affiliations

  1. St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia

    Andrey Ronzhin

  2. Saint Petersburg State University of Aerospace Instrumentation, St. Petersburg, Russia

    Vladislav Shishlakov

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Murhij, Y., Serebrenny, V. (2021). Hand Gestures Recognition Model for Augmented Reality Robotic Applications. In: Ronzhin, A., Shishlakov, V. (eds) Proceedings of 15th International Conference on Electromechanics and Robotics "Zavalishin's Readings". Smart Innovation, Systems and Technologies, vol 187. Springer, Singapore. https://doi.org/10.1007/978-981-15-5580-0_15

Download citation

Publish with us

Policies and ethics

Search

Navigation