Skip to main content
View expanded cover

Electromechanics and Robotics pp 271–283Cite as

Pilot Studies on Avrora Unior Car-Like Robot Control Using Gestures

Part of the Smart Innovation, Systems and Technologies book series (SIST,volume 232)

Abstract

Gesture recognition is not only an important communication channel in human-human interaction but it also allows a human to communicate with other intelligent devices. This paper presents a concept for controlling the car-like robot Avrora Unior locomotion using gestures. We created a list of 18 control commands that contains basic and compound commands. A group of 17 volunteers used this list to create individual control gestures independently. A small part of the obtained dataset of gestures was used with the Teachable machine service in order to preliminary evaluate a possibility of constructing a full-scale model and to train it appropriately. The obtained model demonstrated acceptable recognition rate. We also attempted to apply SURF and FLANN techniques for matching with the direct matching approach and the skeleton-based approach, but the matching results were not satisfactory.

Keywords

  • Gesture-based control
  • Car-like robot control
  • Machine learning
  • Posture matching
  • New dataset
  • Avrora unior car

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-981-16-2814-6_24
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   219.00
Price excludes VAT (USA)
  • ISBN: 978-981-16-2814-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   279.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    https://kpfu.ru/eng/itis/research/laboratory-of-intelligent-robotic-systems.

References

  1. Qadri, M.T., Asif, M.: Automatic number plate recognition system for vehicle identification using optical character recognition. In: 2009 International Conference on Education Technology and Computer, pp. 335–338. IEEE (2009)

    Google Scholar 

  2. He, L., Chao, Y., Suzuki, K., Wu, K.: Fast connected-component labeling. Pattern Recognit 42(9), 1977–1987 (2009)

    CrossRef  Google Scholar 

  3. Nguyen, H., Maclagan, S.J., Nguyen, T.D., Nguyen, T., Flemons, P., Andrews, K., Ritchie, E.G., Phung, D.: Animal recognition and identification with deep convolutional neural networks for automated wildlife monitoring. In: 2017 IEEE international conference on data science and advanced Analytics (DSAA), pp. 40–49. IEEE (2017)

    Google Scholar 

  4. Ray, S., Das, S., Sen, A.: An intelligent vision system for monitoring security and surveillance of atm. In: 2015 Annual IEEE India Conference (INDICON), pp. 1–5. IEEE (2015)

    Google Scholar 

  5. Sutoyo, R., Harefa, J., Chowanda, A.: Unlock screen application design using face expression on android smartphone. In: MATEC Web of Conferences. vol. 54, p. 05001, EDP Sciences (2016)

    Google Scholar 

  6. Cuevas, E., Díaz, M., Manzanares, M., Zaldivar, D., Perez-Cisneros, M.: An improved computer vision method for white blood cells detection. Computational and Mathematical Methods in Medicine (2013)

    Google Scholar 

  7. Lee, H., Chen, Y.P.P.: Image based computer aided diagnosis system for cancer detection. Expert Syst Appl 42(12), 5356–5365 (2015)

    CrossRef  Google Scholar 

  8. Al-Kaff, A., Moreno, F.M., de la Escalera, A., Armingol, J.M.: Intelligent vehicle for search, rescue and transportation purposes. In: 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), pp. 110–115. IEEE (2017)

    Google Scholar 

  9. Perez-Grau, F., Ragel, R., Caballero, F., Viguria, A., Ollero, A.: Semi-autonomous teleoperation of uavs in search and rescue scenarios. In: 2017 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1066–1074. IEEE (2017)

    Google Scholar 

  10. Shirwalkar, S., Singh, A., Sharma, K., Singh, N.: Telemanipulation of an industrial robotic arm using gesture recognition with kinect. In: 2013 International Conference on Control, Automation, Robotics and Embedded Systems (CARE), pp. 1–6. IEEE (2013)

    Google Scholar 

  11. Rashid, M., Han, X.: Gesture control of zigbee connected smart home internet of things. In: 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV), pp. 667–670. IEEE (2016)

    Google Scholar 

  12. Hsiao, H.S., Chen, J.C.: Using a gesture interactive game-based learning approach to improve preschool children’s learning performance and motor skills. Comput Educat 95, 151–162 (2016)

    CrossRef  Google Scholar 

  13. Rahman, A.M., Hossain, M.A., Parra, J., El Saddik, A.: Motion-path based gesture interaction with smart home services. In: Proceedings of the 17th ACM international conference on Multimedia, pp. 761–764 (2009)

    Google Scholar 

  14. Hussain, S., Schaffner, S., Moseychuck, D.: Applications of wireless sensor networks and rfid in a smart home environment. In: 2009 Seventh Annual Communication Networks and Services Research Conference, pp. 153–157. IEEE (2009)

    Google Scholar 

  15. Muñoz-Salinas, R., Medina-Carnicer, R., Madrid-Cuevas, F.J., Carmona-Poyato, A.: Depth silhouettes for gesture recognition. Pattern Recognit Lett 29(3), 319–329 (2008)

    CrossRef  Google Scholar 

  16. Pal, M.: Random forest classifier for remote sensing classification. Int J Remote Sensing 26(1), 217–222 (2005)

    CrossRef  Google Scholar 

  17. Rautaray, S.S.: Real time hand gesture recognition system for dynamic applications. Int J UbiComp (IJU) 3(1) (2012)

    Google Scholar 

  18. Vivek Veeriah, J., Swaminathan, P.: Robust hand gesture recognition algorithm for simple mouse control. Int J Comput Commun Eng 2(2), 219–221 (2013)

    CrossRef  Google Scholar 

  19. Galin, R., Meshcheryakov, R.: Review on human–robot interaction during collaboration in a shared workspace. In: International Conference on Interactive Collaborative Robotics, pp. 63–74. Springer (2019)

    Google Scholar 

  20. Malov, D., Edemskii, A., Saveliev, A.: Architecture of proactive localization service for cyber-physical system’s users. In: International Conference on Interactive Collaborative Robotics, pp. 10–18. Springer (2019)

    Google Scholar 

  21. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput Vision Image Understanding 110(3), 346–359 (2008)

    CrossRef  Google Scholar 

  22. Goel, A., Saxena, S.C., Bhanot, S.: Modified functional link artificial neural network. Int J Electri Comput Eng 1(1), 22–30 (2006)

    Google Scholar 

  23. Tang, G., Webb, P.: The design and evaluation of an ergonomic contactless gesture control system for industrial robots. J Robotics (2018)

    Google Scholar 

  24. Chen, S., Ma, H., Yang, C., Fu, M.: Hand gesture based robot control system using leap motion. In: International Conference on Intelligent Robotics and Applications, pp. 581–591. Springer (2015)

    Google Scholar 

  25. Mikadlicki, K., Pajor, M.: Real-time gesture control of a CNC machine tool with the use Microsoft Kinect sensor. Int J Sci Eng Res 6(9), 538–543 (2015)

    Google Scholar 

  26. Grif, H.S., Farcas, C.C.: Mouse cursor control system based on hand gesture. Procedia Technol 22, 657–661 (2016)

    CrossRef  Google Scholar 

  27. Song, S., Yan, D., Xie, Y.: Design of control system based on hand gesture recognition. In: 2018 IEEE 15th International Conference on Networking, Sensing and Control (ICNSC), pp. 1–4. IEEE (2018)

    Google Scholar 

  28. Phyo, A.S., Fukuda, H., Lam, A., Kobayashi, Y., Kuno, Y.: A human-robot interaction system based on calling hand gestures. In: International Conference on Intelligent Computing, pp. 43–52. Springer (2019)

    Google Scholar 

  29. Gao, X., Shi, L., Wang, Q.: The design of robotic wheelchair control system based on hand gesture control for the disabled. In: 2017 International Conference on Robotics and Automation Sciences (ICRAS), pp. 30–34. IEEE (2017)

    Google Scholar 

  30. Zhang, B., Yang, M., Yuan, W., Wang, C., Wang, B.: A novel system for guiding unmanned vehicles based on human gesture recognition. In: 2020 IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 345–350. IEEE (2020)

    Google Scholar 

  31. Zhang, Z.: Microsoft Kinect sensor and its effect. IEEE Multimedia 19(2), 4–10 (2012)

    CrossRef  Google Scholar 

  32. Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE Transa Cybernet 43(5), 1318–1334 (2013)

    CrossRef  Google Scholar 

  33. Safin, R., Lavrenov, R., Tsoy, T., Svinin, M., Magid, E.: Real-time video server implementation for a mobile robot. In: 2018 11th International Conference on Developments in eSystems Engineering (DeSE), pp. 180–185. IEEE (2018)

    Google Scholar 

  34. Magid, E., Lavrenov, R., Khasianov, A.: Modified spline-based path planning for autonomous ground vehicle. In: ICINCO (2), pp. 132–141 (2017)

    Google Scholar 

  35. Lavrenov, R., Zakiev, A.: Tool for 3d gazebo map construction from arbitrary images and laser scans. In: 2017 10th International Conference on Developments in eSystems Engineering (DeSE), pp. 256–261. IEEE (2017)

    Google Scholar 

  36. Imameev, D., Shabalina, K., Sagitov, A., Su, K.L., Magid, E.: Modelling Autonomous Parallel Parking Procedure for Car-Like Robot Avrora Unior in Gazebo Simulator, pp. 428–431 (2020)

    Google Scholar 

  37. Safin, R., Garipova, E., Lavrenov, R., Li, H., Svinin, M., Magid, E.: Hardware and software video encoding comparison. In: 2020 59th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), pp. 924–929. IEEE (2020)

    Google Scholar 

  38. Imameev, D., Zakiev, A., Tsoy, T., Bai, Y., Svinin, M., Magid, E.: Lidar-based parking spot search algorithm. In: Thirteenth International Conference on Machine Vision. vol. 11605, p. 1160502. International Society for Optics and Photonics (2021)

    Google Scholar 

  39. Shabalina, K., Sagitov, A., Su, K.L., Hsia, K.H., Magid, E.: Avrora unior car-like robot in gazebo environment. In: International Conference on Artificial Life and Robotics, pp. 116–119 (2019)

    Google Scholar 

  40. Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: Openpose: realtime multi-person 2d pose estimation using part affinity fields. IEEE Trans pattern Anal Mach Intell 43(1), 172–186 (2019)

    CrossRef  Google Scholar 

  41. Carney, M., Webster, B., Alvarado, I., Phillips, K., Howell, N., Griffith, J., Jongejan, J., Pitaru, A., Chen, A.: Teachable machine: Approachable web-based tool for exploring machine learning classification. In: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–8 (2020)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Russian Foundation for Basic Research (RFBR), project ID 19-58-70002. Forth and fifth authors acknowledge the support of the Japan Science and Technology Agency, the JST Strategic International Collaborative Research Program, Project No. 18065977.

Author information

Authors and Affiliations

  1. Laboratory of Intelligent Robotics Systems (LIRS), Institute of Information Technology and Intelligent Systems, Kazan Federal University, Kazan, Russia

    Nikita Nikiforov, Tatyana Tsoy, Ramil Safin & Evgeni Magid

  2. Information Science and Engineering Department, College of Information Science and Engineering, Ritsumeikan University, 1 -1 -1 Noji -higashi, Kusatsu, Shiga, 525-8577, Japan

    Yang Bai & Mikhail Svinin

Authors
  1. Nikita Nikiforov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tatyana Tsoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramil Safin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mikhail Svinin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Evgeni Magid
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia

    Dr. Andrey Ronzhin

  2. State University of Airspace Instrumentation, St. Petersburg, Russia

    Dr. Vladislav Shishlakov

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Nikiforov, N., Tsoy, T., Safin, R., Bai, Y., Svinin, M., Magid, E. (2022). Pilot Studies on Avrora Unior Car-Like Robot Control Using Gestures. In: Ronzhin, A., Shishlakov, V. (eds) Electromechanics and Robotics. Smart Innovation, Systems and Technologies, vol 232. Springer, Singapore. https://doi.org/10.1007/978-981-16-2814-6_24

Download citation