Abstract
Gesture recognition is not only an important communication channel in human-human interaction but it also allows a human to communicate with other intelligent devices. This paper presents a concept for controlling the car-like robot Avrora Unior locomotion using gestures. We created a list of 18 control commands that contains basic and compound commands. A group of 17 volunteers used this list to create individual control gestures independently. A small part of the obtained dataset of gestures was used with the Teachable machine service in order to preliminary evaluate a possibility of constructing a full-scale model and to train it appropriately. The obtained model demonstrated acceptable recognition rate. We also attempted to apply SURF and FLANN techniques for matching with the direct matching approach and the skeleton-based approach, but the matching results were not satisfactory.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
https://kpfu.ru/eng/itis/research/laboratory-of-intelligent-robotic-systems.
References
Qadri, M.T., Asif, M.: Automatic number plate recognition system for vehicle identification using optical character recognition. In: 2009 International Conference on Education Technology and Computer, pp. 335–338. IEEE (2009)
He, L., Chao, Y., Suzuki, K., Wu, K.: Fast connected-component labeling. Pattern Recognit 42(9), 1977–1987 (2009)
Nguyen, H., Maclagan, S.J., Nguyen, T.D., Nguyen, T., Flemons, P., Andrews, K., Ritchie, E.G., Phung, D.: Animal recognition and identification with deep convolutional neural networks for automated wildlife monitoring. In: 2017 IEEE international conference on data science and advanced Analytics (DSAA), pp. 40–49. IEEE (2017)
Ray, S., Das, S., Sen, A.: An intelligent vision system for monitoring security and surveillance of atm. In: 2015 Annual IEEE India Conference (INDICON), pp. 1–5. IEEE (2015)
Sutoyo, R., Harefa, J., Chowanda, A.: Unlock screen application design using face expression on android smartphone. In: MATEC Web of Conferences. vol. 54, p. 05001, EDP Sciences (2016)
Cuevas, E., DÃaz, M., Manzanares, M., Zaldivar, D., Perez-Cisneros, M.: An improved computer vision method for white blood cells detection. Computational and Mathematical Methods in Medicine (2013)
Lee, H., Chen, Y.P.P.: Image based computer aided diagnosis system for cancer detection. Expert Syst Appl 42(12), 5356–5365 (2015)
Al-Kaff, A., Moreno, F.M., de la Escalera, A., Armingol, J.M.: Intelligent vehicle for search, rescue and transportation purposes. In: 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), pp. 110–115. IEEE (2017)
Perez-Grau, F., Ragel, R., Caballero, F., Viguria, A., Ollero, A.: Semi-autonomous teleoperation of uavs in search and rescue scenarios. In: 2017 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1066–1074. IEEE (2017)
Shirwalkar, S., Singh, A., Sharma, K., Singh, N.: Telemanipulation of an industrial robotic arm using gesture recognition with kinect. In: 2013 International Conference on Control, Automation, Robotics and Embedded Systems (CARE), pp. 1–6. IEEE (2013)
Rashid, M., Han, X.: Gesture control of zigbee connected smart home internet of things. In: 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV), pp. 667–670. IEEE (2016)
Hsiao, H.S., Chen, J.C.: Using a gesture interactive game-based learning approach to improve preschool children’s learning performance and motor skills. Comput Educat 95, 151–162 (2016)
Rahman, A.M., Hossain, M.A., Parra, J., El Saddik, A.: Motion-path based gesture interaction with smart home services. In: Proceedings of the 17th ACM international conference on Multimedia, pp. 761–764 (2009)
Hussain, S., Schaffner, S., Moseychuck, D.: Applications of wireless sensor networks and rfid in a smart home environment. In: 2009 Seventh Annual Communication Networks and Services Research Conference, pp. 153–157. IEEE (2009)
Muñoz-Salinas, R., Medina-Carnicer, R., Madrid-Cuevas, F.J., Carmona-Poyato, A.: Depth silhouettes for gesture recognition. Pattern Recognit Lett 29(3), 319–329 (2008)
Pal, M.: Random forest classifier for remote sensing classification. Int J Remote Sensing 26(1), 217–222 (2005)
Rautaray, S.S.: Real time hand gesture recognition system for dynamic applications. Int J UbiComp (IJU) 3(1) (2012)
Vivek Veeriah, J., Swaminathan, P.: Robust hand gesture recognition algorithm for simple mouse control. Int J Comput Commun Eng 2(2), 219–221 (2013)
Galin, R., Meshcheryakov, R.: Review on human–robot interaction during collaboration in a shared workspace. In: International Conference on Interactive Collaborative Robotics, pp. 63–74. Springer (2019)
Malov, D., Edemskii, A., Saveliev, A.: Architecture of proactive localization service for cyber-physical system’s users. In: International Conference on Interactive Collaborative Robotics, pp. 10–18. Springer (2019)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput Vision Image Understanding 110(3), 346–359 (2008)
Goel, A., Saxena, S.C., Bhanot, S.: Modified functional link artificial neural network. Int J Electri Comput Eng 1(1), 22–30 (2006)
Tang, G., Webb, P.: The design and evaluation of an ergonomic contactless gesture control system for industrial robots. J Robotics (2018)
Chen, S., Ma, H., Yang, C., Fu, M.: Hand gesture based robot control system using leap motion. In: International Conference on Intelligent Robotics and Applications, pp. 581–591. Springer (2015)
Mikadlicki, K., Pajor, M.: Real-time gesture control of a CNC machine tool with the use Microsoft Kinect sensor. Int J Sci Eng Res 6(9), 538–543 (2015)
Grif, H.S., Farcas, C.C.: Mouse cursor control system based on hand gesture. Procedia Technol 22, 657–661 (2016)
Song, S., Yan, D., Xie, Y.: Design of control system based on hand gesture recognition. In: 2018 IEEE 15th International Conference on Networking, Sensing and Control (ICNSC), pp. 1–4. IEEE (2018)
Phyo, A.S., Fukuda, H., Lam, A., Kobayashi, Y., Kuno, Y.: A human-robot interaction system based on calling hand gestures. In: International Conference on Intelligent Computing, pp. 43–52. Springer (2019)
Gao, X., Shi, L., Wang, Q.: The design of robotic wheelchair control system based on hand gesture control for the disabled. In: 2017 International Conference on Robotics and Automation Sciences (ICRAS), pp. 30–34. IEEE (2017)
Zhang, B., Yang, M., Yuan, W., Wang, C., Wang, B.: A novel system for guiding unmanned vehicles based on human gesture recognition. In: 2020 IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 345–350. IEEE (2020)
Zhang, Z.: Microsoft Kinect sensor and its effect. IEEE Multimedia 19(2), 4–10 (2012)
Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE Transa Cybernet 43(5), 1318–1334 (2013)
Safin, R., Lavrenov, R., Tsoy, T., Svinin, M., Magid, E.: Real-time video server implementation for a mobile robot. In: 2018 11th International Conference on Developments in eSystems Engineering (DeSE), pp. 180–185. IEEE (2018)
Magid, E., Lavrenov, R., Khasianov, A.: Modified spline-based path planning for autonomous ground vehicle. In: ICINCO (2), pp. 132–141 (2017)
Lavrenov, R., Zakiev, A.: Tool for 3d gazebo map construction from arbitrary images and laser scans. In: 2017 10th International Conference on Developments in eSystems Engineering (DeSE), pp. 256–261. IEEE (2017)
Imameev, D., Shabalina, K., Sagitov, A., Su, K.L., Magid, E.: Modelling Autonomous Parallel Parking Procedure for Car-Like Robot Avrora Unior in Gazebo Simulator, pp. 428–431 (2020)
Safin, R., Garipova, E., Lavrenov, R., Li, H., Svinin, M., Magid, E.: Hardware and software video encoding comparison. In: 2020 59th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), pp. 924–929. IEEE (2020)
Imameev, D., Zakiev, A., Tsoy, T., Bai, Y., Svinin, M., Magid, E.: Lidar-based parking spot search algorithm. In: Thirteenth International Conference on Machine Vision. vol. 11605, p. 1160502. International Society for Optics and Photonics (2021)
Shabalina, K., Sagitov, A., Su, K.L., Hsia, K.H., Magid, E.: Avrora unior car-like robot in gazebo environment. In: International Conference on Artificial Life and Robotics, pp. 116–119 (2019)
Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: Openpose: realtime multi-person 2d pose estimation using part affinity fields. IEEE Trans pattern Anal Mach Intell 43(1), 172–186 (2019)
Carney, M., Webster, B., Alvarado, I., Phillips, K., Howell, N., Griffith, J., Jongejan, J., Pitaru, A., Chen, A.: Teachable machine: Approachable web-based tool for exploring machine learning classification. In: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–8 (2020)
Acknowledgements
This work was supported by the Russian Foundation for Basic Research (RFBR), project ID 19-58-70002. Forth and fifth authors acknowledge the support of the Japan Science and Technology Agency, the JST Strategic International Collaborative Research Program, Project No. 18065977.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Nikiforov, N., Tsoy, T., Safin, R., Bai, Y., Svinin, M., Magid, E. (2022). Pilot Studies on Avrora Unior Car-Like Robot Control Using Gestures. In: Ronzhin, A., Shishlakov, V. (eds) Electromechanics and Robotics. Smart Innovation, Systems and Technologies, vol 232. Springer, Singapore. https://doi.org/10.1007/978-981-16-2814-6_24
Download citation
DOI: https://doi.org/10.1007/978-981-16-2814-6_24
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-2813-9
Online ISBN: 978-981-16-2814-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)