Abstract
The lack of caregivers in an aging society is a major social problem. Without assistance, many of the elderly and disabled are unable to perform daily tasks. One important daily activity is shopping in supermarkets. Pushing a shopping cart and moving it from shelf to shelf is tiring and laborious, especially for customers with certain disabilities or the elderly. To alleviate this problem, we develop a person following shopping support robot using a Kinect camera that can recognize customer shopping actions or activities. Our robot can follow within a certain distance behind the customer. Whenever our robot detects the customer performing a “hand in shelf” action in front of a shelf it positions itself beside the customer with a shopping basket so that the customer can easily put his or her product in the basket. Afterwards, the robot again follows the customer from shelf to shelf until he or she is done with shopping. We conduct our experiments in a real supermarket to evaluate its effectiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kobayashi, Y., Yamazaki, S., Takahashi, H., Fukuda, H., Kuno, Y.: Robotic shopping trolley for supporting the elderly. In: Lightner, N.J. (ed.) AHFE 2018. AISC, vol. 779, pp. 344–353. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-94373-2_38
Islam, M.M., Lam, A., Fukuda, H., Kobayashi, Y., Kuno, Y.: An intelligent shopping support robot: understanding shopping behavior from 2D skeleton data using GRU network. ROBOMECH J. 6(1), 1–10 (2019). https://doi.org/10.1186/s40648-019-0150-1
Islam, Md.M., Lam, A., Fukuda, H., Kobayashi, Y., Kuno, Y.: A person-following shopping support robot based on human pose skeleton data and LiDAR sensor. In: Huang, D.-S., Huang, Z.-K., Hussain, A. (eds.) ICIC 2019. LNCS (LNAI), vol. 11645, pp. 9–19. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26766-7_2
Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. arXiv preprint arXiv:1812.08008 (2018)
Microsoft Kinect SDK. http://www.microsoft.com/en-us/kinectforwindows/
Beymer, D., Konolige, K.: Tracking people from a mobile platform. In: Siciliano, B., Dario, P. (eds.) Experimental Robotics VIII Springer Tracts in Advanced Robotics, vol. 5. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-36268-1_20
Calisi, D., Iocchi, L., Leone, R.: Person following through appearance models and stereo vision using a mobile robot. In: VISAPP (Workshop on on Robot Vision), pp. 46–56, March 2007
Takafumi, S., Takashi, Y., Manabu, N., Hideichi, N., Seiji, T., Nobuto, M.: Person following robot with vision-based and sensor fusion tracking algorithm. In: Zhihui, X. (ed.) Computer Vision, p. 538 (2008)
Ilias, B., Nagarajan, R., Murugappan, M., Helmy, K., Awang Omar, A.S., Abdul Rahman, M.A.: Hospital nurse following robot: hardware development and sensor integration. Int. J. Med. Eng. Inform. 6(1), 1–13 (2014)
Hiroi, Y., Matsunaka, S., Ito, A.: A mobile robot system with semi-autonomous navigation using simple and robust person following behavior. J. Man Mach. Technol. (JMMT) 1(1), 44–62 (2012)
Germa, T., Lerasle, F., Ouadah, N., Cadenat, V., Devy, M.: Vision and RFID-based person tracking in crowds from a mobile robot. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5591–5596. IEEE (2009)
Nishimura S., Takemura H., Mizoguchi H.: Development of attachable modules for robotizing daily items-person following shopping cart robot. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), pp 1506–1511. IEEE, New York (2007)
Kohtsuka, T., Onozato, T., Tamura, H., Katayama, S., Kambayashi, Y.: Design of a control system for robot shopping carts. In: König, A., Dengel, A., Hinkelmann, K., Kise, K., Howlett, R.J., Jain, L.C. (eds.) KES 2011. LNCS (LNAI), vol. 6881, pp. 280–288. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23851-2_29
Hu, Y., Cao, L., Lv, F., Yan, S., Gong, Y., Huang, T.S.: Action detection in complex scenes with spatial and temporal ambiguities. In: IEEE 12th International Conference on Computer Vision, pp. 128–135. IEEE (2009)
Haritaoglu, I., Beymer, D., Flickner, M.: Ghost 3D: detecting body posture and parts using stereo. In: Proceedings of the Workshop on Motion and Video Computing, pp. 175–180. IEEE (2002)
Lao, W., Han, J., De With, P.H.: Automatic video-based human motion analyzer for consumer surveillance system. Consum. Electron. IEEE Trans. 55(2), 591–598 (2009)
Acknowledgement
This work was supported by JSPS KAKENHI Grant Number JP26240038.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Islam, M.M., Lam, A., Fukuda, H., Kobayashi, Y., Kuno, Y. (2020). Person-Following Shopping Support Robot Using Kinect Depth Camera Based on 3D Skeleton Tracking. In: Huang, DS., Premaratne, P. (eds) Intelligent Computing Methodologies. ICIC 2020. Lecture Notes in Computer Science(), vol 12465. Springer, Cham. https://doi.org/10.1007/978-3-030-60796-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-60796-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60795-1
Online ISBN: 978-3-030-60796-8
eBook Packages: Computer ScienceComputer Science (R0)