Robust Children Behavior Tracking for Childcare Assisting Robot by Using Multiple Kinect Sensors

  • Bin ZhangEmail author
  • Tomoaki Nakamura
  • Rena Ushiogi
  • Takayuki Nagai
  • Kasumi Abe
  • Takashi Omori
  • Natsuki Oka
  • Masahide Kaneko
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9979)


Recently, the requirement for the high qualified childcare schools keeps increasing, but the number of qualified nursery teachers is far from enough. Developing a childcare assisting robot is highly necessary to help the works of nursery teachers. To work like a human nursery teacher, the first challenge for the robot is to understand the behaviors of the children automatically so that the robot can give adaptive reactions to the children. In this paper, we developed a robust children behavior tracking system by using multiple Kinect sensors. Each of the child is detected and recognized by integrating his/her personal features of face, color and motion. The tracking process is realized by using the Markov Chain Monte Carlo (MCMC) particle filter. The experiments are conducted in a childcare school to show the usefulness of our system.


Childcare assisting robot Children behavior tracking Children recognition MCMC particle filter 



This work was supported by Grant-in-Aid for Scientific Research on Innovative Areas 26118003. Special thanks to all the members in the nursery school.


  1. 1.
    Shiomi, M., Hagita, N.: Preliminary investiga-tion of supporting child-care at an intelligent playroom. In: The 2nd International Conference on Human Agent Interaction (2014)Google Scholar
  2. 2.
    Hwang, I., Jang, H., Nachman, L., Song, J.: Exploring inter-child behavioral relativity in a shared social environment: a field study in a kindergarten. In: Proceedings of the 12th ACM International Conference on Ubiquitous Computing, pp. 271–280 (2010)Google Scholar
  3. 3.
    Srivastava, M., Muntz, R., Potkonjak, M.: Smart kindergarten: sensor-based wireless networks for smart developmental problem solving environments. In: The 7th Annual International Conference on Mobile Computing and Networking, pp. 132–138 (2001)Google Scholar
  4. 4.
    Tanaka, F., Matsuzoe, S.: Children teach a care-receiving robot to promote their learning: Field experiments in a classroom for vocabulary learning. J. Hum. Robot Interact. 1(1), 78–95 (2012)CrossRefGoogle Scholar
  5. 5.
    Hieida, C., Abe, K., Attamimi, M., Shimotomai, T., Nagai, T., Omori, T.: Physical embodied communication between robots and children: An approach for relationship building by holding hands. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3291–3298 (2014)Google Scholar
  6. 6.
    Shiomi, M., Hagita, N.: Social acceptance of a childcare support robot system. In: 24th IEEE International Symposium on Robot and Human Interactive Communication, pp. 13–18 (2015)Google Scholar
  7. 7.
    Brscic, D., Kanda, T., Ikeda, T., Miyashita, T.: Person tracking in large public spaces using 3-D range sensors. IEEE Trans. Hum. Mach. Syst. 43, 522–534 (2013)CrossRefGoogle Scholar
  8. 8.
    Choi, W., Pantofaru, C., Savarese, S.: A general framework for tracking multiple people from a moving camera. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1157–1591 (2013)Google Scholar
  9. 9.
    Rusu, R., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation, pp. 1–4 (2011)Google Scholar
  10. 10.
    He, L., Chao, Y., Suzuki, K.: A run-based two-scan labeling algorithm. IEEE Trans. Image Process. 17(5), 749–756 (2008)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Takigawa, E., Hosoi, S., Kawade, M.: Gender, age, human race identification and estimation technology of facial attributes-OKAO vision. Image Lab 15(4) (2004)Google Scholar
  12. 12.
    Bradski, G.R.: Real time face and object tracking as a component of a perceptual user interface. In: 4th IEEE Workshop on Applications of Computer Vision, pp. 214–219 (1998)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Bin Zhang
    • 1
    Email author
  • Tomoaki Nakamura
    • 1
  • Rena Ushiogi
    • 2
  • Takayuki Nagai
    • 1
  • Kasumi Abe
    • 1
  • Takashi Omori
    • 3
  • Natsuki Oka
    • 4
  • Masahide Kaneko
    • 1
  1. 1.The University of Electro-CommunicationsChofuJapan
  2. 2.Otsuma Women’s UniversityChiyodaJapan
  3. 3.Tamagawa UniversityMachidaJapan
  4. 4.Kyoto Institute of TechnologyKyotoJapan

Personalised recommendations