A Remote Puppet Control System for Humanoid Communication Robot

  • Toshiyuki HaramakiEmail author
  • Hiroaki Nishino
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1035)


When a speaker communicates a message to the listener, it can be expected to be transmitted more accurately by attaching an appropriate gesture to message. That theory is valid, whether the speaker and the listener are near or far away. However, for a speaker to deliver messages and gestures to a remote listener synchronously, it is necessary to have a real-time conversation system using video images. Therefore, in this research, we propose a mechanism for presenting gestures synchronized with gestures at minimal cost when sending messages to remote listeners. To realize that, we develop a method to move the robot near the speaker instead of conveying the gesture of the speaker on the video shot. In this proposed method, motion data of the hand of the speaker is acquired using a motion sensor, the data is processing, converting into information for moving the motor corresponding to the joint of the robot, and the information is transmitting to the remote robot, the robot moves with motion according to the intention of the speaker. To realize the proposed method, we prepared a mechanism to convert the hand gesture of the speaker into the motion data of the robot in real-time and manipulate the robot in a faraway place. In our system, we realize a function to transmit information of extreme urgency such as disaster information all at once to robots installed in each household. Based on these requirements, we developed a prototype of an information presentation system. By utilizing this system, we expect the speaker’s message to communicate with the multiple listeners effectively.


  1. 1.
    Shimizu, D., Haramaki, T., Nishino, H.: A mobile wireless network visualizer for assisting administrators. In: Proceedings of the 6th International Conference on Emerging Internet, Data and Web Technologies, pp. 651–662, February 2018CrossRefGoogle Scholar
  2. 2.
    Yatsuda, A., Haramaki, T., Nishino, H.: An unsolicited heat stroke alert system for the elderly. In: Proceedings of the 2017 IEEE International Conference on Consumer Electronics - Taiwan, pp. 347–348, June 2017Google Scholar
  3. 3.
    Okazaki, S., Haramaki, T., Nishino, H.: A safe driving support method using olfactory stimuli. In: Proceedings of the 12th International Conference on Complex, Intelligent and Software Intensive Systems, pp. 958–967, June 2018Google Scholar
  4. 4.
    Haramaki, T., Yatsuda, A., Nishino, H.: A robot assistant in an edge-computing-based safe driving support system. In: Proceedings of the 21st International Conference on Network-Based Information Systems, pp. 144–155, September 2018Google Scholar
  5. 5.
    Haramaki, T., Goto, K., Tsutsumi, H., Yatsuda, A., Nishino, H.: A real-time robot motion generation system based on human gesture. In: Proceedings of the 13th International Conference on Broad-Band Wireless Computing, Communication and Applications, pp. 135–146, October 2018Google Scholar
  6. 6.
    Matsui, D., Minato, T., Mac Dorman, K., Ishiguro, H.: Generating natural motion in an android by mapping human motion. In: Proceedings of the 18th IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3301–3308, July 2005Google Scholar
  7. 7.
    Patsadu, O., Nukoolkit, C., Watanapa, B.: Human gesture recognition using Kinect camera. In: Proceedings of the 9th International Conference on Computer Science and Software Engineering, pp. 28–32, May 2012Google Scholar
  8. 8.
    Cicirelli, G., Attolico, C., Guaragnella, C., D’Orazio, T.: A kinect-based gesture recognition approach for a natural human robot interface. Int. J. Adv. Robot. Syst. 12(3), 22 (2015)CrossRefGoogle Scholar
  9. 9.
    Williams, K., Breazeal, C.: Reducing driver task load and promoting sociability through an affective intelligent driving agent (AIDA). In: Proceedings of IFIP Conference on Human-Computer Interaction, pp. 619–626, September 2013CrossRefGoogle Scholar
  10. 10.
    Zeng, L., Einert, B., Pitkin, A., Weber, G.: HapticRein: design and development of an interactive haptic rein for a guidance robot. In: Proceedings of International Conference on Computers Helping People with Special Needs, pp. 94–101, July 2018Google Scholar
  11. 11.
    Peng, L., Hou, Z., Peng, L., Luo, L., Wang, W.: Robot assisted rehabilitation of the arm after stroke: prototype design and clinical evaluation. Sci. China Inf. Sci. 60, 073201:1–073201:7 (2017)Google Scholar
  12. 12.
    Miyachi, T., Iga, S., Furuhata, T.: Human robot communication with facilitators for care robot innovation. In: Proceedings of International Conference on Knowledge Based and Intelligent Information and Engineering Systems, pp. 1254–1262, September 2017CrossRefGoogle Scholar
  13. 13.
    Maimone, A., Fuchs, H.: Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras. In: Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 137–146, October 2011Google Scholar
  14. 14.
    Adalgeirsson, S.O., Breazeal, C.: MeBot: a robotic platform for socially embodied telepresence. In: Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction, pp. 15–22, March 2010Google Scholar
  15. 15.
    Wang, J., Tsao, V., Fels, S., Chan, B.: Tippy the telepresence robot. In: Proceedings of International Conference on Entertainment Computing, pp. 358–361, October 2011Google Scholar
  16. 16.
    Tachi, S.: Real-time remote robotics-toward networked telexistence. IEEE Trans. Comput. Graph. Appl. 18(6), 6–9 (1998)CrossRefGoogle Scholar
  17. 17.
    Tachi, S., Minamizawa, K., Furukawa, M., Fernando, C.L.: Telexistence - from 1980 to 2012. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5440–5441, October 2012Google Scholar
  18. 18.
    TELEXISTENCE inc. Accessed June 2019
  19. 19.
    Leap Motion. Accessed June 2019
  20. 20.
    Sota. Accessed June 2019. (in Japanese)
  21. 21.
    MQTT. Accessed June 2019
  22. 22.
    Mosquitto. Accessed June 2019
  23. 23.
    Eclipse Paho. Accessed June 2019

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Division of Computer Science and Intelligent Systems, Faculty of Science and TechnologyOita UniversityOitaJapan

Personalised recommendations