Robot-Supported Pointing Interaction for Intelligent Environments

  • Mark Prediger
  • Andreas Braun
  • Alexander Marinc
  • Arjan Kuijper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8530)


A natural interaction with appliances in smart environment is a highly desired form of controlling the surroundings using intuitively learned interpersonal means of communication. Hand and arm gestures, recognized by depth cameras, are a popular representative of this interaction paradigm. However they usually require stationary units that limit applicability in larger environments. To overcome this problem we are introducing a self-localizing mobile robot system that autonomously follows the user in the environment, in order to recognize performed gestures independent from the current user position. We have realized a prototypical implementation using a custom robot platform and evaluated the system with various users.


Gesture recognition service robots smart environments 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Valli, A.: The design of natural interaction. Multimed. Tools Appl. 38, 295–305 (2008)CrossRefGoogle Scholar
  2. 2.
    Braun, A., Kamieth, F.: Passive identification and control of arbitrary devices in smart environments. In: Jacko, J.A. (ed.) Human-Computer Interaction, Part III, HCII 2011. LNCS, vol. 6763, pp. 147–154. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. Commun. ACM. 56, 116–124 (2013)CrossRefGoogle Scholar
  4. 4.
    Caon, M., Yue, Y., Tscherrig, J., Mugellini, E., Abou Khaled, O.: Context-aware 3d gesture interaction based on multiple kinects. In: Ambient 2011, First Int. Conf. Ambient Comput. Appl. Serv. Technol., pp. 7–12 (2011)Google Scholar
  5. 5.
    Forlizzi, J., Disalvo, C.: Service Robots in the Domestic Environment: A Study of the Roomba Vacuum in the Home. In: Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, pp. 258–265 (2006)Google Scholar
  6. 6.
    Graf, B., Parlitz, C., Hägele, M.: Robotic home assistant Care-O-bot® 3-product vision and innovation platform. In: Jacko, J.A. (ed.) HCI International 2009, Part II. LNCS, vol. 5611, pp. 312–320. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Tonin, L., Carlson, T., Leeb, R., Millán, J.d.R.: Brain-controlled telepresence robot by motor-disabled people. In: Conf. Proc. IEEE Eng. Med. Biol. Soc., pp. 4227–4230 (2011)Google Scholar
  8. 8.
    Nickel, K., Stiefelhagen, R.: Visual recognition of pointing gestures for human–robot interaction. Image Vis. Comput. 25, 1875–1884 (2007)CrossRefGoogle Scholar
  9. 9.
    Grisetti, G., Stachniss, C., Burgard, W.: Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters. IEEE Trans. Robot. 23, 34–46 (2007)CrossRefGoogle Scholar
  10. 10.
    Newcombe, R.A., Davison, A.J., Izadi, S., Kohli, P., Hilliges, O., Shotton, J., Molyneaux, D., Hodges, S., Kim, D., Fitzgibbon, A.: KinectFusion: Real-time dense surface mapping and tracking. In: Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 127–136. IEEE (2011)Google Scholar
  11. 11.
    Doisy, G., Jevti, A., Lucet, E., Edan, Y.: Adaptive Person-Following Algorithm Based on Depth Images and Mapping*. In: Proceedings of Workshop on Robot Motion Planning (2012)Google Scholar
  12. 12.
    Ko, N., Simmons, R.: The lane-curvature method for local obstacle avoidance. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1615–1621 (1998)Google Scholar
  13. 13.
    Gockley, R., Forlizzi, J., Simmons, R.: Natural person-following behavior for social robots. In: Proceeding of the ACM/IEEE International Conference on Human-Robot Interaction - HRI 2007, pp. 17–24. ACM Press, New York (2007)CrossRefGoogle Scholar
  14. 14.
    Hiroi, Y., Matsunaka, S., Ito, A.: Mobile Robot System with Semi-Autonomous Navigation Using Simple and Robust Person Following Behavior. J. Man, Mach. Technol. 1, 44–62 (2012)CrossRefGoogle Scholar
  15. 15.
    Majewski, M., Braun, A., Marinc, A., Kuijper, A.: Visual Support System for Selecting Reactive Elements in Intelligent Environments. In: International Conference on Cyberworlds, pp. 251–255 (2012)Google Scholar
  16. 16.
    Marinc, A., Stockloew, C., Tazari, S.: 3D Interaction in AAL Environments Based on Ontologies. Fourth Ambient Assisted Living Congress (2012)Google Scholar
  17. 17.
    Wilson, A., Shafer, S.: XWand: UI for intelligent spaces. In: Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 545–552. ACM (2003)Google Scholar
  18. 18.
    Braun, A., Fischer, A., Marinc, A., Stocklöw, C., Majewski, M.: Context-based bounding volume morphing in pointing gesture applications. In: Kurosu, M. (ed.) Human-Computer Interaction, Part IV, HCII 2013. LNCS, vol. 8007, pp. 147–156. Springer, Heidelberg (2013)Google Scholar
  19. 19.
    Arulampalam, M.S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50, 174–188 (2002)CrossRefGoogle Scholar
  20. 20.
    MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, p. 14 (1967)Google Scholar
  21. 21.
    Quigley, M., Conley, K.: ROS: An open-source Robot Operating System. ICRA Work. Open Source Softw. 3 (2009)Google Scholar
  22. 22.
    Fox, D., Burgard, W., Thrun, S.: The dynamic window approach to collision avoidance. IEEE Robot. Autom. Mag. 4, 23–33 (1997)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Mark Prediger
    • 1
  • Andreas Braun
    • 2
  • Alexander Marinc
    • 2
  • Arjan Kuijper
    • 1
    • 2
  1. 1.Technische Universität DarmstadtDarmstadtGermany
  2. 2.Fraunhofer Institute for Computer Graphics Research IGDDarmstadtGermany

Personalised recommendations