Skip to main content
SpringerLink
Log in

Gesture Interfaces with Depth Sensors

  • Chapter
Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8200))

Abstract

Computers and other electronic devices shrink and the need for a human interface remains. This generates a tremendous interest in alternative interfaces such as touch-less gesture interfaces, which can create a large, generic interface with a small piece of hardware. However, the acceptance of novel interfaces is hard to predict and may challenge the required computer-vision algorithms in terms of robustness, latency, precision, and the complexity of the problems involved.

In this article, we provide an overview of current gesture interfaces that are based on depth sensors. The focus is on the algorithms and systems that operate in the near range and can recognize hand gestures of increasing complexity, from simple wipes to the tracking of a full hand-skeleton.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Pavlovic, V., Sharma, R., Huang, T.: Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 677–695 (1997)

    Article  Google Scholar 

  2. Haker, M., Böhme, M., Martinetz, T., Barth, E.: Deictic gestures with a time-of-flight camera. In: Kopp, S., Wachsmuth, I. (eds.) GW 2009. LNCS, vol. 5934, pp. 110–121. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  3. Droeschel, D., Stuckler, J., Behnke, S.: Learning to interpret pointing gestures with a time-of-flight camera. In: 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 481–488 (2011)

    Google Scholar 

  4. Kolb, A., Barth, E., Koch, R., Larsen, R.: Time-of-flight cameras in computer graphics. Computer Graphics Forum 29(1), 141–159 (2010)

    Article  Google Scholar 

  5. Böhme, M., Haker, M., Martinetz, T., Barth, E.: A facial feature tracker for human-computer interaction based on 3D Time-of-Flight cameras. International Journal of Intelligent Systems Technologies and Applications 5(3/4), 264–273 (2008)

    Article  Google Scholar 

  6. Haker, M., Böhme, M., Martinetz, T., Barth, E.: Self-organizing maps for pose estimation with a time-of-flight camera. In: Kolb, A., Koch, R. (eds.) Dyn3D 2009. LNCS, vol. 5742, pp. 142–153. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  7. Böhme, M., Haker, M., Martinetz, T., Barth, E.: Head tracking with combined face and nose detection. In: Proceedings of the IEEE International Symposium on Signals, Circuits & Systems (ISSCS), Iaşi, Romania (2009)

    Google Scholar 

  8. Böhme, M., Haker, M., Riemer, K., Martinetz, T., Barth, E.: Face detection using a time-of-flight camera. In: Kolb, A., Koch, R. (eds.) Dyn3D 2009. LNCS, vol. 5742, pp. 167–176. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  9. Böhme, M., Haker, M., Martinetz, T., Barth, E.: Shading constraint improves accuracy of time-of-flight measurements. Computer Vision and Image Understanding 114, 1329–1335 (2010)

    Article  Google Scholar 

  10. Holte, M., Moeslund, T., Fihl, P.: View-invariant gesture recognition using 3D optical flow and harmonic motion context. Computer Vision and Image Understanding 114(12), 1353–1361 (2010), Special issue on Time-of-Flight Camera Based Computer Vision

    Google Scholar 

  11. Haubner, N., Schwanecke, U., Dorner, R., Lehmann, S., Luderschmidt, J.: Recognition of dynamic hand gestures with time-of-flight cameras. In: Dörner, R., Krömker, D. (eds.) Self Integrating Systems for Better Living Environments: First Workshop, Sensyble. Number 1, pp. 7–13 (2010)

    Google Scholar 

  12. Kollorz, E., Penne, J., Hornegger, J., Barke, A.: Gesture recognition with a time-of-flight camera. Int. J. Intell. Syst. Technol. Appl. 5(3/4), 334–343 (2008)

    Google Scholar 

  13. Mo, Z., Neumann, U.: Real-time hand pose recognition using low-resolution depth images. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1499–1505 (2006)

    Google Scholar 

  14. Suryanarayan, P., Subramanian, A., Mandalapu, D.: Dynamic hand pose recognition using depth data. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 3105–3108 (August 2010)

    Google Scholar 

  15. Keskin, C., Kirac, F., Kara, Y., Akarun, L.: Randomized decision forests for static and dynamic hand shape classification. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 31–36 (2012)

    Google Scholar 

  16. Keskin, C., Kirac, F., Kara, Y., Akarun, L.: Real time hand pose estimation using depth sensors. In: Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K. (eds.) Consumer Depth Cameras for Computer Vision. Advances in Computer Vision and Pattern Recognition, pp. 119–137. Springer, London (2013)

    Chapter  Google Scholar 

  17. Kurakin, A., Zhang, Z., Liu, Z.: A real time system for dynamic hand gesture recognition with a depth sensor. In: 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO), pp. 1975–1979 (2012)

    Google Scholar 

  18. Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Efficient model-based 3D tracking of hand articulations using kinect. In: British Machine Vision Conference, Dundee, UK, vol. 2 (2011)

    Google Scholar 

  19. Doliotis, P., Athitsos, V., Kosmopoulos, D., Perantonis, S.: Hand shape and 3D pose estimation using depth data from a single cluttered frame. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Fowlkes, C., Wang, S., Choi, M.-H., Mantler, S., Schulze, J., Acevedo, D., Mueller, K., Papka, M. (eds.) ISVC 2012, Part I. LNCS, vol. 7431, pp. 148–158. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  20. Ren, Z., Yuan, J., Zhang, Z.: Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In: Proceedings of the 19th ACM International Conference on Multimedia (MM 2011), pp. 1093–1096. ACM, New York (2011)

    Google Scholar 

  21. Caputo, M., Denker, K., Dums, B., Umlauf, G.: 3D hand gesture recognition based on sensor fusion of commodity hardware. In: Reiterer, H., Deussen, O. (eds.) Mensch & Computer 2012: Interaktiv Informiert Allgegenwärtig und Allumfassend!?, München (Oldenbourg Verlag), pp. 293–302 (2012)

    Google Scholar 

  22. Reyes, M., Dominguez, G., Escalera, S.: Feature weighting in dynamic time warping for gesture recognition in depth data. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1182–1188 (2011)

    Google Scholar 

  23. Zetzsche, C., Barth, E., Wegmann, B.: The importance of intrinsically two-dimensional image features in biological vision and picture coding. In: Watson, A.B. (ed.) Digital Images and Human Vision, pp. 109–138. MIT Press (October 1993)

    Google Scholar 

  24. Erol, A., Bebis, G., Nicolescu, M., Boyle, R.D., Twombly, X.: Vision-based hand pose estimation: A review. Computer Vision and Image Understanding 108(12), 52–73 (2007), Special Issue on Vision for Human-Computer Interaction

    Google Scholar 

  25. Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 780–785 (1997)

    Article  Google Scholar 

  26. Campbell, L., Becker, D., Azarbayejani, A., Bobick, A., Pentland, A.: Invariant features for 3-D gesture recognition. In: Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, pp. 157–162 (1996)

    Google Scholar 

  27. Ballan, L., Taneja, A., Gall, J., Van Gool, L., Pollefeys, M.: Motion capture of hands in action using discriminative salient points. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 640–653. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  28. Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Markerless and efficient 26-DOF hand pose recovery. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 744–757. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  29. Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., Moore, R.: Real-time human pose recognition in parts from single depth images. Commun. ACM 56(1), 116–124 (2013)

    Article  Google Scholar 

  30. Lahamy, H., Litchi, D.: Real-time hand gesture recognition using range cameras. In: Canadian Geomatics Conference (CGC), vol. 10 (2010)

    Google Scholar 

  31. Malassiotis, S., Tsalakanidou, F., Mavridis, N., Giagourta, V., Grammalidis, N., Strintzis, M.: A face and gesture recognition system based on an active stereo sensor. In: Proceedings of the 2001 International Conference on Image Processing, vol. 3, pp. 955–958 (2001)

    Google Scholar 

  32. Breuer, P., Eckes, C., Müller, S.: Hand gesture recognition with a novel IR time-of-flight range camera–A pilot study. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2007. LNCS, vol. 4418, pp. 247–260. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  33. Zhu, X., Wong, K.Y.K.: Single-frame hand gesture recognition using color and depth kernel descriptors. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 2989–2992 (November 2012)

    Google Scholar 

  34. Liu, X., Fujimura, K.: Hand gesture recognition using depth data. In: Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 529–534 (2004)

    Google Scholar 

  35. Van den Bergh, M., Van Gool, L.: Combining RGB and ToF cameras for real-time 3D hand gesture interaction. In: IEEE Workshop on Applications of Computer Vision (WACV), pp. 66–72 (2011)

    Google Scholar 

  36. Trindade, P., Lobo, J., Barreto, J.: Hand gesture recognition using color and depth images enhanced with hand angular pose data. In: 2012 IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), pp. 71–76 (2012)

    Google Scholar 

  37. Ghobadi, S., Loepprich, O., Hartmann, K., Loffeld, O.: Hand segmentation using 2D/3D images. In: Cree, M.J. (ed.) IVCNZ 2007 Conference, University of Waikato, pp. 64–69 (2007)

    Google Scholar 

  38. Holte, M., Moeslund, T., Fihl, P.: Fusion of range and intensity information for view invariant gesture recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2008), pp. 1–7 (2008)

    Google Scholar 

  39. Uebersax, D., Gall, J., Van den Bergh, M., Van Gool, L.: Real-time sign language letter and word recognition from depth data. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 383–390 (2011)

    Google Scholar 

  40. Hernandez-Vela, A., Bautista, M., Perez-Sala, X., Ponce, V., Baro, X., Pujol, O., Angulo, C., Escalera, S.: BoVDW: Bag-of-visual-and-depth-words for gesture recognition. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 449–452 (2012)

    Google Scholar 

  41. Song, Y., Demirdjian, D., Davis, R.: Multi-signal gesture recognition using temporal smoothing hidden conditional random fields. In: 2011 IEEE International Conference on Automatic Face Gesture Recognition and Workshops (FG 2011), pp. 388–393 (2011)

    Google Scholar 

  42. Haker, M., Barth, E., Martinetz, T.: Method for the real-time-capable, computer-assisted analysis of an image sequence containing a variable pose, International patent WO/2010/130245 (filed: May 6, 2010)

    Google Scholar 

  43. Andersen, M.R., Jensen, T., Lisouski, P., Mortensen, A.K., Hansen, M.K., Gregersen, T., Ahrendt, P.: Kinect depth sensor evaluation for computer vision applications. Technical report ECE-TR-6, Department of Engineering Electrical and Computer Engineering, Aarhus University (2012)

    Google Scholar 

  44. State, A., Coleca, F., Barth, E., Martinetz, T.: Hand tracking with an extended self-organizing map. In: Estevez, P.A., Principe, J.C., Zegers, P. (eds.) Advances in Self-Organizing Maps. AISC, vol. 198, pp. 115–124. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  45. Coleca, F., Klement, S., Martinetz, T., Barth, E.: Real-time skeleton tracking for embedded systems. In: Proceedings of Multimedia Content and Mobile Devices SPIE Conference, vol. 8667 (2013)

    Google Scholar 

  46. Guan, H., Feris, R., Turk, M.: The isometric self-organizing map for 3D hand pose estimation. In: 7th International Conference on Automatic Face and Gesture Recognition (FGR 2006), pp. 263–268 (2006)

    Google Scholar 

  47. Caridakis, G., Karpouzis, K., Drosopoulos, A., Kollias, S.: Somm: Self organizing markov map for gesture recognition. Pattern Recognition Letters 31(1), 52–59 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Neuro- and Bioinformatics, University of Lübeck, 160 Ratzeburger Allee, 23562, Lübeck, Germany

    Foti Coleca, Thomas Martinetz & Erhardt Barth

  2. gestigon GmbH, Maria-Goeppert Straße 1, 23562, Lübeck, Germany

    Foti Coleca

Authors
  1. Foti Coleca
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Martinetz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erhardt Barth
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Pattern Recognition Group, University of Siegen, Siegen, Germany

    Marcin Grzegorzek

  2. Max-lanck-Institute, Graphics, Vision & Video Gruop, Saarbrücken, Germany

    Christian Theobalt

  3. Multimedia Information Processing Group, University of Kiel, Kiel, Germany

    Reinhard Koch

  4. Computer Graphics and Multimedia Systems Group, University of Siegen, Siegen, Germany

    Andreas Kolb

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Coleca, F., Martinetz, T., Barth, E. (2013). Gesture Interfaces with Depth Sensors. In: Grzegorzek, M., Theobalt, C., Koch, R., Kolb, A. (eds) Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications. Lecture Notes in Computer Science, vol 8200. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-44964-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-44964-2_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-44963-5

  • Online ISBN: 978-3-642-44964-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation