Skip to main content
SpringerLink
Log in

Architecture and applications of the FingerMouse: a smart stereo camera for wearable computing HCI

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

In this paper we present a visual input HCI system for wearable computers, the FingerMouse. It is a fully integrated stereo camera and vision processing system, with a specifically designed ASIC performing stereo block matching at 5 Mpixel/s (e.g. QVGA 320  ×  240 at 30 fps) and a disparity range of 47, consuming 187 mW (78 mW in the ASIC). It is button-sized (43 mm  ×  18 mm) and can be worn on the body, capturing the user’s hand and processing in real-time its coordinates as well as a 1-bit image of the hand segmented from the background. Alternatively, the system serves as a smart depth camera, delivering foreground segmentation and tracking, depth maps and standard images, with a processing latency smaller than 1 ms. This paper describes the FingerMouse functionality and its applications, and how the specific architecture outperforms other systems in size, latency and power consumption.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Crowley JL, Coutaz J, Bérard F (2000) Perceptual user interfaces: things that see. Commun ACM 43:54–ff

    Article  Google Scholar 

  2. Card SK, Newell A, Moran TP (1983) The psychology of human–computer interaction. Lawrence Erlbaum, Mahwah

    Google Scholar 

  3. de la Hamette P, Tröster G (2006) Fingermouse—a button size visual hand tracking and segmentation device. In: Proceedings of the 19th international conference on architecture of computing systems-ARCS 2006, pp 31–41

  4. Habili N, Lim CC, Moini A (2004) Segmentation of the face and hands in sign language video sequences using color and motion cues. IEEE Trans Circ Syst Video Technol

  5. Gandy M, Starner T, Auxier J, Ashbrook D (2000) The gesture pendant: a self-illuminating, wearable, infrared computer vision system for home automation control and medical monitoring. In: ISWC ’00—Proceedings of the 4th IEEE international symposium on wearable computers. IEEE Computer Society, Washington DC, pp 87

  6. de la Hamette P, von Waldkirch M, Tröster G (2004) Laser triangulation as a means of robust visual input for wearable computers. In: ISWC ’04—Proceedings of the 4th IEEE international symposium on wearable computers, Doctoral Colloquium. IEEE Computer Society, Washington DC, pp 18–20

  7. Blake A, Isard M (1998) Active contours, 1st edn. Springer Berlin Heidelberg, New York, ISBN: 3-540-76217-5

  8. Grewal, MS, Andrews AS (1993) Kalman filtering: theory and practice. Prentice-Hall, Englewood Cliffs

  9. Blake A, Curwen R, Andrew Z (1993) A framework for spatio-temporal control in the tracking of visual contours. Int J Comput Vis

  10. de la Hamette P, Lukowicz P, Tröster G, Svoboda T (2002) Fingermouse: awearable hand tracking system. In: Adjunct proceedings of the 4th international conference on ubiquitous computing

  11. Hartley RI, Zisserman A (2004) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, London, ISBN: 0521540518

  12. Faugeras O (1999) Three-dimensional computer vision: a geometric viewpoint, 3rd edn. MIT Press, Cambridge. ISBN: 0-262-06158-9

  13. Frisius G (1544) Camera obscura. Historic publication on projective camera geometry

  14. Porter RB, Bergmann NW (1997) A generic implementation framework for fpga based stereo matching. IEEE TENCON

  15. Woodfill J, Herzen BV (1997) Real-time stereo vision on the parts reconfigurable computer. The 5th annual IEEE symposium on FPGAs for custom computing machines

  16. Corke P, Dunn P (1999) Frame-rate stereopsis using non-parametric transforms and programmable logic. In: IEEE international conference on robotics and automation

  17. Kanade T, Yoshida A, Oda K, Kano H, Tanaka M (1996) A stereo machine for video-rate dense depth mapping and its new applications. In: IEEE computer society conference on computer vision and pattern recognition

  18. Konolige K (1997) Small vision systems: hardware and implementation. Eighth international symposium on robotics

  19. Egnal G, Wildes RP (2002) Detecting binocular half-occlusions: empirical comparisons of five approaches. IEEE Trans Pattern Anal Mach Intell

  20. Ayache N, Lustman F (1991) Trinocular stereo vision for robotics. IEEE Trans Pattern Anal Mach Intell

  21. Fua P (1991) A parallel stereo algorithm that produces dense depth maps and preserves image features. Technical report 1369, Unite de Recherche, INRIA-Sophia Antipolis

  22. Faugeras O, Hotz B, Mathieu H et al (1993) Real time correlation-based stereo: algorithm, implementations and applications. Technical report 2013, INRIA

  23. Anandan P (1989) A computational framework and an algorithm for the measurement of visual motion. Int J Comput Vis 283–310

  24. Ohm JR, Grüneberg K, Izquierdo E, Karl M Real time hardware system for stereoscopic videoconference with viewpoint adaptation. Image Commun 14, special issue on 3D technology

  25. Demirdjian D Estereo: a c++ library for real-time stereo estimation. (http://www.sourceforge.net/projects/estereo/)

  26. Yang R, Pollefeys M (2003) Multi-resolution real-time stereo on commodity graphics hardware. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, 2003

  27. Amft O, Lauffer M, Ossevoort S, Macaluso F, Lukowicz P, Tröster G (2004) Design of the qbic wearable computing platform. In: ASAP 2004—Proceedings of the 15th IEEE international conference on application-specific systems, architectures and processors

  28. Starner T, Weaver J, Pentland A (1997) A wearable computer based american sign language recognizer. In: Digest of papers, first international symposium on wearable computers

Download references

Acknowledgments

We would like to thank Andreas Burg and Marc Wegmüller for their assistance in the ASIC design process. Most of the hardware implementations of the new prototype were done within student projects. Therefore, we thank the students Roman Gmünder, Julian Heeb, Thomas Koch and Sven Kuonen for their hard work. We also thank Martin Lanz, for his assistance with the die-bonding. For her help in the scenario photography, we thank Claire Muller.

Author information

Authors and Affiliations

  1. Wearable Computing Lab, ETH Zurich, Switzerland

    Patrick de la Hamette & Gerhard Tröster

Authors
  1. Patrick de la Hamette
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerhard Tröster
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Patrick de la Hamette.

Rights and permissions

Reprints and permissions

About this article

Cite this article

de la Hamette, P., Tröster, G. Architecture and applications of the FingerMouse: a smart stereo camera for wearable computing HCI. Pers Ubiquit Comput 12, 97–110 (2008). https://doi.org/10.1007/s00779-006-0109-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-006-0109-0

Keywords

Search

Navigation