Advertisement

A Virtual Reality-Based Framework for Experiments on Perception of Manual Gestures

  • Sebastian Ullrich
  • Jakob T. Valvoda
  • Marc Wolter
  • Gisela Fehrmann
  • Isa Werth
  • Ludwig Jaeger
  • Torsten Kuhlen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5085)

Abstract

This work contributes an integrated and flexible approach to sign language processing in virtual environments that allows for interactive experimental evaluations with high ecological validity. Initial steps deal with real-time tracking and processing of manual gestures. Motion data is stereoscopically rendered in immersive virtual environments with varying spatial and representational configurations. Besides flexibility, the most important aspect is the seamless integration within a VR-based neuropsychological experiment software. Ongoing studies facilitated with this system contribute to the understanding of the cognition of sign language. The system is beneficial for experimenters because of the controlled and immersive three-dimensional environment enabling experiments with visual depth perception that can not be achieved with video presentations.

Keywords

Virtual Reality Virtual Environment Motion Data Sign Language Hand Gesture 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Fehrmann, G., Jäger, L.: Sprachbewegung und Raumerinnerung. Zur topographischen Medialität der Gebärdensprachen. In: der Bewegung, K. Kinästhetische Wahrnehmung und Probehandeln in virtuellen Welten. Publikationen zur Zeitschrift für Germanistik, vol. 8, pp. 311–341. Peter Lang, Bern (2004)Google Scholar
  2. 2.
    Lee, J., Kunii, T.L.: Visual translation: from native language to sign language. In: Proc. of IEEE Workshop on Visual Languages, USA, pp. 103–109 (1992)Google Scholar
  3. 3.
    Lebourque, T., Gibet, S.: High level specification and control of communication gestures: the GESSYCA system. In: Proc. of IEEE Computer Animation, Switzerland, pp. 24–35 (1999)Google Scholar
  4. 4.
    Heloir, A., Gibet, S., Multon, F., Courty, N.: Captured Motion Data Processing for Real Time Synthesis of Sign Language. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS, vol. 3881, pp. 168–171. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Cox, S., Lincoln, M., Tryggvason, J., Nakisa, M., Wells, M., Tutt, M., Abbott, S.: TESSA, a system to aid communication with deaf people. In: Proc. of ASSETS 2002, Fifth International ACM SIGCAPH Conference on Assistive Technologies, Scotland, pp. 205–212 (2002)Google Scholar
  6. 6.
    Kennaway, R.: Synthetic Animation of Deaf Signing Gestures. In: Wachsmuth, I., Sowa, T. (eds.) GW 2001. LNCS, vol. 2298, pp. 146–157. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Rezzoug, N., Gorce, P., Héloir, A., Gibet, S., Courty, N., Kamp, J.F., Multon, F., Pelachaud, C.: Virtual humanoids endowed with expressive communication gestures: The HuGEx project. In: Proc. of IEEE International Conference on Systems, Man, and Cybernetics, Taiwan (2006)Google Scholar
  8. 8.
    Ullrich, S., Valvoda, J.T., Prescher, A., Kuhlen, T.: Comprehensive Architecture for Simulation of the Human Body based on Functional Anatomy. In: Proc. of BVM 2007, Germany (2007)Google Scholar
  9. 9.
    Valvoda, J.T., Kuhlen, T., Bischof, C.: Interactive Virtual Humanoids for Virtual Environments. In: Short Paper Proc. of Eurographics Symposium on Virtual Environments, Portugal, pp. 9–12 (2006)Google Scholar
  10. 10.
    Valvoda, J.T., Kuhlen, T., Wolter, M., et al.: NeuroMan: A Comprehensive Software System for Neuropsychological Experiments. CyberPsychology and Behaviour 8(4), 366–367 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Sebastian Ullrich
    • 1
  • Jakob T. Valvoda
    • 1
  • Marc Wolter
    • 1
  • Gisela Fehrmann
    • 2
  • Isa Werth
    • 2
  • Ludwig Jaeger
    • 2
  • Torsten Kuhlen
    • 1
  1. 1.Virtual Reality GroupRWTH Aachen UniversityGermany
  2. 2.Deaf & Sign Language Research Team AachenRWTH Aachen UniversityGermany

Personalised recommendations