Virtual Reality

, Volume 4, Issue 2, pp 103–113 | Cite as

Creating natural language interfaces to VR systems

  • S. S. Everett
  • K. Wauchope
  • M. A. Pérez Quiñones


Two research projects are described that explore the use of spoken natural language interfaces to virtual reality (VR) systems. Both projects combine off-the-shelf speech recognition and synthesis technology with in-house command interpreters that interface to the VR applications. Details about the interpreters and other technical aspects of the projects are provided, together with a discussion of some of the design decisions involved in the creation of speech interfaces. Questions and issues raised by the projects are presented as inspiration for future work. These issues include: requirements for object and information representation in VR models to support natural language interfaces; use of the visual context to establish the interaction context; difficulties with referencing events in the virtual world; and problems related to the usability of speech and natural language interfaces in general.


Natural language processing Natural language interfaces Speech interfaces Speech interface design Speech recognition Virtual reality 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tate D, Sibert L, King T. Using virtual environments for firefighter training. IEEE Computer Graphics and Applications 1997; 17(6): 23–29Google Scholar
  2. 2.
    Zoltan-Ford E. How to get people to say and type what computers can understand. International Journal of Man-Machine Studies 1991; 34; 527–547Google Scholar
  3. 3.
    Solan B, Hill T. An application of object-oriented problem solving in electronic warfare simulation. Naval Research Laboratory Report NRL/FR/5707-93-9551, Washington DC, 1993Google Scholar
  4. 4.
    Wauchope K. Eucalyptus: integrating natural language input with a graphical user interface. Naval Research Laboratory Report NRL/FR/5510-94-9711, Washington DC, 1994Google Scholar
  5. 5.
    Wauchope K, Everett S, Perzanowski D, Marsh E. Natural language in four spatial interfaces. In: Proceedings of the Fifth Conference on Applied Natural Language Processing 1997, San Francisco, CA: Morgan Kaufmann; 8–11Google Scholar
  6. 6.
    Grishman R. PROTEUS parser reference manual. PROTEUS Project Memorandum #4, Department of Computer Science, Courant Institute of Mathematical Sciences, New York University, New York 1986Google Scholar
  7. 7.
    Wauchope K. A tandem semantic interpreter for incremental parse selection. Naval Research Laboratory Report 9288. Washington DC, 1990Google Scholar
  8. 8.
    Rosenstein L. MacApp: first commercially successful framework. In: Object-Oriented Application Frameworks. Lewis T, ed. Manning Publications Co., 1995Google Scholar
  9. 9.
    Oviatt S. Multimodal interfaces for dynamic interactive maps. In: Proceedings of CHI'96; NY: ACM Press, 95–102Google Scholar
  10. 10.
    Oviatt S, DeAngeli A, Kuhn K. Integration and synchronization of input modes during multimodal human-computer interaction. In: Proceedings of CHI '97; NY: ACM Press, 415–422Google Scholar
  11. 11.
    Preece J. Human-computer interaction. Reading, MA: Addison-Wesley Publishing Company, 1994Google Scholar
  12. 12.
    Tullis T. Screen design. In: Handbook of Human-Computer Interaction. Helander M, ed. Amsterdam: Elsevier Science Publishers BV (North-Holland), 1988; 377–411Google Scholar
  13. 13.
    Ogden W. Using Natural Language Interfaces. In Handbook of Human-Computer Interaction, Helander M ed. Amsterdam: Elsevier Science Publishers BV (North-Holland), 1988; 281–299Google Scholar

Copyright information

© Springer-Verlag London Ltd 1999

Authors and Affiliations

  • S. S. Everett
    • 2
  • K. Wauchope
    • 2
  • M. A. Pérez Quiñones
    • 1
  1. 1.Dept. Electrical & Computer EngineeringUniversidad de Puerto RicoMayagüezPuerto Rico
  2. 2.Naval Research LaboratoryNavy Center for Applied Research in Artificial IntelligenceWashington, DCUSA

Personalised recommendations