Using Natural Dialogs as the Basis for Speech Interface Design

  • Nicole Yankelovich
Part of the Signals and Communication Technology book series (SCT)


The main premise of this chapter is that speech applications will be more usable if they follow the conventions of natural human dialog. Understanding how natural dialogs take place in the application domain serves as an effective starting point for a speech user interface design. This chapter reports on natural dialog studies that take place in the earliest stages of an application life cycle, prior to any system design or functional specification. Not only do these studies help in the design of grammars, feedback, and prompts, but they also point out instances where speech technology cannot be effectively applied. Natural dialog studies help to uncover common patterns of behavior and can reveal unexpected insights that significantly improve application design. Occasionally, observing participants in a natural setting can even trigger the rethinking of basic design assumptions. The natural dialog studies described in this chapter were conducted in the process of designing four experimental applications built by the Speech Applications Group at Sun Microsystems Laboratories. Each of these applications attempted to push the bounds of conversational interaction, given the constraints of current speech technology. Of the four case studies described, two are for speech-only systems and two involve multimodal systems: (1) The SpeechActs Calendar, a telephone-based system to access an online calendar, (2) The Office Monitor, a walk-up speech interface which offers information and takes messages when an office occupant is out, (3) The Automated Customer Service Representative, a multimodal version of the Lands’ End catalog that allows users to query with speech and see the results on an interactive television display, and (4) Multimodal Drawing, a speech-enhanced user interface to a drawing editor. The study descriptions conclude with a summary of lessons learned about general speech user interface concepts.


Speech applications user interface design user research 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Clark, H. H. (1992). Arenas of language use, Chicago: University of Chicago Press.Google Scholar
  2. Delomier, D., Meunier, A., & Morel, M.A. (1998). Linguistic features of human machine oral dialog. Paper presented at Eurospeech ’89, Paris, France.Google Scholar
  3. Fraser, N. M., & Gilbert, G. N. (1991). Simulating speech systems. Computer Speech and Language, 5.Google Scholar
  4. Gould, J. D., Conti, J., & Hovanyecz, T. (1982). Composing letters with a simulated listening typewriter. Proceedings of the 1982 Conference on Human Factors in Computing Systems, Gaithersburg, MD.Google Scholar
  5. Guyomard, M., & Siroux, J. (1987). Experimentation in the specification of oral dialog. In NATO-ASI Recent Advances in Speech Understanding and Dialogue Systems.Google Scholar
  6. Martin, P., Crabbe, F., Adams, S., Baatz, E., & Yankelovich, N. (1996), SpeechActs: A spoken-language framework. IEEE Computer, 29(7).Google Scholar
  7. Nass, C., & Gong, Li (2000). Speech interfaces from an evolutionary perspective. Communications of the ACM, 43(9).Google Scholar
  8. Norman, D. A., & Draper, S. W. (1986). User centered system design: New perspectives on human-computer interaction. Hillsdale, NJ: Erlbaum.Google Scholar
  9. Schmandt, C. (1994). Voice communication with computers: Conversational systems. New York: Van Nostrand Reinhold.Google Scholar
  10. Yankelovich, N., & McLain, C. D. (1996). Office Monitor, CHI ‘96 Conference Companion, 1996 ACM Conference on Human Factors in Computing Systems. New York: ACM.Google Scholar
  11. Yankelovich, N., Levow, G.-A., & Marx, M. (1995). Designing SpeechActs: Issues in speech user interfaces. Proceedings of CHI ‘95 Conference on Human Factors in Computing Systems. New York: ACM.Google Scholar
  12. Yankelovich, N. (1994). Talking vs taking: Speech access to remote computers, CHI ‘94 Conference Companion, 1994 ACM Conference on Human Factors in Computing Systems. New York: ACM.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2008

Authors and Affiliations

  • Nicole Yankelovich
    • 1
  1. 1.Sun Microsystems IncBurlingtonUSA

Personalised recommendations