Assistive Technology and Artificial Intelligence pp 109-125 | Cite as
An augmentative communication interface based on conversational schemata
Abstract
Many people with severe speech and motor impairments make use of augmentative and alternative communication (AAC) systems. These systems can employ a variety of techniques to organize stored words, phrases, and sentences, and to make them available to the user. It is argued in this chapter that an AAC system should make better use of the regularities in an individual's conversational experiences and the expectations that the individual normally brings into a conversational context.
An interface and methodology are described for organizing and retrieving sentences appropriate to a particular conversational context, sentences that were possibly developed from earlier conversations. These conversations are represented according to the schema structures discussed by Schank as a model for memory and cognitive organization [16]. The interface allows the user to proceed with minimal effort through conversations that follow the schema closely, and facilitates the derivation of new schemata when the conversation diverges from existing ones. This interface, called SchemaTalk, is intended to operate in parallel with and to complement a user's existing augmentative communication system. The results of preliminary investigations into the effectiveness of the interface and methodology have been encouraging; further investigations are planned. Of interest for future study is how the use of schematized text might influence the way that augmented communicators are perceived by their conversational partners. Possible design modifications to improve the usability of the interface are also under investigation.
Keywords
Fast Food Restaurant Communicative Competence Conversational Context Conversational Partner Natural Language InterfacePreview
Unable to display preview. Download preview PDF.
References
- 1.N. Alm, A. Morrison, and J.L. Arnott. A communication system based on scripts, plans, and goals for enabling non-speaking people to conduct telephone conversations. In Proceedings of IEEE Systems, Man and Cybernetics, pages 2408–2412, Vancouver, Canada, 1995.Google Scholar
- 2.N. Alm, A.F. Newell, and J.L. Arnott. A communication aid which models conversational patterns. In Proceedings of the RESNA 10th Annual Conference, pages 127–129, San Jose, CA, 1987.Google Scholar
- 3.N. Alm, A.F. Newell, and J.L. Arnott. Database design for storing and accessing personal conversational material. In Proceedings of the RESNA 12th Annual Conference, pages 147–148, New Orleans, LA, 1989.Google Scholar
- 4.B. Baker. Minspeak. Byte, pages 186–202, 1982.Google Scholar
- 5.J. L. Bedrosian, L. Hoag, D. Johnson, and S. N. Calculator. Communicative competence as perceived by adults with severe speech impairments associated with cerebral palsy. Manuscript submitted for publication, 1997.Google Scholar
- 6.T.L. Carpenter, Jr., K.F. McCoy, and C.A. Pennington. Schema-based organization of reusable text in AAC: User-interface considerations. In Proceedings of the 20th Annual RESNA Conference, pages 57–59, Pittsburgh, PA, 1997.Google Scholar
- 7.R. E. Cullingford and J. L. Kolodner. Interactive advice giving. In Proceedings of the 1986 IEEE International Conference on Systems, Man and Cybernetics, pages 709–714, Atlanta, GA, 1986.Google Scholar
- 8.P.W. Demasco and K.F. McCoy. Generating text from compressed input: An intelligent interface for people with severe motor impairments. Communications of the ACM, 35(5):68–78, 1992.CrossRefGoogle Scholar
- 9.P. S. Elder and C. Goossens'. Engineering Training Environments for Interactive Augmentative Communication: Strategies for adolescents and adults who are moderately/severely developmentally delayed. Clinician Series. Southeast Augmentative Communication Conference Publications, Birmingham, AL, 1994.Google Scholar
- 10.L. Hoag, J. L. Bedrosian, D. Johnson, and B. Molineux. Variables affecting perceptions of social aspects of the communicative competence of an adult AAC user. Augmentative and Alternative Communication, 10:129–137, 1994.CrossRefGoogle Scholar
- 11.K. Kellermann, S. Broetzmann, T.-S. Lim, and K. Kitao. The conversation MOP: Scenes in the stream of discourse. Discourse Processes, 12:27–61, 1989.Google Scholar
- 12.H. H. Koester and S. P. Levine. Quantitative indicators of cognitive load during use of a word prediction system. In Proceedings of the RESNA '94 Annual Conference, pages 118–120, Nashville, TN, 1994.Google Scholar
- 13.J. Light. Interaction involving individuals using augmentative and alternative communication systems: State of the art and future directions. Augmentative and Alternative Communication, 4(2):66–82, 1988.CrossRefGoogle Scholar
- 14.R. Miikkulainen. Subsymbolic Natural Language Processing: An integrated model of scripts, lexicon, and memory. MIT Press, Cambridge, MA, 1993.Google Scholar
- 15.R. C. Schank and R. P. Abelson. Scripts, plans, goals and understanding: An inquiry into human knowledge structures. Erlbaum, Hillsdale, NJ, 1977.MATHGoogle Scholar
- 16.R.C. Schank. Dynamic Memory: A theory of reminding and learning in computers and people. Cambridge University Press, NY, 1982.Google Scholar
- 17.R.C. Schank. Tell Me A Story: A new look at real and artificial memory. Charles Scribner's Sons, NY, 1990.Google Scholar
- 18.E.H. Turner and R.E. Cullingford. Using conversation MOPs in natural language interfaces. Discourse Processes, 12:63–90, 1989.CrossRefGoogle Scholar
- 19.P. Vanderheyden. Organization of pre-stored text in alternative and augmentative communication systems: An interactive schema-based approach. Master's thesis, University of Delaware, Newark, DE, 1995.Google Scholar