The synthetic teammate project
- 157 Downloads
- 14 Citations
Abstract
The main objective of the Synthetic Teammate project is to develop language and task enabled synthetic agents capable of being integrated into team training simulations. To achieve this goal, the agents must be able to closely match human behavior. The initial application for the synthetic teammate research is creation of an agent able to perform the functions of a pilot for an Unmanned Aerial Vehicle (UAV) simulation as part of a three-person team. The agent, or synthetic teammate, is being developed in the ACT-R cognitive architecture. The major components include: language comprehension and generation, dialog management, agent-environment interaction, and situation assessment. Initial empirical results suggest that the agent-environment interaction is a good approximation to human behavior in the UAV environment, and we are planning further empirical tests of the synthetic teammate operating with human teammates. This paper covers the project’s modeling approach, challenges faced, progress made toward an integrated synthetic teammate, and lessons learned during development.
Keywords
Synthetic teammate Language comprehension/generation Dialog management Situation model Agent-environment interactionPreview
Unable to display preview. Download preview PDF.
References
- Altmann G, Steedman M (1988) Interaction with context during human sentence processing. Cognition 30:191–238 CrossRefGoogle Scholar
- Anderson JR (2007) How can the human mind occur in the physical Universe? Oxford University Press, New York CrossRefGoogle Scholar
- Anderson JR, Fincham JM, Douglass S (1997) The role of examples and rules in the acquisition of a cognitive skill. J Exp Psychol Learn Mem Cogn 23:932–945 CrossRefGoogle Scholar
- Anderson JR, Bothell D, Byrne M, Douglass S, Lebiere C, Qin Y (2004) An integrated theory of the mind. Psychol Rev 111(4):1036–1060 CrossRefGoogle Scholar
- Ball J (1991). PM, propositional model, a computational psycholinguistic model of language comprehension based on a relational analysis of written english. UMI Dissertation Information Service, Ann Arbor, MI Google Scholar
- Ball J (2004a). A cognitively plausible model of language comprehension. In: Proceedings of the 13th conference on behavior representation in modeling and simulation, pp 305–316 Google Scholar
- Ball J (2004b) Software agents with natural language capabilities–where are we? In: Symposium conducted at the 13th conference on behavior representation in modeling and simulation, Arlington, VA Google Scholar
- Ball J (2006) Can NLP systems be a cognitive black box? (Is cognitive science relevant to AI problems?) In: Paper presented at the AAAI spring symposium: between a rock and a hard place, cognitive science principles meet AI hard problems (Technical Report SS-06-02). AAAI Press, Menlo Park Google Scholar
- Ball J (2007a) Construction-driven language processing. In: Vosniadou S, Kayser D, Protopapas A (eds) Proceedings of the 2nd european cognitive science conference. LEA, New York, pp 722–727 Google Scholar
- Ball J (2007b) A bi-polar theory of nominal and clause structure and function. Ann Rev Cogn Linguist 5(1):27–54 Google Scholar
- Ball J (2008) A naturalistic, functional approach to modeling language comprehension. In: Paper presented at the AAAI Fall Symposium, Naturally Inspired Artificial Intelligence (Technical Report FS-08-06). AAAI Press, Menlo Park Google Scholar
- Ball J (2010) Simplifying the mapping from referring expression to referent in a conceptual semantics of reference. In: Proceedings of the 32nd annual meeting of the cognitive science society Google Scholar
- Ball J, Heiberg A, Silber R (2007) Toward a large-scale model of language comprehension in ACT-R 6. In: Lewis R, Polk T, Laird J (eds) Proceedings of the 8th international conference on cognitive modeling. Psychology Press, New York, pp 173–179 Google Scholar
- Boersma P, Hayes B (2001) Empirical tests of the gradual learning algorithm. Linguist Inq 32:45–86 CrossRefGoogle Scholar
- Byrne MD (2001) ACT-R/PM and menu selection: applying a cognitive architecture to HCI. Int J Human-Comput Stud 55(1):41–84 CrossRefGoogle Scholar
- Byrne MD, Kirlik A (2005) Using computational cognitive modeling to diagnose possible sources of aviation error. Int J Aviat Psychol 15(2):135–155 CrossRefGoogle Scholar
- Byrne MD, Wood SD, Sukaviriya P, Foley JD, Kieras DE (1994) Automating interface evaluation. In: Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence. ACM, New York, pp 232–237 CrossRefGoogle Scholar
- Cassimatis N, Bello P, Langley P (2008) Ability, breadth, and parsimony in computational models of higher-order cognition. Cogn Sci 32:1304–1322 CrossRefGoogle Scholar
- Christianson K, Hollingsworth A, Halliwell J, Ferreira F (2001) Thematic roles assigned along the garden path linger. Cogn Psychol 42:368–407 CrossRefGoogle Scholar
- Colmerauer A, Roussel P (1996) The birth of Prolog. In: Bergin T, Gibson R (eds) History of programming languages II. ACM Press/Addison-Wesley, New York, pp 331–367 Google Scholar
- Cooke N, Shope S (2005) Synthetic task environments for teams: CERTT’s UAV-STE. Handbook on human factors and ergonomics methods, vol 46. CRC Press, Boca Raton Google Scholar
- Cooke NJ, Kiekel PA, Helm EE (2001) Measuring team knowledge during skill acquisition of a complex task. Int J Cogn Ergon 5(3):297–315. Special section on knowledge acquisition CrossRefGoogle Scholar
- Cooke NJ, Gorman JC, Duran JL, Taylor AR (2007) Team cognition in experienced command-and-control teams. J Exp Psychol Appl 13(3):146–157. Special issue on capturing expertise across domains CrossRefGoogle Scholar
- Cooke NJ, Gorman JC, Kiekel PA (2008) Communication as team-level cognitive processing. In: Letsky M, Warner N, Fiore S, Smith CAP (eds) Macrocognition in teams: theories and methodologies. Ashgate, Hants, pp 51–64 Google Scholar
- Core MG, Allen JF (1997) Coding dialogs with the DAMSL annotation scheme. Paper presented at the AAAI fall symposium on communicative action in humans and machines, November 8–10, 1997, Cambridge, MA Google Scholar
- Douglass SA (2007) A computational model of situated action. Carnegie Mellon University, Pittsburgh (Doctoral dissertation) Google Scholar
- Douglass SA (2010) Rule & Automata Modeling Language (RaAML) (in preparation) Google Scholar
- Douglass S, Ball J, Rodgers S (2009) Large declarative memories in ACT-R. In: Proceedings of the 9th international conference on cognitive modeling 2009, Manchester, UK Google Scholar
- Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Hum Factors 37(1):32–64 CrossRefGoogle Scholar
- Ericsson S (2004) Dynamic optimisation of information enrichment in dialogue. In: Proceedings of the 8th international workshop on formal semantics and pragmatics of dialogue. Catalog, Barcelona Google Scholar
- Freiman M, Ball J (2008) Computational cognitive modeling of reading comprehension at the word level. In: Proceedings of the 38th western conference on linguistics. University of California, Davis, Davis, pp 34–45 Google Scholar
- Freiman M, Ball J (2010) Improving the reading rate of Double-RLanguage. In: Proceedings of the 10th international conference on cognitive modeling (to appear) Google Scholar
- Gallagher HL, Frith CD (2003) Functional imaging of ‘theory of mind’. Trends Cogn Sci 7(2):77–83 CrossRefGoogle Scholar
- Gibson E, Pearlmutter NJ (1998) Constraints on sentence comprehension. Trends Cogn Sci 2(7):262–268 CrossRefGoogle Scholar
- Gorman JC, Cooke NJ, Winner JL (2006) Measuring team situation awareness in decentralized command and control systems. Ergonomics 49:1312–1325 CrossRefGoogle Scholar
- Grodner D, Gibson E, Argaman V, Babyonyshev M (2003) Against repair-based reanalysis in sentence comprehension. J Psycholinguist Res 32(2):141–166 CrossRefGoogle Scholar
- Hobbs JR (1985) Ontological promiscuity. In: Proceedings of the 23rd annual meeting of the association for computational linguistics. Chicago, IL, pp 61–69 Google Scholar
- Hobbs JR (2003) Discourse and inference. Retrieved from http://www.isi.edu/~hobbs/disinf-tc.html
- Jones RM, Laird JE, Nielsen PE, Coulter KJ, Kenny P, Koss FV (1999) Automated intelligent pilots for combat flight simulation. AI Mag 20(1):27–41 Google Scholar
- Kamp H, Ryle U (1993) From discourse to logic: introduction to model-theoretic semantics of natural language, formal logic and discourse representation theory. Studies in linguistics and philosophy. Kluwer Academic, Dordrecht Google Scholar
- Kieras DE (1988) Towards a practical GOMS model methodology for user interface design. In: Helander M (ed) The handbook of human-computer interaction. North-Holland, Amsterdam, pp 135–158 Google Scholar
- Kieras D, Meyer DE (1997) An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Hum-Comput Interact 12(4):391–438 CrossRefGoogle Scholar
- Kintsch W (1998) Comprehension, a paradigm for cognition. Cambridge University Press, New York Google Scholar
- Klahr D, Chase WG, Lovelace EA (1983) Structure and process in alphabetic retrieval. J Exp Psychol Learn Mem Cogn 9(3):462–477 CrossRefGoogle Scholar
- Kosslyn S (2006) The case for mental imagery. Oxford University Press, New York CrossRefGoogle Scholar
- Laird JE, Jones RM (1998) Building advanced autonomous AI systems for large scale real time simulations. In: Proceedings of the 1998 computer game developers’ conference. Freeman, Long Beach, pp 365–378 Google Scholar
- Landauer T, Dumais S (1997) A solution to Plato’s problem: the latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychol Rev 104(2):211–240 CrossRefGoogle Scholar
- Langacker RW (1987) Foundations of cognitive grammar, vol I: theoretical prerequisites. Stanford University Press, Stanford Google Scholar
- Langacker RW (1991) Foundations of cognitive grammar, vol II: descriptive application. Stanford University Press, Stanford Google Scholar
- Lebiere C, Wray R (eds) (2006) Between a rock and a hard place: cognitive science principles meet AI-hard problems. AAAI Press, Menlo Park. Papers from the AAAI spring symposium Google Scholar
- Lovett MC (1998) Choice. In: Anderson JR, Lebiere C (eds) The atomic components of thought. Erlbaum, Mahwah, pp 255–296 Google Scholar
- Marcus M, Badler N, Joshi A, Pappas G, Pereira F, Romero M, McCallum A, Potts C, Yanco H (2008) SUBTLE (Situation Understanding Bot Through Language and Environment) project program review. University of Massachusetts, Amherst Google Scholar
- Matessa M (2000) Simulating adaptive communication. Carnegie Mellon University, Pittsburgh (Doctoral dissertation) Google Scholar
- McClelland JL, Rumelhart DE (1981) An interactive activation model of context effects in letter perception. Part I. An account of basic findings. Psychol Rev 88(5):375–407 CrossRefGoogle Scholar
- McDonald D (1999) A rational reconstruction of Genaro. In: Proceedings of the RAGS Workshop, Edinburgh Google Scholar
- Miller G (1995) WordNet: a lexical database for English. Commun ACM 38(11):39–41 CrossRefGoogle Scholar
- Myers CW (2009) An account of model inspiration, integration, & sub-task validation. In: Proceedings of the 9th international conference on cognitive modeling, Manchester, UK Google Scholar
- Prince A, Smolensky P (1993/2004) Optimality theory: constraint interaction in generative grammar. Wiley-Blackwell, New York Google Scholar
- Ritter FE, Van Rooy D, St Amant R (2002) A user modelling design tool based on a cognitive architecture for comparing interfaces. In: Computer-aided design of user interfaces III. Proceedings of the 4th international conference on computer-aided design of user interfaces CADUI’2002, Valenciennes, France, 15–17 May 2002. Kluwer Academics, Dordrecht, pp 111–118 Google Scholar
- Scolaro J, Santarelli T (2002) Cognitive modeling teamwork, taskwork, and instructional behavior in synthetic teammates. In: Proceedings of the 11th conference on computer generated forces and behavioral representation. Institute for Simulation and Training, Orlando Google Scholar
- Seidenberg MS, McClelland JL (1989) A distributed, developmental model of word recognition and naming. Psychol Rev 96(4):523–568 CrossRefGoogle Scholar
- Stokes J (2001) Speech interaction and human behavior representations (HBRs). In: Proceedings of 10th conference on computer generated forces and behavioral representation. SISO, Inc, Norfolk, pp 467–476 Google Scholar
- Tambe M, Johnson WL, Jones RM, Koss F, Laird JE, Rosenbloom PS, Schwamb K (1995) Intelligent agents for interactive simulation environments. AI Mag 16(1):15–40 Google Scholar
- Tanenhaus MK, Spivey-Knowlton MJ, Eberhard KM, Sedivy JC (1995) Integration of visual and linguistic information in spoken language comprehension. Science 268(5217):1632–1634 CrossRefGoogle Scholar
- Traum DR, Allen JF (1994) Discourse obligations in dialogue processing. In: Proceedings of the 32nd annual meeting of the association for computational linguistics. Association for Computational Linguistics, Las Cruces, Morristown Google Scholar
- Traum DR, Rickel J, Gratch J, Marsella S (2003) Negotiation over tasks in hybrid human-agent teams for simulation-based training. In: Proceedings of the second international joint conference on Autonomous agents and multiagent systems. ACM, Melbourn, Australia, New York, pp 441–448. CrossRefGoogle Scholar
- van Dijk T, Kintsch W (1983) Strategies of discourse comprehension. Academic Press, New York Google Scholar
- Varges S (2006) Overgeneration and ranking for spoken dialogue systems. In: Proceedings of the 4th international natural language generation conference, Sydney, Australia, July 2006. Association for Computational Linguistics, pp 20–22 Google Scholar
- Vosse T, Kempen G (2000) Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar. Cognition 75:105–143 CrossRefGoogle Scholar
- Walker MA, Whittaker SJ, Stent A, Maloor P, Moore J, Johnston M, Vasireddy G (2004) Generation and evaluation of user tailored responses in multimodal dialogue. Cogn Sci 28(5):811–840 CrossRefGoogle Scholar
- Yang Y, Bello P (2005) Some empirical results concerning deontic reasoning: models, schema, or both? In: Proceedings of the 27th annual meeting of the cognitive science society. Erlbaum, Mahway, pp 2393–2398 Google Scholar
- Zachary W, Santarelli T, Lyons D, Bergondy M, Johnston J (2001) Using a community of intelligent synthetic entities to support operational team training. In: Proceedings of the tenth conference on computer generated forces and behavioral representation. Orlando, FL: Institute for Simulation and Training, University of Central Florida, pp 215–224 Google Scholar
- Zwann R, Radvansky G (1998) Situation models in language comprehension and memory. Psychol Bull 123(2):162–185 CrossRefGoogle Scholar