Computational and Mathematical Organization Theory

, Volume 16, Issue 3, pp 271–299 | Cite as

The synthetic teammate project

  • Jerry Ball
  • Christopher Myers
  • Andrea Heiberg
  • Nancy J. Cooke
  • Michael Matessa
  • Mary Freiman
  • Stuart Rodgers
Article
First Online:

Abstract

The main objective of the Synthetic Teammate project is to develop language and task enabled synthetic agents capable of being integrated into team training simulations. To achieve this goal, the agents must be able to closely match human behavior. The initial application for the synthetic teammate research is creation of an agent able to perform the functions of a pilot for an Unmanned Aerial Vehicle (UAV) simulation as part of a three-person team. The agent, or synthetic teammate, is being developed in the ACT-R cognitive architecture. The major components include: language comprehension and generation, dialog management, agent-environment interaction, and situation assessment. Initial empirical results suggest that the agent-environment interaction is a good approximation to human behavior in the UAV environment, and we are planning further empirical tests of the synthetic teammate operating with human teammates. This paper covers the project’s modeling approach, challenges faced, progress made toward an integrated synthetic teammate, and lessons learned during development.

Keywords

Synthetic teammate Language comprehension/generation Dialog management Situation model Agent-environment interaction 
This is a preview of subscription content, log in to check access

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Altmann G, Steedman M (1988) Interaction with context during human sentence processing. Cognition 30:191–238 CrossRefGoogle Scholar
  2. Anderson JR (2007) How can the human mind occur in the physical Universe? Oxford University Press, New York CrossRefGoogle Scholar
  3. Anderson JR, Fincham JM, Douglass S (1997) The role of examples and rules in the acquisition of a cognitive skill. J Exp Psychol Learn Mem Cogn 23:932–945 CrossRefGoogle Scholar
  4. Anderson JR, Bothell D, Byrne M, Douglass S, Lebiere C, Qin Y (2004) An integrated theory of the mind. Psychol Rev 111(4):1036–1060 CrossRefGoogle Scholar
  5. Ball J (1991). PM, propositional model, a computational psycholinguistic model of language comprehension based on a relational analysis of written english. UMI Dissertation Information Service, Ann Arbor, MI Google Scholar
  6. Ball J (2004a). A cognitively plausible model of language comprehension. In: Proceedings of the 13th conference on behavior representation in modeling and simulation, pp 305–316 Google Scholar
  7. Ball J (2004b) Software agents with natural language capabilities–where are we? In: Symposium conducted at the 13th conference on behavior representation in modeling and simulation, Arlington, VA Google Scholar
  8. Ball J (2006) Can NLP systems be a cognitive black box? (Is cognitive science relevant to AI problems?) In: Paper presented at the AAAI spring symposium: between a rock and a hard place, cognitive science principles meet AI hard problems (Technical Report SS-06-02). AAAI Press, Menlo Park Google Scholar
  9. Ball J (2007a) Construction-driven language processing. In: Vosniadou S, Kayser D, Protopapas A (eds) Proceedings of the 2nd european cognitive science conference. LEA, New York, pp 722–727 Google Scholar
  10. Ball J (2007b) A bi-polar theory of nominal and clause structure and function. Ann Rev Cogn Linguist 5(1):27–54 Google Scholar
  11. Ball J (2008) A naturalistic, functional approach to modeling language comprehension. In: Paper presented at the AAAI Fall Symposium, Naturally Inspired Artificial Intelligence (Technical Report FS-08-06). AAAI Press, Menlo Park Google Scholar
  12. Ball J (2010) Simplifying the mapping from referring expression to referent in a conceptual semantics of reference. In: Proceedings of the 32nd annual meeting of the cognitive science society Google Scholar
  13. Ball J, Heiberg A, Silber R (2007) Toward a large-scale model of language comprehension in ACT-R 6. In: Lewis R, Polk T, Laird J (eds) Proceedings of the 8th international conference on cognitive modeling. Psychology Press, New York, pp 173–179 Google Scholar
  14. Boersma P, Hayes B (2001) Empirical tests of the gradual learning algorithm. Linguist Inq 32:45–86 CrossRefGoogle Scholar
  15. Byrne MD (2001) ACT-R/PM and menu selection: applying a cognitive architecture to HCI. Int J Human-Comput Stud 55(1):41–84 CrossRefGoogle Scholar
  16. Byrne MD, Kirlik A (2005) Using computational cognitive modeling to diagnose possible sources of aviation error. Int J Aviat Psychol 15(2):135–155 CrossRefGoogle Scholar
  17. Byrne MD, Wood SD, Sukaviriya P, Foley JD, Kieras DE (1994) Automating interface evaluation. In: Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence. ACM, New York, pp 232–237 CrossRefGoogle Scholar
  18. Cassimatis N, Bello P, Langley P (2008) Ability, breadth, and parsimony in computational models of higher-order cognition. Cogn Sci 32:1304–1322 CrossRefGoogle Scholar
  19. Christianson K, Hollingsworth A, Halliwell J, Ferreira F (2001) Thematic roles assigned along the garden path linger. Cogn Psychol 42:368–407 CrossRefGoogle Scholar
  20. Colmerauer A, Roussel P (1996) The birth of Prolog. In: Bergin T, Gibson R (eds) History of programming languages II. ACM Press/Addison-Wesley, New York, pp 331–367 Google Scholar
  21. Cooke N, Shope S (2005) Synthetic task environments for teams: CERTT’s UAV-STE. Handbook on human factors and ergonomics methods, vol 46. CRC Press, Boca Raton Google Scholar
  22. Cooke NJ, Kiekel PA, Helm EE (2001) Measuring team knowledge during skill acquisition of a complex task. Int J Cogn Ergon 5(3):297–315. Special section on knowledge acquisition CrossRefGoogle Scholar
  23. Cooke NJ, Gorman JC, Duran JL, Taylor AR (2007) Team cognition in experienced command-and-control teams. J Exp Psychol Appl 13(3):146–157. Special issue on capturing expertise across domains CrossRefGoogle Scholar
  24. Cooke NJ, Gorman JC, Kiekel PA (2008) Communication as team-level cognitive processing. In: Letsky M, Warner N, Fiore S, Smith CAP (eds) Macrocognition in teams: theories and methodologies. Ashgate, Hants, pp 51–64 Google Scholar
  25. Core MG, Allen JF (1997) Coding dialogs with the DAMSL annotation scheme. Paper presented at the AAAI fall symposium on communicative action in humans and machines, November 8–10, 1997, Cambridge, MA Google Scholar
  26. Douglass SA (2007) A computational model of situated action. Carnegie Mellon University, Pittsburgh (Doctoral dissertation) Google Scholar
  27. Douglass SA (2010) Rule & Automata Modeling Language (RaAML) (in preparation) Google Scholar
  28. Douglass S, Ball J, Rodgers S (2009) Large declarative memories in ACT-R. In: Proceedings of the 9th international conference on cognitive modeling 2009, Manchester, UK Google Scholar
  29. Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Hum Factors 37(1):32–64 CrossRefGoogle Scholar
  30. Ericsson S (2004) Dynamic optimisation of information enrichment in dialogue. In: Proceedings of the 8th international workshop on formal semantics and pragmatics of dialogue. Catalog, Barcelona Google Scholar
  31. Freiman M, Ball J (2008) Computational cognitive modeling of reading comprehension at the word level. In: Proceedings of the 38th western conference on linguistics. University of California, Davis, Davis, pp 34–45 Google Scholar
  32. Freiman M, Ball J (2010) Improving the reading rate of Double-RLanguage. In: Proceedings of the 10th international conference on cognitive modeling (to appear) Google Scholar
  33. Gallagher HL, Frith CD (2003) Functional imaging of ‘theory of mind’. Trends Cogn Sci 7(2):77–83 CrossRefGoogle Scholar
  34. Gibson E, Pearlmutter NJ (1998) Constraints on sentence comprehension. Trends Cogn Sci 2(7):262–268 CrossRefGoogle Scholar
  35. Gorman JC, Cooke NJ, Winner JL (2006) Measuring team situation awareness in decentralized command and control systems. Ergonomics 49:1312–1325 CrossRefGoogle Scholar
  36. Grodner D, Gibson E, Argaman V, Babyonyshev M (2003) Against repair-based reanalysis in sentence comprehension. J Psycholinguist Res 32(2):141–166 CrossRefGoogle Scholar
  37. Hobbs JR (1985) Ontological promiscuity. In: Proceedings of the 23rd annual meeting of the association for computational linguistics. Chicago, IL, pp 61–69 Google Scholar
  38. Hobbs JR (2003) Discourse and inference. Retrieved from http://www.isi.edu/~hobbs/disinf-tc.html
  39. Jones RM, Laird JE, Nielsen PE, Coulter KJ, Kenny P, Koss FV (1999) Automated intelligent pilots for combat flight simulation. AI Mag 20(1):27–41 Google Scholar
  40. Kamp H, Ryle U (1993) From discourse to logic: introduction to model-theoretic semantics of natural language, formal logic and discourse representation theory. Studies in linguistics and philosophy. Kluwer Academic, Dordrecht Google Scholar
  41. Kieras DE (1988) Towards a practical GOMS model methodology for user interface design. In: Helander M (ed) The handbook of human-computer interaction. North-Holland, Amsterdam, pp 135–158 Google Scholar
  42. Kieras D, Meyer DE (1997) An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Hum-Comput Interact 12(4):391–438 CrossRefGoogle Scholar
  43. Kintsch W (1998) Comprehension, a paradigm for cognition. Cambridge University Press, New York Google Scholar
  44. Klahr D, Chase WG, Lovelace EA (1983) Structure and process in alphabetic retrieval. J Exp Psychol Learn Mem Cogn 9(3):462–477 CrossRefGoogle Scholar
  45. Kosslyn S (2006) The case for mental imagery. Oxford University Press, New York CrossRefGoogle Scholar
  46. Laird JE, Jones RM (1998) Building advanced autonomous AI systems for large scale real time simulations. In: Proceedings of the 1998 computer game developers’ conference. Freeman, Long Beach, pp 365–378 Google Scholar
  47. Landauer T, Dumais S (1997) A solution to Plato’s problem: the latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychol Rev 104(2):211–240 CrossRefGoogle Scholar
  48. Langacker RW (1987) Foundations of cognitive grammar, vol I: theoretical prerequisites. Stanford University Press, Stanford Google Scholar
  49. Langacker RW (1991) Foundations of cognitive grammar, vol II: descriptive application. Stanford University Press, Stanford Google Scholar
  50. Lebiere C, Wray R (eds) (2006) Between a rock and a hard place: cognitive science principles meet AI-hard problems. AAAI Press, Menlo Park. Papers from the AAAI spring symposium Google Scholar
  51. Lovett MC (1998) Choice. In: Anderson JR, Lebiere C (eds) The atomic components of thought. Erlbaum, Mahwah, pp 255–296 Google Scholar
  52. Marcus M, Badler N, Joshi A, Pappas G, Pereira F, Romero M, McCallum A, Potts C, Yanco H (2008) SUBTLE (Situation Understanding Bot Through Language and Environment) project program review. University of Massachusetts, Amherst Google Scholar
  53. Matessa M (2000) Simulating adaptive communication. Carnegie Mellon University, Pittsburgh (Doctoral dissertation) Google Scholar
  54. McClelland JL, Rumelhart DE (1981) An interactive activation model of context effects in letter perception. Part I. An account of basic findings. Psychol Rev 88(5):375–407 CrossRefGoogle Scholar
  55. McDonald D (1999) A rational reconstruction of Genaro. In: Proceedings of the RAGS Workshop, Edinburgh Google Scholar
  56. Miller G (1995) WordNet: a lexical database for English. Commun ACM 38(11):39–41 CrossRefGoogle Scholar
  57. Myers CW (2009) An account of model inspiration, integration, & sub-task validation. In: Proceedings of the 9th international conference on cognitive modeling, Manchester, UK Google Scholar
  58. Prince A, Smolensky P (1993/2004) Optimality theory: constraint interaction in generative grammar. Wiley-Blackwell, New York Google Scholar
  59. Ritter FE, Van Rooy D, St Amant R (2002) A user modelling design tool based on a cognitive architecture for comparing interfaces. In: Computer-aided design of user interfaces III. Proceedings of the 4th international conference on computer-aided design of user interfaces CADUI’2002, Valenciennes, France, 15–17 May 2002. Kluwer Academics, Dordrecht, pp 111–118 Google Scholar
  60. Scolaro J, Santarelli T (2002) Cognitive modeling teamwork, taskwork, and instructional behavior in synthetic teammates. In: Proceedings of the 11th conference on computer generated forces and behavioral representation. Institute for Simulation and Training, Orlando Google Scholar
  61. Seidenberg MS, McClelland JL (1989) A distributed, developmental model of word recognition and naming. Psychol Rev 96(4):523–568 CrossRefGoogle Scholar
  62. Stokes J (2001) Speech interaction and human behavior representations (HBRs). In: Proceedings of 10th conference on computer generated forces and behavioral representation. SISO, Inc, Norfolk, pp 467–476 Google Scholar
  63. Tambe M, Johnson WL, Jones RM, Koss F, Laird JE, Rosenbloom PS, Schwamb K (1995) Intelligent agents for interactive simulation environments. AI Mag 16(1):15–40 Google Scholar
  64. Tanenhaus MK, Spivey-Knowlton MJ, Eberhard KM, Sedivy JC (1995) Integration of visual and linguistic information in spoken language comprehension. Science 268(5217):1632–1634 CrossRefGoogle Scholar
  65. Traum DR, Allen JF (1994) Discourse obligations in dialogue processing. In: Proceedings of the 32nd annual meeting of the association for computational linguistics. Association for Computational Linguistics, Las Cruces, Morristown Google Scholar
  66. Traum DR, Rickel J, Gratch J, Marsella S (2003) Negotiation over tasks in hybrid human-agent teams for simulation-based training. In: Proceedings of the second international joint conference on Autonomous agents and multiagent systems. ACM, Melbourn, Australia, New York, pp 441–448. CrossRefGoogle Scholar
  67. van Dijk T, Kintsch W (1983) Strategies of discourse comprehension. Academic Press, New York Google Scholar
  68. Varges S (2006) Overgeneration and ranking for spoken dialogue systems. In: Proceedings of the 4th international natural language generation conference, Sydney, Australia, July 2006. Association for Computational Linguistics, pp 20–22 Google Scholar
  69. Vosse T, Kempen G (2000) Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar. Cognition 75:105–143 CrossRefGoogle Scholar
  70. Walker MA, Whittaker SJ, Stent A, Maloor P, Moore J, Johnston M, Vasireddy G (2004) Generation and evaluation of user tailored responses in multimodal dialogue. Cogn Sci 28(5):811–840 CrossRefGoogle Scholar
  71. Yang Y, Bello P (2005) Some empirical results concerning deontic reasoning: models, schema, or both? In: Proceedings of the 27th annual meeting of the cognitive science society. Erlbaum, Mahway, pp 2393–2398 Google Scholar
  72. Zachary W, Santarelli T, Lyons D, Bergondy M, Johnston J (2001) Using a community of intelligent synthetic entities to support operational team training. In: Proceedings of the tenth conference on computer generated forces and behavioral representation. Orlando, FL: Institute for Simulation and Training, University of Central Florida, pp 215–224 Google Scholar
  73. Zwann R, Radvansky G (1998) Situation models in language comprehension and memory. Psychol Bull 123(2):162–185 CrossRefGoogle Scholar

Copyright information

© US Government 2010

Authors and Affiliations

  • Jerry Ball
    • 1
  • Christopher Myers
    • 2
  • Andrea Heiberg
    • 3
  • Nancy J. Cooke
    • 4
  • Michael Matessa
    • 5
  • Mary Freiman
    • 6
  • Stuart Rodgers
    • 7
  1. 1.Air Force Research LaboratoryMesaUSA
  2. 2.Air Force Research LaboratoryWright-Patterson Air Force BaseUSA
  3. 3.General MotorsMesaUSA
  4. 4.Cognitive Engineering Research InstituteMesaUSA
  5. 5.AlionBoulderUSA
  6. 6.L3 CommunicationsMesaUSA
  7. 7.AGS TechNetDaytonUSA

Personalised recommendations