Skip to main content
SpringerLink
Log in

The synthetic teammate project

  • Published:
Computational and Mathematical Organization Theory Aims and scope Submit manuscript

Abstract

The main objective of the Synthetic Teammate project is to develop language and task enabled synthetic agents capable of being integrated into team training simulations. To achieve this goal, the agents must be able to closely match human behavior. The initial application for the synthetic teammate research is creation of an agent able to perform the functions of a pilot for an Unmanned Aerial Vehicle (UAV) simulation as part of a three-person team. The agent, or synthetic teammate, is being developed in the ACT-R cognitive architecture. The major components include: language comprehension and generation, dialog management, agent-environment interaction, and situation assessment. Initial empirical results suggest that the agent-environment interaction is a good approximation to human behavior in the UAV environment, and we are planning further empirical tests of the synthetic teammate operating with human teammates. This paper covers the project’s modeling approach, challenges faced, progress made toward an integrated synthetic teammate, and lessons learned during development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Altmann G, Steedman M (1988) Interaction with context during human sentence processing. Cognition 30:191–238

    Article  Google Scholar 

  • Anderson JR (2007) How can the human mind occur in the physical Universe? Oxford University Press, New York

    Book  Google Scholar 

  • Anderson JR, Fincham JM, Douglass S (1997) The role of examples and rules in the acquisition of a cognitive skill. J Exp Psychol Learn Mem Cogn 23:932–945

    Article  Google Scholar 

  • Anderson JR, Bothell D, Byrne M, Douglass S, Lebiere C, Qin Y (2004) An integrated theory of the mind. Psychol Rev 111(4):1036–1060

    Article  Google Scholar 

  • Ball J (1991). PM, propositional model, a computational psycholinguistic model of language comprehension based on a relational analysis of written english. UMI Dissertation Information Service, Ann Arbor, MI

  • Ball J (2004a). A cognitively plausible model of language comprehension. In: Proceedings of the 13th conference on behavior representation in modeling and simulation, pp 305–316

  • Ball J (2004b) Software agents with natural language capabilities–where are we? In: Symposium conducted at the 13th conference on behavior representation in modeling and simulation, Arlington, VA

  • Ball J (2006) Can NLP systems be a cognitive black box? (Is cognitive science relevant to AI problems?) In: Paper presented at the AAAI spring symposium: between a rock and a hard place, cognitive science principles meet AI hard problems (Technical Report SS-06-02). AAAI Press, Menlo Park

    Google Scholar 

  • Ball J (2007a) Construction-driven language processing. In: Vosniadou S, Kayser D, Protopapas A (eds) Proceedings of the 2nd european cognitive science conference. LEA, New York, pp 722–727

    Google Scholar 

  • Ball J (2007b) A bi-polar theory of nominal and clause structure and function. Ann Rev Cogn Linguist 5(1):27–54

    Google Scholar 

  • Ball J (2008) A naturalistic, functional approach to modeling language comprehension. In: Paper presented at the AAAI Fall Symposium, Naturally Inspired Artificial Intelligence (Technical Report FS-08-06). AAAI Press, Menlo Park

    Google Scholar 

  • Ball J (2010) Simplifying the mapping from referring expression to referent in a conceptual semantics of reference. In: Proceedings of the 32nd annual meeting of the cognitive science society

  • Ball J, Heiberg A, Silber R (2007) Toward a large-scale model of language comprehension in ACT-R 6. In: Lewis R, Polk T, Laird J (eds) Proceedings of the 8th international conference on cognitive modeling. Psychology Press, New York, pp 173–179

    Google Scholar 

  • Boersma P, Hayes B (2001) Empirical tests of the gradual learning algorithm. Linguist Inq 32:45–86

    Article  Google Scholar 

  • Byrne MD (2001) ACT-R/PM and menu selection: applying a cognitive architecture to HCI. Int J Human-Comput Stud 55(1):41–84

    Article  Google Scholar 

  • Byrne MD, Kirlik A (2005) Using computational cognitive modeling to diagnose possible sources of aviation error. Int J Aviat Psychol 15(2):135–155

    Article  Google Scholar 

  • Byrne MD, Wood SD, Sukaviriya P, Foley JD, Kieras DE (1994) Automating interface evaluation. In: Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence. ACM, New York, pp 232–237

    Chapter  Google Scholar 

  • Cassimatis N, Bello P, Langley P (2008) Ability, breadth, and parsimony in computational models of higher-order cognition. Cogn Sci 32:1304–1322

    Article  Google Scholar 

  • Christianson K, Hollingsworth A, Halliwell J, Ferreira F (2001) Thematic roles assigned along the garden path linger. Cogn Psychol 42:368–407

    Article  Google Scholar 

  • Colmerauer A, Roussel P (1996) The birth of Prolog. In: Bergin T, Gibson R (eds) History of programming languages II. ACM Press/Addison-Wesley, New York, pp 331–367

    Google Scholar 

  • Cooke N, Shope S (2005) Synthetic task environments for teams: CERTT’s UAV-STE. Handbook on human factors and ergonomics methods, vol 46. CRC Press, Boca Raton

    Google Scholar 

  • Cooke NJ, Kiekel PA, Helm EE (2001) Measuring team knowledge during skill acquisition of a complex task. Int J Cogn Ergon 5(3):297–315. Special section on knowledge acquisition

    Article  Google Scholar 

  • Cooke NJ, Gorman JC, Duran JL, Taylor AR (2007) Team cognition in experienced command-and-control teams. J Exp Psychol Appl 13(3):146–157. Special issue on capturing expertise across domains

    Article  Google Scholar 

  • Cooke NJ, Gorman JC, Kiekel PA (2008) Communication as team-level cognitive processing. In: Letsky M, Warner N, Fiore S, Smith CAP (eds) Macrocognition in teams: theories and methodologies. Ashgate, Hants, pp 51–64

    Google Scholar 

  • Core MG, Allen JF (1997) Coding dialogs with the DAMSL annotation scheme. Paper presented at the AAAI fall symposium on communicative action in humans and machines, November 8–10, 1997, Cambridge, MA

  • Douglass SA (2007) A computational model of situated action. Carnegie Mellon University, Pittsburgh (Doctoral dissertation)

    Google Scholar 

  • Douglass SA (2010) Rule & Automata Modeling Language (RaAML) (in preparation)

  • Douglass S, Ball J, Rodgers S (2009) Large declarative memories in ACT-R. In: Proceedings of the 9th international conference on cognitive modeling 2009, Manchester, UK

  • Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Hum Factors 37(1):32–64

    Article  Google Scholar 

  • Ericsson S (2004) Dynamic optimisation of information enrichment in dialogue. In: Proceedings of the 8th international workshop on formal semantics and pragmatics of dialogue. Catalog, Barcelona

    Google Scholar 

  • Freiman M, Ball J (2008) Computational cognitive modeling of reading comprehension at the word level. In: Proceedings of the 38th western conference on linguistics. University of California, Davis, Davis, pp 34–45

    Google Scholar 

  • Freiman M, Ball J (2010) Improving the reading rate of Double-RLanguage. In: Proceedings of the 10th international conference on cognitive modeling (to appear)

  • Gallagher HL, Frith CD (2003) Functional imaging of ‘theory of mind’. Trends Cogn Sci 7(2):77–83

    Article  Google Scholar 

  • Gibson E, Pearlmutter NJ (1998) Constraints on sentence comprehension. Trends Cogn Sci 2(7):262–268

    Article  Google Scholar 

  • Gorman JC, Cooke NJ, Winner JL (2006) Measuring team situation awareness in decentralized command and control systems. Ergonomics 49:1312–1325

    Article  Google Scholar 

  • Grodner D, Gibson E, Argaman V, Babyonyshev M (2003) Against repair-based reanalysis in sentence comprehension. J Psycholinguist Res 32(2):141–166

    Article  Google Scholar 

  • Hobbs JR (1985) Ontological promiscuity. In: Proceedings of the 23rd annual meeting of the association for computational linguistics. Chicago, IL, pp 61–69

  • Hobbs JR (2003) Discourse and inference. Retrieved from http://www.isi.edu/~hobbs/disinf-tc.html

  • Jones RM, Laird JE, Nielsen PE, Coulter KJ, Kenny P, Koss FV (1999) Automated intelligent pilots for combat flight simulation. AI Mag 20(1):27–41

    Google Scholar 

  • Kamp H, Ryle U (1993) From discourse to logic: introduction to model-theoretic semantics of natural language, formal logic and discourse representation theory. Studies in linguistics and philosophy. Kluwer Academic, Dordrecht

    Google Scholar 

  • Kieras DE (1988) Towards a practical GOMS model methodology for user interface design. In: Helander M (ed) The handbook of human-computer interaction. North-Holland, Amsterdam, pp 135–158

    Google Scholar 

  • Kieras D, Meyer DE (1997) An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Hum-Comput Interact 12(4):391–438

    Article  Google Scholar 

  • Kintsch W (1998) Comprehension, a paradigm for cognition. Cambridge University Press, New York

    Google Scholar 

  • Klahr D, Chase WG, Lovelace EA (1983) Structure and process in alphabetic retrieval. J Exp Psychol Learn Mem Cogn 9(3):462–477

    Article  Google Scholar 

  • Kosslyn S (2006) The case for mental imagery. Oxford University Press, New York

    Book  Google Scholar 

  • Laird JE, Jones RM (1998) Building advanced autonomous AI systems for large scale real time simulations. In: Proceedings of the 1998 computer game developers’ conference. Freeman, Long Beach, pp 365–378

    Google Scholar 

  • Landauer T, Dumais S (1997) A solution to Plato’s problem: the latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychol Rev 104(2):211–240

    Article  Google Scholar 

  • Langacker RW (1987) Foundations of cognitive grammar, vol I: theoretical prerequisites. Stanford University Press, Stanford

    Google Scholar 

  • Langacker RW (1991) Foundations of cognitive grammar, vol II: descriptive application. Stanford University Press, Stanford

    Google Scholar 

  • Lebiere C, Wray R (eds) (2006) Between a rock and a hard place: cognitive science principles meet AI-hard problems. AAAI Press, Menlo Park. Papers from the AAAI spring symposium

    Google Scholar 

  • Lovett MC (1998) Choice. In: Anderson JR, Lebiere C (eds) The atomic components of thought. Erlbaum, Mahwah, pp 255–296

    Google Scholar 

  • Marcus M, Badler N, Joshi A, Pappas G, Pereira F, Romero M, McCallum A, Potts C, Yanco H (2008) SUBTLE (Situation Understanding Bot Through Language and Environment) project program review. University of Massachusetts, Amherst

    Google Scholar 

  • Matessa M (2000) Simulating adaptive communication. Carnegie Mellon University, Pittsburgh (Doctoral dissertation)

    Google Scholar 

  • McClelland JL, Rumelhart DE (1981) An interactive activation model of context effects in letter perception. Part I. An account of basic findings. Psychol Rev 88(5):375–407

    Article  Google Scholar 

  • McDonald D (1999) A rational reconstruction of Genaro. In: Proceedings of the RAGS Workshop, Edinburgh

  • Miller G (1995) WordNet: a lexical database for English. Commun ACM 38(11):39–41

    Article  Google Scholar 

  • Myers CW (2009) An account of model inspiration, integration, & sub-task validation. In: Proceedings of the 9th international conference on cognitive modeling, Manchester, UK

  • Prince A, Smolensky P (1993/2004) Optimality theory: constraint interaction in generative grammar. Wiley-Blackwell, New York

    Google Scholar 

  • Ritter FE, Van Rooy D, St Amant R (2002) A user modelling design tool based on a cognitive architecture for comparing interfaces. In: Computer-aided design of user interfaces III. Proceedings of the 4th international conference on computer-aided design of user interfaces CADUI’2002, Valenciennes, France, 15–17 May 2002. Kluwer Academics, Dordrecht, pp 111–118

    Google Scholar 

  • Scolaro J, Santarelli T (2002) Cognitive modeling teamwork, taskwork, and instructional behavior in synthetic teammates. In: Proceedings of the 11th conference on computer generated forces and behavioral representation. Institute for Simulation and Training, Orlando

    Google Scholar 

  • Seidenberg MS, McClelland JL (1989) A distributed, developmental model of word recognition and naming. Psychol Rev 96(4):523–568

    Article  Google Scholar 

  • Stokes J (2001) Speech interaction and human behavior representations (HBRs). In: Proceedings of 10th conference on computer generated forces and behavioral representation. SISO, Inc, Norfolk, pp 467–476

    Google Scholar 

  • Tambe M, Johnson WL, Jones RM, Koss F, Laird JE, Rosenbloom PS, Schwamb K (1995) Intelligent agents for interactive simulation environments. AI Mag 16(1):15–40

    Google Scholar 

  • Tanenhaus MK, Spivey-Knowlton MJ, Eberhard KM, Sedivy JC (1995) Integration of visual and linguistic information in spoken language comprehension. Science 268(5217):1632–1634

    Article  Google Scholar 

  • Traum DR, Allen JF (1994) Discourse obligations in dialogue processing. In: Proceedings of the 32nd annual meeting of the association for computational linguistics. Association for Computational Linguistics, Las Cruces, Morristown

    Google Scholar 

  • Traum DR, Rickel J, Gratch J, Marsella S (2003) Negotiation over tasks in hybrid human-agent teams for simulation-based training. In: Proceedings of the second international joint conference on Autonomous agents and multiagent systems. ACM, Melbourn, Australia, New York, pp 441–448.

    Chapter  Google Scholar 

  • van Dijk T, Kintsch W (1983) Strategies of discourse comprehension. Academic Press, New York

    Google Scholar 

  • Varges S (2006) Overgeneration and ranking for spoken dialogue systems. In: Proceedings of the 4th international natural language generation conference, Sydney, Australia, July 2006. Association for Computational Linguistics, pp 20–22

  • Vosse T, Kempen G (2000) Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar. Cognition 75:105–143

    Article  Google Scholar 

  • Walker MA, Whittaker SJ, Stent A, Maloor P, Moore J, Johnston M, Vasireddy G (2004) Generation and evaluation of user tailored responses in multimodal dialogue. Cogn Sci 28(5):811–840

    Article  Google Scholar 

  • Yang Y, Bello P (2005) Some empirical results concerning deontic reasoning: models, schema, or both? In: Proceedings of the 27th annual meeting of the cognitive science society. Erlbaum, Mahway, pp 2393–2398

    Google Scholar 

  • Zachary W, Santarelli T, Lyons D, Bergondy M, Johnston J (2001) Using a community of intelligent synthetic entities to support operational team training. In: Proceedings of the tenth conference on computer generated forces and behavioral representation. Orlando, FL: Institute for Simulation and Training, University of Central Florida, pp 215–224

  • Zwann R, Radvansky G (1998) Situation models in language comprehension and memory. Psychol Bull 123(2):162–185

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Air Force Research Laboratory, Mesa, AZ, USA

    Jerry Ball

  2. Air Force Research Laboratory, Wright-Patterson Air Force Base, OH, USA

    Christopher Myers

  3. General Motors, Mesa, AZ, USA

    Andrea Heiberg

  4. Cognitive Engineering Research Institute, Mesa, AZ, USA

    Nancy J. Cooke

  5. Alion, Boulder, CO, USA

    Michael Matessa

  6. L3 Communications, Mesa, AZ, USA

    Mary Freiman

  7. AGS TechNet, Dayton, OH, USA

    Stuart Rodgers

Authors
  1. Jerry Ball
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher Myers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Heiberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nancy J. Cooke
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Matessa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mary Freiman
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stuart Rodgers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jerry Ball.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ball, J., Myers, C., Heiberg, A. et al. The synthetic teammate project. Comput Math Organ Theory 16, 271–299 (2010). https://doi.org/10.1007/s10588-010-9065-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10588-010-9065-3

Keywords

Search

Navigation