Skip to main content
SpringerLink
Log in

PerFECT: An Automated Framework for Training on the Fly

  • Chapter
  • First Online:
Human-in-the-Loop Simulations

  • 1294 Accesses

Abstract

Currently available cognitive training systems can highly benefit from more adaptable and encapsulated frameworks that include better performance assessment methods, robust feedback mechanisms and automated mechanisms that reduce the manual intervention and curriculum management required during training sessions. In short, there is an ardent need for an automated human in the loop training system that can effectively train cognitive skills required for military operations. An automated training system would be extremely beneficial if it can be easily coupled with a synthetic learning environment to function autonomously is an entirely data driven manner. Such a system would enable rapid deployment of key training scenarios, skills and tactics to war fighters and help them maintain a superior level of competence in the battlefield. An automated framework for training on the fly also known as performance feedback engine for conflict training (PerFECT) which includes key components for simulating training scenarios, measuring trainee’s performance, providing relevant feedback and dynamic curriculum management is discussed in this chapter. First, the training system comprises of custom plug-in interface that allows components of the training framework to readily interface with a simulated virtual learning environment. Second, it has a “Performance Evaluator” that enables automated, real-time and objective evaluation of a trainee’s performance grounded within an objective framework known as time window and enables run-time evaluation of performance skills based on a skills matrix. Third, PerFECT has a “Feedback System” that can provide contextual and immediate feedback to trainees based on process measures. Finally, PerFECT includes a “Curriculum Manager” that dynamically selects appropriate training scenario from a template library with varying levels of complexity. The selection algorithm for training scenario is based on the trainee’s historical performance scores and complexity of the earlier scenarios. We also present the initial findings from a pilot study which helps illustrate the capabilities of the framework and conclude with future directions in this area of research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Anderson PL, Rothbaum BO, Hodges L (2001) Virtual reality: using the virtual world to improve quality of life in the real world. Bulletin of the Menninger Clin 65:78–91

    Article  Google Scholar 

  • Bainbridge WS (2007) The scientific research potential of virtual worlds. Science 317:472–476

    Article  Google Scholar 

  • Balzer WK, Hammer LB, Sumner KE, Birchenough TR, Martens SP, Raymark PH (1994) Effects of cognitive feedback components, display format, and elaboration on performance. Organ Behav Hum Decis Process 58:369–385

    Article  Google Scholar 

  • Barnett J (2009) Virtual Environments and unmanned systems: human system integration issues. In: Cohn J, Schmorrow D (eds) The PSI handbook of virtual environments for training and education: developments for military and beyond. Praeger Security International, Westport

    Google Scholar 

  • Bewley WL, Chung GKWK, Delacruz GC, Baker EL (2009) Assessment models and tools for virtual environment training. In: Schmorrow D, Cohn J, Nicholson D (eds) The PSI handbook of virtual environments for training and education: developments for military and beyond, vol 1. Praeger Security International, Westport

    Google Scholar 

  • Billings CE (1997) Aviation automation: the search for a human-centered approach. Erlbaum, Mahwah

    Google Scholar 

  • Cannon-Bowers JA, Salas E (1997) A framework for developing team performance measures in training. In: Brannick MT, Salas E, Prince E (eds) Team performance assessment and measurement: theory, methods, and applications. Erlbaum, Hillsdale

    Google Scholar 

  • Cannon-Bowers JA, Salas E (2001) Reflections on shared cognition. J Orgmet Ch 22:195–202

    Google Scholar 

  • Cannon-Bowers JA, Bowers CA, Sanchez A (2008) Using synthetic learning Environments to train teams. In: Sessa VI, London M (eds) Work group learning: understanding, improving and assessing how groups learn in organizations. Lawrence Erbaum Associates, New York

    Google Scholar 

  • Canon-Bowers JA, Burns JJ, Salas E, Pruitt JS (1998) Advanced technology in scenario-based training. In: Cannon-Bowers JA, Salas E (eds) Making decisions under stress: implications for individual and team training. APA, Washington

    Chapter  Google Scholar 

  • Charness N, Schultetus RS (1999) Knowledge and expertise. In: Durso FT, Nickerson RS, Schvaneveldt RW et al (eds) Handbook of applied cognition. Wiley, West Sussex

    Google Scholar 

  • Cohn J, Nicholson D, Schmorrow D (2009) Integrated systems, training evaluations, and future directions. The PSI handbook of virtual environments for training and education: developments for military and beyond. Praeger Security International, Westport

    Google Scholar 

  • Department of the Army (2002) Battle drills for the infantry rifle platoon and squad, ARTEP 7–8

    Google Scholar 

  • Dieterle E, Clarke J (2008) Multi-user virtual environments for teaching and learning. In: Pagani M (ed) Encyclopedia of multimedia technology and networking, 3rd edn. Idea Group, Hershey

    Google Scholar 

  • Endsley MR (1995) Measurement of situation awareness in dynamic systems. Hum Factors 37(1):65–84

    Article  Google Scholar 

  • Ericsson KA, Krampe RT, Tesch-Romer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100:363–406

    Article  Google Scholar 

  • Foltz P, LaVoie N, Oberbreckling R, Rosenstein M (2009) Automated performance assessment of teams in virtual environments. In: Schmorrow D, Cohn J, Nicholson D (eds) The PSI handbook of virtual environments for training and education: developments for military and beyond, vol 1. Praeger Security International, Westport

    Google Scholar 

  • Gordon SE (1994) Systematic training program design: maximizing effectiveness and minimizing liability. Prentice Hall, Englewoods Cliffs

    Google Scholar 

  • Green DM, Swets JA (1988) Signal detection theory and psychophysics. Peninsula Publishing, Los Altos

    Google Scholar 

  • Grier RA, Warm JS, Dember WN, Matthews G, Galinsky TL, Szalma JL, Parasuraman R (2003) The vigilance decrement reflects limitations in effortful attention, not mindlessness. Hum Factors 45:349–359

    Article  Google Scholar 

  • Grubb PL, Warm JS, Dember WN, Berch DB (1995) Effects of multiple-signal discrimination on vigilance performance and perceived workload. Hum Factors Ergon Soc Annu Meet Proc 39:1360–1364

    Google Scholar 

  • Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): results of experimental and theoretical research. In: Hancock PA, Meshkati N (eds) Human Mental Workload. North Holland Press, Amsterdam pp 139–183

    Chapter  Google Scholar 

  • Johnston JH, Smith-Jentsch KA, Cannon- Bowers JA (1997) Performance measurement tools for enhancing team decision making. In: Brannick MT, Salas E, Prince C (Eds) Assessment and measurement of team performance: Theory, research, and applications. NJ: Erlbaum, Hillsdale pp 45–62

    Google Scholar 

  • Kirwan B, Ainsworth LK (1992) A guide to task analysis. Taylor & Francis, Bristol

    Google Scholar 

  • Kluger AN, DeNisi A (1996) The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 119:254–284

    Article  Google Scholar 

  • Kozlowski SWJ, Toney RJ, Mullins ME, Weissbein DA, Brown KG, Bell BS (2001) Developing adaptability: a theory for the design of integrated-embedded training systems. In: Salas E (ed) Advances in human performance and cognitive engineering research, vol 1. JAI/Elsevier Science, Amsterdam

    Google Scholar 

  • Lampton DR, Bliss JP, Morris CS (2002) Human performance measurement in virtual environments. In: Stanney KM (ed) Handbook of virtual environment: design, implementation, and applications. Erlbaum, Mahwah

    Google Scholar 

  • Lee J, Moray N (1992) Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35(10):1243–1270

    Article  Google Scholar 

  • Lewandowsky S, Little DR, Kalish M (2007) Knowledge and expertise. In: Durso FT, Nickerson R, Dumais S, Lewandowsky S, Perfect T (eds) Handbook of applied cognition, 2nd edn. Wiley, West Sussex

    Google Scholar 

  • Loomis JM, Blascovich JJ, Beall AC (1999) Immersive virtual enviornment technology as a basic research tool in psychology. Behav Res Methods instrum comput 31:557–564

    Article  Google Scholar 

  • Mantovani F (2001) VR learning: potential and challenges for the use of 3D environments in education and training. In: Riva G, Galimberti C (eds) Towards cyberpsychology. IOS Press, Amsterdam

    Google Scholar 

  • Meister D (2004) Conceptual foundations of human factors measurement. Earlbaum, Mahwah

    Google Scholar 

  • Moray N, Inagaki T, Itoh M (2000) Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. J Exp Psychol Appl 6:44–58

    Article  Google Scholar 

  • Mosier KL, Skitka LJ, Heers S, Burdick M (1997) Automation bias: decision making and performance in high-tech cockpits. Int J Aviat Psychol 8:47–63

    Article  Google Scholar 

  • Nicholson D, Schmorrow D, Cohn J (2009) The PSI handbook of virtual environments for training and education: developments for military and beyond, vol 2. VE components and training technologies. Praeger Security International, Westport

    Google Scholar 

  • Oser RL, Gualtieri JW, Cannon-Bowers JA, Salas E (1999) Training team problem-solving skills: an event-based approach. Comp Hum Behav 15:441–462

    Article  Google Scholar 

  • Parasuraman R, Riley V (1997) Humans and automation: use, misuse, disuse, abuse. Hum Factors 39:230–253

    Article  Google Scholar 

  • Rasmussen JR (1983) Skills, rules and knowledge: signals, signs, symbols, and other distinctions in human performance models. IEEE trans syst man cybern 13:257–266

    Google Scholar 

  • Rasmussen J (1986) Information processing and human-machine interaction: an approach to cognitive engineering. Elsevier, New York

    Google Scholar 

  • Riva G (1997) Virtual reality in neuro-psycho-physiology: cognitive, clinical and methodological issues in assessment and rehabilitation. IOS Press, Amsterdam

    Google Scholar 

  • Roman PA, Brown D (2008) Games—just how serious are they? Interservice/industry training, simulation and education conference (I/ITSEC). Academic Press, Orlando

    Google Scholar 

  • Rothrock L (2001) Using time windows to evaluate operator performance. Int J Cogn Ergon 5:1–21

    Article  Google Scholar 

  • Salas E, Cannon-Bowers JA (1997) Methods, tools, and strategies for team training. In: Quinones MA, Ehrenstein A (eds) Training for a rapidly changing workplace: applications of psychological research. APA, Washington

    Google Scholar 

  • Salas E, Oser RL, Cannon-Bowers JA, Daskarolis-Kring E (2002) Team training in virtual environments: an event-based approach. In: Quiñones MA, Ehrensstein A (eds) Training for a rapidly changing workplace: applications of psychological research. APA, Washington

    Google Scholar 

  • Salas E, Cannon-Bowers JA (2001). The science of training: A decade of progress. Ann Rev Psychol 52:471–499

    Article  Google Scholar 

  • Salas E, Priest HA, Wilson KA, Burke CS (2006) Scenario-based training: improving military mission performance and adaptability. In: Adler AB, Castro CA, Britt TW (eds) Military life: the psychology of serving in peace and combat. Praeger Security International, Westport

    Google Scholar 

  • Sarter NB, Woods DD (1997) Team play with a powerful and independent agent: operational experiences and surprises on the Airbus A-320. Hum Factors 39:553–569

    Article  Google Scholar 

  • Schmorrow D, Cohn J, Nicholson D (2009) The PSI handbook of virtual environments for training and education: developments for military and beyond, vol 1. Learning, requirements and metrics. Praeger Security International, Westport

    Google Scholar 

  • Schultetus S, Charness N (1999) Recall vs position evaluation revisited: the importance of position-specific memory in chess skill. Am J Psychol 112(4):555–569

    Article  Google Scholar 

  • Shaffer DW, Resnick M (1999) "Thick" authenticity: new media and authentic learning. J Interact Learn Res 10:195–215

    Google Scholar 

  • Smithers JW, Wohers AJ, London M (1995) A field study of reactions to normative versus individualized upward feedback. Group Organ Manag 20:61–89

    Article  Google Scholar 

  • Sterling BA, Burns CA (2004) Skills required for platoon leaders in the objective force unit of action. Army Research Lab Aberdeen Proving Ground, MD

    Google Scholar 

  • Swets JA (1996) Signal detection theory and ROC analysis in psychology and diagnostics: collected papers. Earlbaum, Mahwah

    MATH  Google Scholar 

  • Tharanathan A, Derby P, Thiruvengada H (under review) Training for metacognition in simulated environments. In:Human-in-the-Loop Simulations. Springer, London

    Google Scholar 

  • Thiruvengada H, Rothrock L (2007) Time windows based team performance measures: a framework to measure team performance in dynamic environments. Cogn Tech Work 9:99–108

    Article  Google Scholar 

  • van Buskirk W, Cornejo J, Astwood R, Russell S, Dorsey D, Dalton J (2009) A theoretical framework for developing systematic instructional guidance for virtual environment training. In: Schmorrow D, Cohn J, Nicholson D (eds) The handbook of virtual environment training. Praeger Security International, Westport

    Google Scholar 

  • van Dongen KW, Tournoij E, van der Zee DC, Schijven MP, Broeders IAMJ (2007) Construct validity of the LapSim: can the LapSim virtual reality simulator distinguish between novies and experts? Surg Endosc 21:1413–1417

    Article  Google Scholar 

  • Verdaasdonk EG, Stassen LP, Monteny LJ, Dankelman J (2005) Validation of a new and simple virtual reality simulator for training of basic endoscopic skills. Surg Endosc 20:1–9

    Google Scholar 

Download references

Acknowledgments

This work supported by DARPA/IPTO under contract# HR0011-09-C-0102. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the DARPA/IPTO. We would like to thank Amy Vanderbilt (DARPA-IPTO), David Montgomery (DARPA-IPTO) and Joseph Traugott (US RDECOM-STTC) for their guidance and mentoring on this project. We would also like to thank Tim Stone (Omega Training, Cubic Corporation) for his support and shaping our understanding of the training provided to the Fire Teams. We would also like to thank members for CATT Lab (University of Maryland) including Michael Pack, Walter Lucman, Michael VanDaniker and Michael Couture for helping us understand the capabilities of the OLIVE Virtual Environment. Second Life is a registered trademark of Linden Research, Inc. OLIVE is a trademark of Forterra Systems Inc (now part of SAIC). All other trademarks used herein are the property of their respective owners.

The views, opinions, and/or findings contained in this article/presentation are those of the author/presenter and should not be interpreted as representing the official views or policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the Department of Defense.

Author information

Authors and Affiliations

  1. Honeywell ACS Labs, 1985 Douglas Drive N, Minneapolis, MN, 55442, USA

    Hari Thiruvengada, Anand Tharanathan & Paul Derby

Authors
  1. Hari Thiruvengada
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anand Tharanathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul Derby
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hari Thiruvengada .

Editor information

Editors and Affiliations

  1. , The Harold and Inge Marcus Department of, The Pennsylvania State University, Leonhard Building 210, State College, 16802, Pennsylvania, USA

    Ling Rothrock

  2. , 405 Russ Engineering Center, Wright State University, Col. Glenn Hwy 3640, Dayton, 45435, Ohio, USA

    S. Narayanan

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Thiruvengada, H., Tharanathan, A., Derby, P. (2011). PerFECT: An Automated Framework for Training on the Fly. In: Rothrock, L., Narayanan, S. (eds) Human-in-the-Loop Simulations. Springer, London. https://doi.org/10.1007/978-0-85729-883-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-883-6_11

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-882-9

  • Online ISBN: 978-0-85729-883-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation