Advertisement

Behavior Research Methods

, Volume 39, Issue 4, pp 852–858 | Cite as

Recoding low-level simulator data into a record of meaningful task performance: The integrated task modeling environment (ITME)

  • Robert KingEmail author
  • Simon Parker
  • Kon Mouzakis
  • Winston Fletcher
  • Patrick Fitzgerald
Article
  • 234 Downloads

Abstract

The Integrated Task Modeling Environment (ITME) is a user-friendly software tool that has been developed to automatically recode low-level data into an empirical record of meaningful task performance. The present research investigated and validated the performance of the ITME software package by conducting complex simulation missions and comparing the task analyses produced by ITME with task analyses produced by experienced video analysts. A very high interrater reliability (≥0.94) existed between experienced video analysts and the ITME for the task analyses produced for each mission. The mean session time:analysis time ratio was 1:24 using video analysis techniques and 1:5 using the ITME. It was concluded that the ITME produced task analyses that were as reliable as those produced by experienced video analysts, and significantly reduced the time cost associated with these analyses.

Keywords

Task Analysis Video Analysis Session Time Human Factor Engineering High Interrater Reliability 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Baskett, F., &Hennessy, J. L. (1993). Microprocessors: From desktops to supercomputers,Science,261, 864–871.CrossRefPubMedGoogle Scholar
  2. Booher, H. R. (1988). Progress of MANPRINT—The Army’s human factors program.Human Factors Society Bulletin,31, 1–3.Google Scholar
  3. Ferguson, R. L. (1990). High performance computer image generation: A marriage of computer graphics and image processing.Optical Engineering,1289, 140–154.Google Scholar
  4. Fisher, C., &Sanderson, P. M. (1993). Exploratory sequential data analysis: Traditions, techniques and tools. Report of the CHI ′92 workshop.SIGCHI Bulletin,25, 31–40.CrossRefGoogle Scholar
  5. Henderson, B. W. (1989). Simulators play a key role in LHX contractor selection.Aviation Week & Space Technology,131 (22), 34–37.Google Scholar
  6. Hennessy, R. T. (1990). Practical Human Performance Testing and Evaluation. In H. R. Booher (Ed.),MANPRINT: An approach to systems integration (pp. 433–470). New York: Reinhold.Google Scholar
  7. Howell, D. C. (1997).Statistical methods for psychology (4th Ed.). Belmont, CA: Duxbury.Google Scholar
  8. King, R. B., &Goss, S. (1995). Automating task analysis.Australian Journal of Psychology,47, 14.Google Scholar
  9. King, R. B., Kieboom, H. M., &Manton, J. G. (1993).Human factors aspects of S-70B-2 and FFG operations (Report No. ARL-RR-16). Melbourne, Australia: Aeronautical Research Laboratory.Google Scholar
  10. Monterey Technologies (2002). All Digital Interactive Video Analysis user manual (Version 3.50). Monterey, CA: Monterey Technologies.Google Scholar
  11. Patrick, J. D., & Mckenna, M. J. (1986). A generalised computer system for sports analysis.Australian Journal of Science & Medicine in Sport,18 (3).Google Scholar
  12. Sanderson, P. M. (1991).ESDA: Exploratory sequential data analysis (Report EPRL-94-01). Urbana: University of Illinois at Urbana-Champagne, Department of Mechanical and Industrial Engineering.Google Scholar
  13. Sanderson, P. M., &Fisher, C. (1994). Exploratory sequential data analysis: Foundations.Human-Computer Interaction,9, 251–317.CrossRefGoogle Scholar
  14. Sanderson, P. M., James, J. M., &Seidler, K. S. (1989). SHAPA: An Interactive Software Environment for Protocol Analysis.Ergonomics,32, 1271–1302.CrossRefGoogle Scholar
  15. Sanderson, P. M., Scott, J. J. P., Johnston, T., Mainzer, J., Watanabe, L. M., &James, J. M. (1994). MacSHAPA and the Enterprise of Exploratory Sequential Data Analysis (ESDA).International Journal of Human-Computer Studies,41, 633–681.CrossRefGoogle Scholar
  16. Shaffer, M. [T.] (1989). Using video for empirically validated task analysis (EVTA) of system-human interaction and performance.MANPRINT Bulletin,4, 12–13.Google Scholar
  17. Shaffer, M. T., Hendy, K. C., &White, L. R. (1988). An empirically validated task analysis (EVTA) of low-level army helicopter operations.Proceedings of the 32nd Annual Meeting of the Human Factors Society,1, 178–182.Google Scholar
  18. Shrout, P. E., &Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability.Psychological Bulletin,2, 420–428.CrossRefGoogle Scholar
  19. Strachan, I. W. (Ed.) (1995).Jane’s simulation and training systems 1995–1996 (8th Ed.). Coulson, U.K.: International Thomson.Google Scholar
  20. Voorhees, J. W., Bucher, N. M., Gossett, T., &Haworth, L. A. (1989). The Crew Station Research and Development Facility.Army Research, Development, & Acquisition Bulletin,2, 22–25.Google Scholar

Copyright information

© Psychonomic Society, Inc. 2007

Authors and Affiliations

  • Robert King
    • 1
    Email author
  • Simon Parker
    • 1
  • Kon Mouzakis
    • 2
  • Winston Fletcher
    • 2
  • Patrick Fitzgerald
    • 2
  1. 1.Air Operations DivisionDefence Science and Technology OrganisationFishermans BendAustralia
  2. 2.Swinburne University of TechnologyHawthornAustralia

Personalised recommendations