Cognition, Technology & Work

, Volume 7, Issue 1, pp 14–28 | Cite as

Problem detection

  • Gary Klein
  • Rebecca Pliske
  • Beth Crandall
  • David D. Woods
Original Article


Problem detection is the process by which people first become concerned that events may be taking an unexpected and undesirable direction that potentially requires action. Previous accounts [e.g., Cowan (Acad Manage Rev 11(4):763–776, 1986)] described problem detection as the accumulation of discrepancies until a threshold was reached. In reviewing incidents taken from a variety of natural settings, we found that discrepancy accumulation did not apply to the incidents we reviewed, because (a) cues to problems may be subtle and context-dependent, and (b) what counts as a discrepancy depends on the problem-solver’s experience and the stance taken in interpreting the situation. In many cases, detecting a problem is equivalent to reconceptualizing the situation.


Critical Incident Problem Detection Inattentional Blindness Experienced Nurse Cognitive Task Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We would like to thank Marvin Thordsen, Rob Hutton, and Patty McDermott for conducting the interviews with wildland firefighters and laparoscopic surgeons, and reviewing the transcripts of taped interviews with surgeons. We would also like to thank Robert Hoffman and several anonymous reviewers for making many helpful suggestions. Erik Hollnagel challenged our formulation and helped us clarify our ideas. We also appreciate the support and encouragement we have received from Fumiya Tanabe. This project was funded by several agreements with Japan Atomic Energy Research Institute.


  1. Anderson JR (1993) Problem solving and learning. Am Psychol 48(1):35–44CrossRefGoogle Scholar
  2. Bechara A, Damasio H, Tranel D, Damasio AR (1997) Deciding advantageously before knowing the advantageous strategy. Science 275:1293–1295CrossRefPubMedGoogle Scholar
  3. Billings R, Milburn T, Schaalman M (1980) A model of crisis perception: a theoretical and empirical analysis. Admin Sci Q 25:300–316Google Scholar
  4. Chow R, Christoffersen K, Woods DD (2000) A model of communication in support of distributed anomaly response and replanning. In: Proceedings of the XIVth Triennial Congress of the International Ergonomics Association and 44th annual meeting of the human factors and ergonomics society, vol 1, pp 34–37Google Scholar
  5. Christoffersen K, Woods DD (2003) Making sense of change: extracting events from dynamic process data (Report ERGO-CSEL 01-TR-02). Institute for Ergonomics/Cognitive Systems Engineering Laboratory, ColumbusGoogle Scholar
  6. Christoffersen K, Woods DD, Blike GT (2001) Extracting event patterns from telemetry data. In: Proceedings of the human factors and ergonomics society 45th annual meeting, Minneapolis/St. Paul, 8–12 October 2001. HFES, Santa Monica, pp 409–413Google Scholar
  7. Cohen MS, Freeman JT, Thompson B (1998) Critical thinking skills in tactical decision making: a model and a training method. In: Cannon-Bowers J, Salas E (eds) Making decisions under stress: implications for individual and team training. APA Press, Washington, pp 155–189Google Scholar
  8. Cook RI, Woods DD, McDonald JS (1991) Human performance in anesthesia: a corpus of cases. Cognitive Systems Engineering Laboratory Report, prepared for the Anesthesia Patient Safety Foundation, April 1991Google Scholar
  9. Cowan DA (1986) Developing a process model of problem recognition. Acad Manage Rev 11(4):763–776Google Scholar
  10. Crandall B, Calderwood R (1989) Clinical assessment skills of experienced neonatal intensive care nurses (No. Contract 1 R43 NR0191101 for The National Center for Nursing, NIH). Klein Associates, FairbornGoogle Scholar
  11. Crandall B, Getchell-Reiter K (1993) Critical decision method: a technique for eliciting concrete assessment indicators from the “intuition” of NICU nurses. Adv Nurs Sci 16(1):42–51Google Scholar
  12. Davis GA (1973) Psychology of problem solving. Basic Books, New YorkGoogle Scholar
  13. De Keyser V, Woods DD (1993) Fixation errors: failures to revise situation assessment in dynamic and risky systems. In: Colombo AG, Saiz de Bustamente A (eds) Advanced systems in reliability modeling. Kluwer, NorwellGoogle Scholar
  14. Dewey J (1896) The reflex arc concept in psychology. Psychol Rev 3:357–370Google Scholar
  15. Dominguez C (1998) Expertise in laparoscopic surgery: anticipation and affordances. Paper presented at the 4th naturalistic decision making conference, Warrenton, VAGoogle Scholar
  16. Dominguez CO, Flach JM, McDermott PL, McKellar DM, Dunn M (2004) The conversion decision in laparoscopic surgery: knowing your limits and limiting your risks. In: Smith K, Shanteau J, Johnson P (eds) Psychological explorations of competence in decision making, Cambridge University Press, NY, pp 7–39Google Scholar
  17. Downs A (1967) Inside bureaucracy. Little and Brown, BostonGoogle Scholar
  18. Duncker K (1945) On problem solving. In: JF Dashiell (ed) Psychological monographs. Translated by LS Lees 58(5, Whole No. 270), pp 1–113Google Scholar
  19. Engdahl RA, Keating RJ (1995) Strategic problem-solving process: a comprehensive model and the need for more focused attention on consensus seeking. Organ Develop J 13(3):80–94Google Scholar
  20. Feltovich PJ, Coulson RL, Spiro RJ (2001) Learners’ (mis)understanding of important and difficult concepts: a challenge to smart machines in education. In: Forbus KD, Feltovich PJ (eds) Smart machines in education. AAAI/MIT Press, Menlo ParkGoogle Scholar
  21. Flanagan JC (1954) The critical incident technique. Psychol Bull 51:327–358PubMedGoogle Scholar
  22. Forbus KD, de Kleer J (1993) Building problem solvers. The MIT Press, CambridgeGoogle Scholar
  23. Gentner D, Stevens AL (eds) (1983) Mental models. Lawrence Erlbaum Associates, MahwahGoogle Scholar
  24. Gibson JJ, Crooks LE (1938) A theoretical field-analysis of automobile driving. Am J Psychol 51:453–471Google Scholar
  25. Gleick J (1987) Chaos: making a new science. Viking Penguin, New YorkGoogle Scholar
  26. Greeno JG, Simon HA (1988) Problem solving and reasoning. In: Atkinson RC, Herrnstein RJ, Lindzey G, Luce RD (eds) Stevens’ handbook of experimental psychology: vol 2, Learning and cognition. Wiley, New YorkGoogle Scholar
  27. Guerlain S, Smith PJ, Obradovich J, Rudmann S, Strohm P, Smith JW, Svirbely J, Sachs L (1999) Interactive critiquing as a form of decision support: an empirical evaluation. Hum Factors 41(1):72–89Google Scholar
  28. Hayes JR (1981) The complete problem solver. The Franklin Institute Press, PhiladelphiaGoogle Scholar
  29. Höffding H (1889) Ueber Wiederkennen, Assocation und psychische Activat. Vierteljahrsschrift fur wissenschaftlische Philosophie 13:420–458Google Scholar
  30. Hoffman RR, Crandall BW, Shadbolt NR (1998) Use of the critical decision method to elicit expert knowledge: a case study in cognitive task analysis methodology. Hum Factors 40(2):254–276Google Scholar
  31. Johnson-Laird PN (1983) Mental models: towards a cognitive science of language, inference, and consciousness (Cognitive Science, No. 6). Harvard University Press, CambridgeGoogle Scholar
  32. Kaempf GL, Klein G, Thordsen ML, Wolf S (1996) Decision making in complex command-and-control environments. Hum Factors 38:220–231Google Scholar
  33. Klein G (1997) Developing expertise in decision making. Thinking Reasoning 3(4):337–352CrossRefGoogle Scholar
  34. Klein GA, Crandall BW (1995) The role of mental simulation in naturalistic decision making. In: Hancock P, Flach J, Caird J, Vicente K (eds) Local applications of the ecological approach to human–machine systems, vol 2. Lawrence Erlbaum Associates, Mahwah, pp 324–358Google Scholar
  35. Klein GA, Hoffman R (1993) Seeing the invisible: perceptual/cognitive aspects of expertise. In: Rabinowitz M (ed) Cognitive science foundations of instruction. Lawrence Erlbaum Associates, Mahwah, pp 203–226Google Scholar
  36. Klein G, Hutton R (1995) The innovators: high-impact researchers at the Armstrong Laboratory Human Engineering Division, Armstrong Laboratory (U) (AL/CF-FR-1995–0027). United States Air Force Armstrong Laboratory, Wright Patterson AFB, OHGoogle Scholar
  37. Klein GA, Calderwood R, Clinton-Cirocco A (1988) Rapid decision making on the fireground (Technical Report No. DTIC No. AD-A199492, US Army Research Institute, Alexander
  38. Klein GA, Calderwood R, MacGregor D (1989) Critical decision method for eliciting knowledge. IEEE Trans Syst Man Cybern 19(3):462–472CrossRefGoogle Scholar
  39. Klein G, Ross KG, Moon BM, Klein DE, Hoffman RR, Hollnagel E (2003) Macrocognition. IEEE Intell Syst 18(3):81–85CrossRefGoogle Scholar
  40. Klein G, Phillips JK, Rall E, Peluso DA (2004) A data/frame theory of sensemaking (submitted for publication)Google Scholar
  41. Klinger DW, Militello LG (2002) Designing for performance: a cognitive systems engineering and cognitive task analysis approach to the modification of the AWACS weapons director interface (submitted 10/2001)Google Scholar
  42. Layton C, Smith PJ, McCoy CE (1994) Design of a cooperative problem-solving system for en-route flight planning: an empirical evaluation. Hum Factors 36(1):94–119Google Scholar
  43. MacCrimmon KR (1973) Managerial decision-making. In: McGuire JW (ed) Contemporary management: issues and viewpoints. Prentice-Hall, Englewood Cliffs, pp 445–495Google Scholar
  44. Mack A (2003) Inattentional blindness: looking without seeing. Curr Dir Psychol Sci 12(5):180–184CrossRefGoogle Scholar
  45. Mumaw RJ, Roth EM, Vicente KJ, Burns CM (2000) There is more to monitoring a nuclear power plant than meets the eye. Hum Factors 42(1):36–55PubMedGoogle Scholar
  46. Newell A, Simon HA (1972) Human problem solving. Prentice-Hall, New JerseyGoogle Scholar
  47. Olson WA, Sarter NB (2001) Management by consent in human–machine systems: when and why it breaks down. Hum Factors 43(2):255–266PubMedGoogle Scholar
  48. Perrow C (1984) Normal accidents: living with high-risk technologies. Basic Books, New YorkGoogle Scholar
  49. Pliske RM, Crandall B, Klein G (2004) Competence in weather forecasting. In: Smith K, Shanteau J, Johnson P (eds) Psychological investigations of competence in decision making. Cambridge University Press, Cambridge, pp 40–70Google Scholar
  50. Polya G (1957) How to solve it, 2nd edn. A Doubleday Anchor Book, Garden CityGoogle Scholar
  51. Roth EM, Woods DD, Pople HE (1992) Cognitive simulation as a tool for cognitive task analysis. Ergonomics 35:1163–1198PubMedGoogle Scholar
  52. Rubinstein MF (1975) Patterns of problem solving. Prentice-Hall, NJGoogle Scholar
  53. Schmitt JF, Klein G (1996) Fighting in the fog: dealing with battlefield uncertainty. Mar Corps Gaz 80:62–69Google Scholar
  54. Schrenk LP (1969) Aiding the decision making—a decision process model. Ergonomics 12:543–557PubMedGoogle Scholar
  55. Smith GF (1989) Defining managerial problems: a framework for prescriptive theorizing. Manage Sci 35(8):963–981Google Scholar
  56. Sorkin RD (1988) Why are people turning off our alarms? J Acoust Soc Am 84:1107–1108Google Scholar
  57. Sorkin RD, Kantowitz BH, Kantowitz SC (1998) Likelihood alarm displays. Hum Factors 30:445–459Google Scholar
  58. Watts-Perotti J, Woods DD (1997) A cognitive analysis of anomaly response in space shuttle mission control (Report CSEL 97-TR-02C). The Ohio State University, ColumbusGoogle Scholar
  59. Weick K (1995) Sensemaking in organizations. Sage Publications, Thousand OaksGoogle Scholar
  60. Wertheimer M (ed) (1959) Productive thinking. Harper and RowGoogle Scholar
  61. Woods DD (1994) Cognitive demands and activities in dynamic fault management: abduction and disturbance management. In: Stanton N (ed) Human factors of alarm design. Taylor and Francis, LondonGoogle Scholar
  62. Woods DD (1995) The alarm problem and directed attention in dynamic fault management. Ergonomics 38(11):2371–2393Google Scholar
  63. Woods DD, O’Brien J, Hanes LF (1987) Human factors challenges in process control: the case of nuclear power plants. In: Salvendy G (ed) Handbook of human factors/ergonomics. Wiley, New YorkGoogle Scholar
  64. Woods DD, Patterson ES, Roth EM (2002) Can we ever escape from data overload? A cognitive systems diagnosis. Cogn Technol Work 4(1):22–36CrossRefGoogle Scholar
  65. Xiao Y (1994) Interacting with complex work environment: a field study and a planning model. PhD thesis, University of TorontoGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2004

Authors and Affiliations

  • Gary Klein
    • 1
  • Rebecca Pliske
    • 2
  • Beth Crandall
    • 1
  • David D. Woods
    • 3
  1. 1.Klein Associates Inc.FairbornUSA
  2. 2.Dominican UniversityRiver ForestUSA
  3. 3.Cognitive Systems Engineering LaboratoryOhio State UniversityColumbusUSA

Personalised recommendations