The Role of the Human Operator in Image-Based Airport Security Technologies

  • Ian Graves
  • Marcus Butavicius
  • Veneta MacLeod
  • Rebecca Heyer
  • Kathryn Parsons
  • Natalie Kuester
  • Agata McCormac
  • Philip Jacques
  • Raymond Johnson
Part of the Studies in Computational Intelligence book series (SCI, volume 338)


Heightened international concerns relating to security and identity management have led to an increased interest in security applications, such as face recognition and baggage and passenger screening at airports. A common feature of many of these technologies is that a human operator is presented with an image and asked to decide whether the passenger or baggage corresponds to a person or item of interest. The human operator is a critical component in the performance of the system and it is of considerable interest to not only better understand the performance of human operators on such tasks, but to also design systems with a human operator in mind. This paper discusses a number of human factors issues which will have an impact on human operator performance in the operational environment, as well as highlighting the variables which must be considered when evaluating the performance of these technologies in scenario or operational trials based on Defence Science and Technology Organisation’s experience in such testing.


False Alarm Face Recognition Human Operator Face Recognition System Aviation Security 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kaine, A.K.: The impact of facial recognition systems on business practices within an operational setting. In: Proceedings of EC-VIP, 4th EURASIP Conference on Video/Image Processing and Multimedia Communications, vol. 2, pp. 315–320 (2003),
  2. 2.
    Butavicius, M.A.: Evaluating and predicting the performance of an identification face recognition system in an operational setting. Australian Society for Operations Research Bulletin 25(2), 2–13 (2006)Google Scholar
  3. 3.
    Sunde, J., Butavicius, M.A., Graves, I., Hemming, D., Ivancevic, V., Johnson, R., Kaine, A., McLindin, B.A., Meaney, K.A.: Methodology for evaluating the operational effectiveness of facial recognition systems. In: Proceedings of EC-VIP, 4th EURASIP Conference on Video/Image Processing and Multimedia Communications, vol. 2, pp. 441–448 (2003),
  4. 4.
    Shea, D.A., Morgan, D.: Detection of explosives on airline passengers: Recommendation of the 9/11 Commission and Related Issues. GAO-07-634 Aviation Security paper (2007)Google Scholar
  5. 5.
  6. 6.
    Hannum, D.W., Parmeter, J.E.: Survey of Commercially Available Explosives Detection Technologies and Equipment. National Institute of Justice 171133. The National Law Enforcement and Corrections Technology Center (1998)Google Scholar
  7. 7.
    Parmeter, J.E., Murray, D.W., Hannum, D.W.: Guide for the Selection of Drug Detectors for Law Enforcement Applications. National Institute of Justice Guide 601- 00. Law Enforcement and Corrections Standards and Testing Program (2000)Google Scholar
  8. 8.
    Singh, S., Singh, M.: Explosives detection systems (EDS) for aviation security. Signal Processing 83, 31–55 (2003)zbMATHCrossRefGoogle Scholar
  9. 9.
    Harel, A.: Biometrics, Identification and Practical Ethics (2001), (last accessed August 19, 2008)
  10. 10.
    Keval, H., Sasse, M.A.: Can we ID from CCTV? Image quality in digital CCTV and face identification performance. In: Proceedings of the SPIE Conference on Mobile Multimedia/Image Processing, Security and Applications, 69820K-69820K-15 (2008)Google Scholar
  11. 11.
    Sheridan, T.B.: Humans and automation: Systems design and research issues. Human Factors and Ergonomics Society/Wiley, Santa Monica/New York (2002)Google Scholar
  12. 12.
    Parasuraman, R., Riley, V.: Humans and automation: use, misuse, disuse and abuse. Human Factors 39, 230–253 (1997)CrossRefGoogle Scholar
  13. 13.
    Wickens, C.D., Lee, J.D., Liu, Y., Gordon Becker, S.E.: An Introduction to Human Factors Engineering, 2nd edn. Pearson, Upper Saddle River (2004)Google Scholar
  14. 14.
    Parasuraman, R.: Human-computer monitoring. Human Factors 29, 695–706 (1987)Google Scholar
  15. 15.
    Vast, R.L., Butavicius, M.A.: A literature review of face recognition for access control: Human versus machine solutions. In: DSTO-TR-1747, Defence Science and Technology Organisation, Edinburgh, SA (2005)Google Scholar
  16. 16.
    Pike, G., Kemp, R., Brace, N.: The Psychology of Human Face Recognition. Paper presented at the IEE Electronics and Communications: Visual Biometrics, London (2000)Google Scholar
  17. 17.
    Burton, A.M., Miller, P., Bruce, V., Hancock, P.J.B., Henderson, Z.: Human and automatic face recognition: a comparison across image formats. Vision Research 41, 3185–3195 (2001)CrossRefGoogle Scholar
  18. 18.
    Hancock, P.J.B., Bruce, V., Burton, A.M.: Recognition of unfamiliar faces. Trends in Cognitive Sciences 4(9), 330–337 (2000)CrossRefGoogle Scholar
  19. 19.
    Lablack, A., Djeraba, C.: Analysis of human behaviour in front of a target scene. In: 19th International Conference on Pattern Recognition, ICPR 2008 (2008)Google Scholar
  20. 20.
    Harrigan, J.A., Taing, K.T.: Fooled by a smile: detecting anxiety in others. Journal of Nonverbal Behavior 21(3), 203–221 (1997)CrossRefGoogle Scholar
  21. 21.
    Harrigan, J.A., Harrigan, K.M., Sale, B.A., Rosenthal, R.: Detecting anxiety and defensiveness from visual and auditory cues. Journal of Personality 64(3), 675–709 (1996)CrossRefGoogle Scholar
  22. 22.
    Hill, H., Bruce, V.: The effects of lighting on the perception of facial surfaces. Journal of Experimental Psychology: Human Perception and Performance 22, 986–1004 (1996)CrossRefGoogle Scholar
  23. 23.
    Liu, X., Gale, A.G.: Effects of time pressure on searching for terrorist threats in Xray air passenger luggage images. In: de Waard, D., Hockey, B., Nickel, P., Brookhuis, K. (eds.) Human Factors Issues in Complex System Performance. Shaker Publishing, Maastricht (2007)Google Scholar
  24. 24.
    Schneier, B.: Beyond Fear: Thinking Sensibly about Security in an Uncertain World. Copernicus Books, New York (2003)Google Scholar
  25. 25.
    Shackel, B.: Applied Ergonomics Handbook. IPC Technology Press, Guildford (1975)Google Scholar
  26. 26.
    Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., Carey, T.: Human- Computer Interaction. Addison-Wesley, Harlow (1994)Google Scholar
  27. 27.
    Murata, A., Hirokazu, I.: Effectiveness of Cognitively Engineered Human Interface Design. In: Proceedings of the 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 20, pp. 2717–2720 (1998)Google Scholar
  28. 28.
    Neiderman, E.C., Fobes, J.L.: Human Factors Considerations for Aviation Security Technologies. In: Proceedings of the 33rd Annual Security Technology Conference (1999)Google Scholar
  29. 29.
    Elliot, E.S., Wills, E.J., Goldstein, A.G.: The effects of discrimination on training on the recognition of white and oriental faces. Bulletin of the Psychonomic Society 2, 71–73 (1973)Google Scholar
  30. 30.
    Goldstein, A.G., Chance, J.E.: Effects of training on Japanese face recognition: reduction of the other-race effect. Bulletin of the Psychonomic Society 23, 211–214 (1985)Google Scholar
  31. 31.
    Cesa, I.L.: Of attractive librarians and lop-sided faces: development and testing of a training procedure for improving the accuracy of eyewitnesses. Dissertation Abstracts International 56, 5221 (1996)Google Scholar
  32. 32.
    Butavicius, M., Parsons, K., McCormac, A., Johnson, R., Questiaux, D., Ayubi, Z., Sunde, J.: Scenario Testing of Body Scanners (Backscatter X-ray, Active and Passive Mm-Wave, THz), LAGs Type 3 (U), DSTO-TN-0883 (2008)Google Scholar
  33. 33.
    Schwaninger, A.: Airport security human factors: From the weakest to the strongest link in airport security screening. In: The 4th International Aviation Security Technology Symposium (2006)Google Scholar
  34. 34.
    Gerrig, R.J., Zimbardo, P.G.: Psychology and Life, 17th edn. Allyn & Bacon, Boston (2005)Google Scholar
  35. 35.
    Schwaninger, A.: Computer based training: a powerful tool to the enhancement of human factors. In: Aviation Security International, pp. 31–36 (February 2004)Google Scholar
  36. 36.
    McCarley, J.S., Kramer, A.F., Wickens, C.D., Vidoni, E.D., Boot, W.R.: Visual Skills in Airport-Security Screening. Psychological Sciences 15, 302–306 (2004)CrossRefGoogle Scholar
  37. 37.
    Schwaninger, A., Hardmeier, D., Hofer, F.: Aviation security screeners visual abilities and visual knowledge measurement. IEEE Aerospace and Electronic Systems 20, 29–35 (2005)Google Scholar
  38. 38.
    Haga, S., Shinoda, H., Mitsuteru, K.: Effects of task difficulty and time-on-task on mental workload. Japanese Psychological Research 3, 134–143 (2002)CrossRefGoogle Scholar
  39. 39.
    Wallach, M.A., Kogan, N., Bem, D.J.: Group Influence on Individual Risk Taking. Journal of bnormal and Social Psychology 65, 75–86 (1962)CrossRefGoogle Scholar
  40. 40.
    Atthowe, J.: Interpersonal decision making: The resolution of a dyadic conflict. Journal of Abnormal and Social Psychology 62, 114–119 (1961)CrossRefGoogle Scholar
  41. 41.
    Endsley, M.R., Jones, W.M.: Situation awareness, information dominance & information warfare. AL/CF-TR-1997-0156, United States Air Force Armstrong Laboratory, Wright-Patterson Air Force Base, OH (1997)Google Scholar
  42. 42.
    Myers, D.G., Lamm, H.: The group polarization phenomenon. Psychological Bulletin 83, 602–627 (1976)CrossRefGoogle Scholar
  43. 43.
    Menner, T., Barrett, D.J.K., Phillips, L., Donnelly, N., Cave, K.R.: Costs in Searching for Two Targets: Dividing Search Across Target Types Could Improve Airport Security Screening. Applied Cognitive Psychology 21, 915–932 (2006)CrossRefGoogle Scholar
  44. 44.
    Parasuraman, R.: Vigilance, monitoring, and search. In: Boff, K.R., Kaufman, L., Thomas, J.P. (eds.) Handbook of Perception and Human Performance, vol. 2. John Wiley & Sons, Inc., Canada (1986)Google Scholar
  45. 45.
    Robert, G., Hockey, J.: Changes in screener efficiency as a function of environmental stress, fatigue, and circadian rhythms. In: Boff, K.R., Kaufman, L., Thomas, J.P. (eds.) Handbook of Perception and Human Performance, vol. 2, John Wiley & Sons, Inc., Canada (1986)Google Scholar
  46. 46.
    Webster, D.M., Richter, L., Kruglanski, A.W.: On leaping to conclusions when feeling tired: mental fatigue effects on impressionable primacy. Journal of Experimental Social Psychology 32, 181–195 (1996)CrossRefGoogle Scholar
  47. 47.
    Harma, M.: Individual differences in toleration to shiftwork: A review. Ergonomics 36(1-3), 101–109 (1993)CrossRefGoogle Scholar
  48. 48.
    Boff, K.R., Kaufman, L., Thomas, J.P. (eds.): Handbook of Perception and Human Performance, vol. 2. John Wiley & Sons, Inc, Canada (1986)Google Scholar
  49. 49.
    Ariely, D., Zakay, D.: A timely account of the role of duration in decision making. Acta Psychologica 108, 187–207 (2001)CrossRefGoogle Scholar
  50. 50.
    Carroll-Mayer, M., Fairweather, B., Stahl, B.C.: Ignotious Per Ignotum: 21st Century Surveillance Technology and the Presumption of Guilt. In: Proceedings of the 21st British and Irish Law, Education and Technology Association Conference, Malta (April 2006)Google Scholar
  51. 51.
    Dror, I.E.: Don’t forget us humans! The Times (July 31, 2006)Google Scholar
  52. 52.
    Butavicius, M., Foster, R., Whittenbury, A., Parsons, K., McCormac, A., MacLeod, V.: Human Factors Study of the ThruVision T400 Passive Terahertz Camera. In: DSTO-TN-0821, Defence Science and Technology Organisation, Edinburgh, SA (2008)Google Scholar
  53. 53.
    Hardmeier, D., Hofer, F., Schwaninger, A.: The x-ray object recognition test (XRay ORT) – A reliable and valid instrument for measuring visual abilities needed in x-ray screening. In: IEEE ICCST Proceedings, vol. 38, pp. 303–308 (2005)Google Scholar
  54. 54.
    Ellis, H.D.: Introduction to aspects of face processing: Ten questions in need of answers. In: Ellis, H., Jeeves, M., Newcombe, F., Young, A. (eds.) Aspects of Face Processing, Nijhoff, Dordrecht, The Netherlands, pp. 3–13 (1986)Google Scholar
  55. 55.
    Gauthier, I., Behrmann, M., Tarr, M.J.: Can face recognition really be dissociated from object recognition? Journal of Cognitive Neuroscience 11, 349–370 (1999)CrossRefGoogle Scholar
  56. 56.
    Gauthier, I., Logothetis, N.K.: Is face recognition so unique after All? Journal of Cognitive Neuropsychology 17, 125–142 (2000)CrossRefGoogle Scholar
  57. 57.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: A literature survey. ACM Computing Surveys 35, 399–458 (2003)CrossRefGoogle Scholar
  58. 58.
    Goren, C., Sarty, M.E., Wu, P.Y.K.: Visual following and pattern discrimination of face-like stimuli by newborn infants. Pediatrics 56, 544–549 (1975)Google Scholar
  59. 59.
    Slater, A., Quinn, P.C.: Face recognition in the newborn infant. Infant and Child Development 10, 21–24 (2001)CrossRefGoogle Scholar
  60. 60.
    Pacini, R., Epstein, S.: The relation of rational and experiential information processing styles to personality, basic beliefs and the ratio-bias phenomenon. Journal of Personality and Social Psychology 76(6), 972–987 (1999)CrossRefGoogle Scholar
  61. 61.
    Furl, N., Phillips, P.J., O’Toole, A.J.: Face recognition algorithms and the other race effect: Computational mechanisms for a developmental contact hypothesis. Cognitive Science 26, 797–815 (2002)CrossRefGoogle Scholar
  62. 62.
    Shapiro, P.N., Penrod, S.: Meta-analysis of facial identification studies. Psychological Bulletin 100, 139–156 (1986)CrossRefGoogle Scholar
  63. 63.
    Malpass, R.S., Lavigneur, H., Weldon, D.E.: Verbal and visual training in face recognition. Perception and Psychophysics 14, 285–292 (1973)CrossRefGoogle Scholar
  64. 64.
    Wild, H.A., Barrett, S.E., Spence, M.J., O’Toole, A.J., Cheng, Y.D., Brooke, J.: Recognition and sex categorization of adults’ and children’s faces: Examining performance in the absence of sex- stereotyped cues. Journal of Experimental Child Psychology 77, 269–291 (2000)CrossRefGoogle Scholar
  65. 65.
    Wright, D.B., Sladden, B.: An own gender bias and the importance of hair in face recognition. Acta Psychologica 114, 101–114 (2003)CrossRefGoogle Scholar
  66. 66.
    de Frias, C.M., Nilsson, L.-G., Herlitz, A.: Sex differences in cognition are stable over a 10-year period in adulthood and old age. Aging, Neuropsychology, and Cognition 13, 574–587 (2006)CrossRefGoogle Scholar
  67. 67.
    Rehnman, J.: The role of gender in face recognition. Department of Psychology, Stockholm University (2007)Google Scholar
  68. 68.
    Rehnman, J., Herlitz, A.: Women remember more faces than men do. Acta Psychologica 124(3), 344–355 (2007)CrossRefGoogle Scholar
  69. 69.
    McKelvie, S.J., Standing, L., St. Jean, D., Law, J.: Gender differences in recognition memory for faces and cars: Evidence for the interest hypothesis. Bulletin of the Psychonomic Society 31, 447–448 (1993)Google Scholar
  70. 70.
    Cellerino, A., Borghetti, D., Sartucci, F.: Sex differences in face gender recognition in humans. Brain Res. Bull. 63(6), 443–449 (2004)CrossRefGoogle Scholar
  71. 71.
    Lewin, C., Herlitz, A.: Sex differences in face recognition- women’s faces make the difference. Brain & Cognition 50, 121–128 (2002)CrossRefGoogle Scholar
  72. 72.
    Shepherd, J.: Social factors in face recognition. In: Davies, G., Ellis, H., Shepherd, J. (eds.) Perceiving and Remembering Faces, pp. 55–79. Academic Press, San Diego (1981)Google Scholar
  73. 73.
    Slone, A.E., Brigham, J.C., Meissner, C.A.: Social and cognitive factors affecting the own-race bias in whites. Basic and applied social Psychology 22(2), 71–84 (2000)Google Scholar
  74. 74.
    Wright, D.B., Stroud, J.S.: Age differences in lineup identification accuracy: People are better with their own age. Law & Human Behavior 26, 641–654 (2002)CrossRefGoogle Scholar
  75. 75.
    Anastasi, J.S., Rhodes, M.G.: Evidence for an own-age bias in face recognition. North American Journal of Psychology 8, 237–253 (2006)Google Scholar
  76. 76.
    Dror, I.E., Fraser-Mackenzie, P.A.F.: Cognitive Biases in Human Perception, Judgment and Decision Making: Bridging Theory and the Real World. In: Rossmo, K. (ed.) Criminal Investigative Failures. Taylor & Francis, London (2008)Google Scholar
  77. 77.
    Bright, D.A., Goodman-Delahunty, J.: The influence of gruesome verbal evidence on mock juror verdicts. Psychiatry, Psychology and Law 11, 154–166 (2004)CrossRefGoogle Scholar
  78. 78.
    Dror, I.E., Peron, A.E., Hind, S., Charlton, D.: When Emotions Get the Better of Us: The Effect of Contextual Top-down Processing on Matching Fingerprints. Applied Cognitive Psychology 19, 799–809 (2005)CrossRefGoogle Scholar
  79. 79.
    Stacey, R.B.: Report on the erroneous fingerprint individualization in the Madrid train bombing case. Journal of Forensic Identification 54, 706–718 (2004)Google Scholar
  80. 80.
    Gilovich, T., Griffin, D., Kahneman, D.: Heuristics and biases: The psychology of intuitive Judgment. Cambridge University Press, New York (2002)Google Scholar
  81. 81.
    Ruthruff, E.: A test of the deadline model for speed–accuracy tradeoffs. Perception & Psychophysics 58(1), 56–64 (1996)CrossRefGoogle Scholar
  82. 82.
    Mansfield, A.J., Wayman, J.L.: Best practices in testing and reporting performance of biometric devices Version 2.01, National Physics laboratory NPL Report CMSC (February 14, 2002)Google Scholar
  83. 83.
    Phillips, P.J., Martin, A., Wilson, C.L., Przbocki, M.: An introduction to evaluating biometric systems. Computer 33(2), 56–63 (2008)CrossRefGoogle Scholar
  84. 84.
    Green, D.M., Swets, J.A.: Signal Detection Theory and Psychophysics. Wiley, New York (1966)Google Scholar
  85. 85.
    Stanislaw, H., Todorov, N.: Calculation of signal detection theory measures. Behavior Research Methods, Instruments & Computers 31(1), 137–149 (1999)CrossRefGoogle Scholar
  86. 86.
    Butavicius, M.A.: Evaluating and predicting the performance of an identification face recognition system in an operational setting. Australian Society for Operations Research Bulletin 25(2), 2–13 (2006)Google Scholar
  87. 87.
    Likert, R.: A Technique for the Measurement of Attitudes. Archives of Psychology 140, 1–55 (1932)Google Scholar
  88. 88.
    Harris, D.H.: How to Really Improve Airport Security. Ergonomics in Design, 1–6 (Winter 2002)Google Scholar
  89. 89.
    Leyk, D., Sievert, A., Heiss, A., Gorges, W., Ridder, D., Alexander, T., Wunderlich, M., Ruther, T.: Validation of a short-term memory test for the recognition of people and faces. Ergonomics 51, 1125–1136 (2008)CrossRefGoogle Scholar
  90. 90.
    Megreya, A.M., Burton, A.M.: Unfamiliar faces are not faces: Evidence from a matching task. Memory & Cognition 34, 865–876 (2006)CrossRefGoogle Scholar
  91. 91.
    Cohen, R.J., Swerdlik, M.E.: Psychological Testing and Assessment, 6th edn. McGraw-Hill, New York (2005)Google Scholar
  92. 92.
    Adini, Y., Wilkonsky, A., Haspel, R., Tsodyks, M., Sagi, D.: Perceptual learning in contrast discrimination: The effect of contrast uncertainty. Journal of Vision 4(12), 993–1005 (2004)CrossRefGoogle Scholar
  93. 93.
    Goldstein, E.B.: Sensation and Perception, 6th edn. Wadsworth, California (2002)Google Scholar
  94. 94.
    Pew, R.W., Mavor, A.S. (eds.): Human-System Integration in the System Development Process: A New Look. National Academies Press, Washington (2007)Google Scholar
  95. 95.
    Jordan, P.W.: Usability Evaluation in Industry. CRC Press, Boca Raton (1996)Google Scholar
  96. 96.
    Dumas, J.S., Loring, B.A.: Moderating Usability Tests: Principles and Practices for Interacting. Morgan Kaufmann, New York (2008)Google Scholar
  97. 97.
    Rubin, J., Chisnell, D.: Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, 2nd edn. Wiley, New York (2008)Google Scholar
  98. 98.
    Sauro, J., Lewis, J.R.: Estimating completion rates from small samples using binomial confidence intervals: Comparisons and recommendations. In: Proceedings of the Human Factors and Ergonomics Society, 49th Annual Meeting, pp. 2100–2104 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ian Graves
    • 1
  • Marcus Butavicius
    • 1
  • Veneta MacLeod
    • 1
  • Rebecca Heyer
    • 1
  • Kathryn Parsons
    • 1
  • Natalie Kuester
    • 1
  • Agata McCormac
    • 1
  • Philip Jacques
    • 1
  • Raymond Johnson
    • 1
  1. 1.Defence Science and Technology Organisation (DSTO)Edinburgh

Personalised recommendations