Fusing Individual Algorithms and Humans Improves Face Recognition Accuracy

  • Alice J. O’Toole
  • Fang Jiang
  • Hervé Abdi
  • P. Jonathon Phillips
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4292)


Recent work indicates that state-of-the-art face recognition algorithms can surpass humans matching identity in pairs of face images taken under different illumination conditions. It has been demonstrated further that fusing algorithm- and human-derived face similarity estimates cuts error rates substantially over the performance of the best algorithms. Here we employed a pattern-based classification procedure to fuse individual human subjects and algorithms with the goal of determining whether strategy differences among humans are strong enough to suggest particular man-machine combinations. The results showed that error rates for the pairwise man-machine fusions were reduced an average of 47 percent when compared to the performance of the algorithms individually. The performance of the best pairwise combinations of individual humans and algorithms was only slightly less accurate than the combination of individual humans with all seven algorithms. The balance of man and machine contributions to the pairwise fusions varied widely, indicating that a one-size-fits-all weighting of human and machine face recognition estimates is not appropriate.


Partial Little Square Face Recognition Face Image Individual Human False Acceptance Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Phillips, P., Flynn, P., Scruggs, T., Bowyer, K., Chang, K., Hoffman, J., Marques, J., Min, J., Worek, W.: Overview of the face recognition grand challeng. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, pp. 947–954 (2005)Google Scholar
  2. 2.
    Phillips, P., Flynn, P., Scruggs, T., Bowyer, K., Worek, W.: Preliminary face recognition grand challenge results. In: Proceedings of the Seventh International Conference on Automatic Face Recognition, pp. 15–24 (2006)Google Scholar
  3. 3.
    O’Toole, A., Phillips, P., Jiang, F., Ayyad, J., Penard, N., Abdi, H.: Face recognition algorithms surpass humans (submitted, 2006)Google Scholar
  4. 4.
    Liu, C.: Capitalize on dimensionality increasing techniques for improving face recognition grand challenge performance. IEEE: Transactions on Pattern Analysis and Machine Intelligence, 725–737 (2006)Google Scholar
  5. 5.
    Xie, C., Savvides, M., Kumar, V.: Kernel correlation filter based redundant class-dependence feature analysis (kcfa) on frgc2.0 data. In: IEEE International Workshop on Analysis and Modeling Faces and Gestures, vol. 1, pp. 32–43 (2005)Google Scholar
  6. 6.
    Husken, M., Brauckmann, B., Gehlen, S., von der Malsburg, C.: Strategies and benefits of fusion of 2d and 3d face recognition. In: Proceedings of the IEEE Workshop on Face Recognition Grand Challenge Experiments. Computer Society Digital Library, 1st edn., vol. 3, p. 174. IEEE Press, Los Alamitos (2005)Google Scholar
  7. 7.
    O’Toole, A., Abdi, H., Futing, T., Phillips, P.: Fusing face recognition algorithms and humans (submitted, 2006)Google Scholar
  8. 8.
    Abdi, H.: Partial least squares regression. In: Beck, M., Bryman, A., Futing, T. (eds.) Encyclopedia for Research Methods in the Social Sciences, pp. 792–795. Sage, Thousand Oaks (2003)Google Scholar
  9. 9.
    McIntosh, A., Lobaugh, N.: Partial least squares analysis of neuroimaging data: applications and advances. Neuroimage 23, 250–263 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Alice J. O’Toole
    • 1
  • Fang Jiang
    • 1
  • Hervé Abdi
    • 1
  • P. Jonathon Phillips
    • 2
  1. 1.The University of Texas at Dallas 
  2. 2.National Institute of Standards and Technology 

Personalised recommendations