Predicting Trait Impressions of Faces Using Classifier Ensembles

  • Sheryl Brahnam
  • Loris Nanni
Part of the Intelligent Systems Reference Library book series (ISRL, volume 1)


In the experiments presented in this chapter, single classifier systems and ensembles are trained to detect the social meanings people perceive in facial morphology. Exploring machine models of people’s impressions of faces has value in the fields of social psychology and human-computer interaction. Our first concern in designing this study was developing a sound ground truth for this problem domain. We accomplished this by collecting a large number of faces that exhibited strong human consensus in a comprehensive set of trait categories. Several single classifier systems and ensemble systems composed of Levenberg-Marquardt neural networks using different methods of collaboration were then trained to match the human perception of the faces in the six trait dimensions of intelligence, maturity, warmth, sociality, dominance, and trustworthiness. Our results show that machine learning methods employing ensembles are as capable as most individual human beings are in their ability to predict the social impressions certain faces make on the average human observer. Single classifier systems did not match human performance as well as the ensembles did. Included in this chapter is a tutorial, suitable for the novice, on the single classifier systems and collaborative methods used in the experiments reported in the study.


Support Vector Machine Face Recognition Face Image Principle Component Analysis Near Neighbor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abate, A.F., Nappi, M., Riccioa, D., Sabatinoa, G.: 2D and 3D face recognition: A survey. Pattern Recognit. Lett. 14, 1885–1906 (2007)CrossRefGoogle Scholar
  2. 2.
    Albright, L., Malloy, T.E., Dong, Q., Kenny, D.A., Fang, X.: Cross-cultural consensus in personality judgments. J. Personal and Soc. Psychol. 3, 558–569 (1997)CrossRefGoogle Scholar
  3. 3.
    Alcock, D., Solano, J., Kayson, W.A.: How individuals’ responses and attractiveness influence aggression. Psychol. Rep. 3(2), 1435–1438 (1998)Google Scholar
  4. 4.
    Alpaydin, E.: Introduction to machine learning. MIT Press, Cambridge (2004)Google Scholar
  5. 5.
    Alt\(\imath \)nçay, H., Demirekler, M.: An information theoretic framework for weight estimation in the combination of probabilistic classifiers for speaker identification. Speech Commun. 4, 255–272 (2000)Google Scholar
  6. 6.
    Bellman, R.: Adaptive control process: A guided tour. Princeton University Press, Princeton (1961)zbMATHGoogle Scholar
  7. 7.
    Berry, D.S., Brownlow, S.: Were the physiognomists right? Personal and Soc. Psychol. Bull. 2, 266–279 (1989)CrossRefGoogle Scholar
  8. 8.
    Berry, D.S., McArthur, L.Z.: Perceiving character in faces: The impact of age-related craniofacial changes on social perception. Psychol. Bull. 1, 3–18 (1986)CrossRefGoogle Scholar
  9. 9.
    Brahnam, S.: Modeling physical personalities for virtual agents by modeling trait impressions of the face: A neural network analysis. The Graduate Center of the City of New York, Department of Computer Science, New York (2002)Google Scholar
  10. 10.
    Breiman, L.: Bagging predictors. Mach. Learn. 2, 123–140 (1996)Google Scholar
  11. 11.
    Breiman, L.: Random forest. Mach. Learn. 1, 5–32 (2001)CrossRefGoogle Scholar
  12. 12.
    Bruce, V.: Recognising faces. Lawrence Erlbaum Associates Publishers, London (1988)Google Scholar
  13. 13.
    Brunelli, R., Poggio, T.: Hyperbf networks for gender classification. In: DARPA Image Understanding Workshop, pp. 311–314 (1992)Google Scholar
  14. 14.
    Brunelli, R., Poggio, T.: Face recognition: Features versus templates. IEEE Trans. Pattern Anal. and Mach. Intell. 10, 1042–1052 (1993)CrossRefGoogle Scholar
  15. 15.
    Brunswik, E.: Perception and the representative design of psychological experiments. University of California Press, Berkeley (1947)Google Scholar
  16. 16.
    Bull, R., Rumsey, N.: The social psychology of facial appearance. Springer, Heidelberg (1988)Google Scholar
  17. 17.
    Burton, A.M., Bruce, V., Dench, N.: What’s the difference between men and women? Evidence from facial measurement. Percept. 2, 153–176 (1993)CrossRefGoogle Scholar
  18. 18.
    Chellappa, R., Wilson, C.L., Sirohey, S.: Human and machine recognition of faces: A survey. Proceedings of the IEEE, pp. 705–740 (1995)Google Scholar
  19. 19.
    Cottrell, G.W., Fleming, M.K.: Face recognition using unsupervised feature extraction. In: International Conference on Neural Networks, pp. 322–325 (1990)Google Scholar
  20. 20.
    Cottrell, G.W., Metcalfe, J.: EMPATH: Face, emotion, and gender recognition using holons. In: Touretzky, D. (ed.) Adv. Neural Inf. Process Syst., pp. 564–571. Morgan & Kaufman, San Mateo (1991)Google Scholar
  21. 21.
    Cover, T.M., Hart, P.E.: Nearest neighbor pattern classificiation. IEEE Trans. Inf. Theory 1, 21–27 (1967)CrossRefGoogle Scholar
  22. 22.
    Cristianini, N., Shawe-Taylor, J.: An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge (2000)Google Scholar
  23. 23.
    Czyz, J., Kittler, J., Vandendorpe, L.: Multiple classifier combination for face-based identity verification. Pattern Recognit. 7, 1459–1469 (2004)CrossRefGoogle Scholar
  24. 24.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, New York (2000)Google Scholar
  25. 25.
    Eagly, A.H., Ashmore, R.D., Makhijan, M.G., Longo, L.C.: What is beautiful is good, but ...: A meta-analytic review of research on the physical attractiveness stereotype. Psychol. Bull. 1, 109–128 (1991)CrossRefGoogle Scholar
  26. 26.
    Edelman, B.E., Valentin, D., Abdi, H.: Sex classification of face areas: How well can a linear neural network predict human performance. J. Biol. Syst. 3, 241–264 (1998)CrossRefGoogle Scholar
  27. 27.
    Efron, B.: The jackknife, the bootstrap and other resampling plans. SIAM, Philadelphia (1982)Google Scholar
  28. 28.
    Enlow, D.H., Hans, M.G.: Essentials of facial growth. W. B. Saunders Company, Philadelphia (1996)Google Scholar
  29. 29.
    Feingold, A.: Good-looking people are not what we think. Psychol. Bull. 2, 304–341 (1992)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Freierman, S.: Constructing a real-life mr. potato head. Faces: The ultimate composite picture. The New York Times D:6 (2000)Google Scholar
  31. 31.
    Golumb, B.A., Lawrence, D.T., Sejnowshi, T.J.: Sexnet: A neural network identifies sex from human faces. Adv. Neural Inf. Process Syst., 572–577 (1991)Google Scholar
  32. 32.
    Guo, G., Li, S.Z., Chan, K.L.: Support vector machines for face recognition. Image and Vis. Comput., 631–638 (2001)Google Scholar
  33. 33.
    Heisele, B., Ho, P., Poggio, T.: Face recognition with support vector machines: Global versus component-based approach. In: The Eighth IEEE International Conference on Computer Vision, Vancouver, BC, pp. 688–694 (2001)Google Scholar
  34. 34.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. and Mach. Intell. 8, 832–844 (1998)Google Scholar
  35. 35.
    Jain, A., Huang, J.: Integrating independent components and linear discriminant analysis for gender classification. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 159–163 (2004)Google Scholar
  36. 36.
    Jain, A.K., Dubes, R.C., Chen, C.C.: Bootstrap techniques for error estimation. IEEE Trans. Pattern Anal. and Mach. Intell. 5, 628–633 (1987)CrossRefGoogle Scholar
  37. 37.
    Jain, A.K., Duin, R.P.W., Mao, J.: Statistical pattern recognition: a review. IEEE Trans. Pattern Anal. and Mach. Intell. 1, 4–37 (2000)CrossRefGoogle Scholar
  38. 38.
    Kanghae, S., Sornil, O.: Face recognition using facial attractiveness. In: The 2nd International Conference on Advances in Information Technology, Bangkok, Thailand (2007)Google Scholar
  39. 39.
    Keating, C.F., Mazur, A., Segall, M.H.: A cross-cultural exploration of physiognomic traits of dominance and happiness. Ethol. and Sociobiol., 41–48 (1981)Google Scholar
  40. 40.
    Kittler, J.: On combining classifiers. IEEE Trans. Pattern Anal. and Mach. Intell. 3, 226–239 (1998)CrossRefGoogle Scholar
  41. 41.
    Kohonen, T.: Associative memory: A system theoretic approach. Springler, Berlin (1977)Google Scholar
  42. 42.
    Kosugi, M.: Human-face search and location in a scene by multi-pyramid architecture for personal identification. Syst. and Comput. Jpn. 6, 27–38 (1995)CrossRefGoogle Scholar
  43. 43.
    Kuncheva, L.I.: Clustering-and-selection model for classier combination. In: Knowledge-Based Intelligent Engineering Systems and Allied Technologies, Brighton, pp. 185–188 (2000)Google Scholar
  44. 44.
    Kuncheva, L.I.: Combining pattern classifiers: Methods and algorithms. Wiley, New York (2004)zbMATHCrossRefGoogle Scholar
  45. 45.
    Kuncheva, L.I.: Diversity in multiple classifier systems. Inf. Fusion 1, 3–4 (2005)CrossRefGoogle Scholar
  46. 46.
    Kuncheva, L.I., Whitaker, C.J.: Measures of Diversity in Classifier Ensembles and their Relationship with the ensemble accuracy. Mach. Learn. 2, 181–207 (2003)CrossRefGoogle Scholar
  47. 47.
    Langlois, J.H., Kalakanis, L., Rubenstein, A.J., Larson, A., Hallam, M., Smoot, M.: Maxims or myths of beauty? A meta-analytic and theoretical review. Psychol. Bull. 3, 390–423 (2000)CrossRefGoogle Scholar
  48. 48.
    Lanitis, A., Taylor, C.J., Cootes, T.F.: Automatic interpretation and coding of face images using flexible models. IEEE Trans. Pattern Anal. and Mach. Intell. 7, 743–756 (1997)CrossRefGoogle Scholar
  49. 49.
    Lanitis, A., Taylor, C.J., Cootes, T.F.: Toward Automatic Simulation of Aging Effects on Face Images. IEEE Trans. Pattern Anal. and Mach. Intell. 4, 442–455 (2002)CrossRefGoogle Scholar
  50. 50.
    Levenberg, K.: A Method for the solution of certain nonlinear problems in least squares. Q Appl. Math. 2, 164–168 (1944)zbMATHMathSciNetGoogle Scholar
  51. 51.
    Ling, C.X., Huang, J., Zhang, H.: Auc: A better measure than accuracy in comparing learning algorithms. In: Canadian Conference on Artificial Intelligence 2003, Halifax, Canada, pp. 329–341 (2003)Google Scholar
  52. 52.
    Lu, X., Jain, A.K.: Ethnicity identification from face images. In: SPIE: Biometric Technology for Human Identification Conference: Biometric Technology for Human Identification, Orlando, FL, pp. 114–123 (2004)Google Scholar
  53. 53.
    Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math., 431–441 (1963)Google Scholar
  54. 54.
    Martínez-Muñoz, G., Suárez, A.: Switching class labels to generate classification ensembles. Pattern Recognit. 10, 1483–1494 (2005)Google Scholar
  55. 55.
    Martinez, A.M., Benavente, R.: The ar face database. CVC Technical Report #24 (1998),
  56. 56.
    McArthur, L.Z., Baron, R.M.: Toward an ecological theory of social perception. Psychol. Rev. 3, 215–238 (1983)CrossRefGoogle Scholar
  57. 57.
    Melville, P., Mooney, R.J.: Constructing diverse classifier ensembles using artificial training examples. In: International Joint Conferences on Artificial Intelligence, pp. 505–510 (2003)Google Scholar
  58. 58.
    Metz, C.E.: Basic principles of ROC analysis. Semin. Nucl. Med. 4, 283–298 (1978)CrossRefGoogle Scholar
  59. 59.
    Mitsumoto, S.T., Kawai, H.: Male/female identification from 8 x 6 very low resolution face images by a neural network. Pattern Recognit. 2, 331–335 (1996)Google Scholar
  60. 60.
    Moghaddam, B., Yang, M.-H.: Gender classification with support vector machines. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition (FG), pp. 306–311 (2000)Google Scholar
  61. 61.
    Moghaddam, B., Yang, M.-H.: Learning gender with support faces. IEEE Trans. Pattern Anal. and Mach. Intell. 5, 306–311 (2002)Google Scholar
  62. 62.
    Mulford, M., Orbell, J., Shatto, C., Stockard, J.: Physical attractiveness, opportunity, and success in everyday exchange. American J. Sociol. 6, 1565–1592 (1998)CrossRefGoogle Scholar
  63. 63.
    Nocedal, J., Wright, S.J.: Numerical optimization. Springer, New York (1999)zbMATHCrossRefGoogle Scholar
  64. 64.
    O’Toole, A.J., Abdi, H., Deffenbacher, K.A., Bartlett, J.C.: Classifying faces by race and sex using an autoassociative memory trained for recognition. In: 13th Annual Conference on Cognitive Science, Hillsdale, NJ, pp. 847–851 (1991)Google Scholar
  65. 65.
    O’Toole, A.J., Deffenbacher, K.A.: The perception of face gender: The role of stimulus structure in recognition and classification. Mem. and Cogn., 146–160 (1997)Google Scholar
  66. 66.
    Oja, E.: Subspace Methods of Pattern Recognition. Research Studies Press Ltd., Letchworth (1983)Google Scholar
  67. 67.
    Oja, E.: Principal components, minor components and linear neural networks. Neural Netw., 927–935 (1992)Google Scholar
  68. 68.
    Opitz, D., Maclin, R.: Popular ensemble methods: an empirical study. J. Artif. Intell. Res., 169–198 (1999)Google Scholar
  69. 69.
    Padgett, C., Cottrell, G.W.: A simple neural network models categorical perception of facial expressions. In: Proceedings of the 20th Annual Cognitive Science Conference, Madison, WI, pp. 806–807 (1998)Google Scholar
  70. 70.
    Phillips, P.J.: Support vector machines applied to face recognition. Adv. Neural Inf. Process Syst., 803–809 (1998)Google Scholar
  71. 71.
    Rosenberg, S.: New approaches to the analysis of personal constructs in person perception. In: Land, A.L., Cole, J.K. (eds.) Nebraska symposium on motivation, pp. 179–242. University of Nebraska Press, Lincoln (1977)Google Scholar
  72. 72.
    Rowley, H.A., Shumeet, B., Kanade, T.: Neural network-based face detection. IEEE Trans. Pattern Anal. and Mach. Intell. 1, 23–38 (1998)CrossRefGoogle Scholar
  73. 73.
    Russell, S., Norvig, P.: Artificial intelligence: A modern approach. Prentice Hall, Upper Saddle River (2002)Google Scholar
  74. 74.
    Sirovich, L., Kirby, M.: Low dimensional procedure for the characterization of human faces. J. Opt. Soc. Am. 3, 519–524 (1987)CrossRefGoogle Scholar
  75. 75.
    Swets, D.L., Weng, J.: Using discriminant eigenfeatures for image retrieval. IEEE Trans. Pattern Anal. and Mach. Intell. 8, 831–837 (1996)CrossRefGoogle Scholar
  76. 76.
    The MathWorks, Using MATLAB: The language of technical computing. The Mathworks, Inc., Natick, MA (2000)Google Scholar
  77. 77.
    Todd, J.T., Mark, L.S.: The perception of human growth. Sci. Am. 2, 132–144 (1980)CrossRefGoogle Scholar
  78. 78.
    Turk, M.A., Pentland, A.P.: Eigenfaces for recognition. J. Cogn. Neurosci. 1, 71–86 (1991)CrossRefGoogle Scholar
  79. 79.
    Turk, M.A., Pentland, A.P.: Face recognition using eigenfaces. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Silver Spring, MD, pp. 586–591 (1991)Google Scholar
  80. 80.
    Valentin, D., Abdi, H., Edelman, B.E., O’Toole, A.J.: Principal component and neural network analyses of face images: What can be generalized in gender classification? J. Math. Psychol. 4, 398–413 (1997)CrossRefGoogle Scholar
  81. 81.
    Valentin, D., Abdi, H., O’Toole, A.J.: Categorization and identification of human face images by neural networks: A review of the linear autoassociative and principal component approaches. J. Biol. Syst. 3, 413–429 (1994)CrossRefGoogle Scholar
  82. 82.
    Valentin, D., Abdi, H., O’Toole, A.J., Cottrell, G.W.: Connectionist models of face processing: A survey. Pattern Recognit. 9, 1209–1230 (1994)CrossRefGoogle Scholar
  83. 83.
    van der Heijden, F., Duin, R.P.W., de Ridder, D., Tax, D.M.J.: Classification, parameter estimation, and state estimation: An engineering approach using MATLAB. John Wiley & Sons, Ltd., Chichester (2004)zbMATHCrossRefGoogle Scholar
  84. 84.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)zbMATHGoogle Scholar
  85. 85.
    Wechsler, H., Gutta, S., Philips, P.J.: Gender and ethnic classification of Face Images. In: 3rd Int. Conf. on Automatic Face and Gesture Recognition, Nara, Japan, pp. 194–199 (1998)Google Scholar
  86. 86.
    Whitaker, C.J., Kuncheva, L.I.: Examining the relationship between majority vote accuracy and diversity in bagging and boosting (2003),
  87. 87.
    Zebrowitz, L.A.: Reading faces: Window to the soul? Westview Press, Boulder (1998)Google Scholar
  88. 88.
    Zebrowitz, L.A., Montepare, J.M.: Impressions of babyfaced individuals across the life span. Dev. Psychol. 6, 1143–1152 (1992)CrossRefGoogle Scholar
  89. 89.
    Zebrowitz, L.A., Montepare, J.M.: Social Psychological Face Perception: Why Appearance Matters. Soc. and Personality Psychol. Compass 3, 1497–1517 (2008)CrossRefGoogle Scholar
  90. 90.
    Zebrowitz, L.A., Montepare, J.M., Lee, H.K.: They don’t all look alike: Individuated impressions of other racial groups. J. Personal and Soc. Psychol. 1, 85–101 (1993)CrossRefGoogle Scholar
  91. 91.
    Zenobi, G., Cunningham, P.: Using diversity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error. In: 12th Conference on Machine Learning, pp. 576–587 (2001)Google Scholar
  92. 92.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: A literature survey. ACM Comp. Surv. 4, 399–458 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Sheryl Brahnam
    • 1
  • Loris Nanni
    • 2
  1. 1.Computer Information SystemsMissouri State UniversitySpringfieldUSA
  2. 2.DEIS, IEIIT—CNR, Università di BolognaBolognaItaly

Personalised recommendations