Advertisement

FaceTracer: A Search Engine for Large Collections of Images with Faces

  • Neeraj Kumar
  • Peter Belhumeur
  • Shree Nayar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5305)

Abstract

We have created the first image search engine based entirely on faces. Using simple text queries such as “smiling men with blond hair and mustaches,” users can search through over 3.1 million faces which have been automatically labeled on the basis of several facial attributes. Faces in our database have been extracted and aligned from images downloaded from the internet using a commercial face detector, and the number of images and attributes continues to grow daily. Our classification approach uses a novel combination of Support Vector Machines and Adaboost which exploits the strong structure of faces to select and train on the optimal set of features for each attribute. We show state-of-the-art classification results compared to previous works, and demonstrate the power of our architecture through a functional, large-scale face search engine. Our framework is fully automatic, easy to scale, and computes all labels off-line, leading to fast on-line search performance. In addition, we describe how our system can be used for a number of applications, including law enforcement, social networks, and personal photo management. Our search engine will soon be made publicly available.

Download to read the full conference paper text

References

  1. 1.
    Omron: OKAO vision (2008), http://www.omron.com/rd/vision/01.html
  2. 2.
    Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3) (1995)Google Scholar
  3. 3.
    Golomb, B.A., Lawrence, D.T., Sejnowski, T.J.: Sexnet: A neural network identifies sex from human faces. NIPS, 572–577 (1990)Google Scholar
  4. 4.
    Belhumeur, P.N., Hespanha, J., Kriegman, D.J.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. In: Buxton, B.F., Cipolla, R. (eds.) ECCV 1996. LNCS, vol. 1065, pp. 45–58. Springer, Heidelberg (1996)Google Scholar
  5. 5.
    Moghaddam, B., Yang, M.-H.: Learning gender with support faces. TPAMI 24(5), 707–711 (2002)Google Scholar
  6. 6.
    Phillips, P., Moon, H., Rizvi, S., Rauss, P.: The FERET evaluation methodology for face-recognition algorithms. TPAMI 22(10), 1090–1104 (2000)Google Scholar
  7. 7.
    Shakhnarovich, G., Viola, P.A., Moghaddam, B.: A unified learning framework for real time face detection and classification. ICAFGR, 14–21 (2002)Google Scholar
  8. 8.
    Baluja, S., Rowley, H.: Boosting sex identification performance. IJCV (2007)Google Scholar
  9. 9.
    Freund, Y., Shapire, R.E.: Experiments with a new boosting algorithm. In: ICML (1996)Google Scholar
  10. 10.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: CVPR (2001)Google Scholar
  11. 11.
    Bartlett, M.S., Littlewort, G., Fasel, I., Movellan, J.R.: Real time face detection and facial expression recognition: Development and applications to human computer interaction. CVPRW 05 (2003)Google Scholar
  12. 12.
    Wang, Y., Ai, H., Wu, B., Huang, C.: Real time facial expression recognition with adaboost. In: ICPR, pp. 926–929 (2004)Google Scholar
  13. 13.
    Datta, R., Li, J., Wang, J.Z.: Content-based image retrieval: Approaches and trends of the new age. Multimedia Information Retrieval, 253–262 (2005)Google Scholar
  14. 14.
    Pentland, A., Picard, R., Sclaroff, S.: Photobook: Content-based manipulation of image databases. IJCV, 233–254 (1996)Google Scholar
  15. 15.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49 (2007)Google Scholar
  16. 16.
    Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: Illumination cone models for face recognition under variable lighting and pose. TPAMI 23(6), 643–660 (2001)Google Scholar
  17. 17.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. In: ICAFGR, pp. 46–51 (2002)Google Scholar
  18. 18.
    Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W.: Overview of the face recognition grand challenge. CVPR, 947–954 (2005)Google Scholar
  19. 19.
    Pentland, A., Moghaddam, B., Starner, T.: View-based and modular eigenspaces for face recognition. CVPR, 84–91 (1994)Google Scholar
  20. 20.
    Huang, J., Shao, X., Wechsler, H.: Face pose discrimination using support vector machines (SVM). In: ICPR, pp. 154–156 (1998)Google Scholar
  21. 21.
    Osuna, E., Freund, R., Girosi, F.: Training support vector machines: An application to face detection. CVPR (1997)Google Scholar
  22. 22.
    Schapire, R., Freund, Y., Bartlett, P., Lee, W.S.: Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics 26(5), 1651–1686 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Drucker, H., Cortes, C.: Boosting decision trees. NIPS, 479–485 (1995)Google Scholar
  24. 24.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/cjlin/libsvm/

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Neeraj Kumar
    • 1
  • Peter Belhumeur
    • 1
  • Shree Nayar
    • 1
  1. 1.Columbia University 

Personalised recommendations

Cite paper