Advertisement

Prototype Discriminative Learning for Face Image Set Classification

  • Wen Wang
  • Ruiping WangEmail author
  • Shiguang Shan
  • Xilin Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10113)

Abstract

This paper presents a novel Prototype Discriminative Learning (PDL) method to solve the problem of face image set classification. We aim to simultaneously learn a set of prototypes for each image set and a linear discriminative transformation to make projections on the target subspace satisfy that each image set can be optimally classified to the same class with its nearest neighbor prototype. For an image set, its prototypes are actually “virtual” as they do not certainly appear in the set but are only assumed to belong to the corresponding affine hull, i.e., affine combinations of samples in the set. Thus, the proposed method not only inherits the merit of classical affine hull in revealing unseen appearance variations implicitly in an image set, but more importantly overcomes its flaw caused by too loose affine approximation via efficiently shrinking each affine hull with a set of discriminative prototypes. The proposed method is evaluated by face identification and verification tasks on three challenging and large-scale databases, YouTube Celebrities, COX and Point-and-Shoot Challenge, to demonstrate its superiority over the state-of-the-art.

Keywords

Linear Projection Deep Convolutional Neural Network Affine Hull False Accept Rate Affine Combination 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgement

This work is partially supported by 973 Program under contract No. 2015CB351802, Natural Science Foundation of China under contracts Nos. 61390511, 61379083, 61272321, and Youth Innovation Promotion Association CAS No. 2015085.

Supplementary material

416261_1_En_23_MOESM1_ESM.pdf (149 kb)
Supplementary material 1 (pdf 148 KB)

References

  1. 1.
    Cevikalp, H., Triggs, B.: Face recognition based on image sets. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010)Google Scholar
  2. 2.
    Hu, Y., Mian, A.S., Owens, R.: Sparse approximated nearest points for image set classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  3. 3.
    Yang, M., Zhu, P., Gool, L.V., Zhang, L.: Face recognition based on regularized nearest points between image sets. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (2013)Google Scholar
  4. 4.
    Wang, W., Wang, R., Shan, S., Chen, X.: Probabilistic nearest neighbor search for robust classification of face image sets. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (2015)Google Scholar
  5. 5.
    Zhu, P., Zhang, L., Zuo, W., Zhang, D.: From point to set: extend the learning of distance metrics. In: IEEE International Conference on Computer Vision (ICCV) (2013)Google Scholar
  6. 6.
    Leng, M., Moutafis, P., Kakadiaris, I.A.: Joint prototype and metric learning for set-to-set matching: application to biometrics. In: IEEE Conference on Biometrics Theory, Applications and Systems (BTAS) (2015)Google Scholar
  7. 7.
    Yamaguchi, O., Fukui, K., Maeda, K.: Face recognition using temporal image sequence. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (1998)Google Scholar
  8. 8.
    Kim, T.K., Kittler, J., Cipolla, R.: Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 29, 1005–1018 (2007)CrossRefGoogle Scholar
  9. 9.
    Hamm, J., Lee, D.D.: Grassmann discriminant analysis: a unifying view on subspace-based learning. In: International Conference on Machine Learning (ICML) (2008)Google Scholar
  10. 10.
    Harandi, M.T., Sanderson, C., Shirazi, S., Lovell, B.C.: Graph embedding discriminant analysis on grassmannian manifolds for improved image set matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  11. 11.
    Wang, R., Shan, S., Chen, X., Gao, W.: Manifold-manifold distance with application to face recognition based on image set. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  12. 12.
    Wang, R., Chen, X.: Manifold discriminant analysis. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  13. 13.
    Cui, Z., Shan, S., Zhang, H., Lao, S., Chen, X.: Image sets alignment for video-based face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  14. 14.
    Chen, S., Sanderson, C., Harandi, M.T., Lovell, B.C.: Improved image set classification via joint sparse approximated nearest subspaces. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  15. 15.
    Li, Z., Liu, J., Tang, J., Lu, H.: Robust structured subspace learning for data representation. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 37, 2085–2098 (2015)CrossRefGoogle Scholar
  16. 16.
    Shakhnarovich, G., Fisher, J.W., Darrell, T.: Face recognition from long-term observations. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 851–865. Springer, Heidelberg (2002). doi: 10.1007/3-540-47977-5_56 CrossRefGoogle Scholar
  17. 17.
    Arandjelović, O., Shakhnarovich, G., Fisher, J., Cipolla, R., Darrell, T.: Face recognition with image sets using manifold density divergence. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2005)Google Scholar
  18. 18.
    Wang, R., Guo, H., Davis, L.S., Dai, Q.: Covariance discriminative learning: a natural and efficient approach to image set classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  19. 19.
    Harandi, M.T., Salzmann, M., Hartley, R.: From manifold to manifold: geometry-aware dimensionality reduction for SPD matrices. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 17–32. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10605-2_2 Google Scholar
  20. 20.
    Huang, Z., Wang, R., Shan, S., Li, X., Chen, X.: Log-Euclidean metric learning on symmetric positive definite manifold with application to image set classification. In: International Conference on Machine Learning (ICML) (2015)Google Scholar
  21. 21.
    Wang, W., Wang, R., Huang, Z., Shan, S., Chen, X.: Discriminant analysis on Riemannian manifold of Gaussian distributions for face recognition with image sets. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  22. 22.
    Chen, Y.-C., Patel, V.M., Phillips, P.J., Chellappa, R.: Dictionary-based face recognition from video. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 766–779. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33783-3_55 CrossRefGoogle Scholar
  23. 23.
    Chen, Y.C., Patel, V.M., Shekhar, S., Chellappa, R., Phillips, P.J.: Video-based face recognition via joint sparse representation. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (2013)Google Scholar
  24. 24.
    Cui, Z., Chang, H., Shan, S., Ma, B., Chen, X.: Joint sparse representation for video-based face recognition. Neurocomputing 135, 306–312 (2014)CrossRefGoogle Scholar
  25. 25.
    Lu, J., Wang, G., Deng, W., Moulin, P.: Simultaneous feature and dictionary learning for image set based face recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 265–280. Springer, Cham (2014). doi: 10.1007/978-3-319-10590-1_18 Google Scholar
  26. 26.
    Hayat, M., Bennamoun, M., An, S.: Learning non-linear reconstruction models for image set classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  27. 27.
    Chen, S., Wiliem, A., Sanderson, C., Lovell, B.C.: Matching image sets via adaptive multi convex hull. arXiv preprint arXiv:1403.0320 (2014)
  28. 28.
    Chen, L.: Dual linear regression based classification for face cluster recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  29. 29.
    Paredes, R., Vidal, E.: Learning prototypes and distances: a prototype reduction technique based on nearest neighbor error minimization. Pattern Recogn. 39, 180–188 (2006)CrossRefzbMATHGoogle Scholar
  30. 30.
    Paredes, R., Vidal, E.: Learning weighted metrics to minimize nearest-neighbor classification error. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 28, 1100–1110 (2006)CrossRefGoogle Scholar
  31. 31.
    Villegas, M., Paredes, R.: Simultaneous learning of a discriminative projection and prototypes for nearest-neighbor classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  32. 32.
    Le, Q.V., Ngiam, J., Coates, A., Lahiri, A., Prochnow, B., Ng, A.Y.: On optimization methods for deep learning. In: International Conference on Machine Learning (ICML) (2011)Google Scholar
  33. 33.
    Garcia, S., Derrac, J., Cano, J.R., Herrera, F.: Prototype selection for nearest neighbor classification: taxonomy and empirical study. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 34, 417–435 (2012)CrossRefGoogle Scholar
  34. 34.
    Ma, M., Shao, M., Zhao, X., Fu, Y.: Prototype based feature learning for face image set classification. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (2013)Google Scholar
  35. 35.
    Kim, M., Kumar, S., Pavlovic, V., Rowley, H.: Face tracking and recognition with visual constraints in real-world videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  36. 36.
    Huang, Z., Wang, R., Shan, S., Chen, X.: Learning euclidean-to-riemannian metric for point-to-set classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  37. 37.
    Ross, B., Phillips, J., Bolme, D., Draper, B., Givens, G., Lui, Y.M., Teli, M.N., Zhang, H., Scruggs, W.T., Bowyer, K., Flynn, P., Cheng, S.: The challenge of face recognition from digital point-and-shoot cameras. In: IEEE Conference on Biometrics Theory, Applications and Systems (BTAS) (2013)Google Scholar
  38. 38.
    Beveridge, J.R., Zhang, H., Draper, B.A., Flynn, P.J., Feng, Z., Huber, P., Kittler, J., Huang, Z., Li, S., Li, Y., Kan, M., Wang, R., Shan, S., Chen, X.: Report on the FG 2015 video person recognition evaluation. In: IEEE Conference and Workshops on Automatic Face and Gesture Recognition (FG) (2015)Google Scholar
  39. 39.
    Zhang, X., Zhang, L., Wang, X.J., Shum, H.Y.: Finding celebrities in billions of web images. IEEE Trans. Multimedia 14, 995–1007 (2012)CrossRefGoogle Scholar
  40. 40.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: ACM International Conference on Multimedia (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Wen Wang
    • 1
    • 2
  • Ruiping Wang
    • 1
    • 2
    • 3
    Email author
  • Shiguang Shan
    • 1
    • 2
    • 3
  • Xilin Chen
    • 1
    • 2
    • 3
  1. 1.Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS)Institute of Computing Technology, CASBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina
  3. 3.Cooperative Medianet Innovation CenterBeijingChina

Personalised recommendations