Pose-Invariant Face Recognition in Surveillance Scenarios Using Extreme Learning Machine Based Domain Adaptation

  • Avishek BhattacharjeeEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1022)


Face Recognition (FR) under adversarial conditions has been a big challenge for researchers in the computer vision community. FR performance deteriorates in surveillance condition due to poor illumination, blur, noise, and pose variation in test samples (probe), when compared to training samples (gallery). Even recent deep learning methods fail to perform well in such conditions. This paper proposes a novel framework called PIFR-EDA (Pose-Invariant Face Recognition using Extreme learning machine based Domain Adaptation) that performs pose-invariant face recognition (PIFR) in cross-domain settings. It consists of two stages where the first stage performs face frontalization using a single unmodified 3D facial model and the second stage performs the task of robust domain adaptation by simultaneously learning a category transformation matrix and an \(\ell _{1,1}\)-regularized sparse extreme learning machine classifier. The proposed method outperforms state-of-the-art shallow and deep methods (in terms of rank-1 recognition rates) when experimented on three real-world face datasets captured using surveillance cameras.


Face recognition \(\ell _{1 , 1}\)-regularized sparse extreme learning machine Face frontalization Domain adaptation 



We would like to thank the faculty and members of the Visualization & Perception Lab, Dept. of CS&E, IIT Madras for their insight, expertise, and support that greatly assisted the research.


  1. 1.
    Aljundi, R., Emonet, R., Muselet, D., Sebban, M.: Landmarks-based kernelized subspace alignment for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 56–63 (2015)Google Scholar
  2. 2.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Incremental face alignment in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1859–1866 (2014)Google Scholar
  3. 3.
    Banerjee, S., Das, S.: Soft-margin learning for multiple feature-kernel combinations with domain adaptation, for recognition in surveillance face dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 169–174 (2016)Google Scholar
  4. 4.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Beijbom, O.: Domain adaptations for computer vision applications. Technical report, University of California, San Diego (2012)Google Scholar
  6. 6.
    Blanz, V., Scherbaum, K., Vetter, T., Seidel, H.P.: Exchanging faces in images. In: Computer Graphics Forum, vol. 23, pp. 669–676 (2004)Google Scholar
  7. 7.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194 (1999)Google Scholar
  8. 8.
    Chen, J.C., Zheng, J., Patel, V.M., Chellappa, R.: Fisher vector encoded deep convolutional features for unconstrained face verification. In: IEEE International Conference on Image Processing, pp. 2981–2985 (2016)Google Scholar
  9. 9.
    Cortes, C., Vapnik, V.: Support vector machine. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  10. 10.
    Dai, W., Yang, Q., Xue, G.R., Yu, Y.: Boosting for transfer learning. In: Proceedings of the International Conference on Machine Learning, pp. 193–200 (2007)Google Scholar
  11. 11.
    Gopalan, R., Li, R., Chellappa, R.: Domain adaptation for object recognition: an unsupervised approach. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 999–1006 (2011)Google Scholar
  12. 12.
    Grgic, M., Delac, K., Grgic, S.: Scface-surveillance cameras face database. Multimed. Tools Appl. 51(3), 863–879 (2011)CrossRefGoogle Scholar
  13. 13.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2003)Google Scholar
  14. 14.
    Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4295–4304 (2015)Google Scholar
  15. 15.
    Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(2), 513–529 (2012)Google Scholar
  16. 16.
    Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)CrossRefGoogle Scholar
  17. 17.
    Kemelmacher-Shlizerman, I., Basri, R.: 3d face reconstruction from a single image using a single reference face shape. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 394–405 (2011)CrossRefGoogle Scholar
  18. 18.
    Kulis, B., Saenko, K., Darrell, T.: What you saw is not what you get: domain adaptation using asymmetric kernel transforms. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1785–1792 (2011)Google Scholar
  19. 19.
    Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38(3), 199–218 (2000)CrossRefGoogle Scholar
  20. 20.
    Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning transferable features with deep adaptation networks. In: Proceedings of the International Conference on Machine Learning (2015)Google Scholar
  21. 21.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  22. 22.
    Parkhi, O.M., Vedaldi, A., Zisserman, A., et al.: Deep face recognition. In: British Machine Vision Conference, vol. 1, p. 6 (2015)Google Scholar
  23. 23.
    Rudrani, S., Das, S.: Face recognition on low quality surveillance images, by compensating degradation. In: International Conference on Image Analysis and Recognition, pp. 212–221 (2011)Google Scholar
  24. 24.
    Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: European Conference on Computer Vision, pp. 213–226 (2010)Google Scholar
  25. 25.
    Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)Google Scholar
  26. 26.
    Sugiyama, M., Nakajima, S., Kashima, H., Buenau, P.V., Kawanabe, M.: Direct importance estimation with model selection and its application to covariate shift adaptation. In: Advances in Neural Information Processing Systems, pp. 1433–1440 (2008)Google Scholar
  27. 27.
    Tang, H., Hu, Y., Fu, Y., Hasegawa-Johnson, M., Huang, T.S.: Real-time conversion from a single 2d face image to a 3d text-driven emotive audio-visual avatar. In: Proceedings of the IEEE International Conference on Multimedia and Expo, pp. 1205–1208 (2008)Google Scholar
  28. 28.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)CrossRefGoogle Scholar
  29. 29.
    Wong, Y., Chen, S., Mau, S., Sanderson, C., Lovell, B.C.: Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition. In: IEEE Biometrics Workshop, Computer Vision and Pattern Recognition Workshops, pp. 81–88 (2011)Google Scholar
  30. 30.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)Google Scholar
  31. 31.
    Yang, F., Wang, J., Shechtman, E., Bourdev, L., Metaxas, D.: Expression flow for 3d-aware face component transfer. ACM Trans. Graph. 30(4), 60 (2011)CrossRefGoogle Scholar
  32. 32.
    Zeng, G., Paris, S., Quan, L., Sillion, F.: Progressive surface reconstruction from images using a local prior. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 2, pp. 1230–1237 (2005)Google Scholar
  33. 33.
    Zhang, L., Zhang, D.: Robust visual knowledge transfer via extreme learning machine-based domain adaptation. IEEE Trans. Image Process. 25(10), 4959–4973 (2016)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Zhang, L., Zuo, W., Zhang, D.: LSDT: latent sparse domain transfer learning for visual adaptation. IEEE Trans. Image Process. 25(3), 1177–1191 (2016)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. (CSUR) 35(4), 399–458 (2003)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Department of CS&EIIT MadrasChennaiIndia

Personalised recommendations