Skip to main content
SpringerLink
Log in

Learning warps based similarity for pose-unconstrained face recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face recognition techniques are widely used in many applications, such as automatic detection of crime scenes from surveillance cameras for public safety. In these real cases, the pose and illumination variances between two matching faces have a big influence on the identification performance. Handling pose changes is an especially challenging task. In this paper, we propose the learning warps based similarity method to deal with face recognition across the pose problem. Warps are learned between two patches from probe faces and gallery faces using the Lucas-Kanade algorithm. Based on these warps, a frontal face registered in the gallery is transformed into a series of non-frontal viewpoints, which enables non-frontal probe face matching with the frontal gallery face. Scale-invariant feature transform (SIFT) keypoints (interest points) are detected from the generated viewpoints and matched with the probe faces. Moreover, based on the learned warps, the probability likelihood is used to calculate the probability of two faces being the same subject. Finally, a hybrid similarity combining the number of matching keypoints and the probability likelihood is proposed to describe the similarity between a gallery face and a probe face. Experimental results show that our proposed method achieves better recognition accuracy than other algorithms it was compared to, especially when the pose difference is within 40 degrees.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  MATH  Google Scholar 

  2. Arashloo SR, Kittler J (2011) Energy normalization for pose-invariant face recognition based on MRF model image matching. IEEE Trans Pattern Anal Mach Intell 33(6):1274–1280

    Article  Google Scholar 

  3. Ashraf AB, Lucey S, Chen T (2008) Learning patch correspondences for improved viewpoint invariant face recognition. Comput Vision Pattern Recogn. CVPR 2008. IEEE Conference on, 2008. IEEE: 1–8

  4. Asthana A, Gedeon T, Goecke R, Sanderson C (2009) Learning-based face synthesis for pose-robust recognition from single image. British Mach Vision Conf, 2009. British Machine Vision Assoc Soc Pattern Recogn: 1–10

  5. Asthana A, Marks TK, Jones MJ, Tieu KH, Rohith M (2011) Fully automatic pose-invariant face recognition via 3D pose normalization. 2011 Int Conf Comput Vision. IEEE: 937–944

  6. Baker S, Matthews I (2004) Lucas-kanade 20 years on: a unifying framework. Int J Comput Vis 56(3):221–255

    Article  Google Scholar 

  7. Blanz V, Vetter T (2003) Face recognition based on fitting a 3D morphable model. IEEE Trans Pattern Anal Mach Intell 25(9):1063–1074

    Article  Google Scholar 

  8. Chai X, Shan S, Chen X, Gao W (2007) Locally linear regression for pose-invariant face recognition. IEEE Trans Image Process 16(7):1716–1725

    Article  MathSciNet  Google Scholar 

  9. Chen D, Cao X, Wen F, Sun J (2013) Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification. Proc IEEE Conf Comput Vision Pattern Recogn: 3025–3032

  10. Gao Y, Lee HJ (2014) Pose unconstrained face recognition based on SIFT and alignment error. Audio, Language Image Process (ICALIP), 2014 Int Conf. IEEE: 277–281

  11. Gao Y, Lee HJ (2015) Cross-pose face recognition based on multiple virtual views and alignment error. Pattern Recogn Lett 65:170–176

    Article  Google Scholar 

  12. Grgic M, Delac K, Grgic S (2011) SCface–surveillance cameras face database. Multimed Tools Appl 51(3):863–879

    Article  Google Scholar 

  13. Ho HT, Chellappa R (2013) Pose-invariant face recognition using Markov random fields. IEEE Trans Image Process 22(4):1573–1584

    Article  MathSciNet  MATH  Google Scholar 

  14. Hua G, Yang M-H, Learned-Miller E, Ma Y, Turk M, Kriegman DJ, Huang TS (2011) Introduction to the special section on real-world face recognition. IEEE Trans Pattern Anal Mach Intell 33(10):1921–1924

    Article  Google Scholar 

  15. Jiang Y, Wang B, Zhou Y, Li W, Liao Q (2015) Patterns of Weber magnitude and orientation for uncontrolled face representation and recognition. Neurocomputing 165:190–201

    Article  Google Scholar 

  16. Li S, Liu X, Chai X, Zhang H, Lao S, Shan S (2012) Morphable displacement field based image matching for face recognition across pose. Europ Conf Comput Vision. Springer: 102–115

  17. Li A, Shan S, Gao W (2012) Coupled bias–variance tradeoff for cross-pose face recognition. IEEE Trans Image Process 21(1):305–315

    Article  MathSciNet  MATH  Google Scholar 

  18. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  19. Mikolaczyk K, Schmid C Scale and affine invariant interest points detectors. International

  20. Mikolajczyk K, Tuytelaars T, Schmid C, Zisserman A, Matas J, Schaffalitzky F, Kadir T, Van Gool L (2005) A comparison of affine region detectors. Int J Comput Vis 65(1-2):43–72

    Article  Google Scholar 

  21. Milborrow S, Nicolls F (2008) Locating facial features with an extended active shape model. Europ Conf Comput Vision. Springer: 504–513

  22. Morel J-M, Yu G (2009) ASIFT: a new framework for fully affine invariant image comparison. SIAM J Imag Sci 2(2):438–469

    Article  MathSciNet  MATH  Google Scholar 

  23. Phillips J (2009) Video challenge problem multiple biometric grand challenge preliminary: results of version 2. Nat Inst Standards Technol

  24. Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The FERET database and evaluation procedure for face-recognition algorithms. Image Vis Comput 16(5):295–306

    Article  Google Scholar 

  25. Prince SJ, Elder JH, Warrell J, Felisberti FM (2008) Tied factor analysis for face recognition across large pose differences. IEEE Trans Pattern Anal Mach Intell 30(6):970–984

    Article  Google Scholar 

  26. Taigman Y, Yang M, Ranzato MA, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. Proc IEEE Conf ComputVision Pattern Recogn: 1701–1708

  27. Vu N-S, Caplier A (2012) Enhanced patterns of oriented edge magnitudes for face recognition and image matching. IEEE Trans Image Process 21(3):1352–1365

    Article  MathSciNet  MATH  Google Scholar 

  28. Wolf L, Hassner T, Taigman Y (2011) Effective unconstrained face recognition by combining multiple descriptors and learned background statistics. IEEE Trans Pattern Anal Mach Intell 33(10):1978–1990

    Article  Google Scholar 

  29. Zhang W, Shan S, Gao W, Chen X, Zhang H (2005) Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. Tenth IEEE Int Conf Comput Vision (ICCV’05) IEEE 1:786–791

    Article  Google Scholar 

  30. Zhang Y, Shao M, Wong EK, Fu Y (2013) Random faces guided sparse many-to-one encoder for pose-invariant face recognition. Proc IEEE Int Conf Comput Vision: 2416–2423

  31. Zhu Z, Luo P, Wang X, Tang X (2013) Deep learning identity-preserving face space. Proc IEEE Int Conf Comput Vision: 113–120

Download references

Acknowledgements

This work was supported by the Brain Korea 21 PLUS Project and the State Scholarship Fund organized by the China Scholarship Council. This work was also supported by the Business for Academic-Industrial Cooperative establishments that were funded by the Korea Small and Medium Business Administration in 2015 (Grants No. C0221114). This research was also supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2016-R0992-16-1023) supervised by the IITP (Institute for Information & communications Technology Promotion). This work was also supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (GR 2016R1D1A3B03931911). This paper was also supported by the project of local colleges’ and universities’ capacity construction of Science and Technology Commission in Shanghai (No. 15590501300) and by the National Natural Science Foundation of China (No. 61461021).

Author information

Authors and Affiliations

  1. Division of Computer Science and Engineering, Chonbuk National University, Deokjin-ku Baekje-dero 567, Jeonju, 54896, Chonbuk, Republic of Korea

    Yongbin Gao

  2. Division of Computer Science and Engineering, Center for Advanced Image and Information Technology, Chonbuk National University, Deokjin-ku Baekje-dero 567, Jeonju, 54896, Chonbuk, Republic of Korea

    Hyo Jong Lee

Authors
  1. Yongbin Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyo Jong Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hyo Jong Lee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, Y., Lee, H.J. Learning warps based similarity for pose-unconstrained face recognition. Multimed Tools Appl 77, 1927–1942 (2018). https://doi.org/10.1007/s11042-017-4359-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4359-9

Keywords

Search

Navigation