Document to Live Facial Identification



The National Institute for Standards and Technology (NIST) highlights that facial recognition (FR) has improved significantly for ideal cases such, where face photographs are full frontal, of good quality, and pose and illumination variations are not significant. However, there are automated face recognition scenarios that involve comparing degraded facial photographs of subjects against their high-resolution counterparts. Such non-ideal scenarios can be encountered in situations where the need is to be able to identify legacy face photographs acquired by a government agency, including examples such as matching of scanned, but degraded, face images present in drivers licenses, refugee documents, and visas against their live photographs for the purpose of establishing or verifying a subject’s identity. The factors impacting the quality of such degraded face photographs include hairstyle, pose and expression variations, lamination and security watermarks , and other artifacts such as camera motion, camera resolution, and compression. In this work, we focus on investigating a set of methodological approaches in order to be able to overcome most of the aforementioned limitations and achieve high identification rate. Thus, we incorporate a combination of preprocessing and heterogeneous face-matching techniques, where comparisons are made between the original (degraded) photograph, the restored photograph, and the high-quality photograph (the mug shot of the live subject). For the purpose of this study, we, first, introduce the restorative building blocks that include threshold-based (TB) denoising, total variational (TV) wavelet inpainting, and exemplar-based inpainting. Next, we empirically assess improvement in image quality , when the aforementioned inpainting methods are applied separately and independently, coupled with TB denoising. Finally, we compare the face-matching performance achieved when using the original degraded, restored, and live photographs and a set of academic and commercial face matchers , including the local binary patterns (LBP) and local ternary patterns (LTP) texture-based operators, combined with different distance metric techniques, as well as a state-of-the-art commercial face matcher. Our results show that the combination of TB denoising, coupled with either of the two inpainting methods selected for the purpose of this study, illustrates significant improvement in rank-1 identification accuracy. It is expected that the proposed restoration approaches discussed in this work can be directly applied to operational scenarios that include border-crossing stations and various transit centers .


Facial Recognition Facial Image Local Binary Pattern Facial Identification Face Matcher 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by the Center for Identification Technology Research and the National Science Foundation under Grant No. - West Virginia University 1066197. We would also like to acknowledge the faculty, students, and staff who assisted us with the work including, but not limited to, the data collection and experiments that led to the work presented in this chapter.


  1. 1.
    Information on the facial recognition vendor test. Technical report, National Institute of Standards and Technology. (2012)
  2. 2.
    Zhou, S.K., Chellappa, R., Zhao, W.: Unconstrained face recognition, pp. 1–8. Springer, Berlin (2006)Google Scholar
  3. 3.
    Bourlai, T., Ross, A., Jain, A.: On matching digital face images against scanned passport photos. In: First IEEE International Conference on Biometrics, Identity and Security (BIDS) (2009)Google Scholar
  4. 4.
    Bourlai, T., Ross, A., Jain, A.: Restoring degraded face images for matching faxed or scanned photos. In: IEEE Transactions on Information Forensics and Security (2011)Google Scholar
  5. 5.
    Albert, A.M., Sethuram, A., Ricanek, K.: Implications of adult facial aging factors on bio-metrics. In: Biometrics: Unique and Diverse Applications in Nature, Science, and Technology. Intech (2011)Google Scholar
  6. 6.
    Ricanek, K., Mahalingam, K., Mahalingam, G., Albert, A.M., Bruegge, R.V.: Human face aging: a perspective analysis from anthropology and biometrics. In: Age Factors in Biometric Processing. The Institution of Engineering and Technology (2013)Google Scholar
  7. 7.
    Andrews, B.: A brief history of us passport photographs. (2009)
  8. 8.
    ICAO: Welcome to the ICAO machine readable travel documents programme, Sept 2012.
  9. 9.
    McMunn, M.: Machine readable travel documents. Int. Civil Aviat. Organ. (ICAO) 1(1), 14–15 (2006)Google Scholar
  10. 10.
    Starovoitov, V.V., Samal, D., Sankur, B.: Matching of faces in camera images and document photographs. In: International Conference on Acoustic, Speech, and Signal Processing, vol. IV, pp. 2349–2352 (2000)Google Scholar
  11. 11.
    Starovoitov, V.V., Samal, D.I., Briliuk, D.V.: Three approaches for face recognition. In: International Conference on Pattern Recognition and Image Analysis, pp. 707–711 (2002)Google Scholar
  12. 12.
    Ramanathan, N., Chellappa, R.: Face verification across age progression. IEEE Trans. Image Process. 15(11), 3349–3362 (2006)CrossRefGoogle Scholar
  13. 13.
    Viola, P.A., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)CrossRefGoogle Scholar
  14. 14.
    Li, Y.P., Kittler, J., Matas, J.: Analysis of the lda-based matching schemes for face verification. British Machine Vision Conference (2000)Google Scholar
  15. 15.
    Mohideen, S.K., Perumal, S.A., Sathik, M.M.: Image de-noising using discrete wavelet transform. Int. J. Comput. Sci. Netw. Secur. 8(1) (2008)Google Scholar
  16. 16.
    Coifman, R.R., Donoho, D.L.: Translation-invariant de-noising. In: Wavelets and Statistics, Springer Lecture Notes in Statistics, vol. 103, pp. 125–150 (1994)Google Scholar
  17. 17.
    Ferrara, M., Franco, A., Maio, D., Maltoni, D.: Face image conformance to iso/icao standards in machine readable travel documents. IEEE Trans. Inf. Forensics Secur. 7(4), 1204–1213 (2012)CrossRefGoogle Scholar
  18. 18.
    Bertalmio, M., Bertozzi, A.L., Caeselles, V., Ballester, C.: Image inpainting. Technical report, University of Minnesota (1999)Google Scholar
  19. 19.
    Chan, T., Shen, J.: Morphologically invariant PDE inpaintings. Technical report, UCLA (2001)Google Scholar
  20. 20.
    Chan, T.F., Kang, S., Shen, J.: Euler’s elastica and curvature based inpainting. SIAM J. Appl. Math. 63, 564–592 (2002)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Bertalmio, M., Bertozzi, A., Shapiro, G.: Navier-stokes, fluid dynamics, and image inpainting and video inpainting. In: IEEE Conference on Computer Vision and Pattern Recognition (2001)Google Scholar
  22. 22.
    Kang, S.H., Chan, T.F., Soatto, S.: Inpainting from multiple view. Technical report, UCLA CAM Report (2002)Google Scholar
  23. 23.
    Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and grey levels. IEEE Trans. Image Process. 10, 1200–1211 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Demanet, L., Song, B., Chan, T.: Image inpainting by correspondence maps: a deterministic approach. Technical report, UCLA CAM Report (2003)Google Scholar
  25. 25.
    Chan, T., Shen, J., Zhou, H.: Total variation wavelet inpainting. SIAM J. Appl. Math. (2006)Google Scholar
  26. 26.
    Bourlai, T., Clark, A.D., Best-Rowden, L.S.: Methodological insights on restoring face photos of multinational passports. In: IEEE International Symposium on Technologies for Homeland Security (IEEE HST) (2013)Google Scholar
  27. 27.
    Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)CrossRefGoogle Scholar
  28. 28.
    Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1–13 (2004)CrossRefGoogle Scholar
  29. 29.
    Hennings-Yeomans, P.H., Baker, S., Kumar, B.V.: Simultaneous super-resolution and feature extraction for recognition of low-resolution faces. In: Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Lane Department of Computer Science and Electrical Engineering, Statler College of Engineering and Mineral ResourcesWest Virginia UniversityMorgantownUSA

Personalised recommendations