Skip to main content

Document to Live Facial Identification

  • 1869 Accesses


The National Institute for Standards and Technology (NIST) highlights that facial recognition (FR) has improved significantly for ideal cases such, where face photographs are full frontal, of good quality, and pose and illumination variations are not significant. However, there are automated face recognition scenarios that involve comparing degraded facial photographs of subjects against their high-resolution counterparts. Such non-ideal scenarios can be encountered in situations where the need is to be able to identify legacy face photographs acquired by a government agency, including examples such as matching of scanned, but degraded, face images present in drivers licenses, refugee documents, and visas against their live photographs for the purpose of establishing or verifying a subject’s identity. The factors impacting the quality of such degraded face photographs include hairstyle, pose and expression variations, lamination and security watermarks , and other artifacts such as camera motion, camera resolution, and compression. In this work, we focus on investigating a set of methodological approaches in order to be able to overcome most of the aforementioned limitations and achieve high identification rate. Thus, we incorporate a combination of preprocessing and heterogeneous face-matching techniques, where comparisons are made between the original (degraded) photograph, the restored photograph, and the high-quality photograph (the mug shot of the live subject). For the purpose of this study, we, first, introduce the restorative building blocks that include threshold-based (TB) denoising, total variational (TV) wavelet inpainting, and exemplar-based inpainting. Next, we empirically assess improvement in image quality , when the aforementioned inpainting methods are applied separately and independently, coupled with TB denoising. Finally, we compare the face-matching performance achieved when using the original degraded, restored, and live photographs and a set of academic and commercial face matchers , including the local binary patterns (LBP) and local ternary patterns (LTP) texture-based operators, combined with different distance metric techniques, as well as a state-of-the-art commercial face matcher. Our results show that the combination of TB denoising, coupled with either of the two inpainting methods selected for the purpose of this study, illustrates significant improvement in rank-1 identification accuracy. It is expected that the proposed restoration approaches discussed in this work can be directly applied to operational scenarios that include border-crossing stations and various transit centers .


  • Facial Recognition
  • Facial Image
  • Local Binary Pattern
  • Facial Identification
  • Face Matcher

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-28501-6_10
  • Chapter length: 23 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   139.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-28501-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   179.99
Price excludes VAT (USA)
Hardcover Book
USD   179.99
Price excludes VAT (USA)
Fig. 10.1
Fig. 10.2
Fig. 10.3
Fig. 10.4
Fig. 10.5
Fig. 10.6
Fig. 10.7
Fig. 10.8
Fig. 10.9


  1. 1.

    Person-related factors are affected here due to the time lapse between the document and ideal images used in comparison.

  2. 2.

    In that work Fax image compression is defined as the process where data (e.g., face images on a document) are transferred via a fax machine using the T.6 data compression, which is performed by a fax software on a controlling computer.

  3. 3.

    This algorithm was provided by L1


  1. Information on the facial recognition vendor test. Technical report, National Institute of Standards and Technology. (2012)

  2. Zhou, S.K., Chellappa, R., Zhao, W.: Unconstrained face recognition, pp. 1–8. Springer, Berlin (2006)

    Google Scholar 

  3. Bourlai, T., Ross, A., Jain, A.: On matching digital face images against scanned passport photos. In: First IEEE International Conference on Biometrics, Identity and Security (BIDS) (2009)

    Google Scholar 

  4. Bourlai, T., Ross, A., Jain, A.: Restoring degraded face images for matching faxed or scanned photos. In: IEEE Transactions on Information Forensics and Security (2011)

    Google Scholar 

  5. Albert, A.M., Sethuram, A., Ricanek, K.: Implications of adult facial aging factors on bio-metrics. In: Biometrics: Unique and Diverse Applications in Nature, Science, and Technology. Intech (2011)

    Google Scholar 

  6. Ricanek, K., Mahalingam, K., Mahalingam, G., Albert, A.M., Bruegge, R.V.: Human face aging: a perspective analysis from anthropology and biometrics. In: Age Factors in Biometric Processing. The Institution of Engineering and Technology (2013)

    Google Scholar 

  7. Andrews, B.: A brief history of us passport photographs. (2009)

  8. ICAO: Welcome to the ICAO machine readable travel documents programme, Sept 2012.

  9. McMunn, M.: Machine readable travel documents. Int. Civil Aviat. Organ. (ICAO) 1(1), 14–15 (2006)

    Google Scholar 

  10. Starovoitov, V.V., Samal, D., Sankur, B.: Matching of faces in camera images and document photographs. In: International Conference on Acoustic, Speech, and Signal Processing, vol. IV, pp. 2349–2352 (2000)

    Google Scholar 

  11. Starovoitov, V.V., Samal, D.I., Briliuk, D.V.: Three approaches for face recognition. In: International Conference on Pattern Recognition and Image Analysis, pp. 707–711 (2002)

    Google Scholar 

  12. Ramanathan, N., Chellappa, R.: Face verification across age progression. IEEE Trans. Image Process. 15(11), 3349–3362 (2006)

    CrossRef  Google Scholar 

  13. Viola, P.A., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)

    CrossRef  Google Scholar 

  14. Li, Y.P., Kittler, J., Matas, J.: Analysis of the lda-based matching schemes for face verification. British Machine Vision Conference (2000)

    Google Scholar 

  15. Mohideen, S.K., Perumal, S.A., Sathik, M.M.: Image de-noising using discrete wavelet transform. Int. J. Comput. Sci. Netw. Secur. 8(1) (2008)

    Google Scholar 

  16. Coifman, R.R., Donoho, D.L.: Translation-invariant de-noising. In: Wavelets and Statistics, Springer Lecture Notes in Statistics, vol. 103, pp. 125–150 (1994)

    Google Scholar 

  17. Ferrara, M., Franco, A., Maio, D., Maltoni, D.: Face image conformance to iso/icao standards in machine readable travel documents. IEEE Trans. Inf. Forensics Secur. 7(4), 1204–1213 (2012)

    CrossRef  Google Scholar 

  18. Bertalmio, M., Bertozzi, A.L., Caeselles, V., Ballester, C.: Image inpainting. Technical report, University of Minnesota (1999)

    Google Scholar 

  19. Chan, T., Shen, J.: Morphologically invariant PDE inpaintings. Technical report, UCLA (2001)

    Google Scholar 

  20. Chan, T.F., Kang, S., Shen, J.: Euler’s elastica and curvature based inpainting. SIAM J. Appl. Math. 63, 564–592 (2002)

    MathSciNet  MATH  Google Scholar 

  21. Bertalmio, M., Bertozzi, A., Shapiro, G.: Navier-stokes, fluid dynamics, and image inpainting and video inpainting. In: IEEE Conference on Computer Vision and Pattern Recognition (2001)

    Google Scholar 

  22. Kang, S.H., Chan, T.F., Soatto, S.: Inpainting from multiple view. Technical report, UCLA CAM Report (2002)

    Google Scholar 

  23. Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and grey levels. IEEE Trans. Image Process. 10, 1200–1211 (2001)

    MathSciNet  CrossRef  MATH  Google Scholar 

  24. Demanet, L., Song, B., Chan, T.: Image inpainting by correspondence maps: a deterministic approach. Technical report, UCLA CAM Report (2003)

    Google Scholar 

  25. Chan, T., Shen, J., Zhou, H.: Total variation wavelet inpainting. SIAM J. Appl. Math. (2006)

    Google Scholar 

  26. Bourlai, T., Clark, A.D., Best-Rowden, L.S.: Methodological insights on restoring face photos of multinational passports. In: IEEE International Symposium on Technologies for Homeland Security (IEEE HST) (2013)

    Google Scholar 

  27. Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)

    CrossRef  Google Scholar 

  28. Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1–13 (2004)

    CrossRef  Google Scholar 

  29. Hennings-Yeomans, P.H., Baker, S., Kumar, B.V.: Simultaneous super-resolution and feature extraction for recognition of low-resolution faces. In: Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2008)

    Google Scholar 

Download references


This work was supported by the Center for Identification Technology Research and the National Science Foundation under Grant No. - West Virginia University 1066197. We would also like to acknowledge the faculty, students, and staff who assisted us with the work including, but not limited to, the data collection and experiments that led to the work presented in this chapter.

Author information

Authors and Affiliations


Corresponding author

Correspondence to A. D. Clark .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Clark, A.D., Whitelam, C., Bourlai, T. (2016). Document to Live Facial Identification. In: Bourlai, T. (eds) Face Recognition Across the Imaging Spectrum. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28499-6

  • Online ISBN: 978-3-319-28501-6

  • eBook Packages: Computer ScienceComputer Science (R0)