Skip to main content
Book cover

Surveillance in Action pp 147–163Cite as

Deep Feature Learning for Classification When Using Single Sensor Multi-wavelength Based Facial Recognition Systems in SWIR Band

Part of the Advanced Sciences and Technologies for Security Applications book series (ASTSA)

Abstract

In this chapter, we propose a convolutional neural network (CNN) based classification framework. Our proposed CNN framework is designed to automatically categorizes face data into individual wavelengths before the face recognition algorithms (pre-processing, feature extraction and matching) are used. Our main objective is to study the impact of classification of multi-wavelength images into individual wavelengths, when using a challenging single sensor multi-wavelength face database in short wavelength infrared (SWIR) band, for the purpose of improving heterogeneous face recognition in law enforcement and surveillance applications. Multi-wavelength database is composed of the face images captured at five different SWIR wavelengths ranging from 1150  nm to 1550 nm in increments of 100 nm. For classification based on CNN networks, there are no pre-trained multi-wavelength models available for our challenging SWIR datasets. To deal with this issue, we trained the models on our database and empirically optimized the model parameters (e.g. epoch and momentum) such that classification is performed more accurately. After classification, a set of face matching experiments is performed where a proposed face matching fusion approach is used indicating that, when fusion is supported by our classification framework, the rank-1 identification rate is significantly improved, namely when no classification is used. For example, face matching rank-1 identification accuracy, when using all data is 63% versus 80% when data is automatically classified into a face dataset where face images were captured at 1550 nm wavelength.

Keywords

  • Short Wavelength Infrared (SWIR)
  • Face Matching
  • Face Images
  • Heterogeneous Face Recognition
  • Multi-wavelength Imaging

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-68533-5_7
  • Chapter length: 17 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   139.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-68533-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   179.99
Price excludes VAT (USA)
Hardcover Book
USD   179.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 7.1
Fig. 7.2
Fig. 7.3
Fig. 7.4
Fig. 7.5
Fig. 7.6
Fig. 7.7
Fig. 7.8
Fig. 7.9
Fig. 7.10

References

  1. Steiner H, Kolb A, Jung N (2016) Reliable face anti-spoofing using multispectral swir imaging. In: Biometrics (ICB), 2016 international conference on, IEEE, pp 1–8

    Google Scholar 

  2. Bertozzi M, Fedriga RI, Miron A, Reverchon JL (2013) Pedestrian detection in poor visibility conditions: would swir help?. In: International conference on image analysis and processing. pp 229–238

    Google Scholar 

  3. Kang J, Borkar A, Yeung A, Nong N, Smith M, Hayes M (2006) Short wavelength infrared face recognition for personalization. In: International conference on image processing ICIP, pp 2757–2760

    Google Scholar 

  4. Kong SG, Heo J, Abidi BR, Paik J, Abidi MA (2005) Recent advances in visual and infrared face recognitiona review. Comput Vis Image Underst 97(1):103–135

    Google Scholar 

  5. Lemoff BE, Martin RB, Sluch M, Kafka KM, McCormick W, Ice R (2013) Long-range night/day human identification using active-swir imaging. In: Proceedings SPIE infrared technology and applications XXXIX, vol 8704, pp 87,042J–87,042J–8

    Google Scholar 

  6. Martin RB, Sluch M, Kafka KM, Ice R, Lemoff BE (2013) Active-SWIR signatures for long-range night/day human detection and identification. In: SPIE defense, security, and sensing, international society for optics and photonics, pp 87,340J–87,340J

    Google Scholar 

  7. Narang N, Bourlai T (2015) Face recognition in the SWIR band when using single sensor multi-wavelength imaging systems. Image Vis Comput 33:26–43

    Google Scholar 

  8. Nicolo F, Schmid N (2012) Long range cross-spectral face recognition: matching SWIR against visible light images. Inf Forensics Secur IEEE Trans 7(6):1717–1726

    Google Scholar 

  9. Whitelam C, Bourlai T (2015b) On designing an unconstrained tri-band pupil detection system for human identification. Mach Vis Appl 26(7–8):1007–1025

    Google Scholar 

  10. Cao ZX, Schmid NA (2014) Recognition performance of cross-spectral periocular biometrics and partial face at short and long standoff distance. Open Trans Inf Process 1:20–32

    Google Scholar 

  11. DeCann B, Ross A, Dawson J (2013) Investigating gait recognition in the short-wave infrared (swir) spectrum: dataset and challenges. In: Proceedings SPIE biometric and surveillance technology for human and activity identification X, vol 8712, pp 87,120J–87,120J–16

    Google Scholar 

  12. Dawson J, Leffel S, Whitelam C, Bourlai T (2016) Collection of multispectral biometric data for cross-spectral identification applications. In: Face recognition across the imaging spectrum. Springer, pp 21–46

    Google Scholar 

  13. Ettenberg MH (2005) A little night vision-InGaAs shortwave infrared emerges as key complement to IR for military imaging. Adv Imag Fort Atkinson 20(3):29–33

    Google Scholar 

  14. Ngo HT, Ives RW, Matey JR, Dormo J, Rhoads M, Choi D (2009) Design and implementation of a multispectral iris capture system. In: Conference record of the forty-third asilomar conference on signals, Systems and Computers. pp 380–384

    Google Scholar 

  15. Steiner H, Sporrer S, Kolb A, Jung N (2015) Design of an active multispectral swir camera system for skin detection and face verification. J Sens 2016

    Google Scholar 

  16. Bourlai T, Narang N, Cukic B, Hornak L (2012) On designing a swir multi-wavelength facial-based acquisition system. In: Proceedings SPIE infrared technology and applications XXXVIII, vol 8353, Baltimore, USA, pp 83,530R–83,530R–14

    Google Scholar 

  17. Stallkamp J, Schlipsing M, Salmen J, Igel C (2012) Man vs computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Networks J 32:323–332

    Google Scholar 

  18. Bourlai T, Mavridis N, Narang N (2016) On designing practical long range near infrared-based face recognition systems. Image Vis Comput 52:25–41

    Google Scholar 

  19. Nowak E, Jurie F, Triggs B (2006) Sampling strategies for bag-of-features image classification. In: European Conference on Computer Vision, Springer

    Google Scholar 

  20. Namin ST, Petersson L (2012) Classification of materials in natural scenes using multi-spectral images. In: IROS, IEEE, pp 1393–1398

    Google Scholar 

  21. Gupta L, Pathangay V, Patra A, Dyana A (2007) Das S (2006) Indoor vs Outdoor scene classification using probabilistic neural network. EURASIP J Adv Signal Proc 1:1–10

    Google Scholar 

  22. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Proc Syst 25:1097–1105

    Google Scholar 

  23. Levi G, Hassner T (2010) Age and gender classification using convolutional neural networks. In: Comput Vis Pattern Recognit CVPR Workshops

    Google Scholar 

  24. Caviedes J, Oberti F (2004) A new sharpness metric based on local kurtosis, edge and energy information. Signal Proc Image Commun 19(2):147–161

    Google Scholar 

  25. Wang Z, Bovik AC, Lu L (2002) Why is image quality assessment so difficult? In: Acoustics, speech, and signal processing (ICASSP), 2002 IEEE international conference on, IEEE, vol 4, pp IV–3313

    Google Scholar 

  26. Luxen M, Forstner W (2002) Characterizing image quality: blind estimation of the point spread function from a single image. Int Arch Photogrammetry Remote Sens Spat Inf Sci 34(3/A):205–210

    Google Scholar 

  27. Vu CT, Phan TD, Chandler DM (2012) \({bf S}_{3}\): a spectral and spatial measure of local perceived sharpness in natural images. IEEE Trans Image Proc 21(3):934–945

    Google Scholar 

  28. Vedaldi A, Lenc K (2015) MatConvNet-convolutional neural networks for MATLAB. In: Proceeding of the ACM international conference on multimedia

    Google Scholar 

  29. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: Proceedings of the British machine vision conference (BMVC)

    Google Scholar 

  30. Ice J, Narang N, Whitelam C, Kalka N, Hornak L, Dawson J, Bourlai T (2012) SWIR imaging for facial image capture through tinted materials. In: SPIE defense, security, and sensing, international society for optics and photonics, pp 83,530S–83,530S

    Google Scholar 

  31. Whitelam C, Bourlai T (2015a) Accurate eye localization in the short waved infrared spectrum through summation range filters. Comput Vis Image Underst 139:59–72

    Google Scholar 

  32. Cao D, Chen C, Piccirilli M, Adjeroh D, Bourlai T, Ross A (2011) Can facial metrology predict gender? In: IJCB, IEEE, pp 1–8

    Google Scholar 

  33. Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. Trans Img Proc 19(6):1635–1650

    Google Scholar 

  34. Cao Z, Schmid NA, Bourlai T (2016) Local operators and measures for heterogeneous face recognition. In: Face recognition across the imaging spectrum. Springer, pp 91–115

    Google Scholar 

Download references

Acknowledgements

This work was sponsored in part through a grant from the Office of Naval Research (ONR), contract N00014-09-C-0495, Distribution A – Approved or Unlimited Distribution. We are also grateful to all faculty and students who assisted us with this work as well as the reviewers for the kind and constructive feedback. The views expressed in this chapter are those of the authors and do not reflect the official policy or position of the Department of Defense, or the U.S. Government.

Author information

Authors and Affiliations

  1. Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, 26506-6109, USA

    Neeru Narang & Thirimachos Bourlai

Authors
  1. Neeru Narang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thirimachos Bourlai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thirimachos Bourlai .

Editor information

Editors and Affiliations

  1. Department of Informatics and Computers, Hellenic Air Force Academy, Attica, Greece

    Prof. Panagiotis Karampelas

  2. Multispectral Imagery Lab, Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, West Virginia, USA

    Thirimachos Bourlai

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Narang, N., Bourlai, T. (2018). Deep Feature Learning for Classification When Using Single Sensor Multi-wavelength Based Facial Recognition Systems in SWIR Band. In: Karampelas, P., Bourlai, T. (eds) Surveillance in Action. Advanced Sciences and Technologies for Security Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-68533-5_7

Download citation