Skip to main content

Face Detection in MWIR Spectrum

Part of the Advanced Sciences and Technologies for Security Applications book series (ASTSA)

Abstract

The capability to perform face recognition in the visible and thermal spectra is of prime interest in many law enforcement and military organizations. Face detection is an important pre-processing step for face recognition. Though many algorithms are available for face detection in the visible spectrum, an assessment of how these algorithms can be retrained for the thermal spectrum is an important study. Current available visible-based face detection algorithms are very effective in daytime conditions, however, when there are extreme changes in illumination conditions like very low-light to no light (night-time), these become challenging. Due to limited amount of data available for researchers from sensors in the thermal band (due to the increased cost of having and operating state of the art thermal sensors), there are only a few proposed algorithms. In this work, we conducted a study to determine the impact of factors such as indoor/outdoor environment, distance from the camera, application of sunscreen, training set size, etc. on training deep-learning models for a face detection system in the thermal spectrum that simultaneously performs face detection and frontal/non-frontal classification. Existing deep learning models such as SSD (Single Shot Multi-box Detector), R-FCN (Region Based Fully Convolutional Network) and R-CNN (Region Based Convolutional Neural Network), are re-trained using thermal images for face detection and pose estimation tasks. Results from each model are compared, and the model with the best performance is further trained and tested on different datasets, including indoor, outdoor at different stand-off distances. The highest accuracy is achieved using a Faster R-CNN model with ResNet-101 and the accuracy is 99.4% after a 10-fold cross-validation. More experiments are performed to further study the efficiency and limitations of this model. The data set we use was collected under constrained indoor and unconstrained outdoor conditions.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-39489-9_8
  • Chapter length: 14 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-39489-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   169.99
Price excludes VAT (USA)
Hardcover Book
USD   169.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3

Notes

  1. 1.

    Release of MWIR Face Dataset: This is currently a private database with availability determined on case-by-case basis. If interested in working with this database, please contact the corresponding author.

References

  1. Biswas SK, Milanfar P (2017) Linear support tensor machine with LSK channels: pedestrian detection in thermal infrared images. IEEE Trans Image Process 26(9):4229–4242

    CrossRef  MathSciNet  Google Scholar 

  2. Herrmann C, Ruf M, Beyerer J (2018, April) CNN-based thermal infrared person detection by domain adaptation. In Proceedings of the Autonomous Systems: Sensors, Vehicles, Security and the Internet of Everything, Orlando, FL, USA, pp. 15–19

    Google Scholar 

  3. Cross SE, Innes B, Roberts MS, Tsuzuki T, Robertson TA, McCormick P (2007) Human skin penetration of sunscreen nanoparticles: in-vitro assessment of a novel micronized zinc oxide formulation. Skin Pharmacol Physiol 148–154

    Google Scholar 

  4. Dai J, Li Y, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. Adv Neural Inf Proces, vol. abs/1605.06409, 379–387

    Google Scholar 

  5. Dowdall J, Pavlidis I, Bebis G (2003) Face detection in the Near-IR spectrum. Image Vis Comput 565–578

    Google Scholar 

  6. Eveland CK, Socolinsky DA, Wolff LB (2003) Tracking human faces in infrared video. Image Vis Comput 579–590

    Google Scholar 

  7. Farfade SS, Saberian MJ, Li LJ (2015) Multi-view face detection using deep convolutional neural networks. In: Proceedings of the 5th ACM on international conference on multimedia retrieval, pp 643–650

    Google Scholar 

  8. Girshick RB (2015) Fast R-CNN. CoRR, pp 91–99

    Google Scholar 

  9. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. CoRR, pp 770–778

    Google Scholar 

  10. Hochreiter S (1998) The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int J Uncertainty Fuzziness Knowledge Based Syst, 6(2):107–116

    CrossRef  MathSciNet  Google Scholar 

  11. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreett M, Adam H (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. CoRR

    Google Scholar 

  12. Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Fischer I, Wojna Z, Song Y, Guadarrama S, Murphy K (2016) Speed/accuracy trade-offs for modern convolutional object detectors. CoRR, pp 7310–7311

    Google Scholar 

  13. Jiang H, Learned-Miller E (2017) Face detection with the faster R-CNN. In: 2017 12th IEEE international conference on automatic face gesture recognition (FG 2017), pp 650–657

    Google Scholar 

  14. Komatsu S, Markman A, Mahalanobis A, Chen K, Javidi B (2017) Three-dimensional integral imaging and object detection using long-wave infrared imaging. Appl Opt D120–D126

    Google Scholar 

  15. Kwaśniewska A, Rumiński J, Rad P (2017) Deep features class activation map for thermal face detection and tracking. In: 2017 10th international conference on human system interactions (HSI). IEEE, pp 41–47

    Google Scholar 

  16. Li H, Lin Z, Shen X, Brandt J, Hua G (2015) A convolutional neural network cascade for face detection. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 5325–5334

    Google Scholar 

  17. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector, pp 21–37

    Google Scholar 

  18. Ma C, Trung N, Uchiyama H, Nagahara H, Shimada A, Taniguchi R (2017) Adapting local features for face detection in thermal image. Sensors 2741

    Google Scholar 

  19. Ma C, Trung NT, Uchiyama H, Nagahara H, Shimada A, Taniguchi RI (2017) Mixed features for face detection in thermal image. Proc SPIE Int Soc Opt Eng 103380E

    Google Scholar 

  20. Murata T, Matsuno S, Mito K, Itakura N, Mizuno T (2017) Investigation of facial region extraction algorithm focusing on temperature distribution characteristics of facial thermal images. In: HCI international 2017 – posters’ extended abstracts, pp 347–352

    Google Scholar 

  21. Ranjan R, Patel VM, Chellappa R (2017) HyperFace: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans Pattern Anal Mach Intell, Vol. abs/1603.01249, 121–135

    Google Scholar 

  22. Reese K, Zheng Y, Elmaghraby AS (2012) A comparison of face detection algorithms in visible and thermal spectrums. In: Object recognition supported by user interaction for service robots

    Google Scholar 

  23. Ren S, He K, Girshick RB, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. CoRR, pp 91–99

    Google Scholar 

  24. Sun X, Wu P, Hoi SCH (2017) Face detection using deep learning: an improved faster R-CNN approach. Neurocomputing 42–50

    Google Scholar 

  25. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2014) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

    Google Scholar 

  26. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2015) Rethinking the inception architecture for computer vision. CoRR, pp 2818–2826

    Google Scholar 

  27. Yang S, Luo P, Loy CC, Tang X (2015) From facial parts responses to face detection: a deep learning approach. In: 2015 IEEE international conference on computer vision (ICCV), pp 3676–3684

    Google Scholar 

  28. Yang S, Luo P, Loy CC, Tang X (2018) Faceness-Net: face detection through deep facial part responses. IEEE Trans Pattern Anal Mach Intell, Vol. abs/1701.08393, 1845–1859

    Google Scholar 

  29. Yang S, Xiong Y, Loy CC, Tang X (2017) Face detection through scale-friendly deep convolutional networks. CoRR

    Google Scholar 

  30. Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett, vol. abs/1604.02878, 1499–1503

    Google Scholar 

  31. Zheng Y (2012) Face detection and eyeglasses detection for thermal face recognition. 83000C

    Google Scholar 

  32. Zhu C, Zheng Y, Luu K, Savvides M (2017) CMS-RCNN: contextual multi-scale region-based CNN for unconstrained face detection, pp 57–79

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thirimachos Bourlai .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Mokalla, S.R., Bourlai, T. (2020). Face Detection in MWIR Spectrum. In: Bourlai, T., Karampelas, P., Patel, V.M. (eds) Securing Social Identity in Mobile Platforms. Advanced Sciences and Technologies for Security Applications. Springer, Cham. https://doi.org/10.1007/978-3-030-39489-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-39489-9_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-39488-2

  • Online ISBN: 978-3-030-39489-9

  • eBook Packages: Computer ScienceComputer Science (R0)