Skip to main content
Book cover

Securing Social Identity in Mobile Platforms pp 61–76Cite as

On Designing a Forensic Toolkit for Rapid Detection of Factors that Impact Face Recognition Performance When Processing Large Scale Face Datasets

Part of the Advanced Sciences and Technologies for Security Applications book series (ASTSA)

Abstract

Due to the overlap between the fields of forensic investigation and biometric recognition, including face recognition, there have been several interesting applications that bridge the gap between the two sciences and better connect the associated communities. These applications have been developed with the intent to assist law enforcement officers with computer assisted and biometrics related capabilities. Thus, utilizing biometric algorithms within the forensics field can support law enforcement investigations in a wide array of applications, including fingerprint comparisons, sketch-to-photo face comparisons, and even find persons of interest via soft biometrics such as scars, marks, and tattoos. In this book chapter, we focus on facial recognition, which can help provide clues when other forensic evidence is not present or available and, most importantly, help investigators eliminate the time consuming processes of interviewing potential witnesses or manually searching through thousands of mugshots to determine a suspect’s identity. To aid in this mission, we propose a software toolkit to automatically and hierarchically categorize face images with a set of binary classifiers using three different attributes, which depending on their true label/condition can affect facial recognition performance. These attributes are: based on facial photo, (1) determining whether a subject’s eyes are open or closed, (2) whether the subject is wearing glasses or not, and (3) whether the facial pose of the subject is either frontal or non-frontal. Our toolkit offers batch processing and therefore can aid forensic operators with a capability to rapidly categorize large scale face datasets in terms of the aforementioned attributes, and thus, determine, which individuals have a higher chance to be identified based on their face information. The proposed forensic toolkit will allow the operators to analyze, enhance, group, or exclude face data before being used for face matching.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-39489-9_4
  • Chapter length: 16 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-39489-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   169.99
Price excludes VAT (USA)
Hardcover Book
USD   169.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. Jain AK, Ross A (2015) Bridging the gap: from biometrics to forensics. Philos Trans R Soc B 370(1674):20140254

    CrossRef  Google Scholar 

  2. Tistarelli M, Grosso E, Meuwly D (2014) Biometrics in forensic science: challenges, lessons and new technologies. In: International workshop on biometric authentication. Springer, pp 153–164

    Google Scholar 

  3. Meuwly D, Veldhuis R (2012) Forensic biometrics: from two communities to one discipline. In: 2012 BIOSIG-proceedings of the international conference of the biometrics special interest group (BIOSIG). IEEE, pp 1–12

    Google Scholar 

  4. Dessimoz D, Champod C (2008) Linkages between biometrics and forensic science. In: Handbook of biometrics. Springer, Boston, pp 425–459

    CrossRef  Google Scholar 

  5. Champod C, Tistarelli M (2017) Biometric technologies for forensic science and policing: state of the art. In: Handbook of biometrics for forensic science. Springer, pp 1–15

    Google Scholar 

  6. Jain AK, Klare B, Park U (2011) Face recognition: some challenges in forensics. In: 2011 IEEE international conference on automatic face & gesture recognition and workshops (FG 2011). IEEE, pp 726–733

    Google Scholar 

  7. Jain AK, Klare B, Park U (2012) Face matching and retrieval in forensics applications. IEEE Multimed 19(1):20

    CrossRef  Google Scholar 

  8. Lee J, Jain A, Tong W et al (2012) Image retrieval in forensics: tattoo image database application. IEEE MultiMed 19(1):40–49

    CrossRef  Google Scholar 

  9. Wing BJ (2013) The ANSI/NIST-ITL standard update for 2011 (data format for the interchange of fingerprint, facial and other biometric information). Int J Biom 5(1):20–29

    Google Scholar 

  10. Bourlai T (2016) Face recognition across the imaging spectrum. Springer, Switzerland

    CrossRef  Google Scholar 

  11. Bourlai T, Narang N, Cukic B, Hornak L (2012) On designing a swir multi-wavelength facial-based acquisition system. In: Infrared technology and applications XXXVIII, vol 8353. International Society for Optics and Photonics, p 83530R

    Google Scholar 

  12. White D, Dunn JD, Schmid AC, Kemp RI (2015) Error rates in users of automatic face recognition software. PLoS One 10(10):e0139827

    CrossRef  Google Scholar 

  13. Jain AK, Li SZ (2011) Handbook of face recognition. Springer, New York

    MATH  Google Scholar 

  14. Ding X, Wang L (2011) Facial landmark localization. In: Handbook of face recognition. Springer, New York, pp 305–322

    CrossRef  Google Scholar 

  15. Whitelam C, Bourlai T (2015) Accurate eye localization in the short waved infrared spectrum through summation range filters. Comput Vis Image Underst 139:59–72

    CrossRef  Google Scholar 

  16. El-Sayed MA, Khafagy MA (2014) An identification system using eye detection based on wavelets and neural networks. 1401.5108

    Google Scholar 

  17. Bourlai T, Jafri Z (2011) Eye detection in the middle-wave infrared spectrum: towards recognition in the dark. In: 2011 IEEE international workshop on information forensics and security. IEEE, pp 1–6

    Google Scholar 

  18. Dutta A, Günther M, El Shafey L, Marcel S, Veldhuis R, Spreeuwers L (2015) Impact of eye detection error on face recognition performance. IET Biom 4(3):137–150

    CrossRef  Google Scholar 

  19. Du C, Su G (2005) Eyeglasses removal from facial images. Pattern Recogn Lett 26(14):2215–2220

    CrossRef  Google Scholar 

  20. Taylor M (2017) The art of facial recognition. https://www.forensicmag.com/article/2017/03/art-facial-recognition. Accessed 18 Feb 2018

  21. Mandal B, Li L, Wang GS, Lin J (2017) Towards detection of bus driver fatigue based on robust visual analysis of eye state. IEEE Trans Intell Transp Syst 18(3):545–557

    CrossRef  Google Scholar 

  22. González-Ortega D, Díaz-Pernas FJ, Antón-Rodríguez M, Martínez-Zarzuela M, Díez-Higuera JF (2013) Real-time vision-based eye state detection for driver alertness monitoring. Pattern Anal Appl 16(3):285–306

    CrossRef  MathSciNet  Google Scholar 

  23. Zhao L, Wang Z, Zhang G, Qi Y, Wang X (2017) Eye state recognition based on deep integrated neural network and transfer learning. Multimed Tools Appl 77(15):1–24

    Google Scholar 

  24. Kim KW, Hong HG, Nam GP, Park KR (2017) A study of deep CNN-based classification of open and closed eyes using a visible light camera sensor. Sensors 17(7):1534

    CrossRef  Google Scholar 

  25. Ji Y, Wang S, Lu Y, Wei J, Zhao Y (2018) Eye and mouth state detection algorithm based on contour feature extraction. J Electron Imaging 27(5):051205

    CrossRef  Google Scholar 

  26. Hassan H, Yaacob S, Radman A, Suandi SA (2016) Eye state detection for driver inattention based on Lucas Kanade optical flow algorithm. In: 2016 6th international conference on intelligent and advanced systems (ICIAS). IEEE, pp 1–6

    Google Scholar 

  27. Radlak K, Smolka B (2012) A novel approach to the eye movement analysis using a high speed camera. In: 2012 2nd international conference on advances in computational tools for engineering applications (ACTEA). IEEE, pp 145–150

    Google Scholar 

  28. Fogelton A, Benesova W (2016) Eye blink detection based on motion vectors analysis. Comput Vis Image Underst 148:23–33

    CrossRef  Google Scholar 

  29. Pauly L, Sankar D (2015) Detection of drowsiness based on hog features and SVM classifiers. In: 2015 IEEE international conference on research in computational intelligence and communication networks (ICRCICN). IEEE, pp 181–186

    Google Scholar 

  30. Dong Y, Zhang Y, Yue J, Hu Z (2016) Comparison of random forest, random ferns and support vector machine for eye state classification. Multimed Tools Appl 75(19):11763–11783

    CrossRef  Google Scholar 

  31. Eddine BD, Dos Santos FN, Boulebtateche B, Bensaoula S (2018) Eyelsd a robust approach for eye localization and state detection. J Signal Process Syst 90(1):99–125

    CrossRef  Google Scholar 

  32. Sharif M, Bhagavatula S, Bauer L, Reiter MK (2016) Accessorize to a crime: real and stealthy attacks on state-of-the-art face recognition. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. ACM, pp 1528–1540

    Google Scholar 

  33. Ying H, Da Su G, Chen JS (2014) Automatic eyeglasses removal of frontal facial images for face recognition. Appl Mech Mater 571:697

    CrossRef  Google Scholar 

  34. Alorf A, Abbott AL (2017) In defense of low-level structural features and SVMs for facial attribute classification: application to detection of eye state, mouth state, and eyeglasses in the wild. In: 2017 IEEE international joint conference on biometrics (IJCB). IEEE, pp 599–607

    Google Scholar 

  35. Mohammad AS, Rattani A, Derahkshani R (2017) Eyeglasses detection based on learning and non-learning based classification schemes. In: 2017 IEEE international symposium on technologies for homeland security (HST). IEEE, pp 1–5

    Google Scholar 

  36. Zhao M, Zhang X, Shi Z, Chen T, Zhang F (2018) Eyeglasses detection, location and frame discriminant based on edge information projection. Multimed Tools Appl 77(12):14931–14949

    CrossRef  Google Scholar 

  37. Lazarus MZ, Gupta S (2016) A low rank model based improved eye detection under spectacles. In: Ubiquitous computing, electronics & mobile communication conference (UEMCON), IEEE annual. IEEE, pp 1–6

    Google Scholar 

  38. Basbrain AM, Al-Taie I, Azeez N, Gan JQ, Clark A (2017) Shallow convolutional neural network for eyeglasses detection in facial images. In: Computer science and electronic engineering (CEEC), 2017. IEEE, pp 157–161

    Google Scholar 

  39. Wang Y, Ou X, Tu L, Liu L (2018) Effective facial obstructions removal with enhanced cycle-consistent generative adversarial networks. In: International conference on artificial intelligence: methodology, systems, and applications. Springer, pp 210–220

    Google Scholar 

  40. Zhang X, Pham DS, Venkatesh S, Liu W, Phung D (2015) Mixed-norm sparse representation for multi view face recognition. Pattern Recognition 48(9):2935–2946.

    CrossRef  Google Scholar 

  41. Ding C, Xu C, Tao D (2015) Multi-task pose-invariant face recognition. IEEE Trans Image Process 24(3):980–993

    CrossRef  MathSciNet  Google Scholar 

  42. Zhang X, Gao Y (2009) Face recognition across pose: A review. Pattern Recogn 42(11):2876–2896

    CrossRef  Google Scholar 

  43. Zhang X, Pham D-S, Venkatesh S, Liu W, Phung D (2015) Mixed-norm sparse representation for multi view face recognition. Pattern Recogn 48(9):2935–2946

    CrossRef  Google Scholar 

  44. Ho HT, Chellappa R (2013) Pose-invariant face recognition using Markov random fields. IEEE Trans Image Process 22(4):1573–1584

    CrossRef  MathSciNet  Google Scholar 

  45. Xu W, Shen Y, Bergmann N, Hu W (2018) Sensor-assisted multi-view face recognition system on smart glass. IEEE Trans Mobile Comput 17(1):197–210

    CrossRef  Google Scholar 

  46. Shao M, Zhang Y, Fu Y (2018) Collaborative random faces-guided encoders for pose-invariant face representation learning. IEEE Trans Neural Netw Learn Syst 29(4):1019–1032

    CrossRef  Google Scholar 

  47. Oh B-S, Toh K-A, Teoh ABJ, Lin Z (2018) An analytic gabor feedforward network for single-sample and pose-invariant face recognition. IEEE Trans Image Process 27(6):2791–2805

    CrossRef  MathSciNet  Google Scholar 

  48. Deng W, Hu J, Wu Z, Guo J (2017) Lighting-aware face frontalization for unconstrained face recognition. Pattern Recogn 68:260–271

    CrossRef  Google Scholar 

  49. Masi I, Chang F-J, Choi J, Harel S, Kim J, Kim K, Leksut J, Rawls S, Wu Y, Hassner T et al (2018) Learning pose-aware models for pose-invariant face recognition in the wild. IEEE Trans Pattern Anal Mach Intell 41(2):379–393

    CrossRef  Google Scholar 

  50. Abaza A, Harrison MA, Bourlai T, Ross A (2014) Design and evaluation of photometric image quality measures for effective face recognition. IET Biom 3(4):314–324

    CrossRef  Google Scholar 

  51. Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503

    CrossRef  Google Scholar 

  52. Viola P, Jones M (2001, December) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001. CVPR 2001, vol 1. IEEE, pp I–I

    Google Scholar 

  53. Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Workshop on faces in’Real-Life’Images: detection, alignment, and recognition

    Google Scholar 

  54. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 1. IEEE, pp 886–893

    Google Scholar 

  55. Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29(1):51–59

    CrossRef  Google Scholar 

  56. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  57. Fix E, Hodges JL Jr (1951) Discriminatory analysis-nonparametric discrimination: consistency properties. Technical report, California Univ Berkeley

    MATH  Google Scholar 

  58. Breiman L (2017) Classification and regression trees. Routledge, Belmont

    CrossRef  Google Scholar 

  59. Dietterich TG (2000) Ensemble methods in machine learning. In: International workshop on multiple classifier systems. Springer, pp 1–15

    Google Scholar 

  60. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

    Google Scholar 

  61. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. West Virginia University, Morgantown, WV, USA

    J. Rose

  2. Lane Department of Computer Science and Electrical Engineering, Multispectral Imagery Lab (MILab), West Virginia University, Morgantown, WV, USA

    T. Bourlai

Authors
  1. J. Rose
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. Bourlai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T. Bourlai .

Editor information

Editors and Affiliations

  1. Lane Department of Computer Science and Electrical Engineering Multispectral Imagery Lab (MILab), West Virginia University, Morgantown, WV, USA

    Prof. Thirimachos Bourlai

  2. Department of Informatics and Computers, Hellenic Air Force Academy, Acharnes Attica, Greece

    Prof. Panagiotis Karampelas

  3. Department of Electrical Engineering, Johns Hopkins University, Baltimore, MD, USA

    Dr. Vishal M. Patel

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Rose, J., Bourlai, T. (2020). On Designing a Forensic Toolkit for Rapid Detection of Factors that Impact Face Recognition Performance When Processing Large Scale Face Datasets. In: Bourlai, T., Karampelas, P., Patel, V.M. (eds) Securing Social Identity in Mobile Platforms. Advanced Sciences and Technologies for Security Applications. Springer, Cham. https://doi.org/10.1007/978-3-030-39489-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-39489-9_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-39488-2

  • Online ISBN: 978-3-030-39489-9

  • eBook Packages: Computer ScienceComputer Science (R0)