Skip to main content
Book cover

Surveillance in Action pp 77–104Cite as

A Study on Human Recognition Using Auricle and Side View Face Images

Part of the Advanced Sciences and Technologies for Security Applications book series (ASTSA)

Abstract

Face profile, the side view of the face, provides biometric discriminative information complimentary to the information provided by frontal view face images. Biometric systems that deal with non-cooperative individuals in unconstrained environments, such as those encountered in surveillance applications, can benefit from profile face images. Part of a profile face image is the human ear, which is referred to as the auricle. Human ears have discriminative information across individuals and thus, are useful for human recognition. In the current literature, there is no clear definition for what a face profile is. In this study, we discuss challenges related to this problem from recognition performance aspect to identify which parts of the head side view provide distinctive identity cues. We perform an evaluation study assessing the recognition performance of the distinct parts of the head side view using four databases (FERET, WVU, UND, and USTB). The contributions of this paper are three-fold: (i) by investigating which parts of the head side view increase the probability of successful human authentication, we determined that ears provide main features in the head side view. The rank-1 identification performance using the ear alone is about 90%. (ii) we examined various feature extraction methods to learn the best features for head side view and auricle recognition including shape-based, namely Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF); and texture-based, namely Multi scale Local Binary Patterns (MLBP), Local Ternary Patterns (LTP). We determined that texture-based techniques perform better considering that the MLBP yielded the best performance with 90.20% rank-1 identification; and (iii) we evaluated the effect of different fusion schemes, at the image, feature, and score levels, on the recognition performance. Weighted Score fusion of face profile and ear has the best score with 91.14% rank-1 identification.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-68533-5_4
  • Chapter length: 28 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   139.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-68533-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   179.99
Price excludes VAT (USA)
Hardcover Book
USD   179.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 4.1
Fig. 4.2
Fig. 4.3
Fig. 4.4
Fig. 4.5
Fig. 4.6
Fig. 4.7
Fig. 4.8
Fig. 4.9
Fig. 4.10
Fig. 4.11
Fig. 4.12
Fig. 4.13

Notes

  1. 1.

    Experimentally overlapped MLBP, 24\(\,\times \,\)24 pixels patches that overlap by 12 pixels, was proven to yield the best performance.

  2. 2.

    Experimentally overlapped LTP, 24\(\,\times \,\)24 pixels patches that overlap by 12 pixels, was proven to yield the best performance.

  3. 3.

    http://www3.nd.edu/~cvrl/CVRL/Data_Sets.html.

  4. 4.

    http://www1.ustb.edu.cn/resb/en/index.htm.

References

  1. Mordini E, Tzovaras D (2012) Second generation biometrics: the ethical, legal and social context. Springer

    Google Scholar 

  2. Jain AK, Li SZ (2005) Handbook of face recognition. Springer, New York

    Google Scholar 

  3. Abaza A, Ross A, Herbert C, Harrison MF, Nixon MS (2013) A survey on ear biometrics. ACM Comput Surv 45:22:1–22:35

    Google Scholar 

  4. Santemiz P, Spreeuwers LJ, Veldhuis RNJ (2010) Side-view face recognition

    Google Scholar 

  5. Gentile JE, Bowyer KW, Flynn PJ (2008) Profile face detection a subset multi-biometric approach. In: Biometrics theory, applications and systems (BTAS)

    Google Scholar 

  6. Holz C, Buthpitiya S, Knaust M (2015) Bodyprint: biometric user identification on mobile devices using the capacitive touchscreen to scan body parts. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, pp 3011–3014

    Google Scholar 

  7. Bicego M, Lagorio A, Grosso E, Tistarelli M (2006) On the use of SIFT features for face authentication. In: Computer vision and pattern recognition workshop

    Google Scholar 

  8. Dreuw P, Steingrube P, Hanselmann H, Ney H, Aachen G (2009) SURF-Face face recognition under viewpoint consistency constraints. In: BMVC

    Google Scholar 

  9. Ojala T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell (PAMI) 28:2037–2041

    CrossRef  Google Scholar 

  10. Tan X, Triggs B (2007) Enhanced local texture feature sets for face recognition under difficult lighting conditions. In: IEEE international workshop on analysis and modeling of faces and gestures (AMFG)

    Google Scholar 

  11. USTB. In: University of Science and Technology Beijing USTB Database. http://www1.ustb.edu.cn/resb/en/index.htm

  12. UND. In: University of Notre Dame UND Databases. http://www3.nd.edu/~cvrl/CVRL/Data_Sets.html

  13. Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The FERET evaluation methodology for face recognition algorithms. IEEE Trans Pattern Anal Mach Intell (PAMI) 22:1090–1104

    CrossRef  Google Scholar 

  14. Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The FERET database and evaluation procedure for face recognition algorithms. Image Vis Comput (IVCJ) 16:295–306

    CrossRef  Google Scholar 

  15. Fahmy G, Elsherbeeny A, Mandala S, AbdelMottaleb M, Ammar H (2006) The effect of lighting direction/condition on the performance of face recognition algorithms. In: SPIE conference on human identification

    Google Scholar 

  16. Gross R, Baker S, Matthews I, Kanade T (2005) Handbook of face recognition: face recognition across pose and illumination. Springer

    Google Scholar 

  17. Zhang X, Gao Y (2009) Face recognition across pose: a review. Pattern Recogn 42:2876–2896

    CrossRef  Google Scholar 

  18. Ghaffary K, Tab F, Danyali H (2011) Profile-based face recognition using the outline curve of the profile Silhouette. In: IJCA special issue on Artificial intelligence techniques—Novel approaches and practical applications, pp 38–43

    Google Scholar 

  19. Kaufman GJ, Breeding KJ (1976) The automatic recognition of human faces from profile Silhouettes. IEEE Trans Syst Man Cybern SMC-6:113–121

    Google Scholar 

  20. Liposcak Z, Loncaric S (1999) A scale-space approach to face recognition from profiles. In: The 8th international conference on computer analysis of images and patterns (CAIP)

    Google Scholar 

  21. Harmon LD, Khan MK, Larsch R, Raming PF (1981) Machine identification of human faces. Pattern Recogn 13:97–110

    CrossRef  Google Scholar 

  22. Wu C, Huang J (1990) Human face profile recognition by computer. Pattern Recogn 23:255–259

    CrossRef  Google Scholar 

  23. Bir B, Xiaoli Z (2004) Face recognition from face profile using dynamic time warping. In: International conference on pattern recognition (ICPR)

    Google Scholar 

  24. Zhou X, Bhanu B (2005) Human recognition based on face profiles in video. In: IEEE computer society conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  25. Sihao D, Qiang Z, Yuan FZ, Dong X (2013) Side-view face authentication based on wavelet and random forest with subsets. In: IEEE international conference on intelligence and security informatics (ISI)

    Google Scholar 

  26. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    CrossRef  Google Scholar 

  27. Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 1097–1105

    Google Scholar 

  28. Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: The IEEE conference on computer vision and pattern recognition, pp 1701–1708

    Google Scholar 

  29. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: The IEEE conference on computer vision and pattern recognition, pp 815–823

    Google Scholar 

  30. Parkhi O, Vedaldi A, Zisserman A (2015) Deep Face Recogn BMVC 1:6–18

    Google Scholar 

  31. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLR

    Google Scholar 

  32. Zhang Y, Mu Z (2017) Ear detection under uncontrolled conditions with multiple scale faster region-based convolutional neural networks. Symmetry 9:53–72

    CrossRef  Google Scholar 

  33. Chang K, Bowyer K, Sarkar S, Victor B (2003) Comparison and combination of ear and face images in appearance-based biometrics. IEEE Trans Pattern Anal Mach Intell (PAMI) 25:1160–1165

    CrossRef  Google Scholar 

  34. Kusuma D, Takashi Y (2006) Ear photo recognition using scale invariant keypoints. In: The 2nd international association of science and technology for development (IASTED)

    Google Scholar 

  35. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis (IJCV) 60:91–110

    CrossRef  Google Scholar 

  36. Kisku D, Mehrotra H, Gupta P, Sing J (2009) SIFT-based ear recognition by fusion of detected key-points from color similarity slice regions. In: The international conference on advances in computational tools for engineering applications (ACTEA)

    Google Scholar 

  37. Feng J, Mu Z (2009) Texture analysis for ear recognition using local feature descriptor and transform filter. In: SPIE pattern recognition and computer vision

    Google Scholar 

  38. Wang Y, Mu Z-C, Zeng H (2008) Block-based and multi-resolution methods for ear recognition using wavelet transform and uniform local binary patterns. In: The 19th international conference on pattern recognition (ICPR)

    Google Scholar 

  39. Yuan L, Mu Z, Liu Y (2006) Multimodal recognition using face profile and ear. In: The 1st international conference on systems and control in aerospace and astronautics (ISSCAA)

    Google Scholar 

  40. Xu X, Mu Z (2007) Multimodal recognition based on fusion of ear and profile face. In: The 4th international conference on image and graphics (ICIG)

    Google Scholar 

  41. Pan X, Cao Y, Xu X, Lu Y, Zhao Y (2008) Ear and face based multimodal recognition based on KFDA. In: International conference on audio, language and image processing (ICALIP)

    Google Scholar 

  42. Rahman MM, Ishikawa S (2005) Proposing a passive biometric system for robotic vision. In: The 10th international symposium on artificial life and robotics (AROB)

    Google Scholar 

  43. Rathore R, Prakash S, Gupta P (2013) Efficient human recognition system using ear and profile face. In: Biometrics theory, applications and systems (BTAS)

    Google Scholar 

  44. Abaza A, Hebert C, Harrison MF (2010) Fast learning ear detection for real-time surveillance. In: Fourth IEEE international conference on biometrics: theory applications and systems (BTAS)

    Google Scholar 

  45. Lowe D (1999) Object recognition from local scale-invariant features. In: IEEE international conference on computer vision (ICCV)

    Google Scholar 

  46. Wang J-G, Li J, Lee CY, Yau W-Y (2010) Dense SIFT and Gabor descriptors-based face representation with applications to gender recognition. In: IEEE international conference on control, automation, robotics and vision (ICARCV)

    Google Scholar 

  47. Bay H, Ess A, Tuytelaars T, Gool LV (2008) SURF speeded up robust features. Comput Vis Image Underst (CVIU) 110:346–359

    CrossRef  Google Scholar 

  48. Ojala T, Pietikinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29:51–59

    CrossRef  Google Scholar 

  49. DeCann B, Ross A (2013) Relating roc and cmc curves via the biometric menagerie. In: Biometrics theory, applications and systems (BTAS)

    Google Scholar 

  50. Jain AK, Flynn P, Ross A (2007) Handbook of biometrics: introduction to biometrics. Springer

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. West Virginia University, Morgantown, USA

    Susan El-Naggar & Thirimachos Bourlai

  2. Cairo University, Cairo, Egypt

    Ayman Abaza

Authors
  1. Susan El-Naggar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayman Abaza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thirimachos Bourlai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Susan El-Naggar .

Editor information

Editors and Affiliations

  1. Department of Informatics and Computers, Hellenic Air Force Academy, Attica, Greece

    Prof. Panagiotis Karampelas

  2. Multispectral Imagery Lab, Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, West Virginia, USA

    Thirimachos Bourlai

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

El-Naggar, S., Abaza, A., Bourlai, T. (2018). A Study on Human Recognition Using Auricle and Side View Face Images. In: Karampelas, P., Bourlai, T. (eds) Surveillance in Action. Advanced Sciences and Technologies for Security Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-68533-5_4

Download citation