A Study on Human Recognition Using Auricle and Side View Face Images

Part of the Advanced Sciences and Technologies for Security Applications book series (ASTSA)


Face profile, the side view of the face, provides biometric discriminative information complimentary to the information provided by frontal view face images. Biometric systems that deal with non-cooperative individuals in unconstrained environments, such as those encountered in surveillance applications, can benefit from profile face images. Part of a profile face image is the human ear, which is referred to as the auricle. Human ears have discriminative information across individuals and thus, are useful for human recognition. In the current literature, there is no clear definition for what a face profile is. In this study, we discuss challenges related to this problem from recognition performance aspect to identify which parts of the head side view provide distinctive identity cues. We perform an evaluation study assessing the recognition performance of the distinct parts of the head side view using four databases (FERET, WVU, UND, and USTB). The contributions of this paper are three-fold: (i) by investigating which parts of the head side view increase the probability of successful human authentication, we determined that ears provide main features in the head side view. The rank-1 identification performance using the ear alone is about 90%. (ii) we examined various feature extraction methods to learn the best features for head side view and auricle recognition including shape-based, namely Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF); and texture-based, namely Multi scale Local Binary Patterns (MLBP), Local Ternary Patterns (LTP). We determined that texture-based techniques perform better considering that the MLBP yielded the best performance with 90.20% rank-1 identification; and (iii) we evaluated the effect of different fusion schemes, at the image, feature, and score levels, on the recognition performance. Weighted Score fusion of face profile and ear has the best score with 91.14% rank-1 identification.


  1. 1.
    Mordini E, Tzovaras D (2012) Second generation biometrics: the ethical, legal and social context. SpringerGoogle Scholar
  2. 2.
    Jain AK, Li SZ (2005) Handbook of face recognition. Springer, New YorkGoogle Scholar
  3. 3.
    Abaza A, Ross A, Herbert C, Harrison MF, Nixon MS (2013) A survey on ear biometrics. ACM Comput Surv 45:22:1–22:35Google Scholar
  4. 4.
    Santemiz P, Spreeuwers LJ, Veldhuis RNJ (2010) Side-view face recognitionGoogle Scholar
  5. 5.
    Gentile JE, Bowyer KW, Flynn PJ (2008) Profile face detection a subset multi-biometric approach. In: Biometrics theory, applications and systems (BTAS)Google Scholar
  6. 6.
    Holz C, Buthpitiya S, Knaust M (2015) Bodyprint: biometric user identification on mobile devices using the capacitive touchscreen to scan body parts. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, pp 3011–3014Google Scholar
  7. 7.
    Bicego M, Lagorio A, Grosso E, Tistarelli M (2006) On the use of SIFT features for face authentication. In: Computer vision and pattern recognition workshopGoogle Scholar
  8. 8.
    Dreuw P, Steingrube P, Hanselmann H, Ney H, Aachen G (2009) SURF-Face face recognition under viewpoint consistency constraints. In: BMVCGoogle Scholar
  9. 9.
    Ojala T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell (PAMI) 28:2037–2041CrossRefGoogle Scholar
  10. 10.
    Tan X, Triggs B (2007) Enhanced local texture feature sets for face recognition under difficult lighting conditions. In: IEEE international workshop on analysis and modeling of faces and gestures (AMFG)Google Scholar
  11. 11.
    USTB. In: University of Science and Technology Beijing USTB Database.
  12. 12.
    UND. In: University of Notre Dame UND Databases.
  13. 13.
    Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The FERET evaluation methodology for face recognition algorithms. IEEE Trans Pattern Anal Mach Intell (PAMI) 22:1090–1104CrossRefGoogle Scholar
  14. 14.
    Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The FERET database and evaluation procedure for face recognition algorithms. Image Vis Comput (IVCJ) 16:295–306CrossRefGoogle Scholar
  15. 15.
    Fahmy G, Elsherbeeny A, Mandala S, AbdelMottaleb M, Ammar H (2006) The effect of lighting direction/condition on the performance of face recognition algorithms. In: SPIE conference on human identificationGoogle Scholar
  16. 16.
    Gross R, Baker S, Matthews I, Kanade T (2005) Handbook of face recognition: face recognition across pose and illumination. SpringerGoogle Scholar
  17. 17.
    Zhang X, Gao Y (2009) Face recognition across pose: a review. Pattern Recogn 42:2876–2896CrossRefGoogle Scholar
  18. 18.
    Ghaffary K, Tab F, Danyali H (2011) Profile-based face recognition using the outline curve of the profile Silhouette. In: IJCA special issue on Artificial intelligence techniques—Novel approaches and practical applications, pp 38–43Google Scholar
  19. 19.
    Kaufman GJ, Breeding KJ (1976) The automatic recognition of human faces from profile Silhouettes. IEEE Trans Syst Man Cybern SMC-6:113–121Google Scholar
  20. 20.
    Liposcak Z, Loncaric S (1999) A scale-space approach to face recognition from profiles. In: The 8th international conference on computer analysis of images and patterns (CAIP)Google Scholar
  21. 21.
    Harmon LD, Khan MK, Larsch R, Raming PF (1981) Machine identification of human faces. Pattern Recogn 13:97–110CrossRefGoogle Scholar
  22. 22.
    Wu C, Huang J (1990) Human face profile recognition by computer. Pattern Recogn 23:255–259CrossRefGoogle Scholar
  23. 23.
    Bir B, Xiaoli Z (2004) Face recognition from face profile using dynamic time warping. In: International conference on pattern recognition (ICPR)Google Scholar
  24. 24.
    Zhou X, Bhanu B (2005) Human recognition based on face profiles in video. In: IEEE computer society conference on computer vision and pattern recognition (CVPR)Google Scholar
  25. 25.
    Sihao D, Qiang Z, Yuan FZ, Dong X (2013) Side-view face authentication based on wavelet and random forest with subsets. In: IEEE international conference on intelligence and security informatics (ISI)Google Scholar
  26. 26.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444CrossRefGoogle Scholar
  27. 27.
    Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 1097–1105Google Scholar
  28. 28.
    Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: The IEEE conference on computer vision and pattern recognition, pp 1701–1708Google Scholar
  29. 29.
    Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: The IEEE conference on computer vision and pattern recognition, pp 815–823Google Scholar
  30. 30.
    Parkhi O, Vedaldi A, Zisserman A (2015) Deep Face Recogn BMVC 1:6–18Google Scholar
  31. 31.
    Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLRGoogle Scholar
  32. 32.
    Zhang Y, Mu Z (2017) Ear detection under uncontrolled conditions with multiple scale faster region-based convolutional neural networks. Symmetry 9:53–72CrossRefGoogle Scholar
  33. 33.
    Chang K, Bowyer K, Sarkar S, Victor B (2003) Comparison and combination of ear and face images in appearance-based biometrics. IEEE Trans Pattern Anal Mach Intell (PAMI) 25:1160–1165CrossRefGoogle Scholar
  34. 34.
    Kusuma D, Takashi Y (2006) Ear photo recognition using scale invariant keypoints. In: The 2nd international association of science and technology for development (IASTED)Google Scholar
  35. 35.
    Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis (IJCV) 60:91–110CrossRefGoogle Scholar
  36. 36.
    Kisku D, Mehrotra H, Gupta P, Sing J (2009) SIFT-based ear recognition by fusion of detected key-points from color similarity slice regions. In: The international conference on advances in computational tools for engineering applications (ACTEA)Google Scholar
  37. 37.
    Feng J, Mu Z (2009) Texture analysis for ear recognition using local feature descriptor and transform filter. In: SPIE pattern recognition and computer visionGoogle Scholar
  38. 38.
    Wang Y, Mu Z-C, Zeng H (2008) Block-based and multi-resolution methods for ear recognition using wavelet transform and uniform local binary patterns. In: The 19th international conference on pattern recognition (ICPR)Google Scholar
  39. 39.
    Yuan L, Mu Z, Liu Y (2006) Multimodal recognition using face profile and ear. In: The 1st international conference on systems and control in aerospace and astronautics (ISSCAA)Google Scholar
  40. 40.
    Xu X, Mu Z (2007) Multimodal recognition based on fusion of ear and profile face. In: The 4th international conference on image and graphics (ICIG)Google Scholar
  41. 41.
    Pan X, Cao Y, Xu X, Lu Y, Zhao Y (2008) Ear and face based multimodal recognition based on KFDA. In: International conference on audio, language and image processing (ICALIP)Google Scholar
  42. 42.
    Rahman MM, Ishikawa S (2005) Proposing a passive biometric system for robotic vision. In: The 10th international symposium on artificial life and robotics (AROB)Google Scholar
  43. 43.
    Rathore R, Prakash S, Gupta P (2013) Efficient human recognition system using ear and profile face. In: Biometrics theory, applications and systems (BTAS)Google Scholar
  44. 44.
    Abaza A, Hebert C, Harrison MF (2010) Fast learning ear detection for real-time surveillance. In: Fourth IEEE international conference on biometrics: theory applications and systems (BTAS)Google Scholar
  45. 45.
    Lowe D (1999) Object recognition from local scale-invariant features. In: IEEE international conference on computer vision (ICCV)Google Scholar
  46. 46.
    Wang J-G, Li J, Lee CY, Yau W-Y (2010) Dense SIFT and Gabor descriptors-based face representation with applications to gender recognition. In: IEEE international conference on control, automation, robotics and vision (ICARCV)Google Scholar
  47. 47.
    Bay H, Ess A, Tuytelaars T, Gool LV (2008) SURF speeded up robust features. Comput Vis Image Underst (CVIU) 110:346–359CrossRefGoogle Scholar
  48. 48.
    Ojala T, Pietikinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29:51–59CrossRefGoogle Scholar
  49. 49.
    DeCann B, Ross A (2013) Relating roc and cmc curves via the biometric menagerie. In: Biometrics theory, applications and systems (BTAS)Google Scholar
  50. 50.
    Jain AK, Flynn P, Ross A (2007) Handbook of biometrics: introduction to biometrics. SpringerGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.West Virginia UniversityMorgantownUSA
  2. 2.Cairo UniversityCairoEgypt

Personalised recommendations