Advertisement

Deep Ear Recognition Pipeline

  • Žiga Emeršič
  • Janez Križaj
  • Vitomir Štruc
  • Peter Peer
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 804)

Abstract

Ear recognition has seen multiple improvements in recent years and still remains very active today. However, it has been approached from recognition and detection perspective separately. Furthermore, deep-learning-based approaches that are popular in other domains have seen limited use in ear recognition and even more so in ear detection. Moreover, to obtain a usable recognition system a unified pipeline is needed. The input in such system should be plain images of subjects and the output identities based only on ear biometrics. We conduct separate analysis through detection and identification experiments on the challenging dataset and, using the best approaches, present a novel, unified pipeline. The pipeline is based on convolutional neural networks (CNN) and presents, to the best of our knowledge, the first CNN-based ear recognition pipeline. The pipeline incorporates both, the detection of ears on arbitrary images of people, as well as recognition on these segmented ear regions. The experiments show that the presented system is a state-of-the-art system and, thus, a good foundation for future real-word ear recognition systems.

Notes

Acknowledgements

This research was supported in parts by the ARRS (Slovenian Research Agency) Research Program P2-0250 (B) Metrology and Biometric Systems, the ARRS Research Program P2-0214 (A) Computer Vision. The authors thank NVIDIA for donating the Titan Xp GPU that was used in the experiments and our colleague Blaž Meden for his help with RefineNet’s Matlab scripts.

References

  1. 1.
    Abaza, A., Hebert, C., Harrison, M.A.F.: Fast learning ear detection for real-time surveillance. In: International Conference on Biometrics: Theory Applications and Systems, pp. 1–6. IEEE (2010)Google Scholar
  2. 2.
    Abaza, A., Ross, A., Hebert, C., Harrison, M.A.F., Nixon, M.: A survey on ear biometrics. ACM Comput. Surv. 45(2), 1–22 (2013)CrossRefGoogle Scholar
  3. 3.
    Alaraj, M., Hou, J., Fukami, T.: A neural network based human identification framework using ear images. In: International Technical Conference of IEEE Region, vol. 10, pp. 1595–1600. IEEE (2010)Google Scholar
  4. 4.
    Ansari, S., Gupta, P.: Localization of ear using outer helix curve of the ear. In: International Conference on Computing: Theory and Applications, pp. 688–692. IEEE (2007)Google Scholar
  5. 5.
    Arbab-Zavar, B., Nixon, M.S.: On shape-mediated enrolment in ear biometrics. In: International Symposium on Visual Computing, pp. 549–558. Springer (2007)Google Scholar
  6. 6.
    Arbab-Zavar, B., Nixon, M.S.: Robust log-Gabor filter for ear biometrics. In: International Conference on Pattern Recognition, pp. 1–4. IEEE (2008)Google Scholar
  7. 7.
    Attarchi, S., Faez, K., Rafiei, A.: A new segmentation approach for ear recognition. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 1030–1037. Springer (2008)Google Scholar
  8. 8.
    Badrinarayanan, V., Handa, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. arXiv:1505.07293 (2015)
  9. 9.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)CrossRefGoogle Scholar
  10. 10.
    Banerjee, S., Chatterjee, A.: Robust multimodal multivariate ear recognition using kernel based simultaneous sparse representation. Eng. Appl. Artif. Intell. 64, 340–351 (2017)CrossRefGoogle Scholar
  11. 11.
    Baoqing, Z., Zhichun, M., Chen, J., Jiyuan, D.: A robust algorithm for ear recognition under partial occlusion. In: Chinese Control Conference, pp. 3800–3804 (2013)Google Scholar
  12. 12.
    Basit, A., Shoaib, M.: A human ear recognition method using nonlinear curvelet feature subspace. Int. J. Comput. Math. 91(3), 616–624 (2014)CrossRefGoogle Scholar
  13. 13.
    Benzaoui, A., Kheider, A., Boukrouche, A.: Ear description and recognition using ELBP and wavelets. In: International Conference on Applied Research in Computer Science and Engineering, pp. 1–6 (2015)Google Scholar
  14. 14.
    Benzaoui, A., Hezil, N., Boukrouche, A.: Identity recognition based on the external shape of the human ear. In: International Conference on Applied Research in Computer Science and Engineering, pp. 1–5. IEEE (2015)Google Scholar
  15. 15.
    Bourouba, H., Doghmane, H., Benzaoui, A., Boukrouche, A.H.: Ear recognition based on multi-bags-of-features histogram. In: International Conference on Control, Engineering Information Technology, pp. 1–6 (2015)Google Scholar
  16. 16.
    Bustard, J.D., Nixon, M.S.: Toward unconstrained ear recognition from two-dimensional images. Trans. Syst. Man Cybern. Part A: Syst. Hum. 40(3), 486–494 (2010)CrossRefGoogle Scholar
  17. 17.
    Carreira-Perpinan, M.A.: Compression neural networks for feature extraction: application to human recognition from ear images. Master’s thesis, Faculty of Informatics, Technical University of Madrid, Spain (1995)Google Scholar
  18. 18.
    Chan, T.S., Kumar, A.: Reliable ear identification using 2-D quadrature filters. Pattern Recogn. Lett. 33(14), 1870–1881 (2012)CrossRefGoogle Scholar
  19. 19.
    Chidananda, P., Srinivas, P., Manikantan, K., Ramachandran, S.: Entropy-cum-Hough-transform-based ear detection using ellipsoid particle swarm optimization. Mach. Vis. Appl. 26(2), 185–203 (2015)CrossRefGoogle Scholar
  20. 20.
    Chowdhury, D.P., Bakshi, S., Guo, G., Sa, P.K.: On applicability of tunable filter bank based feature for ear biometrics: a study from constrained to unconstrained. J. Med. Syst. 42(1), 11 (2018)CrossRefGoogle Scholar
  21. 21.
    Cummings, A.H., Nixon, M.S., Carter, J.N.: A novel ray analogy for enrolment of ear biometrics. In: International Conference on Biometrics: Theory Applications and Systems, pp. 1–6. IEEE (2010)Google Scholar
  22. 22.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference on Computer Vision and Patten Recognition, pp. 886–893. IEEE (2005)Google Scholar
  23. 23.
    Damar, N., Fuhrer, B.: Ear recognition using multi-scale histogram of oriented gradients. In: Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 21–24 (2012)Google Scholar
  24. 24.
    Dewi, K., Yahagi, T.: Ear photo recognition using scale invariant keypoints. In: Computational Intelligence, pp. 253–258 (2006)Google Scholar
  25. 25.
    Dodge, S., Mounsef, J., Karam, L.: Unconstrained ear recognition using deep neural networks. IET Biom. (2018)Google Scholar
  26. 26.
    Dogucan, Y., Fevziye, E., Ekenel, H.: Domain adaptation for ear recognition using deep convolutional neural networks. IET Biom. 7(3), 199–206 (2018)CrossRefGoogle Scholar
  27. 27.
    Ear Recognition Laboratory at the University of Science & Technology Beijing: Introduction to USTB Ear Image Databases (2002). http://www1.ustb.edu.cn/resb/en/index.htm. Accessed 15 Mar 2018
  28. 28.
    Earnest, H., Segundo, P., Sarkar, S.: Employing fusion of learned and handcrafted features for unconstrained ear recognition. IET Biom. 7(3), 215–223 (2018)CrossRefGoogle Scholar
  29. 29.
    Emeršič, Ž., Gabriel, L.L., Štruc, V., Peer, P.: Convolutional encoder-decoder networks for pixel-wise ear detection and segmentation. IET Biom. 7(3), 175–184 (2018)CrossRefGoogle Scholar
  30. 30.
    Emeršič, Ž., Meden, B., Peer, P., Štruc, V.: Evaluation and analysis of ear recognition models: performance, complexity and resource requirements. Neural Comput. Appl. 1–16Google Scholar
  31. 31.
    Emeršič, Ž., Meden, B., Peer, P., Štruc, V.: Covariate analysis of descriptor-based ear recognition techniques. In: 2017 International Conference and Workshop on Bioinspired Intelligence (IWOBI), pp. 1–9. IEEE (2017)Google Scholar
  32. 32.
    Emeršič, Ž., Meden, B., Peer, P., Štruc, V.: Evaluation and analysis of ear recognition models: performance, complexity and resource requirements. Neural Comput. Appl. 1–16 (2018)Google Scholar
  33. 33.
    Emeršič, Ž., Peer, P.: Ear biometric database in the wild. In: 2015 4th International Work Conference on Bioinspired Intelligence (IWOBI), pp. 27–32. IEEE (2015)Google Scholar
  34. 34.
    Emeršič, Ž., Štepec, D., Štruc, V., Peer, P.: Training convolutional neural networks with limited training data for ear recognition in the wild. In: 12th IEEE International Conference on Automatic Face and Gesture (FG 2017) (2017)Google Scholar
  35. 35.
    Emeršič, Ž., Štepec, D., Štruc, V., Peer, P., George, A., Ahmad, A., Omar, E., Boult, T.E., Safdari, R., Zhou, Y., Zafeiriou, S., Yaman, D., Eyiokur, F.I., Ekenel, H.K.: The unconstrained ear recognition challenge. In: International Joint Conference on Biometrics (IJCB) (2017)Google Scholar
  36. 36.
    Emeršič, Ž., Štruc, V., Peer, P.: Ear recognition: more than a survey. Neurocomputing 255, 26–39 (2017)CrossRefGoogle Scholar
  37. 37.
    Ganesh, M.R., Krishna, R., Manikantan, K., Ramachandran, S.: Entropy based binary particle swarm optimization and classification for ear detection. Eng. Appl. Artif. Intell. 27, 115–128 (2014)CrossRefGoogle Scholar
  38. 38.
    Gourier, N., Hall, D., Crowley, J.L.: Estimating face orientation from robust detection of salient facial structures. In: FG Net Workshop on Visual Observation of Deictic Gestures, vol. 6 (2004)Google Scholar
  39. 39.
    Guo, Y., Xu, Z.: Ear recognition using a new local matching approach. In: International Conference on Image Processing, pp. 289–292. IEEE (2008)Google Scholar
  40. 40.
    Han, J., Zhang, D., Cheng, G., Liu, N., Xu, D.: Advanced deep-learning techniques for salient and category-specific object detection: a survey. IEEE Signal Process. Mag. 35(1), 84–100 (2018)CrossRefGoogle Scholar
  41. 41.
    Hansley, E.E., Segundo, M.P., Sarkar, S.: Employing fusion of learned and handcrafted features for unconstrained ear recognition. arXiv:1710.07662 (2017)
  42. 42.
    He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: European Conference on Computer Vision, pp. 630–645. Springer (2016)Google Scholar
  43. 43.
    He, X., Yu, Z., Wang, T., Lei, B.: Skin lesion segmentation via deep RefineNet. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 303–311. Springer (2017)Google Scholar
  44. 44.
    He, X., Yu, Z., Wang, T., Lei, B., Shi, Y.: Dense deconvolution net: multi path fusion and dense deconvolution for high resolution skin lesion segmentation. Technol. Health Care 1–10 (2018)Google Scholar
  45. 45.
    Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861 (2017)
  46. 46.
    Islam, S.M., Bennamoun, M., Davies, R.: Fast and fully automatic ear detection using cascaded AdaBoost. In: Workshop on Applications of Computer Vision, pp. 1–6. IEEE (2008)Google Scholar
  47. 47.
    Jacobs, R.A.: Increased rates of convergence through learning rate adaptation. Neural Netw. 1(4), 295–307 (1988)CrossRefGoogle Scholar
  48. 48.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. In: ACM International Conference on Multimedia, pp. 675–678. ACM (2014)Google Scholar
  49. 49.
    Kannala, J., Rahtu, E.: BSIF: Binarized statistical image features. In: International Conference on Pattern Recognition, pp. 1363–1366. IEEE (2012)Google Scholar
  50. 50.
    Križaj, J., Štruc, V., Pavešic, N.: Adaptation of SIFT features for robust face recognition. In: Image Analysis and Recognition, pp. 394–404. Springer (2010)Google Scholar
  51. 51.
    Kumar, A., Wu, C.: Automated human identification using ear imaging. Pattern Recogn. 45(3), 956–968 (2012)CrossRefGoogle Scholar
  52. 52.
    Lin, G., Milan, A., Shen, C., Reid, I.: RefineNet: Multi-path refinement networks for high-resolution semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 5 (2017)Google Scholar
  53. 53.
    Lu, Z., Jiang, X., Kot, A.: Deep coupled resnet for low-resolution face recognition. Signal Process. Lett. 25(4), 526–530 (2018)CrossRefGoogle Scholar
  54. 54.
    Meraoumia, A., Chitroub, S., Bouridane, A.: An automated ear identification system using Gabor filter responses. In: International Conference on New Circuits and Systems, pp. 1–4. IEEE (2015)Google Scholar
  55. 55.
    Messer, K., Matas, J., Kittler, J., Luettin, J., Maitre, G.: XM2VTSDB: the extended M2VTS database. In: International Conference on Audio and Video-Based Biometric Person Authentication, vol. 964, pp. 965–966 (1999)Google Scholar
  56. 56.
    Moody, J., Hanson, S., Krogh, A., Hertz, J.A.: A simple weight decay can improve generalization. Adv. Neural Inf. Process. Syst. 4, 950–957 (1995)Google Scholar
  57. 57.
    University of Notre Dame: Face database (2015). https://sites.google.com/a/nd.edu/public-cvrl/data-sets. Accessed 01 Mar 2018
  58. 58.
    Ojansivu, V., Heikkilä, J.: Blur insensitive texture classification using local phase quantization. In: International Conference on Image and Signal Processing, pp. 236–243 (2008)Google Scholar
  59. 59.
    Ojansivu, V., Rahtu, E., Heikkilä, J.: Rotation invariant local phase quantization for blur insensitive texture analysis. In: International Conference on Pattern Recognition, pp. 1–4. IEEE (2008)Google Scholar
  60. 60.
    Omara, I., Wu, X., Zhang, H., Du, Y., Zuo, W.: Learning pairwise SVM on hierarchical deep features for ear recognition. IET Biom. (2018)Google Scholar
  61. 61.
    Pflug, A., Busch, C., Ross, A.: 2D ear classification based on unsupervised clustering. In: International Joint Conference on Biometrics, pp. 1–8. IEEE (2014)Google Scholar
  62. 62.
    Pflug, A., Busch, C.: Ear biometrics: a survey of detection, feature extraction and recognition methods. IET Biom. 1(2), 114–129 (2012)CrossRefGoogle Scholar
  63. 63.
    Pflug, A., Paul, P.N., Busch, C.: A comparative study on texture and surface descriptors for ear biometrics. In: International Carnahan Conference on Security Technology, pp. 1–6. IEEE (2014)Google Scholar
  64. 64.
    Pflug, A., Winterstein, A., Busch, C.: Robust localization of ears by feature level fusion and context information. In: International Conference on Biometrics, pp. 1–8 (2013)Google Scholar
  65. 65.
    Phillips, P.J., Wechsler, H., Huang, J., Rauss, P.J.: The FERET database and evaluation procedure for face-recognition algorithms. Image Vis. Comput. 16(5), 295–306 (1998)CrossRefGoogle Scholar
  66. 66.
    Pietikäinen, M., Hadid, A., Zhao, G., Ahonen, T.: Computer Vision Using Local Binary Patterns. Computational Imaging and Vision. Springer (2011)Google Scholar
  67. 67.
    Prakash, S., Gupta, P.: An efficient ear localization technique. Image Vis. Comput. 30(1), 38–50 (2012)CrossRefGoogle Scholar
  68. 68.
    Prakash, S., Gupta, P.: An efficient ear recognition technique invariant to illumination and pose. Telecommun. Syst. 52(3), 1435–1448 (2013)CrossRefGoogle Scholar
  69. 69.
    Prakash, S., Gupta, P.: Ear Biometrics in 2D and 3D: Localization and Recognition, vol. 10. Springer (2015)Google Scholar
  70. 70.
    Prakash, S., Jayaraman, U., Gupta, P.: Ear localization from side face images using distance transform and template matching. In: Workshops on Image Processing Theory, Tools and Applications, pp. 1–8 (2008)Google Scholar
  71. 71.
    Prakash, S., Jayaraman, U., Gupta, P.: Connected component based technique for automatic ear detection. In: International Conference on Image Processing, pp. 2741–2744. IEEE (2009)Google Scholar
  72. 72.
    Prakash, S., Jayaraman, U., Gupta, P.: A skin-color and template based technique for automatic ear detection. In: International Conference on Advances in Pattern Recognition, pp. 213–216. IEEE (2009)Google Scholar
  73. 73.
    Proença, H., Alexandre, L.A.: The NICE.I: Noisy iris challenge evaluation—Part I. In: International Conference on Biometrics: Theory, Applications, and Systems, pp. 1–4. IEEE (2007)Google Scholar
  74. 74.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  75. 75.
    Sarangi, P.P., Panda, M., Mishra, B.S.P., Dehuri, S.: An automated ear localization technique based on modified Hausdorff distance. In: International Conference on Computer Vision and Image Processing, pp. 1–12 (2016)Google Scholar
  76. 76.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. In: International Conference on Automatic Face and Gesture Recognition, pp. 53–58. IEEE (2002)Google Scholar
  77. 77.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)
  78. 78.
    Szkuta, B.R., Sanabria, L.A., Dillon, T.S.: Electricity price short-term forecasting using artificial neural networks. IEEE Trans. Power Syst. 14(3), 851–857 (1999)CrossRefGoogle Scholar
  79. 79.
    Tian, L., Mu, Z.: Ear recognition based on deep convolutional network. In: International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 437–441. IEEE (2016)Google Scholar
  80. 80.
    University of Sheffield: The Sheffield (previously UMIST) face database (1998). https://www.sheffield.ac.uk/eee/research/iel/research/face. Accessed 01 May 2016
  81. 81.
    Urooj, A., Borji, A.: Analysis of hand segmentation in the wild. In: Conference on Computer Vision and Pattern Recognition, IEEE. pp. 4710–4719 (2018)Google Scholar
  82. 82.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Conference on Computer Vision and Pattern Recognition, pp. I–I. IEEE (2001)Google Scholar
  83. 83.
    Vu, N.S., Caplier, A.: Face recognition with patterns of oriented edge magnitudes. In: European Conference on Computer Vision, pp. 313–326 (2010)CrossRefGoogle Scholar
  84. 84.
    Wahab, N.K.A., Hemayed, E.E., Fayek, M.B.: HEARD: an automatic human ear detection technique. In: International Conference on Engineering and Technology, pp. 1–7 (2012)Google Scholar
  85. 85.
    Zhang, Y., Mu, Z., Yuan, L., Yu, C.: Ear verification under uncontrolled conditions with convolutional neural networks. IET Biom. 7(3), 185–198 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Žiga Emeršič
    • 1
  • Janez Križaj
    • 2
  • Vitomir Štruc
    • 2
  • Peter Peer
    • 1
  1. 1.Computer Vision Laboratory, Faculty of Computer and Information ScienceUniversity of LjubljanaLjubljanaSlovenia
  2. 2.Laboratory of Artificial Perception, Systems and Cybernetics, Faculty of Electrical EngineeringUniversity of LjubljanaLjubljanaSlovenia

Personalised recommendations