Skip to main content
SpringerLink
Log in
Book cover

Deep Biometrics pp 161–190Cite as

Constellation-Based Deep Ear Recognition

  • Chapter
  • First Online:

Part of the book series: Unsupervised and Semi-Supervised Learning ((UNSESUL))

Abstract

This chapter introduces COM-Ear, a deep constellation model for ear recognition. Different from competing solutions, COM-Ear encodes global as well as local characteristics of ear images and generates descriptive ear representations that ensure competitive recognition performance. The model is designed as dual-path convolutional neural network (CNN), where one path processes the input in a holistic manner, and the second captures local images characteristics from image patches sampled from the input image. A novel pooling operation, called patch-relevant-information pooling, is also proposed and integrated into the COM-Ear model. The pooling operation helps to select features from the input patches that are locally important and to focus the attention of the network to image regions that are descriptive and important for representation purposes. The model is trained in an end-to-end manner using a combined cross-entropy and center loss. Extensive experiments on the recently introduced Extended Annotated Web Ears (AWEx).

The authors “Dejan Štepec” and “Žiga Emeršič” contributed equally to this work.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Note that we study the influence of the size of the patches in the experimental section.

  2. 2.

    https://github.com/aleju/imgaug.

References

  1. A.F. Abate, M. Nappi, D. Riccio, S. Ricciardi, Ear recognition by means of a rotation invariant descriptor, in International Conference on Pattern Recognition, vol. 4 (IEEE, Piscataway, 2006), pp. 437–440

    Google Scholar 

  2. A. Abaza, A. Ross, C. Hebert, M.A.F. Harrison, M. Nixon, A survey on ear biometrics. ACM Comput. Surv. 45(2), 1–22 (2013)

    Article  Google Scholar 

  3. M. Abdel-Mottaleb, J. Zhou, Human ear recognition from face profile images, in Advances in Biometrics (Springer, Berlin, 2006), pp. 786–792

    Google Scholar 

  4. C. Akin, U. Kacar, M. Kirci, A multi-biometrics for twins identification based speech and ear. CoRR abs/1801.09056 (2018)

    Google Scholar 

  5. A.A. Almisreb, N. Jamil, N.M. Din, Utilizing AlexNet deep transfer learning for ear recognition, in International Conference on Information Retrieval and Knowledge Management (IEEE, Piscataway, Mar 2018), pp. 1–5

    Google Scholar 

  6. H.A. Alshazly, M. Hassaballah, M. Ahmed, A.A. Ali, Ear biometric recognition using gradient-based feature descriptors, in Proceedings of the International Conference on Advanced Intelligent Systems and Informatics, 2018, ed. by A.E. Hassanien, M.F. Tolba, K. Shaalan, A.T. Azar. Advances in Intelligent Systems and Computing (Springer, Berlin, 2019), pp. 435–445

    Google Scholar 

  7. B. Arbab-Zavar, M.S. Nixon, Robust log-Gabor filter for ear biometrics, in International Conference on Pattern Recognition (IEEE, Piscataway, 2008), pp. 1–4

    Google Scholar 

  8. Z. Baoqing, M. Zhichun, J. Chen, D. Jiyuan, A robust algorithm for ear recognition under partial occlusion, in Chinese Control Conference (2013), pp. 3800–3804

    Google Scholar 

  9. A. Basit, M. Shoaib, A human ear recognition method using nonlinear curvelet feature subspace. Int. J. Comput. Math. 91(3), 616–624 (2014)

    Article  MATH  Google Scholar 

  10. A. Benzaoui, A. Kheider, A. Boukrouche, Ear description and recognition using ELBP and wavelets, in International Conference on Applied Research in Computer Science and Engineering (2015), pp. 1–6

    Google Scholar 

  11. M. Burge, W. Burger, Ear biometrics, in Biometrics, ed. by A.K. Jain, R. Bolle, S. Pankanti (IEEE, Piscataway, 1996), pp. 273–285

    Google Scholar 

  12. J.D. Bustard, M.S. Nixon, Toward unconstrained ear recognition from two-dimensional images. IEEE Trans. Syst. Man Cybern. A: Syst. Humans 40(3), 486–494 (2010)

    Article  Google Scholar 

  13. T.S. Chan, A. Kumar, Reliable ear identification using 2-D quadrature filters. Pattern Recogn. Lett. 33(14), 1870–1881 (2012)

    Article  Google Scholar 

  14. K. Chang, K.W. Bowyer, S. Sarkar, B. Victor, Comparison and combination of ear and face images in appearance-based biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1160–1165 (2003)

    Article  Google Scholar 

  15. M. Choraś, Perspective methods of human identification: ear biometrics. Opto-Electr. Rev. 16(1), 85–96 (2008)

    Google Scholar 

  16. M. Choras, R.S. Choras, Geometrical algorithms of ear contour shape representation and feature extraction, in International Conference on Intelligent Systems Design and Applications (IEEE, Piscataway, 2006), pp. 451–456

    Google Scholar 

  17. D.P. Chowdhury, S. Bakshi, G. Guo, P.K. Sa, On applicability of tunable filter bank based feature for ear biometrics: a study from constrained to unconstrained. J. Med. Syst. 42(1), 11/1–20 (2017)

    Google Scholar 

  18. M. Chowdhury, R. Islam, J. Gao, Robust ear biometric recognition using neural network, in Conference on Industrial Electronics and Applications (IEEE, Piscataway, Jun 2017), pp. 1855–1859

    Google Scholar 

  19. N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, in Conference on Computer Vision and Pattern Recognition (IEEE, Piscataway, 2005), pp. 886–893

    Google Scholar 

  20. N. Damar, B. Fuhrer, Ear recognition using multi-scale histogram of oriented gradients, in Conference on Intelligent Information Hiding and Multimedia Signal Processing (2012), pp. 21–24

    Google Scholar 

  21. J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, L. Fei-Fei, Imagenet: a large-scale hierarchical image database, in Conference on Computer Vision and Pattern Recognition (IEEE, Piscataway, 2009), pp. 248–255

    Google Scholar 

  22. K. Dewi, T. Yahagi, Ear photo recognition using scale invariant keypoints, in Computational Intelligence (2006), pp. 253–258

    Google Scholar 

  23. S. Dodge, J. Mounsef, L. Karam, Unconstrained ear recognition using deep neural networks. IET Biom. 7(3), 207–214 (2018)

    Article  Google Scholar 

  24. Ž. Emeršič, D. Štepec, V. Štruc, P. Peer, Training convolutional neural networks with limited training data for ear recognition in the wild, in International Conference on Automatic Face and Gesture Recognition – Workshop on Biometrics in the Wild (IEEE, Piscataway, 2017), pp. 987–994

    Google Scholar 

  25. Ž. Emeršič, D. Štepec, V. Štruc, P. Peer, A. George, A. Ahmad, E. Omar, T.E. Boult, R. Safdari, Y. Zhou, S. Zafeiriou, D. Yaman, F.I. Eyiokur, H.K. Ekenel, The unconstrained ear recognition challenge, in International Joint Conference on Biometrics (IEEE/IAPR, Piscataway, 2017), pp. 715–724

    Google Scholar 

  26. Ž. Emeršič, V. Štruc, P. Peer, Ear recognition: more than a survey. Neurocomputing 255, 26–39 (2017)

    Article  Google Scholar 

  27. Ž. Emeršič, N.O. Playà, V. Štruc, P. Peer, Towards accessories-aware ear recognition, in International Work Conference on Bioinspired Intelligence (IEEE, Piscataway, Jul 2018), pp. 1–8

    Google Scholar 

  28. Ž. Emeršič, B. Meden, P. Peer, V. Štruc, Evaluation and analysis of ear recognition models: performance, complexity and resource requirements. Neural Comput. Applic., 1–16 (2018). https://doi.org/10.1007/s00521-018-3530-1

  29. Ž. Emeršič, J. Križaj, V. Štruc, P. Peer, Deep ear recognition pipeline, in Recent Advances in Computer Vision, ed. by M. Hassaballah, K.M. Hosny, vol. 804 (Springer, Berlin, 2019), pp. 333–362

    Google Scholar 

  30. F.I. Eyiokur, D. Yaman, H.K. Ekenel, Domain adaptation for ear recognition using deep convolutional neural networks. IET Biom. 7(3), 199–206 (2017)

    Article  Google Scholar 

  31. T. Fan, Z. Mu, R. Yang, Multi-modality recognition of human face and ear based on deep learning, in International Conference on Wavelet Analysis and Pattern Recognition (Jul 2017), pp. 38–42

    Google Scholar 

  32. P.L. Galdámez, W. Raveane, A. González Arrieta, A brief review of the ear recognition process using deep neural networks. J. Appl. Log. 24, 62–70 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  33. Y. Guo, Z. Xu, Ear recognition using a new local matching approach, in International Conference on Image Processing (IEEE, Piscataway, 2008), pp. 289–292

    Google Scholar 

  34. E.E. Hansley, M.P. Segundo, S. Sarkar, Employing fusion of learned and handcrafted features for unconstrained ear recognition. IET Biom. 7(3), 215–223 (2018)

    Article  Google Scholar 

  35. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Conference on Computer Vision and Pattern Recognition (IEEE, Piscataway, 2016), pp. 770–778

    Google Scholar 

  36. A.G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. Adam, MobileNets: efficient convolutional neural networks for mobile vision applications, CoRR abs/1704.04861 (2017)

    Google Scholar 

  37. D.J. Hurley, M.S. Nixon, J.N. Carter, Automatic ear recognition by force field transformations, in Colloquium on Visual Biometrics (IET, London, 2000), pp. 7/1–5

    Google Scholar 

  38. A. Jain, A. Ross, K. Nandakumar, Introduction to Biometrics (Springer, Berlin, 2011)

    Book  Google Scholar 

  39. U. Kacar, M. Kirci, Ear recognition with score-level fusion based on CMC in long-wave infrared spectrum, CoRR abs/1801.09054 (2018)

    Google Scholar 

  40. J. Kannala, E. Rahtu, BSIF: binarized statistical image features, in International Conference on Pattern Recognition (IEEE, Piscataway, 2012), pp. 1363–1366

    Google Scholar 

  41. I. Kemelmacher-Shlizerman, S.M. Seitz, D. Miller, E. Brossard, The MegaFace benchmark: 1 million faces for recognition at scale, in Conference on Computer Vision and Pattern Recognition, pp. 4873–4882 (IEEE, Piscataway, 2016)

    Google Scholar 

  42. J. Križaj, V. Štruc, N. Pavešić, Adaptation of SIFT features for robust face recognition, in Image Analysis and Recognition (Springer, Berlin, 2010), pp. 394–404

    Google Scholar 

  43. D. Laptev, N. Savinov, J.M. Buhmann, M. Pollefeys, TI-POOLING: transformation-invariant pooling for feature learning in convolutional neural networks, in Conference on Computer Vision and Pattern Recognition (IEEE, Piscataway, 2016), pp. 289–297

    Google Scholar 

  44. D.G. Lowe, Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  45. S. Mahto, T. Arakawa, T. Koshinak, Ear acoustic biometrics using inaudible signals and its application to continuous user authentication, in European Signal Processing Conference (Sep 2018), pp. 1407–1411

    Google Scholar 

  46. B. Moreno, A. Sánchez, J. F. Vélez, On the use of outerear images for personal identification in security applications, in: Proceedings of the International Carnahan Conference onSecurity Technology, IEEE, 1999, pp. 469–476.

    Google Scholar 

  47. Z. Mu, L. Yuan, Z. Xu, D. Xi, S. Qi, Shape and structural feature based ear recognition, in Advances in Biometric Person Authentication (Springer, Berlin, pp. 663–670, 2004)

    Chapter  Google Scholar 

  48. I. Naseem, R. Togneri, M. Bennamoun, Sparse representation for ear biometrics, in Advances in Visual Computing (Springer, Berlin, 2008), pp. 336–345

    Book  Google Scholar 

  49. H. Nejati, L. Zhang, T. Sim, E. Martinez-Marroquin, G. Dong, Wonder ears: identification of identical twins from ear images, in International Conference on Pattern Recognition (IEEE, Piscataway, 2012), pp. 1201–1204

    Google Scholar 

  50. T. Ojala, M. Pietikainen, D. Harwood, Performance evaluation of texture measures with classification based on Kullback discrimination of distributions, in International Conference on Pattern Recognition, Computer Vision & Image Processing, vol. 1 (IEEE/IAPR, Piscataway, 1994), pp. 582–585

    Google Scholar 

  51. V. Ojansivu, J. Heikkilä, Blur insensitive texture classification using local phase quantization, in Image and Signal Processing (Springer, Berlin, 2008), pp. 236–243

    Google Scholar 

  52. V. Ojansivu, E. Rahtu, J. Heikkilä, Rotation invariant local phase quantization for blur insensitive texture analysis, in International Conference on Pattern Recognition (IEEE, Piscataway, 2008), pp. 1–4

    Google Scholar 

  53. I. Omara, G. Xiao, M. Amrani, Z. Yan, W. Zuo, Deep features for efficient multi-biometric recognition with face and ear images, in International Conference on Digital Image Processing, vol. 10420 (International Society for Optics and Photonics, Bellingham, Jul 2017), pp. 1–6

    Google Scholar 

  54. I. Omara, X. Wu, H. Zhang, Y. Du, W. Zuo, Learning pairwise SVM on hierarchical deep features for ear recognition. IET Biom. 7(6), 557–566 (2018)

    Article  Google Scholar 

  55. R.N. Othman, F. Alizadeh, A. Sutherland, A novel approach for occluded ear recognition based on shape context, in International Conference on Advanced Science and Engineering (IEEE, Piscataway, Oct 2018), pp. 93–98

    Google Scholar 

  56. A. Pflug, C. Busch, Ear biometrics: a survey of detection, feature extraction and recognition methods. IET Biom. 1(2), 114–129 (2012)

    Article  Google Scholar 

  57. A. Pflug, P.N. Paul, C. Busch, A comparative study on texture and surface descriptors for ear biometrics, in International Carnahan Conference on Security Technology (IEEE, Piscataway, 2014), pp. 1–6

    Google Scholar 

  58. M. Pietikäinen, A. Hadid, G. Zhao, T. Ahonen, Computer vision using local binary patterns, in Computational Imaging and Vision (Springer, Berlin, 2011)

    Book  Google Scholar 

  59. S. Prakash, P. Gupta, An efficient ear recognition technique invariant to illumination and pose. Telecommun. Syst. 52(3), 1435–1448 (2013)

    Article  Google Scholar 

  60. M. Rahman, M.R. Islam, N.I. Bhuiyan, B. Ahmed, A. Islam, Person identification using ear biometrics. Int. J. Comput. Intern. Manag. 15(2), 1–8 (2007)

    Google Scholar 

  61. M. Ribič, Ž. Emeršič, V. Štruc, P. Peer, Influence of alignment on ear recognition: case study on AWE dataset, in International Electrotechnical and Computer Science Conference, vol. 25-B (IEEE, Piscataway, 2016), pp. 131–134

    Google Scholar 

  62. U. Saeed, M.M. Khan, Combining ear-based traditional and soft biometrics for unconstrained ear recognition. J. Electr. Imaging 27(5), 051220/1–10 (2018)

    Article  MathSciNet  Google Scholar 

  63. A. Sepas-Moghaddam, F. Pereira, P.L. Correia, Ear recognition in a light field imaging framework: a new perspective. IET Biom. 7(3), 224–231 (2018)

    Article  Google Scholar 

  64. K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)

    Google Scholar 

  65. H. Sinha, R. Manekar, Y. Sinha, P.K. Ajmera, Convolutional neural network-based human identification using outer ear images, in Soft Computing for Problem Solving, ed. by J.C. Bansal, K.N. Das, A. Nagar, K. Deep, A.K. Ojha. Advances in Intelligent Systems and Computing (Springer, Singapore, 2019), pp. 707–719

    Google Scholar 

  66. T. Theoharis, G. Passalis, G. Toderici, I.A. Kakadiaris, Unified 3D face and ear recognition using wavelets on geometry images. Pattern Recogn. 41(3), 796–804 (2008)

    Article  MATH  Google Scholar 

  67. L. Tian, Z. Mu, Ear recognition based on deep convolutional network, in International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (IEEE, Piscataway, Oct 2016), pp. 437–441

    Google Scholar 

  68. B. Victor, K. Bowyer, S. Sarkar, An evaluation of face and ear biometrics, in International Conference on Pattern Recognition, vol. 1 (IEEE, Piscataway, 2002), pp. 429–432

    Google Scholar 

  69. R. Vidyasri, B. Priyalakshmi, M.R. Raja, S. Priyanka, Recognition of ear based on partial features fusion. Int. J. Biom. 10(2), 105–120 (2018)

    Google Scholar 

  70. N.S. Vu, A. Caplier, Face recognition with patterns of oriented edge magnitudes, in European Conference on Computer Vision (Springer, Berlin, 2010), pp. 313–326

    Google Scholar 

  71. Y. Wen, K. Zhang, Z. Li, Y. Qiao, A discriminative feature learning approach for deep face recognition, in European Conference on Computer Vision (Springer, Berlin, 2016), pp. 499–515

    Google Scholar 

  72. Z. Xie, Z. Mu, Ear recognition using LLE and IDLLE algorithm, in International Conference on Pattern Recognition (IEEE, Piscataway, 2008), pp. 1–4

    Google Scholar 

  73. X. Xu, Z. Mu, L. Yuan, Feature-level fusion method based on KFDA for multimodal recognition fusing ear and profile face, in International Conference on Wavelet Analysis and Pattern Recognition (IEEE, Piscataway, 2007), pp. 1306–1310

    Google Scholar 

  74. T. Ying, W. Shining, L. Wanxiang, Human ear recognition based on deep convolutional neural network, in Chinese Control And Decision Conference (Jun 2018), pp. 1830–1835

    Google Scholar 

  75. L. Yuan, Z.C. Mu, Ear recognition based on 2D images, in Conference on Biometrics: Theory, Applications and Systems (IEEE, Piscataway, 2007), pp. 1–5

    Google Scholar 

  76. L. Yuan, H. Zhao, Y. Zhang, Z. Wu, Ear alignment based on convolutional neural network, in Biometric Recognition, ed. by J. Zhou, Y. Wang, Z. Sun, Z. Jia, J. Feng, S. Shan, K. Ubul, Z. Guo, pp. 562–571. Lecture Notes in Computer Science (Springer, Berlin, 2018)

    Chapter  Google Scholar 

  77. H.J. Zhang, Z.C. Mu, W. Qu, L.M. Liu, C.Y. Zhang, A novel approach for ear recognition based on ICA and RBF network, in International Conference on Machine Learning and Cybernetics, vol. 7 (IEEE, Piscataway, 2005), pp. 4511–4515

    Google Scholar 

  78. Y. Zhang, Z. Mu, L. Yuan, H. Zeng, L. Chen, 3D ear normalization and recognition based on local surface variation. Appl. Sci. 7(1), 104/1–21 (2017)

    Google Scholar 

  79. Y. Zhang, Z. Mu, L. Yuan, C. Yu, Ear verification under uncontrolled conditions with convolutional neural networks. IET Biom. 7(3), 185–198 (2018)

    Article  Google Scholar 

  80. Y. Zhou, S. Zaferiou, Deformable models of ears in-the-wild for alignment and recognition, in International Conference on Automatic Face and Gesture Recognition (IEEE, Piscataway, May 2017), pp. 626–633

    Google Scholar 

Download references

Acknowledgements

This research was supported in parts by the ARRS (Slovenian Research Agency) Research Program P2-0250 (B) Meteorology and Biometric Systems, the ARRS Research Program P2-0214 (A) Computer Vision. The authors thank NVIDIA for donating the Titan Xp GPU that was used in the experiments.

This work was also partially supported by the European Commission through the Horizon 2020 research and innovation program under grants 688201 (M2DC) and 690907 (IDENTITY).

Author information

Author notes

    Authors and Affiliations

    1. XLAB d.o.o., Ljubljana, Slovenia

      Dejan Štepec

    2. Computer Vision Laboratory, Faculty of Computer and Information Science, University of Ljubljana, Ljubljana, Slovenia

      Žiga Emeršič & Peter Peer

    3. Laboratory of Artificial Perception, Systems and Cybernetics, Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia

      Vitomir Štruc

    Authors
    1. Dejan Štepec
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Žiga Emeršič
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Peter Peer
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Vitomir Štruc
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Žiga Emeršič .

    Editor information

    Editors and Affiliations

    1. School of Computing and Communication, InfoLab21, Lancaster University, Lancaster, UK

      Richard Jiang

    2. School of Information Technology, Deakin University, Waurn Ponds, VIC, Australia

      Chang-Tsun Li

    3. Electrical Engineering and Computer Science, School of Electronics, Queen’s University Belfast, Belfast, UK

      Danny Crookes

    4. Technical University of Denmark, Kgs. Lyngby, Denmark

      Weizhi Meng

    5. GREYC research Lab, ENSICAEN, Caen, France

      Christophe Rosenberger

    Rights and permissions

    Reprints and permissions

    Copyright information

    © 2020 Springer Nature Switzerland AG

    About this chapter

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this chapter

    Štepec, D., Emeršič, Ž., Peer, P., Štruc, V. (2020). Constellation-Based Deep Ear Recognition. In: Jiang, R., Li, CT., Crookes, D., Meng, W., Rosenberger, C. (eds) Deep Biometrics. Unsupervised and Semi-Supervised Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-32583-1_8

    Download citation

    • DOI: https://doi.org/10.1007/978-3-030-32583-1_8

    • Published:

    • Publisher Name: Springer, Cham

    • Print ISBN: 978-3-030-32582-4

    • Online ISBN: 978-3-030-32583-1

    • eBook Packages: EngineeringEngineering (R0)

    Publish with us

    Policies and ethics

    Search

    Navigation