Skip to main content
Log in

Average biased ReLU based CNN descriptor for improved face retrieval

  • 1154T: Content-Based Multimedia Indexing in the era of Artificial Intelligence
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The convolutional neural networks (CNN), including AlexNet, GoogleNet, VGGNet, etc. extract features for many computer vision problems which are very discriminative. The trained CNN model over one dataset performs reasonably well whereas on another dataset of similar type the hand-designed feature descriptor outperforms the same trained CNN model. The Rectified Linear Unit (ReLU) layer discards some values in order to introduce the non-linearity. In this paper, it is proposed that the discriminative ability of deep image representation using trained model can be improved by Average Biased ReLU (AB-ReLU) at the last few layers. Basically, AB-ReLU improves the discriminative ability in two ways: 1) it exploits some of the discriminative and discarded negative information of ReLU and 2) it also neglects the irrelevant and positive information used in ReLU. The VGGFace model trained in MatConvNet over the VGG-Face dataset is used as the feature descriptor for face retrieval over other face datasets. The proposed approach is tested over six challenging, unconstrained and robust face datasets (PubFig, LFW, PaSC, AR, FERET and ExtYale) and also on a large scale face dataset (PolyUNIR) in retrieval framework. It is observed that the AB-ReLU outperforms the ReLU when used with a pre-trained VGGFace model over the face datasets. The validation error by training the network after replacing all ReLUs with AB-ReLUs is also observed to be favorable over each dataset. The AB-ReLU even outperforms the state-of-the-art activation functions, such as Sigmoid, ReLU, Leaky ReLU and Flexible ReLU over all seven face datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://www.robots.ox.ac.uk/~vgg/software/vgg_face/

  2. http://www.robots.ox.ac.uk/~vgg/data/vgg_face/

  3. http://www.comp.polyu.edu.hk/~biometrics/NIRFace/polyudb_face.htm

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  Google Scholar 

  2. Bansal A, Castillo C, Ranjan R, Chellappa R (2017) The do’s and don’ts for cnn-based face verification. arXiv:1705.07426

  3. Beveridge JR, Phillips PJ, Bolme DS, Draper BA, Givens GH, Lui YM, Teli MN, Zhang H, Scruggs WT, Bowyer KW et al (2013) The challenge of face recognition from digital point-and-shoot cameras. In: 2013 IEEE sixth international conference on biometrics: theory, applications and systems (BTAS). IEEE, pp 1–8

  4. Chakraborty S, Singh S, Chakraborty P (2016) Local gradient hexa pattern: A descriptor for face recognition and retrieval. IEEE Trans Circuits Systems Video Technol

  5. Chakraborty S, Singh SK, Chakraborty P (2017) Centre symmetric quadruple pattern: A novel descriptor for facial image recognition and retrieval. Pattern Recognition Letters

  6. Chakraborty S, Singh SK, Chakraborty P (2017) Local directional gradient pattern: a local descriptor for face recognition. Multimed Tools Appl 76 (1):1201–1216

    Article  Google Scholar 

  7. Clevert DA, Mayr A, Unterthiner T, Hochreiter S (2015) Rectified factor networks. In: Advances in neural information processing systems, pp 1855–1863

  8. Clevert DA, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (elus). arXiv:1511.072891511.07289

  9. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009. IEEE, pp 248–255

  10. Dubey SR (2019) Face retrieval using frequency decoded local descriptor. Multimed Tools Appl 78(12):16411–16431

    Article  Google Scholar 

  11. Dubey SR (2019) Local directional relation pattern for unconstrained and robust face retrieval. Multimed Tools Appl

  12. Dubey SR, Mukherjee S (2018) Ldop: Local directional order pattern for robust face retrieval. arXiv:1803.07441

  13. Dubey SR, Singh SK, Singh RK (2014) Rotation and illumination invariant interleaved intensity order-based local descriptor. IEEE Trans Image Process 23(12):5323–5333

    Article  MathSciNet  Google Scholar 

  14. Dubey SR, Singh SK, Singh RK (2015) Local diagonal extrema pattern: a new and efficient feature descriptor for ct image retrieval. IEEE Signal Processing Letters 22(9):1215–1219

    Article  Google Scholar 

  15. Dubey SR, Singh SK, Singh RK (2015) Local wavelet pattern: A new feature descriptor for image retrieval in medical ct databases. IEEE Trans Image Process 24(12):5892–5903

    Article  MathSciNet  Google Scholar 

  16. Dubey SR, Singh SK, Singh RK (2016) Local bit-plane decoded pattern: a novel feature descriptor for biomedical image retrieval. IEEE Journal of Biomedical and Health Informatics 20(4):1139–1147

    Article  Google Scholar 

  17. Dubey SR, Singh SK, Singh RK (2016) Multichannel decoded local binary patterns for content-based image retrieval. IEEE Trans Image Process 25 (9):4018–4032

    Article  MathSciNet  Google Scholar 

  18. Ge Y, Jiang S, Xu Q, Jiang C, Ye F (2018) Exploiting representations from pre-trained convolutional neural networks for high-resolution remote sensing image retrieval. Multimed Tools Appl 1–27

  19. Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Article  Google Scholar 

  20. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp. 1026–1034

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  22. Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Tech. rep. Technical Report, vol 07-49. University of Massachusetts, Amherst

    Google Scholar 

  23. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105

  24. Kumar N, Berg AC, Belhumeur PN, Nayar SK (2009) Attribute and simile classifiers for face verification. In: Computer Vision, 2009 IEEE 12th International Conference on, pp. 365–372. IEEE

  25. Lee KC, Ho J, Kriegman DJ (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on pattern analysis and machine intelligence 27(5):684–698

    Article  Google Scholar 

  26. Li Y, Wan L, Fu T, Hu W (2019) Piecewise supervised deep hashing for image retrieval. Multimed Tools Appl 1–21

  27. Liu P, Guo JM, Wu CY, Cai D (2017) Fusion of deep learning and compressed domain features for content-based image retrieval. IEEE Trans Image Process 26(12):5706–5717

    Article  MathSciNet  Google Scholar 

  28. Ma X, Jiang X (2019) Multimedia image quality assessment based on deep feature extraction. Multimed Tools Appl, 1–12

  29. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proc. ICML, vol 30

  30. Martinez AM (1998) The ar face database. CVC technical report

  31. Martínez AM, Kak AC (2001) Pca versus lda. IEEE Trans Pattern Anal Mach Intell 23(2):228–233

    Article  Google Scholar 

  32. Murala S, Maheshwari R, Balasubramanian R (2012) Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21(5):2874–2886

    Article  MathSciNet  Google Scholar 

  33. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814

  34. Parkhi OM, Vedaldi A, Zisserman A, et al. (2015) Deep face recognition. In: BMVC, vol 1, p 6

  35. Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The feret evaluation methodology for face-recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104

    Article  Google Scholar 

  36. Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The feret database and evaluation procedure for face-recognition algorithms. Image And Vision Computing 16(5):295–306

    Article  Google Scholar 

  37. Qiu S, Xu X, Cai B (2018) Frelu: Flexible rectified linear units for improving convolutional neural networks. In: 2018 24th International conference on pattern recognition (ICPR). IEEE, pp 1223–1228

  38. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823

  39. Shamsolmoali P, Jain DK, Zareapoor M, Yang J, Alam MA (2019) High-dimensional multimedia classification using deep cnn and extended residual units. Multimed Tools Appl 78(17):23867–23882

    Article  Google Scholar 

  40. Sharma S, Dubey SR, Singh SK, Saxena R, Singh RK (2015) Identity verification using shape and geometry of human hands. Expert Syst Appl 42(2):821–832

    Article  Google Scholar 

  41. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  42. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  43. Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: Closing the gap to human-level performance in face verification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1701–1708

  44. Tzeng E, Hoffman J, Darrell T, Saenko K (2015) Simultaneous deep transfer across domains and tasks. In: Proceedings of the IEEE international conference on computer vision, pp 4068–4076

  45. Vedaldi A, Lenc K (2015) Matconvnet: Convolutional neural networks for matlab. In: Proceedings of the 23rd ACM international conference on multimedia. ACM, pp 689–692

  46. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001. CVPR 2001, vol 1. IEEE, pp I–I

  47. Wan J, Wang D, Hoi SCH, Wu P, Zhu J, Zhang Y, Li J (2014) Deep learning for content-based image retrieval: A comprehensive study. In: Proceedings of the 22nd ACM international conference on Multimedia. ACM, pp 157–166

  48. Wang Y, Wang G, Chen C, Pan Z (2019) Multi-scale dilated convolution of convolutional neural network for image denoising. Multimed Tools Appl 1–16

  49. Wen Y, Zhang K, Li Z, Qiao Y (2016) A discriminative feature learning approach for deep face recognition. In: European conference on computer vision. Springer, pp 499–515

  50. Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv:1505.00853

  51. Zhang B, Zhang L, Zhang D, Shen L (2010) Directional binary code with application to polyu near-infrared face database. Pattern Recogn Lett 31 (14):2337–2344

    Article  Google Scholar 

  52. Zhou H, Li Z (2019) Deep networks with non-static activation function. Multimed Tools Appl 78(1):197–211

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This research is funded by IIIT Sri City, India through the Faculty Seed Research Grant. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the GeForce Titan X Pascal used for this research.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shiv Ram Dubey.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dubey, S.R., Chakraborty, S. Average biased ReLU based CNN descriptor for improved face retrieval. Multimed Tools Appl 80, 23181–23206 (2021). https://doi.org/10.1007/s11042-020-10269-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10269-x

Keywords

Navigation