Skip to main content

Advertisement

SpringerLink
Log in

A novel deep learning system for facial feature extraction by fusing CNN and MB-LBP and using enhanced loss function

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Extensive use of surveillance cameras for human tracking and observation have been fostering the research on face recognition technique for individual identification in an unconstrained environment. However, face recognition is a challenging task in an unconstrained environment, where the captured images are affected by illumination effect, varying poses, noise and occlusion. The main objective of this research is to improve the accuracy and processing time in extracting facial features by using the fusion of deep learning and handcrafted architecture for recognizing individuals in unconstrained conditions, thereby providing accurate information about the individuals to security systems. The proposed system consists of Multi-Block Local Binary Pattern (MB-LBP) modules for extracting the handcrafted features and Convolutional Neural Network (CNN) for extracting the high-level distinctive features. The features from both modules are fused and passed through fully connected layer with Softmax classifier to identify individuals. The results show that the enhanced algorithm based on Softmax loss function aided classifier with regularization improves the accuracy and processing time for face recognition. The proposed model improves accuracy by 94.37% against 90.01% for the state-of-the-art solution. In addition to that, it improves the processing time of 307 ms against 357 ms. The proposed system focuses on fusing hand-crafted and deep learned features to extract face features accurately and thus improving the accuracy and overall performance of the proposed system in an unconstrained environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2.
Fig 3.
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Alghamdi A, Mohamed H, Hassan U, Abdel-Raheem A, Khan M, Hany S.K, Abd El-Latif AA (2020)Detection of myocardial infarction based on novel deep transfer learning methods for urban healthcare in smart cities. Multimed Tools Appl 1–22

  2. Al-Waisy A, Qahwaji R, Ipson S, Al-Fahdawi S (2017) A multimodal deep learning framework using local feature representations for face recognition. Mach Vis Appl 29(1):35–54. https://doi.org/10.1007/s00138-017-0870-2

    Article  Google Scholar 

  3. Bong K, Choi S, Kim C, Han D, Yoo H (2018) A low-power convolutional neural network face recognition processor and a CIS integrated with always-on face detector. IEEE J Solid State Circuits 53(1):115–123. https://doi.org/10.1109/jssc.2017.2767705

    Article  Google Scholar 

  4. Chaudhry S, Chandra R (2017) Face detection and recognition in an unconstrained environment for mobile visual assistive system. Appl Soft Comput 53:168–180. https://doi.org/10.1016/j.asoc.2016.12.035

    Article  Google Scholar 

  5. Chen G, Shao Y, Tang C, Jin Z, Zhang J (2018) Deep transformation learning for face recognition in the unconstrained scene. Mach Vis Appl 29(3):513–523. https://doi.org/10.1007/s00138-018-0907-1

    Article  Google Scholar 

  6. Ding C, Tao D (2018) Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans Pattern Anal Mach Intell 40(4):1002–1014. https://doi.org/10.1109/tpami.2017.2700390

    Article  Google Scholar 

  7. Ding Y, Cheng Y, Cheng X, Li B, You X, Yuan X (2017) Noise-resistant network: a deep-learning method for face recognition under noise. EURASIP Journal on Image and Video Processing 2017(1). https://doi.org/10.1186/s13640-017-0188-z

  8. El-Rahiem BA, Ahmed MAO, Reyad O, El-Rahaman HA, Amin M, El-Samie FA (2019) An efficient deep convolutional neural network for visual image classification. The International Conference on Advanced Machine Learning Technologies and Applications (AMLTA)

  9. Hu H, Li Y, Zhu Z, Zhou G (2018), October. CNNAuth: continuous authentication via two-stream convolutional neural networks. In 2018 IEEE international conference on networking, architecture and storage (NAS) (pp. 1-9). IEEE

  10. Kamencay P, Benco M, Mizdos T, Radil R (2017) A new method for face recognition using convolutional neural network. Advances in Electrical and Electronic Engineering 15(4). https://doi.org/10.15598/aeee.v15i4.2389

  11. Leng B, Liu Y, Yu K, Xu S, Yuan Z, Qin J (2016) Cascade shallow CNN structure for face verification and identification. Neurocomputing 215:232–240. https://doi.org/10.1016/j.neucom.2015.08.134

    Article  Google Scholar 

  12. Leng L, Li M, Kim C, Bi X (2017) Dual-source discrimination power analysis for multi-instance contactless palmprint recognition. Multimed Tools Appl 76, 333–354 (2017)

  13. Li J, Qiu T, Wen C, Xie K, Wen F (2018) Robust face recognition using the deep C2D-CNN model based on decision-level fusion. Sensors 18(7):2080. https://doi.org/10.3390/s18072080

    Article  Google Scholar 

  14. Li Y, Lu Z, Li J, Deng Y (2018) Improving deep learning feature with facial texture feature for face recognition. Wirel Pers Commun 103:1195–1206. https://doi.org/10.1007/s11277-018-5377-2

    Article  Google Scholar 

  15. Li Y, Hu H, Zhu Z, Zhou G (n.d.) SCANet: sensor-based continuous authentication with two-stream convolutional neural networks. ACM Transactions on Sensor Networks (TOSN)

  16. Liu J, Fang C, Wu C (2016) A fusion face recognition approach based on 7-layer deep learning neural network. Journal of Electrical and Computer Engineering 2016:1–7. https://doi.org/10.1155/2016/8637260

    Article  Google Scholar 

  17. Moon H, Seo C, Pan S (2016) A face recognition system based on convolution neural network using multiple distance face. Soft Comput 21(17):4995–5002. https://doi.org/10.1007/s00500-016-2095-0T

    Article  Google Scholar 

  18. Nielsen MA (2015) Neural networks and deep learning. San Francisco, CA, USA, Determination press

  19. Peng J, Li Q, Abd El-Latif AA, Wang N, Niu X. (2013) Finger vein recognition with gabor wavelets and local binary patterns. IEICE TRANSACTIONS on Information and Systems, 96(8) (2013), pp.1886–1889

  20. Tripathi B (2017) On the complex domain deep machine learning for face recognition. Appl Intell 47(2):382–396. https://doi.org/10.1007/s10489-017-0902-7

    Article  Google Scholar 

  21. Wang N, Qiong L, Ahmed A, Xuehu Y, Xiamu N (2013) A novel hybrid multibiometrics based on the fusion of dual iris, visible and thermal face images. In 2013 International Symposium on Biometrics and Security Technologies, pp. 217–223. IEEE

  22. Wang N, Qiong L, Abd El-Latif AA, Jialiang P, Xiamu N (2014) An enhanced thermal face recognition method based on multiscale complex fusion for Gabor coefficients. Multimed Tools Appl 72(3):2339–2358

    Article  Google Scholar 

  23. Zhao J, Lv Y, Zhou Z, Cao F (2017) A novel deep learning algorithm for incomplete face recognition: low-rank-recovery network. Neural Netw 94:115–124. https://doi.org/10.1016/j.neunet.2017.06.013

    Article  Google Scholar 

Download references

Funding

This research was not externally funded.

Author information

Authors and Affiliations

  1. School of Computing and Mathematics, Charles Sturt University, Sydney Campus, Bathurst, Australia

    Raj Silwal, Abeer Alsadoon & P. W. C. Prasad

  2. Department of Islamic Sciences, Al Iraqia University, Baghdad, Iraq

    Omar Hisham Alsadoon

  3. Sheridan College, Oakville, Ontario, Canada

    Ammar Al-Qaraghuli

Authors
  1. Raj Silwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abeer Alsadoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. W. C. Prasad
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Omar Hisham Alsadoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ammar Al-Qaraghuli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abeer Alsadoon.

Ethics declarations

Conflict of interest

No conflict of interest is expected in this research

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 4 Table of Abbreviations
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Silwal, R., Alsadoon, A., Prasad, P.W.C. et al. A novel deep learning system for facial feature extraction by fusing CNN and MB-LBP and using enhanced loss function. Multimed Tools Appl 79, 31027–31047 (2020). https://doi.org/10.1007/s11042-020-09559-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09559-1

Keywords

Search

Navigation