Skip to main content
SpringerLink
Log in

A lightweight CNN-based algorithm and implementation on embedded system for real-time face recognition

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Deep learning has become the main solution for face recognition applications due to its high accuracy and robustness. In recent years, a batch of research on lightweight convolutional neural networks (CNNs) has emerged, bringing new ideas to the economic application of face recognition systems. In this paper, a lightweight face recognition algorithm is proposed to reduce the number of parameters and calculations of the face feature extraction network. The most important part of our approach lies in designing a novel inverted residual shuffle unit (IR-Shuffle). After being trained by ArcFace loss on a GPU workstation, our model built on improved IR-Shuffle blocks of size 1.45 MB achieves an accuracy of 98.65%. In terms of running time, our model is 5 ms faster than the current fastest MobileFaceNet, with only about 0.5% drop in accuracy. The proposed algorithm is implemented and optimized on the Jetson Nano embedded platform, and accurate and real-time deployment of the face recognition system is realized. The system takes 37 ms to perform the complete face detection and recognition and is robust to complex backgrounds and ambient light changes. Experimental results show that our system is of practical application value.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Bouvrie, J.: Notes on convolutional neural networks (2006)

  2. Krizhevsky, A., Sutskever, I., Hinton GE Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1097–1105 (2012)

  3. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  4. Huang, G.B., Lee, H., Learned-Miller, E.: Learning hierarchical representations for face verification with convolutional deep belief networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2518–2525 (2012)

  5. Howard, AG., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. (2017) arXiv preprint arXiv:1704.04861

  6. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, LC.: Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4510–4520 (2018)

  7. Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: An extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 6848–6856 (2018)

  8. Ma, N., Zhang, X., Zheng, HT., Sun, J.: Shufflenet v2: Practical guidelines for efficient cnn architecture design. In: Proceedings of the European Conference on Computer Vision, pp 116–131 (2018)

  9. Huang, GB., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database forstudying face recognition in unconstrained environments (2008)

  10. Yi, D., Lei, Z., Liao, S., Li, SZ.: Learning face representation from scratch. (2014) arXiv preprint arXiv:1411.7923

  11. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: Closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1701–1708 (2014)

  12. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. (2014) arXiv preprint arXiv:1409.1556

  13. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9 (2015)

  14. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. (2015) arXiv preprint arXiv:1502.03167

  15. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2818–2826 (2016a)

  16. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, inception-resnet and the impact of residual connections on learning. (2016b) arXiv preprint arXiv:1602.07261

  17. Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1891–1898 (2014)

  18. Zhang, Y., Tsang, IW., Luo, Y., Hu, CH., Lu, X., Yu, X.: Copy and paste gan: Face hallucination from shaded thumbnails. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7355–7364 (2020)

  19. Zhang, Y., Tsang, I., Luo, Y., Hu, C., Lu, X., Yu, X.: Recursive copy and paste gan: Face hallucination from shaded thumbnails. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)

  20. Gan, Y., Luo, Y., Yu, X., Zhang, B., Yang, Y.: Vidface: A full-transformer solver for video facehallucination with unaligned tiny snapshots. (2021) arXiv preprint arXiv:2105.14954

  21. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 815–823 (2015)

  22. Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: Advances in Neural Information Processing Systems, pp 1988–1996 (2014)

  23. Sun, Y., Wang, X., Tang, X.: Deeply learned face representations are sparse, selective, and robust. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2892–2900 (2015)

  24. Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: European Conference on Computer Vision, pp 499–515 (2016)

  25. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: Deep hypersphere embedding for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 212–220 (2017)

  26. Wang, F., Cheng, J., Liu, W., Liu, H.: Additive margin softmax for face verification. IEEE Signal Process. Lett. 25(7), 926–930 (2018)

    Article  Google Scholar 

  27. Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., Liu, W.: Cosface: Large margin cosine loss for deep face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5265–5274 (2018b)

  28. Potluri, S., Fasih, A., Vutukuru, LK., Al Machot, F., Kyamakya, K.: CNN based high performance computing for real time image processing on GPU. In: Proceedings of the Joint INDS’11 & ISTET’11, IEEE, pp 1–7 (2011)

  29. Wang, S., Zhang, X., Zhang, Y., Wang, L., Yang, J., Wang, W.: A survey on mobile edge networks: Convergence of computing, caching and communications. IEEE Access 5, 6757–6779 (2017)

    Article  Google Scholar 

  30. Hu, Y.C., Patel, M., Sabella, D., Sprecher, N., Young, V.: Mobile edge computing-a key technology towards 5G. ETSI White Paper 11(11), 1–16 (2015)

    Google Scholar 

  31. Hong, K., Lillethun, D., Ramachandran, U., Ottenwälder, B., Koldehofe, B.: Mobile fog: A programming model for large-scale applications on the internet of things. In: Proceedings of the Second ACM SIGCOMM Workshop on Mobile Cloud Computing, pp 15–20 (2013)

  32. Kim, S., Howe, P., Moreau, T., Alaghi, A., Ceze, L., Sathe, V.S.: Energy-efficient neural network acceleration in the presence of bit-level memory errors. IEEE Trans. Circuits Syst. I Regul. Pap. 65(12), 4285–4298 (2018)

    Article  Google Scholar 

  33. Jo, J., Kim, S., Park, I.C.: Energy-efficient convolution architecture based on rescheduled dataflow. IEEE Trans. Circuits Syst. I Regul. Pap. 65(12), 4196–4207 (2018)

    Article  Google Scholar 

  34. Kim, S., Lee, J., Kang, S., Lee, J., Yoo, H.J.: A power-efficient CNN accelerator with similar feature skipping for face recognition in mobile devices. IEEE Trans. Circuits Syst. I Regul. Pap. 67(4), 1181–1193 (2020)

    Article  Google Scholar 

  35. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034 (2015)

  36. Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4690–4699 (2019)

  37. Lin, TY., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2980–2988 (2017)

  38. Deng, J., Guo, J., Zhou, Y., Yu, J., Kotsia, I., Zafeiriou, S.: Retinaface: Single-stage dense face localisation in the wild. (2019) arXiv preprint arXiv:1905.00641

  39. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch. In: 31st Conference on Neural Information Processing Systems (2017)

  40. Chen, S., Liu, Y., Gao, X., Han, Z.: Mobilefacenets: Efficient CNNs for accurate real-time face verification on mobile devices. In: Chinese Conference on Biometric Recognition, pp 428–438 (2018)

  41. Wu, B., Wan, A., Yue, X., Jin, P., Zhao, S., Golmant, N., Gholaminejad, A., Gonzalez, J., Keutzer, K.: Shift: A zero flop, zero parameter alternative to spatial convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 9127–9135 (2018)

  42. Martindez-Diaz, Y., Luevano, LS., Mendez-Vazquez, H., Nicolas-Diaz, M., Chang, L., Gonzalez-Mendoza, M.: Shufflefacenet: A lightweight face architecture for efficient and highly-accurate face recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp 0–0 (2019)

  43. Lai, L., Suda, N., Chandra, V.: Deep convolutional neural network inference with floating-point weights and fixed-point activations. (2017) arXiv preprint arXiv:1703.03073

  44. Wang, M., Deng, W.: Deep visual domain adaptation: A survey. Neurocomputing 312, 135–153 (2018)

    Article  Google Scholar 

  45. Luo, Y., Liu, P., Guan, T., Yu, J., Yang, Y.: Significance-aware information bottleneck for domain adaptive semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 6778–6787 (2019)

  46. Luo, Y., Liu, P., Zheng, L., Guan, T., Yu, J., Yang, Y.: Category-level adversarial adaptation for semantic segmentation using purified features. IEEE Transactions on Pattern Analysis and Machine Intelligence(2021)

Download references

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Soochow University, Suzhou, 215006, Jiangsu, China

    Zhongyue Chen, Jiangqi Chen, Guangliu Ding & He Huang

Authors
  1. Zhongyue Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiangqi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangliu Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. He Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhongyue Chen.

Ethics declarations

Conflict of interest

This study was supported by Natural Science Research of Jiangsu Higher Education Institutions of China (21KJB510021). The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Communicated by F. Wu.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Zhongyue Chen and Jiangqi Chen contributed equally to this work.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Z., Chen, J., Ding, G. et al. A lightweight CNN-based algorithm and implementation on embedded system for real-time face recognition. Multimedia Systems 29, 129–138 (2023). https://doi.org/10.1007/s00530-022-00973-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-00973-z

Keywords

Search

Navigation