Skip to main content
SpringerLink
Log in

Multi-task aided face recognition network with convolution kernel spatial collaboration

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Most face recognition networks based on convolutional neural networks are easily affected by nonlinear factors of expression and posture, and the single task cannot adapt to multi-task scenarios. In addition, there is still room for further utilization of convolution kernels. Therefore, this paper proposes a multi-task aided face recognition network with convolution kernel spatial collaboration. The network is based on GhostNet, and the auxiliary task branch of expression and posture is added, which reduces the influence of nonlinear factors of expression and posture on face recognition, and enables the network to recognize expression and posture so that the network can adapt to the situation of more tasks. On this basis, the convolution of convolution is improved to increase its generality on grouped convolution, and it is applied to the cheap operation of Ghost Module so that the network can further use the convolution kernel to learn additional feature maps. Finally, a multi-task feature fusion module is proposed, which combines the shared features extracted from the backbone network with the features of the branch network, and further utilizes the features between different tasks to improve the performance of each task. The proposed method was compared with the existing methods based on deep learning on different datasets. The experimental results show that the proposed method has better recognition performance in face recognition, and the model size and inference speed are close to the lightweight network, which is suitable for more tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The dataset in this paper comes from [26, 27]:

References

  1. He, K., Zhang, X., Ren, S. et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. Las Vegas (2016)

  2. Xiangyu, Z., Xinyu, Z., Mengxia, O.L., et al.: ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the lEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856. NewYork: IEEE (2018)

  3. Guo, Y., Abudiriyimu, A., Yadik, N., et al.: Multi-national face classification and recognition based on MobileNet network. J. Image Signal Process. 9(3), 146–155 (2020)

    Article  Google Scholar 

  4. Han, K., Wang, Y., Tian, Q. et al.: Ghostnet: More features from cheap operations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1580–1589, Seattle (2020)

  5. Chen, S., Liu, Y., Gao, X. et al.: Mobilefacenets: efficient CNNS for accurate real-time face verification on mobile devices. In: Chinese Conference on Biometric Recognition, pp. 428–438, Cham (2018)

  6. Martindez-Diaz, Y., Luevano, L.S., Mendez-Vazquez, H. et al.: Shufflefacenet: a lightweight face architecture for efficient and highly-accurate face recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul (2019)

  7. Cao, Q., Shen, L., Xie, W. et al.: Vggface2: a dataset for recognising faces across pose and age. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition(FG 2018), pp. 67–74, Xi'an (2018)

  8. Yan, M., Zhao, M., Xu, Z. et al.: Vargfacenet: an efficient variable group convolutional neural network for lightweight face recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul (2019)

  9. Wang, Q., Guo, G.: LS-CNN: characterizing local patches at multiple scales for face recognition. IEEE Trans. Inf. Forensics Secur.15, 1640–1653 (2020)

    Article  Google Scholar 

  10. Tamilselvi, M., Karthikeyan, S.: An ingenious face recognition system based on HRPSM_CNN under unrestrained environmental condition. Alex. Eng. J. 61(6), 6 (2022)

    Article  Google Scholar 

  11. He, L., He, L.L., Peng, L.J.: CFormerFaceNet: efficient lightweight network merging a CNN and transformer for face recognition. Appl. Sci. Basel 13(11), 5 (2023)

    Google Scholar 

  12. Nam, G.P., Choi, H., et al.: PSI-CNN: a pyramid-based scale-invariant CNN architecture for face recognition robust to various image resolutions. Appl. Sci. Basel 8(9), 9 (2018)

    Google Scholar 

  13. Long, Y.X.: A face recognition algorithm based on intermediate layers connected by the CNN. J. Circuits Syst. Comput. 31(06), 2250107 (2022)

    Article  Google Scholar 

  14. Xie, Z.H., Niu, J.Y., et al.: Regularization and attention feature distillation base on light CNN for hyperspectral face recognition. Multimed. Tools Appl. 81(14), 6 (2022)

    Article  Google Scholar 

  15. Wang, H., Wang, Y.T., Zhou, Z., Ji, X. et al.: CosFace: large margin cosine loss for deep face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5265–5274. Salt Lake City, USA (2018)

  16. Deng, J., Guo, J., Zafeiriou, S.: ArcF ace: additive angular margin Loss for deep face recognition. In: Proceedings of 2019 IEEE/CVF Conference onComputer Vision and Pattern Recognition, pp. 4685–4694. Long Beach, USAIEEE (2019)

  17. Zhao R.Z. et al.: Convolution of convolution: let kernels spatially collaborate. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 651–660. Vancouver Canada (2023)

  18. Ghali, A., Chouraqui, S., et al.: Few-shot learning CNN optimized using combined 2D-DWT injection and evolutionary optimization algorithms for human face recognition. Int. J. Wavelets Multiresolut. Inf. Process. (2023). https://doi.org/10.1142/S0219691323500248

    Article  Google Scholar 

  19. Khalifa, A., Al-Hamadi, A.: JAMsFace: joint adaptive margins loss for deep face recognition. Neural Comput. Appl. 35(26), 9 (2023)

    Article  Google Scholar 

  20. Ardekani, P.F., Tale, S.Z., Parseh, M.J.: Face mask recognition using a custom CNN and data augmentation. Signal Image Video Process. 18, 9 (2023)

    Google Scholar 

  21. Pasquale, F., Antonio, G., Alessia, S., Mario, V.: Multi-task learning on the edge for effective gender, age, ethnicity and emotion recognition. Eng. Appl. Artif. Intell. 118 (2022)

  22. Yu, W.M., Xu, H.: Co-attentive multi-task convolutional neural network for facial expression recognition. Pattern Recognit. 123, 108401 (2022)

    Article  Google Scholar 

  23. Mao, L.B., Yan, Y., Xue, J.H., Wang, H.Z.: Deep multi-task multi-label CNN for effective facial attribute classification. IEEE Trans. Affect. Comput.13(2), 818–828 (2022)

    Article  Google Scholar 

  24. Ranjan, R., Patel, V.M., Chellappa, R.: HyperFace: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans. Pattern Anal. Mach. Intell. 41(1), 121–135 (2018)

    Article  Google Scholar 

  25. Liu, Y.F., Chen, J.H., Qiu, Y.: Joint multi-patch and multi-task CNNs for robust face recognition. IEICE Trans. Inf. Syst. E103D(10), 2178–2187 (2020)

    Article  Google Scholar 

  26. Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1615–1618 (2003)

    Article  Google Scholar 

  27. Gao, W., Cao, B., Shan, S.G., et al.: The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans. Syst. Man Cybern.38(1), 149–161 (2008)

    Article  Google Scholar 

  28. Howard, A., Sandler, M., Chu, G., Chen, L.C. et al.: Searching for MobileNetV3. In: International Conference on Computer Vision, Seoul, Korea, (2019)

  29. Ding, X.H., Zhang, X.Y., Ma, N.N. et al.: RepVGG: making VGG-style convnets great again. In: IEEE Conference on Computer Vision and Pattern Recognition. Online, (2021)

  30. Tang, Y.H., Han, K., Guo, J.Y., Xu, C., Xu, G., Wang, Y.H.: GhostNetV2: enhance cheap operation with long-range attention. arXiv:2211.12905 (2022)

Download references

Funding

There is no fund for this article.

Author information

Authors and Affiliations

  1. College of Physics and Electronic Engineering, Northwest Normal University, Lanzhou, 730070, China

    Chunman Yan & Zhen Zheng

Authors
  1. Chunman Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhen Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZZ was responsible for the experiment and paper writing, and CY was responsible for the supervision and guidance of the paper throughout.

Corresponding author

Correspondence to Chunman Yan.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Ethical approval

This article is not a human/animal study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, C., Zheng, Z. Multi-task aided face recognition network with convolution kernel spatial collaboration. SIViP 18, 3361–3372 (2024). https://doi.org/10.1007/s11760-024-02999-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-024-02999-4

Keywords

Search

Navigation