Skip to main content
SpringerLink
Log in

Improved Network for Face Recognition Based on Feature Super Resolution Method

  • Research Article
  • Brain-Inspired Intelligence
  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

Abstract

Low-resolution face images can be found in many practical applications. For example, faces captured from surveillance videos are typically in small sizes. Existing face recognition deep networks, trained on high-resolution images, perform poorly in recognizing low-resolution faces. In this work, an improved multi-branch network is proposed by combining ResNet and feature super-resolution modules. ResNet is for recognizing high-resolution facial images and extracting features from both high- and low-resolution images. Feature super-resolution modules are inserted before the classifier of ResNet for low-resolution facial images. They are used to increase feature resolution. The proposed method is effective and simple. Experimental results show that the recognition accuracy for high-resolution face images is high, and the recognition accuracy for low-resolution face images is improved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Y. Y. Zheng, J. Yao. Multi-angle face detection based on DP-Adaboost. International Journal of Automation and Computing, vol. 12, no. 4, pp. 421–431, 2015. DOI: https://doi.org/10.1007/s11633-014-0872-8.

    Article  Google Scholar 

  2. L. Wang, R. F. Li, K. Wang, J. Chen. Feature representation for facial expression recognition based on FACS and LBP. International Journal of Automation and Computing, vol. 11, no. 5, pp. 459–468, 2014. DOI: https://doi.org/10.1007/s11633-014-0835-0.

    Article  Google Scholar 

  3. H. S. Du, Q. P. Hu, D. F. Qiao, I. Pitas. Robust face recog-nition via low-rank sparse representation-based classification. International Journal of Automation and Computing, vol. 12, no. 6, pp. 579–587, 2015. DOI: https://doi.org/10.1007/s11633-015-0901-2.

    Article  Google Scholar 

  4. I. Masi, Y. Wu, T. Hassner, P. Natarajan. Deep face recognition: A survey. In Proceedings of the 31st SIBGRAPI Conference on Graphics, Patterns and Images, IEEE, Parana, Brazil, pp. 471–478, 2018. DOI:: https://doi.org/10.1109/SIB-GRAPI.2018.00067.

    Google Scholar 

  5. B. Prihasto, S. Choirunnisa, M. I. Nurdiansyah, S. Mathulaprangsan, V. C. M. Chu, S. H. Chen, J. C. Wang. A survey of deep face recognition in the wild. In Proceedings of International Conference on Orange Technologies, IEEE, Melbourne, Australia, pp. 76–79, 2016. DOI:: https://doi.org/10.1109/ICOT.2016.8278983.

    Google Scholar 

  6. W. Zhao, R. Chellappa, P. J. Phillips, A. Rosenfeld. Face recognition: A literature survey. ACM Computing Surveys, vol. 35, no. 4, pp. 399–458, 2003. DOI:: https://doi.org/10.1145/954339.954342.

    Article  Google Scholar 

  7. X. Z. Zhang, Y. S. Gao. Face recognition across pose: A review. Pattern Recognition, vol. 42, no. 11, pp. 2876–2896, 2009. DOI:: https://doi.org/10.1016/j.patcog.2009.04.017.

    Article  Google Scholar 

  8. C. H. Lin, Z. H. Wang, G. J. Jong. A de-identification face recognition using extracted thermal features based on deep learning. IEEE Sensors Journal, vol. 20, no. 16, pp. 9510–9517, 2020. DOI::https://doi.org/10.1109/JSEN.2020.2986098.

    Article  Google Scholar 

  9. W. Yang, H. W. Gao, Y. Q. Jiang, J. H. Yu, J. Sun, J. G. Liu, Z. J. Ju. A cascaded feature pyramid network with non-backward propagation for facial expression recogntion. IEEE Sensors Journal, vol. 21, no. 10, pp. 11382–11392, 2021. DOI: https://doi.org/10.1109/JSEN.2020.2997182.

    Article  Google Scholar 

  10. Y. Taigman, M. Yang, M. A. Ranzato, L. Wolf. DeepFace: Closing the gap to human-level performance in face verification. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Columbus, USA, pp. 1701–1708, 2014. DOI: https://doi.org/10.1109/CVPR.2014.220.

    Google Scholar 

  11. F. Schroff, D. Kalenichenko, J. Philbin. FaceNet: A unified embedding for face recognition and clustering. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 815–823, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298682.

    Google Scholar 

  12. O. M. Parkhi, A. Vedaldi, A. Zisserman. Deep face recognition. In Proceedings of the British Machine Vision Conference, Swansea, UK, 2015. DOI: https://doi.org/10.5244/C.29.41.

    Google Scholar 

  13. J. K. Deng, J. Guo, N. N. Xue, S. Zafeiriou. ArcFace: Additive angular margin loss for deep face recognition. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Long Beach, USA, pp. 4685–4694, 2019. DOI: https://doi.org/10.1109/CVPR.2019.00482.

    Google Scholar 

  14. G. B. Huang, M. Ramesh, T. Berg, E. Learned-Miller. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, Technical Report, Technical Report 07-49, University of Massachusetts, Amherst, USA, 2007.

    Google Scholar 

  15. K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CVPR.2016.90.

    Google Scholar 

  16. Y. Taigman, M. Yang, M. A. Ranzato, L. Wolf. Web-scale training for face identification. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 2746–2754, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298891.

    Google Scholar 

  17. Q. Cao, L. Shen, W. D. Xie, O. M. Parkhi, A. Zisserman. VGGFace2: A dataset for recognising faces across pose and age. In Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition, IEEE, Xi’an, China, 2018. DOI: https://doi.org/10.1109/FG.2018.00020.

    Google Scholar 

  18. Y. D. Guo, L. Zhang, Y. X. Hu, X. D. He, J. F. Gao. MS-Celeb-1M: A dataset and benchmark for large-scale face recognition. In Proceedings of the 14th European Conference on Computer Vision, Springer, Amsterdam, Netherlands, pp. 87–102, 2016. DOI: https://doi.org/10.1007/978-3-319-46487-9_6.

    Google Scholar 

  19. D. Yi, Z. Lei, S. C. Liao, S. Z. Li. Learning face representation from scratch. [Online], Available: https://arxiv.org/abs/1411.7923, 2014.

    Google Scholar 

  20. Y. M. Lui, D. Bolme, B. A. Draper, J. R. Beveridge, G. Givens, P. J. Phillips. A meta-analysis of face recognition covariates. In Proceedings of the 3rd International Conference on Biometrics: Theory, Applications, and Systems, IEEE, Washington, USA, 2009. DOI: https://doi.org/10.1109/BTAS.2009.5339025.

    MATH  Google Scholar 

  21. W. W. Zou, P. C. Yuen. Very low resolution face recognition problem. IEEE Transactions on Image Processing, vol. 21, no. 1, pp. 327–340, 2012. DOI: https://doi.org/10.1109/TIP.2011.2162423.

    Article  MathSciNet  Google Scholar 

  22. J. D. van Ouwerkerk. Image super-resolution survey. Image and Vision Computing, vol. 24, no. 10, pp. 1039–1052, 2006. DOI: https://doi.org/10.1016/j.imavis.2006.02.026.

    Article  Google Scholar 

  23. J. Y. Wu, S. Y. Ding, W. Xu, H. Y. Chao. Deep joint face hallucination and recognition. [Online], Available: https://arxiv.org/abs/1611.08091, 2016.

    Google Scholar 

  24. Z. Lu, X. D. Jiang, A. Kot. Deep coupled ResNet for low-resolution face recognition. IEEE Signal Processing Letters, vol. 25, no. 4, pp. 526–530, 2018. DOI: https://doi.org/10.1109/LSP.2018.2810121..

    Article  Google Scholar 

  25. A. J. Shah, S. B. Gupta. Image super resolution-A survey. In Proceedings of the 1st International Conference on Emerging Technology Trends in Electronics, Communication & Networking, IEEE, Surat, India, 2012. DOI: https://doi.org/10.1109/ET2ECN.2012.6470098.

    Google Scholar 

  26. Z. Y. Cheng, X. T. Zhu, S. G. Gong. Low-resolution face recognition. [Online], Available: https://arxiv.org/abs/1811.08965, 2019.

    Book  Google Scholar 

  27. P. Li, L. Prieto, D. Mery, P. J. Flynn. On low-resolution face recognition in the wild: Comparisons and new techniques. IEEE Transactions on Information Forensics and Security, vol. 14, no. 8, pp. 2000–2012, 2019. DOI: https://doi.org/10.1109/TIFS.2018.2890812..

    Article  Google Scholar 

  28. L. S. Luevano, L. Chang, H. Méndez-Vázquez, Y. Martínez-Díaz, M. González-Mendoza. A study on the performance of unconstrained very low resolution face recognition: Analyzing current trends and new research directions. IEEE Access, vol. 9, pp. 75470–75493, 2021. DOI: https://doi.org/10.1109/ACCESS.2021.3080712.

    Article  Google Scholar 

  29. K. Grm, W. J. Scheirer, V. Štruc. Face hallucination using cascaded super-resolution and identity priors. IEEE Transactions on Image Processing, vol. 29, pp. 2150–2165, 2019. DOI: https://doi.org/10.1109/TIP.2019.2945835.

    Article  Google Scholar 

  30. K. Nguyen, C. Fookes, S. Sridharan, M. Tistarelli, M. Nixon. Super-resolution for biometrics: A comprehensive survey. Pattern Recognition, vol. 78, pp. 23–42, 2018. DOI: https://doi.org/10.1016/j.patcog.2018.01.002.

    Article  Google Scholar 

  31. S. Banerjee, S. Das. LR-GAN for degraded face recognition. Pattern Recognition Letters, vol. 116, pp. 246–253, 2018. DOI: https://doi.org/10.1016/j.patrec.2018.10.034.

    Article  Google Scholar 

  32. A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, A. A. Bharath. Generative adversarial networks: An overview. IEEE Signal Processing Magazine, vol. 35, no. 1, pp. 53–65, 2018. DOI: https://doi.org/10.1109/MSP.2017.2765202.

    Article  Google Scholar 

  33. V. K. Ha, J. C. Ren, X. Y. Xu, S. Zhao, G. Xie, V. Masero, A. Hussain. Deep learning based single image super-resolution: A survey. International Journal of Automation and Computing, vol. 16, no. 4, pp. 413–426, 2019. DOI: https://doi.org/10.1007/s11633-019-1183-x..

    Article  Google Scholar 

  34. S. Z. Zhu, S. F. Liu, C. C. Loy, X. O. Tang. Deep cascaded bi-network for face hallucination. In Proceedings of the 14th European Conference on Computer Vision, Springer, Amsterdam, The Netherlands, pp. 614–630, 2016. DOI: https://doi.org/10.1007/978-3-319-46454-1_37.

    Google Scholar 

  35. X. Yu, B. Fernando, R. Hartley, F. Porikli. Semantic face hallucination: Super-resolving very low-resolution face images with supplementary attributes. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 11, pp. 2926–2943, 2020. DOI: https://doi.org/10.1109/TPAMI.2019.2916881.

    Google Scholar 

  36. E. Zangeneh, M. Rahmati, Y. Mohsenzadeh. Low resolution face recognition using a two-branch deep convolutional neural network architecture. Expert Systems with Applications, vol. 139, Article number 112854, 2020. DOI: https://doi.org/10.1016/j.eswa.2019.112854.

  37. W. M. Tan, B. Yan, B. Bare. Feature super-resolution: Make machine see more clearly. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake City, USA, pp. 3994–4002, 2018. DOI: https://doi.org/10.1109/CVPR.2018.00420.

    Google Scholar 

  38. F. Wang, L. R. Chen, C. Li, S. Y. Huang, Y. J. Chen, C. Qian, C. C. Loy. The devil of face recognition is in the noise. In Proceedings of the 15th European Conference on Computer Vision, Springer, Munich, Germany, pp. 780–795, 2018. DOI: https://doi.org/10.1007/978-3-030-01240-3_47.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical & Computer Engineering, Rutgers, The State University of New Jersey, Piscataway, 08854, USA

    Ling-Yi Xu & Zoran Gajic

Authors
  1. Ling-Yi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zoran Gajic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zoran Gajic.

Additional information

Colored figures are available in the online version at https://link.springer.com/journal/11633

Ling-Yi Xu received the B. Sc. degree in control theory and control engineering from University of Science and Technology, China in 2014. She received the M. Sc. degree in control theory and control engineering at State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, China in 2017, and received the M. Sc. degree in electrical and computer engineering from Rutgers, The State University of New Jersey, USA in 2017. Currently, she is a Ph. D. degree candidate with Department of Electrical and Computer Engineering, Rutgers, The State University of New Jersey, USA.

Her research interests include computer vision, machine learning, control systems and robotics.

Zoran Gajic received the Diploma in Engineering (five year program) and Magister of Science (two year program) degrees in electrical engineering from University of Belgrade, Serbia, received the M. Sc. degree in applied mathematics, and the Ph. D. degree in systems science engineering under direction of Professor Hassan Khalil from Department of Electrical Engineering and System Science, Michigan State University, USA in 1984. He was a visiting professor with Princeton University, USA in 2003, and the American University of Sharjah, UAE in 2011. He is currently a professor of Department of Electrical and Computer Engineering with Rutgers, The State University of New Jersey, where he has been involved in teaching linear systems and signals, controls, communication networks, optical networks, reinforcement learning, and electrical circuit courses since 1984. He has authored/co-authored close to 100 journal papers, primarily published in IEEE Transactions on Automatic Control and the IFAC Automatica, and eight books on linear systems and linear and bilinear control systems published by Academic Press, Prentice Hall, Marcel Dekker, Taylor and Francis, and Springer Verlag. His Prentice Hall book Linear Dynamic Systems and Signals was translated into the Chinese by Jiaotong University Press in 2004. His 1995 Academic Press book Lyapunov Matrix Equation in Systems Stability and Control was republished in 2008 by Dover Publications. Dr. Gajic has supervised 18 doctoral dissertations and 25 master theses. Eleven of his former doctoral students hold faculty positions with respected world universities. He has delivered four plenary lectures at international conferences and presented close to 150 conference papers. Dr. Gajic has served on editorial boards for nine journals and as a guest editor for six journal special issues. From 2003 to 2020, he was the Electrical and Computer Engineering Graduate Program Director. Presently, he serves on the American Association of University Professors National Council. Dr. Gajic is a Life Senior Master of the U. S. Chess Federation and a Master of the World Chess Federation.

His research interests include control systems, reinforcement learning, energy systems (fuel and solar cells, wind turbines, electric power grids), wireless communications, and networking.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, LY., Gajic, Z. Improved Network for Face Recognition Based on Feature Super Resolution Method. Int. J. Autom. Comput. 18, 915–925 (2021). https://doi.org/10.1007/s11633-021-1309-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-021-1309-9

Keywords

Search

Navigation