Skip to main content
SpringerLink
Log in

Gaussian noise robust face hallucination via average filtering based data fidelity and locality regularization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In surveillance scenarios, the captured face images are often of low-resolution and contaminated by Gaussian noise. Noise creates problems in weighted linear representation, an essential process in attaining the reconstruction coefficients through conventional least square representation-based face hallucination models. To address this problem, a new face hallucination framework using average filtering-based data fidelity and locality regularization is proposed in this paper. In the proposed framework, an additional fidelity term is introduced which is accomplished through average filtered input and dictionary images. It helps in minimizing the reconstruction error. Moreover, the average filtering-based similarity matrix is introduced in the regularization term which succors the proposed framework in achieving more accurate locality, a quintessential component for face hallucination. The experimental analysis investigated on widely used public human face databases and locally captured CCTV footages reveal the better performance of the proposed framework over the compared state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability Statements

Data sharing is not applicable to this article as no datasets were generated. However, the source code of this work is available on email request to ershyamrajput@gmail.com

Notes

  1. Due to the public unavailability of the source code of [28], I could not produce the results of this model for all noise levels considered in this paper. I found results for only σ = 5, 10, and 15 in their paper.

References

  1. An L, Bhanu B (2014) Face image super-resolution using 2d {CCA}. Signal Process 103:184–194

    Article  Google Scholar 

  2. Baker S, Kanade T (2000) Hallucinating faces. In: Proceedings fourth IEEE international conference on automatic face and gesture recognition (Cat. No. PR00580), pp 83–88

  3. Cao Q, Lin L, Shi Y, Liang X, Li G (2017) Attention-aware face hallucination via deep reinforcement learning. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 1656–1664

  4. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 1, pp 1–8

  5. Chen Y, Tai Y, Liu X, Shen C, Yang J (2018) Fsrnet: end-to-end learning face super-resolution with facial priors. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 2492–2501

  6. Ding X, Hu R, He Z, Liang C, Wang Z (2021) Face hallucination based on degradation analysis for robust manifold. Neurocomputing

  7. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  8. Farrugia RA, Guillemot C (2017) Face hallucination using linear models of coupled sparse support. IEEE Trans Image Process 26(9):4562–4577. https://doi.org/10.1109/TIP.2017.2717181

    Article  MathSciNet  MATH  Google Scholar 

  9. Guleryuz OG (2007) Weighted averaging for denoising with overcomplete dictionaries. IEEE Trans Image Process 16(12):3020–3034. https://doi.org/10.1109/TIP.2007.908078

    Article  MathSciNet  Google Scholar 

  10. Hao N, Liao H, Qiu Y, Yang J (2016) Face super-resolution reconstruction and recognition using non-local similarity dictionary learning based algorithm. IEEE/CAA J Automatica Sinica 3(2):213–224

    Article  MathSciNet  Google Scholar 

  11. Hosseini H, Hessar F, Marvasti F (2015) Real-time impulse noise suppression from images using an efficient weighted-average filtering. IEEE Signal Process Lett 22(8):1050–1054

    Article  Google Scholar 

  12. Huang H, He H, Fan X, Zhang J (2010) Super-resolution of human face image using canonical correlation analysis. Pattern Recogn 43(7):2532–2543

    Article  MATH  Google Scholar 

  13. Huang S, Sun J, Yang Y, Fang Y, Lin P, Que Y (2018) Robust single-image super-resolution based on adaptive edge-preserving smoothing regularization. IEEE Trans Image Process 27(6):2650–2663

    Article  MathSciNet  MATH  Google Scholar 

  14. Hui Z, Liu W, Lam KM (2017) A novel correspondence-based face-hallucination method. Image Vis Comput 60:171–184

    Article  Google Scholar 

  15. Jiang J, Chen C, Huang K, Cai Z, Hu R (2016) Noise robust position-patch based face super-resolution via tikhonov regularized neighbor representation. Inform Sci 367–368:354–372

    Article  Google Scholar 

  16. Jiang J, Chen C, Ma J, Wang Z, Wang Z, Hu R (2017) Srlsp: a face image super-resolution algorithm using smooth regression with local structure prior. IEEE Trans Multimed 19(1):27–40

    Article  Google Scholar 

  17. Jiang J, Hu R, Wang Z, Han Z (2014) Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans Image Process 23(10):4220–4231

    Article  MathSciNet  MATH  Google Scholar 

  18. Jiang J, Hu R, Wang Z, Han Z (2014) Noise robust face hallucination via locality-constrained representation. IEEE Trans Multimed 16(5):1268–1281

    Article  Google Scholar 

  19. Jiang J, Hu R, Wang Z, Han Z, Ma J (2016) Facial image hallucination through coupled-layer neighbor embedding. IEEE Trans Circuits Syst Video Technol 26(9):1674–1684

    Article  Google Scholar 

  20. Jiang J, Ma J, Chen C, Jiang X, Wang Z (2016) Noise robust face image super-resolution through smooth sparse representation. IEEE Trans Cybern PP(99):1–12

    Google Scholar 

  21. Jiang J, Wang C, Liu X, Ma J (2021) Deep learning-based face super-resolution: a survey

  22. Jiang J, Yu Y, Tang S, Ma J, Aizawa A, Aizawa K (2020) Context-patch face hallucination based on thresholding locality-constrained representation and reproducing learning. IEEE Trans Cybern 50 (1):324–337

    Article  Google Scholar 

  23. Jung C, Jiao L, Liu B, Gong M (2011) Position-patch based face hallucination using convex optimization. IEEE Signal Process Lett 18(6):367–370

    Article  Google Scholar 

  24. Li Y, Cai C, Qiu G, Lam KM (2014) Face hallucination based on sparse local-pixel structure. Pattern Recogn 47(3):1261–1270

    Article  Google Scholar 

  25. Liu C, Shum HY, Zhang CS (2001) A two-step approach to hallucinating faces: global parametric model and local nonparametric model. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR 2001), vol 1, pp 192–198

  26. Liu D, Wang Z, Wen B, Yang J, Han W, Huang T (2016) Robust single image super-resolution via deep networks with sparse prior. IEEE Trans Image Process 25(7):3194–3207

    Article  MathSciNet  MATH  Google Scholar 

  27. Liu L, Chen CLP, Li S, Tang YY, Chen L (2017) Robust face hallucination via locality-constrained bi-layer representation. IEEE Trans Cybern PP(99):1–13

    Google Scholar 

  28. Liu L, Feng Q, Chen CLP, Wang Y (2021) Noise robust face hallucination based on smooth correntropy representation. IEEE Transactions on Neural Networks and Learning Systems, 1–13. https://doi.org/10.1109/TNNLS.2021.3071982

  29. Liu L, Li S, Chen CLP (2018) Quaternion locality-constrained coding for color face hallucination. IEEE Trans Cybern 48(5):1474–1485. https://doi.org/10.1109/TCYB.2017.2703134

    Article  Google Scholar 

  30. Liu L, Li S, Philip Chen CL (2019) Iterative relaxed collaborative representation with adaptive weights learning for noise robust face hallucination. IEEE Trans Circuits Syst Video Technol 29(5):1284–1295. https://doi.org/10.1109/TCSVT.2018.2829758

    Article  Google Scholar 

  31. Lu T, Xiong Z, Zhang Y, Wang B, Lu T (2017) Robust face super-resolution via locality-constrained low-rank representation. IEEE Access 5:13,103–13,117. https://doi.org/10.1109/ACCESS.2017.2717963https://doi.org/10.1109/ACCESS.2017.2717963

    Article  Google Scholar 

  32. Lu T, Zeng K, Jiang J, Zhang Y, He W, Wang Z, Zhou H (2018) Face hallucination using manifold-regularized group locality-constrained representation. In: 2018 25th IEEE International conference on image processing (ICIP), pp 2511–2515. https://doi.org/10.1109/ICIP.2018.8451499

  33. Luo J, Zhu Y, Magnin IE (2009) Denoising by averaging reconstructed images: application to magnetic resonance images. IEEE Trans Biomed Eng 56(3):666–674. https://doi.org/10.1109/TBME.2009.2012256https://doi.org/10.1109/TBME.2009.2012256

    Article  Google Scholar 

  34. Ma X, Song H, Qian X (2015) Robust framework of single-frame face superresolution across head pose, facial expression, and illumination variations. IEEE Trans Human-Mach Syst 45(2):238–250. https://doi.org/10.1109/THMS.2014.2375329

    Article  Google Scholar 

  35. Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recogn 43 (6):2224–2236

    Article  Google Scholar 

  36. Nagar S, Jain A, Singh PK, Kumar A (2021) Mixed-noise robust face super-resolution through residual-learning based error suppressed nearest neighbor representation. Inform Sci 546:121–145

    Article  MathSciNet  Google Scholar 

  37. Park SW, Savvides M (2007) Breaking the limitation of manifold analysis for super-resolution of facial images. In: 2007 IEEE International conference on acoustics, speech and signal processing - ICASSP ’07, vol 1, pp I–573–I–576

  38. Pei X, Guan Y, Cai P, Dong T (2018) Face hallucination via gradient constrained sparse representation. IEEE Access 6:4577–4586

    Article  Google Scholar 

  39. Rajput SS (2022) Mixed gaussian-impulse noise robust face hallucination via noise suppressed low-and-high resolution space-based neighbor representation. Multimed Tools Applic 81:15,997–16,019

    Article  Google Scholar 

  40. Rajput SS, Arya K, Singh V (2018) Robust face super-resolution via iterative sparsity and locality-constrained representation. Inform Sci 463–464:227–244

    Article  Google Scholar 

  41. Rajput SS, Arya K (2019) Noise robust face hallucination via outlier regularized least square and neighbor representation. IEEE Trans Biometr Behav Identity Sci 1(4):252–263

    Article  Google Scholar 

  42. Rajput SS, Arya K (2019) A robust facial image super-resolution model via mirror-patch based neighbor representation. Multimed Tools Applic 78:25,407–25,426

    Article  Google Scholar 

  43. Rajput SS, Arya K (2020) A robust face super-resolution algorithm and its application in low-resolution face recognition system. Multimed Tools Applic 79:23,909–23,934

    Article  Google Scholar 

  44. Rajput SS, Arya K, Bohat VK (2019) Face image super-resolution using differential evolutionary algorithm. In: Verma NK, Ghosh AK (eds) Computational intelligence: theories, applications and future directions - volume ii. Springer, Singapore, pp 635–644

  45. Rajput SS, Arya K, Singh V, Bohat VK (2018) Face hallucination techniques: a survey. In: 2018 Conference on information and communication technology (CICT), pp 1–6

  46. Rajput SS, Bohat VK, Arya K (2018) Grey wolf optimization algorithm for facial image super-resolution. Applied Intelligence

  47. Rajput SS, Singh A, Arya K, Jiang J (2018) Noise robust face hallucination algorithm using local content prior based error shrunk nearest neighbors representation. Signal Process 147:233–246

    Article  Google Scholar 

  48. Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):23–38

    Article  Google Scholar 

  49. Shi J, Liu X, Zong Y, Qi C, Zhao G (2018) Hallucinating face image by regularization models in high-resolution feature space. IEEE Trans Image Process 27(6):2980–2995. https://doi.org/10.1109/TIP.2018.2813163

    Article  MathSciNet  MATH  Google Scholar 

  50. Song Y, Zhang J, He S, Bao L, Yang Q (2017) Learning to hallucinate face images via component generation and enhancement

  51. Su Y, Bai X, Li W, Jing P, Zhang J, Liu J (2018) Graph regularized low-rank tensor representation for feature selection. J Vis Commun Image Represent 56:234–244

    Article  Google Scholar 

  52. Thomaz CE, Giraldi GA (2010) A new ranking method for principal components analysis and its application to face image analysis. Image Vis Comput 28(6):902–913

    Article  Google Scholar 

  53. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: 2010 IEEE Computer society conference on computer vision and pattern recognition, pp 3360–3367

  54. Wang X, Tang X (2005) Hallucinating face by eigentransformation. IEEE Trans Syst Man Cybern Part C (Applic Rev) 35(3):425–434

    Article  Google Scholar 

  55. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  56. Wang Z, Hu R, Wang S, Jiang J (2014) Face hallucination via weighted adaptive sparse regularization. IEEE Trans Circuits Syst Video Technol 24(5):802–813

    Article  Google Scholar 

  57. Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: 2015 IEEE International conference on computer vision (ICCV), pp 370–378

  58. Yang J, Tang H, Ma Y, Huang T (2008) Face hallucination via sparse coding. In: 2008 15th IEEE International conference on image processing, pp 1264–1267

  59. Yang J, Wright J, Huang T, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  60. Yu K, Zhang T, Gong Y (2009) Nonlinear learning using local coordinate coding. In: Bengio Y, Schuurmans D, Lafferty J, Williams A, Culotta C (eds) Advances in neural information processing systems 22, pp 2223–2231

  61. Yu X, Porikli F (2016) Ultra-resolving face images by discriminative generative networks. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision – ECCV 2016. Springer International Publishing, Cham, pp 318–333

  62. Zhang J, Li X, Jing P, Liu J, Su Y (2018) Low-rank regularized heterogeneous tensor decomposition for subspace clustering. IEEE Signal Process Lett 25(3):333–337

    Article  Google Scholar 

  63. Zhang L, Zhang L, Mou X, Zhang D (2011) Fsim: A feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386. https://doi.org/10.1109/TIP.2011.2109730

    Article  MathSciNet  MATH  Google Scholar 

  64. Zhu S, Liu S, Loy CC, Tang X (2016) Deep cascaded bi-network for face hallucination

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, National Institute of Technology Patna, Patna-, 800005, India

    Shyam Singh Rajput

Authors
  1. Shyam Singh Rajput
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shyam Singh Rajput.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rajput, S.S. Gaussian noise robust face hallucination via average filtering based data fidelity and locality regularization. Appl Intell 53, 7917–7930 (2023). https://doi.org/10.1007/s10489-022-03901-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03901-w

Keywords

Search

Navigation