The Journal of Supercomputing

, Volume 71, Issue 6, pp 2035–2049 | Cite as

Face hallucination and recognition in social network services

  • Feng Jiang
  • Seungmin Rho
  • Bo-Wei Chen
  • Xiaodan Du
  • Debin Zhao


Due to the rapid growth of social network services such as Facebook and Twitter, incorporation of face recognition in these large-scale web services is attracting much attention in both academia and industry. The major problem in such applications is that visual data provided are in general noisy, low resolution, prone to degradation due to lighting and other adverse effects. This paper proposes a novel face hallucination method with an aim of improving face recognition performance with the photography in social media data, which is modeled as a progressive process which explores the specific face characteristics and priors based on frequency bands analysis. In the first stage of our algorithm, initially estimated middle-resolution images are generated based on a patch-based learning method in discrete cosine transformation (DCT) domain as the first-scale restoration image. According to the relationship between the high-resolution face images and their lower resolution ones, the DC coefficients and AC coefficients are estimated separately. In the second stage, aiming at generating more refined high-resolution face images, a DCT up-sample algorithm emphasizing on low-frequency bands preserving is applied. Meanwhile, an interpolation-based method is presented in spatial domain to obtain high-frequency bands. Extensive experiments show that the proposed algorithm effectively helps improving the recognition performance of low-resolution faces.


Social media data Face hallucination Learning-based  Frequency bands 



This work was supported in part by the National Science Council of the Republic of China under Grant No. 103-2917-I-564- 058, and the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2013R1A1A2061978), the National Natural Science Foundation of China under Grant No. 61272386 and 61100096. We are grateful for Zhang kindly providing source code of [29].


  1. 1.
    Acquisti A, Gross R, Stutzman F (2011) Faces of facebook: privacy in the age of augmented reality. BlackHat USAGoogle Scholar
  2. 2.
    Anastassiou D, Jensen K (1995) Sub-pixel edge localization and the interpolation of still images. IEEE Trans Image Process 4(3):285C295Google Scholar
  3. 3.
    Baker S, Kanade T (2000) Hallucinating faces. In: Proceedings fourth IEEE international conference on automatic face and gesture recognition, IEEE, pp 83–88Google Scholar
  4. 4.
    Belhumeur PN, Hespanha JP, Kriegman D (1997) Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720CrossRefGoogle Scholar
  5. 5.
    Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society conference on computer vision and pattern recognition. CVPR 2004, vol 1, IEEE, pp I-IGoogle Scholar
  6. 6.
    Chang TL, Liu TL, Chuang JH (2006) Direct energy minimization for super-resolution on nonlinear manifolds. In: Computer vision-ECCV 2006, Springer, pp 281–294Google Scholar
  7. 7.
    Cohen S, Hamilton JT, Turner F (2011) Computational journalism. Commun ACM 54(10):66–71CrossRefGoogle Scholar
  8. 8.
    Dantone M, Bossard L, Quack T, Van Gool L (2011) Augmented faces. In: 2011 IEEE International conference on computer vision workshops (ICCV workshops), IEEE, pp 24–31Google Scholar
  9. 9.
    Gao W, Cao B, Shan S, Chen X, Zhou D, Zhang X, Zhao D (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans Syst Man Cybern Part A Syst Hum 38(1):149Google Scholar
  10. 10.
    He X, Niyogi P (2003) Locality preserving projections. NIPS 16:234–241Google Scholar
  11. 11.
    Hu Y, Lam KM, Qiu G, Shen T (2011) From local pixel structure to global image super-resolution: a new face hallucination framework. IEEE Trans Image Process 20(2):433–445CrossRefMathSciNetGoogle Scholar
  12. 12.
    Jung SH, Mitra SK, Mukherjee D (1996) Subband dct: definition, analysis, and applications. IEEE Trans Circuits Syst Video Technol 6(3):273–286CrossRefGoogle Scholar
  13. 13.
    Lazer D, Pentland AS, Adamic L, Aral S, Barabasi AL, Brewer D, Christakis N, Contractor N, Fowler J, Gutmann M et al (2009) Life in the network: the coming age of computational social science. Science (New York, NY) 323(5915):721CrossRefGoogle Scholar
  14. 14.
    Li Y, Lin X (2004) Face hallucination with pose variation. In: Sixth IEEE international conference on automatic face and gesture recognition, IEEE, pp 723–728Google Scholar
  15. 15.
    Liu C, Shum HY, Freeman WT (2007) Face hallucination: theory and practice. Int J Comput Vis 75(1):115–134CrossRefGoogle Scholar
  16. 16.
    Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recognit 43(6):2224–2236CrossRefGoogle Scholar
  17. 17.
    Mavridis N, Kazmi W, Toulis P (2010) Friends with faces: how social networks can enhance face recognition and vice versa. In: Computational social network analysis, Springer, pp 453–482Google Scholar
  18. 18.
    Mukherjee J, Mitra SK (2002) Image resizing in the compressed domain using subband DCT. IEEE Trans Circuits Syst Video Technol 12(7):620–627CrossRefGoogle Scholar
  19. 19.
    Park JS, Lee SW (2008) An example-based face hallucination method for single-frame, low-resolution facial images. IEEE Trans Image Process 17(10):1806–1816CrossRefMathSciNetGoogle Scholar
  20. 20.
    Stone Z, Zickler T, Darrell T (2010) Toward large-scale face recognition using social network context. Proc IEEE 98(8):1408–1415CrossRefGoogle Scholar
  21. 21.
    Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: IEEE Computer Society conference on computer vision and pattern recognition, IEEE, pp 586–591Google Scholar
  22. 22.
    Wang X, Tang X (2005) Hallucinating face by eigentransformation. IEEE Trans Syst Man Cybern Part C: Appl Rev 35(3):425–434CrossRefGoogle Scholar
  23. 23.
    Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227CrossRefGoogle Scholar
  24. 24.
    Wu Z, Yu H, Chen CW (2010) A new hybrid DCT-Wiener-based interpolation scheme for video intra frame up-sampling. IEEE Signal Process Lett 17(10):827–830CrossRefGoogle Scholar
  25. 25.
    Xu X, Liu W, Li L (2013) Face hallucination: how much it can improve face recognition. In: 2013 3rd Australian control conference (AUCC), IEEE, pp 93–98Google Scholar
  26. 26.
    Yang J, Tang H, Ma Y, Huang T (2008) Face hallucination via sparse coding. In: 15th IEEE international conference on image processing. ICIP 2008, IEEE, pp 1264–1267Google Scholar
  27. 27.
    Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873CrossRefMathSciNetGoogle Scholar
  28. 28.
    Zhang D, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238CrossRefGoogle Scholar
  29. 29.
    Zhang W, Cham WK (2011) Hallucinating face in the DCT domain. IEEE Trans Image Process 20(10):2769–2779CrossRefMathSciNetGoogle Scholar
  30. 30.
    Zhao W, Chellappa R, Phillips PJ, Rosenfeld A (2003) Face recognition: a literature survey. ACM Comput Surv (CSUR) 35(4):399–458CrossRefGoogle Scholar
  31. 31.
    Zhuang Y, Zhang J, Wu F (2007) Hallucinating faces: LPH super-resolution and neighbor reconstruction for residue compensation. Pattern Recognit 40(11):3178–3194CrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Feng Jiang
    • 1
  • Seungmin Rho
    • 2
  • Bo-Wei Chen
    • 3
  • Xiaodan Du
    • 1
  • Debin Zhao
    • 1
  1. 1.School of Computer ScienceHarbin Institute of TechnologyHarbin China
  2. 2.Department of MultimediaSungkyul UniversityAnyangKorea
  3. 3.Department of Electrical EngineeringNational Cheng Kung UniversityTainanTaiwan

Personalised recommendations