International Journal of Computer Vision

, Volume 126, Issue 6, pp 597–614 | Cite as

Hallucinating Compressed Face Images

Article
  • 554 Downloads

Abstract

A face hallucination algorithm is proposed to generate high-resolution images from JPEG compressed low-resolution inputs by decomposing a deblocked face image into structural regions such as facial components and non-structural regions like the background. For structural regions, landmarks are used to retrieve adequate high-resolution component exemplars in a large dataset based on the estimated head pose and illumination condition. For non-structural regions, an efficient generic super resolution algorithm is applied to generate high-resolution counterparts. Two sets of gradient maps extracted from these two regions are combined to guide an optimization process of generating the hallucination image. Numerous experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art hallucination methods on JPEG compressed face images with different poses, expressions, and illumination conditions.

Keywords

Face hallucination Super resolution JPEG compression Image denoising Landmark points 

Notes

Acknowledgements

This work is supported by NSF CAREER Grant 1149783, and gifts from Adobe and Nvidia.

References

  1. Baker, S., & Kanade, T. (2002). Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9), 1167–1183.CrossRefGoogle Scholar
  2. Barnes, C., Shechtman, E., Goldman, D. B., & Finkelstein, A. (2010). The generalized PatchMatch correspondence algorithm. In Proceedings of European conference on computer vision.Google Scholar
  3. Buades, A., Coll, B., & Morel, J. M. (2005). A non-local algorithm for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition.Google Scholar
  4. Choi, I., Kim, S., Brown, M., & Tai, Y. W. (2013). A learning-based approach to reduce JPEG artifacts in image matting. In Proceedings of IEEE international conference on computer vision.Google Scholar
  5. Figueiredo, M. A. T., Dias, J. B., Oliveira, J. P., & Nowak, R. (2006). On total variation denoising: A new majorization-minimization algorithm and an experimental comparison with wavelet denoising. In Proceedings of IEEE international conference on image processing.Google Scholar
  6. Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images. IEEE Transactions on Image Processing, 16(5), 1395–1411.MathSciNetCrossRefGoogle Scholar
  7. Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2008). Multi-PIE. In Proceedings of IEEE conference on automatic face and gesture recognition.Google Scholar
  8. Jia, K., & Gong, S. (2005). Multi-modal tensor face for simultaneous super-resolution and recognition. In Proceedings of IEEE international conference on computer vision.Google Scholar
  9. Jiang, J., Hu, R., Wang, Z., & Han, Z. (2014). Noise robust face hallucination via locality-constrained representation. IEEE Transactions on Multimedia, 16(5), 1268–1281.CrossRefGoogle Scholar
  10. Kim, K. I., & Kwon, Y. (2008). Example-based learning for single-image super-resolution and JPEG artifact removal. Max-Planck-Institut Technical Report.Google Scholar
  11. Kumar, N., Berg, A. C., Belhumeur, P. N., & Nayar, S. K. (2009). Attribute and simile classifiers for face verification. In Proceedings of IEEE international conference on computer vision.Google Scholar
  12. Li, Y., Guo, F., Tan, R. T., & Brown, M. S. (2014). A contrast enhancement framework with JPEG artifacts suppression. In Proceedings of European conference on computer vision.Google Scholar
  13. Liang, Y., Lai, J. H., Yuen, P. C., Zou, W. W., & Cai, Z. (2014). Face hallucination with imprecise-alignment using iterative sparse representation. Pattern Recognition, 47(10), 3327–3342.CrossRefGoogle Scholar
  14. Liu, C., Shum, H. Y., & Freeman, W. T. (2007). Face hallucination: Theory and practice. International Journal of Computer Vision, 75(1), 115–134.CrossRefGoogle Scholar
  15. Liu, S., & Bovik, A. C. (2002). Efficient DCT-domain blind measurement and reduction of blocking artifacts. IEEE Transactions on Circuits and Systems for Video Technology, 12(12), 1139–1149.CrossRefGoogle Scholar
  16. Liu, S., & Yang, M. H. (2014). Compressed face hallucination. In Proceedings of IEEE international conference on image processing.Google Scholar
  17. Ma, X., Zhang, J., & Qi, C. (2010). Hallucinating face by position-patch. Pattern Recognition, 43(6), 2224–2236.CrossRefGoogle Scholar
  18. Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009). Non-local sparse models for image restoration. In Proceedings of IEEE international conference on computer vision.Google Scholar
  19. Park, J. S., & Lee, S. W. (2008). An example-based face hallucination method for single-frame, low-resolution facial images. IEEE Transactions on Image Processing, 17(10), 1806–1816.MathSciNetCrossRefMATHGoogle Scholar
  20. Singh, S., Kumar, V., & Verma, H. K. (2007). Reduction of blocking artifacts in JPEG compressed images. Digital Signal Processing, 17(1), 225–243.CrossRefGoogle Scholar
  21. Tappen, M. F., & Liu, C. (2012). A Bayesian approach to alignment-based image hallucination. In Proceedings of European conference on computer vision.Google Scholar
  22. Timofte, R., Smet, V. D., & Gool, L. V. (2014). A+: Adjusted anchored neighborhood regression for fast super-resolution. In Proceedings of Asian conference on computer vision.Google Scholar
  23. Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International Journal of Computer Vision, 57(2), 137–154.CrossRefGoogle Scholar
  24. Voska, R., Mediachancecom. (2001). JpgQ—jpeg quality estimator. www.mediachance.com.
  25. Wang, N., Tao, D., Gao, X., Li, X., & Li, J. (2014). A comprehensive survey to face hallucination. International Journal of Computer Vision, 106(1), 9–30.CrossRefGoogle Scholar
  26. Wang, X., & Tang, X. (2005). Hallucinating face by eigentransformation. IEEE Transactions on Systems, Man, and Cybernetics, 35(3), 425–434.CrossRefGoogle Scholar
  27. Wang, Z., Bovik, A., Sheikh, H., & Simoncelli, E. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.CrossRefGoogle Scholar
  28. Xiong, X., & la Torre, F. D. (2013). Supervised descent method and its application to face alignment. In Proceedings of IEEE conference on computer vision and pattern recognition.Google Scholar
  29. Xiong, Z., Sun, X., & Wu, F. (2010). Robust web image/video super-resolution. IEEE Transactions on Image Processing, 19(8), 2017–2028.MathSciNetCrossRefMATHGoogle Scholar
  30. Yang, CY., Liu, S., & Yang, M. H. (2013). Structured face hallucination. In Proceedings of IEEE conference on computer vision and pattern recognition.Google Scholar
  31. Yang, J., Wright, J., Huang, T., & Ma, Y. (2008). Image super-resolution via sparse representation of raw image patches. In Proceedings of IEEE conference on computer vision and pattern recognition.Google Scholar
  32. Yang, J., Wright, J., Huang, T., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 19(11), 2861–2873.MathSciNetCrossRefMATHGoogle Scholar
  33. Zhai, G., Zhang, W., Yang, X., Lin, W., & Xu, Y. (2008). Efficient deblocking with coefficient regularization, shape-adaptive filtering, and quantization constraint. IEEE Transactions on Multimedia, 10(5), 735–745.CrossRefGoogle Scholar
  34. Zhu, X., & Ramanan, D. (2012). Face detection, pose estimation, and landmark localization in the wild. In Proceedings of IEEE conference on computer vision and pattern recognition.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Electrical Engineering and Computer Science, School of EngineeringUniversity of California at MercedMercedUSA

Personalised recommendations