Advertisement

The Visual Computer

, Volume 33, Issue 11, pp 1443–1452 | Cite as

Artistic stylization of face photos based on a single exemplar

  • Zili Yi
  • Yang Li
  • Songyuan Ji
  • Minglun Gong
Original Article

Abstract

In this paper, we propose a unified framework for fully automatic face photo stylization based on a single style exemplar. Constrained by the “single-exemplar” condition, where the numbers and varieties of patch samples are limited, we introduce flexibility in sample selection while preserving identity and content of the input photo. Based on the observation that many styles are characterized by unique color selections and texture patterns, we employ a two-phase procedure. The first phase searches a dense and semantic-aware correspondence between the input and the exemplar images, so that colors in the exemplar can be transferred to the input. The second phase conducts edge-preserving texture transfer, which preserves edges and contours of the input and mimics the textures of the exemplar at multiple scales. Experimental results demonstrate compelling visual effects and notable improvements over other state-of-the-art methods which are adapted for the same task.

Keywords

Face stylization Non-photorealistic rendering Texture synthesis 

References

  1. 1.
    Burnt picture. http://bbs.wenxuecity.com/art/3047.html. Accessed 29 Feb 2016
  2. 2.
    Rollip. http://www.rollip.com/start/?src=ws. Accessed 17 Feb 2016
  3. 3.
    Barnes, C., Shechtman, E., Goldman, D.B., Finkelstein, A.: The generalized patchmatch correspondence algorithm. In: Computer Vision–ECCV 2010, pp. 29–43. Springer, New York (2010)Google Scholar
  4. 4.
    Beier, T., Neely, S.: Feature-based image metamorphosis. In: ACM SIGGRAPH Computer Graphics, vol. 26, pp. 35–42. ACM, New York (1992)Google Scholar
  5. 5.
    Choi, H.C., Sibbing, D., Kobbelt, L.: Nonparametric facial feature localization using segment-based eigenfeatures. Comput Intell Neurosci 501, 164290 (2015)Google Scholar
  6. 6.
    Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., Xu, Y.Q.: Color harmonization. In: ACM Transactions on Graphics (TOG), vol. 25, pp. 624–630. ACM, New York (2006)Google Scholar
  7. 7.
    Darabi, S., Shechtman, E., Barnes, C., Goldman, D.B., Sen, P.: Image melding: combining inconsistent images using patch-based synthesis. ACM Trans. Graph. 31(4), 82 (2012)CrossRefGoogle Scholar
  8. 8.
    Efros, A., Leung, T.K., et al.: Texture synthesis by non-parametric sampling. In: Proceedings of the Seventh IEEE International Conference on Computer Vision (ICCV), vol. 2, pp. 1033–1038. IEEE (1999)Google Scholar
  9. 9.
    Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 341–346. ACM, New York (2001)Google Scholar
  10. 10.
    Gooch, B., Coombe, G., Shirley, P.: Artistic vision: painterly rendering using computer vision techniques. In: Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, pp. 83–ff. ACM, New York (2002)Google Scholar
  11. 11.
    HaCohen, Y., Shechtman, E., Goldman, D.B., Lischinski, D.: Optimizing color consistency in photo collections. ACM Trans Graph 32(4), 38 (2013)CrossRefGoogle Scholar
  12. 12.
    Henstock, P.V., Chelberg, D.M.: Automatic gradient threshold determination for edge detection . IEEE Trans. Image Process 5(5), 784–787 (1996)Google Scholar
  13. 13.
    Kagaya, M., Brendel, W., Deng, Q., Kesterson, T., Todorovic, S., Neill, P.J., Zhang, E.: Video painting with space-time-varying style parameters. Vis Comput Graph IEEE Trans 17(1), 74–87 (2011)CrossRefGoogle Scholar
  14. 14.
    Kaspar, A., Neubert, B., Lischinski, D., Pauly, M., Kopf, J.: Self tuning texture optimization. In: Computer Graphics Forum, vol. 34, pp. 349–359. Wiley Online Library, New York (2015)Google Scholar
  15. 15.
    Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. In: ACM SIGGRAPH, pp. 277–286. ACM, New York (2003)Google Scholar
  16. 16.
    Lewis, J.P.: Texture synthesis for digital painting. In: ACM SIGGRAPH Computer Graphics. vol. 18, pp. 245–252. ACM, New York (1984)Google Scholar
  17. 17.
    Li, H., Liu, G., Ngan, K.N.: Guided face cartoon synthesis. Multimed. IEEE Trans. 13(6), 1230–1239 (2011)CrossRefGoogle Scholar
  18. 18.
    Li, H., Mould, D.: Contrast-aware halftoning. In: Computer Graphics Forum, vol. 29, pp. 273–280. Wiley Online Library, New York (2010)Google Scholar
  19. 19.
    Li, H., Mould, D.: Artistic tessellations by growing curves. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering, pp. 125–134. ACM, New York (2011)Google Scholar
  20. 20.
    Li, H., Mould, D.: Content-sensitive screening in black and white. In: GRAPP, pp. 166–172 (2011)Google Scholar
  21. 21.
    Li, H., Mould, D.: Structure-preserving stippling by priority-based error diffusion. In: Proceedings of Graphics Interface, pp. 127–134. Canadian Human-Computer Communications Society, Canada (2011)Google Scholar
  22. 22.
    Liang, L., Liu, C., Xu, Y.Q., Guo, B., Shum, H.Y.: Real-time texture synthesis by patch-based sampling. ACM Trans. Graph. 20(3), 127–150 (2001)CrossRefGoogle Scholar
  23. 23.
    Lu, C., Xu, L., Jia, J.: Combining sketch and tone for pencil drawing production. In: Proceedings of the Symposium on Non-Photorealistic Animation and Rendering, pp. 65–73. Eurographics Association, Manchester (2012)Google Scholar
  24. 24.
    Neumann, L., Neumann, A.: Color style transfer techniques using hue, lightness and saturation histogram matching. In: Computational Aesthetics, pp. 111–122. Citeseer (2005)Google Scholar
  25. 25.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. In: ACM Transactions on Graphics (TOG), vol. 22, pp. 313–318. ACM, New York (2003)Google Scholar
  26. 26.
    Shih, Y., Paris, S., Barnes, C., Freeman, W.T., Durand, F.: Style transfer for headshot portraits. ACM Trans Graph 33(4), 148 (2014)CrossRefGoogle Scholar
  27. 27.
    Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1955–1967 (2009)CrossRefGoogle Scholar
  28. 28.
    Wexler, Y., Shechtman, E., Irani, M.: Space-time completion of video. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 463–476 (2007)CrossRefGoogle Scholar
  29. 29.
    Zhang, W., Wang, X., Tang, X.: Lighting and pose robust face sketch synthesis. In: ECCV 2010, pp. 420–433. Springer, New York (2010)Google Scholar
  30. 30.
    Zhang, Y., Dong, W., Deussen, O., Huang, F., Li, K., Hu, B.G.: Data-driven face cartoon stylization. ACM, New york (2014)Google Scholar
  31. 31.
    Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 2879–2886. IEEE (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceMemorial University of NewfoundlandSt. John’sCanada

Personalised recommendations