Advertisement

Real-Time Exemplar-Based Face Sketch Synthesis

  • Yibing Song
  • Linchao Bao
  • Qingxiong Yang
  • Ming-Hsuan Yang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8694)

Abstract

This paper proposes a simple yet effective face sketch synthesis method. Similar to existing exemplar-based methods, a training dataset containing photo-sketch pairs is required, and a K-NN photo patch search is performed between a test photo and every training exemplar for sketch patch selection. Instead of using the Markov Random Field to optimize global sketch patch selection, this paper formulates face sketch synthesis as an image denoising problem which can be solved efficiently using the proposed method. Real-time performance can be obtained on a state-of-the-art GPU. Meanwhile quantitative evaluations on face sketch recognition and user study demonstrate the effectiveness of the proposed method. In addition, the proposed method can be directly extended to the temporal domain for consistent video sketch synthesis, which is of great importance in digital entertainment.

Keywords

Face Hallucination Texture Synthesis 

References

  1. 1.
    Adobe: Adobe photoshop cs6Google Scholar
  2. 2.
    Aleix, M., Robert, B.: The ar face database. Tech. Rep. CVC Technical Report 24, Purdue University (1998)Google Scholar
  3. 3.
    Altman, N.S.: An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician (1992)Google Scholar
  4. 4.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.: Patchmatch: a randomized correspondence algorithm for structural image editing. In: SIGGRAPH (2009)Google Scholar
  5. 5.
    Berger, I., Shamir, A., Mahler, M., Carter, E., Hodgins, J.: Style and abstraction in portrait sketching. In: SIGGRAPH (2013)Google Scholar
  6. 6.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: CVPR (2005)Google Scholar
  7. 7.
    Chen, H., Xu, Y.Q., Shum, H.Y., Zhu, S.C., Zheng, N.N.: Example-based facial sketch generation with non-parametric sampling. In: ICCV (2001)Google Scholar
  8. 8.
    Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-d transform-domain collaborative filtering. TIP (2007)Google Scholar
  9. 9.
    Delac, K., Grgic, M., Grgic, S.: Independent comparative study of pca, ica, and lda on the feret data set. IJIST (2005)Google Scholar
  10. 10.
    Freeman, W.T., Tenenbaum, J.B., Pasztor, E.: An example-based approach to style translation for line drawings. MERL Technical Report (1999)Google Scholar
  11. 11.
    Gastal, E.S., Oliveira, M.M.: Domain transform for edge-aware image and video processing. In: SIGGRAPH (2011)Google Scholar
  12. 12.
    Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies. In: SIGGRAPH (2001)Google Scholar
  13. 13.
    Liu, Q., Tang, X., Jin, H., Lu, H., Ma, S.: A nonlinear approach for face sketch synthesis and recognition. In: CVPR (2005)Google Scholar
  14. 14.
    Lu, C., Xu, L., Jia, J.: Combining sketch and tone for pencil drawing production. In: NPAR (2012)Google Scholar
  15. 15.
    Paige, C.C., Saunders, M.A.: Lsqr: An algorithm for sparse linear equations and sparse least squares. ACM Transactions on Mathematical Software (1982)Google Scholar
  16. 16.
    Tang, X., Wang, X.: Face sketch synthesis and recognition. In: CVPR (2003)Google Scholar
  17. 17.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: CVPR (2001)Google Scholar
  18. 18.
    Wang, S., Zhang, L.: Y., L., Pan, Q.: Semi-coupled dictionary learning with applications in image super-resolution and photo-sketch synthesis. In: CVPR (2012)Google Scholar
  19. 19.
    Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. PAMI (2009)Google Scholar
  20. 20.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. PAMI (2009)Google Scholar
  21. 21.
    Xu, Z., Chen, H., Zhu, S.C., Luo, J.: A hierarchical compositional model for face representation and sketching. PAMI (2008)Google Scholar
  22. 22.
    Zhang, W., Wang, X., Tang, X.: Lighting and pose robust face sketch synthesis. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 420–433. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  23. 23.
    Zhang, W., Wang, X., Tang, X.: Coupled information-theoretic encoding for face photo-sketch recognition. In: CVPR (2011)Google Scholar
  24. 24.
    Zhou, H., Kuang, Z., Wong, K.: Markov weight fields for face sketch synthesis. In: CVPR (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Yibing Song
    • 1
    • 2
  • Linchao Bao
    • 1
    • 2
  • Qingxiong Yang
    • 1
    • 2
  • Ming-Hsuan Yang
    • 3
  1. 1.Department of Computer Science, Multimedia Software Engineering Research Centre (MERC)City University of Hong KongHong Kong, China
  2. 2.MERC-ShenzhenGuangdongHong Kong, China
  3. 3.Department of Electrical Engineering and Computer ScienceUniversity of California at MercedMercedUSA

Personalised recommendations