Skip to main content
SpringerLink
Log in

Semantic portrait color transfer with internet images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We present a novel color transfer method for portraits by exploring their high-level semantic information. First, a database is set up which consists of a collection of portrait images download from the Internet, and each of them is manually segmented using image matting as a preprocessing step. Second, we search the database using Face++ to find the images with similar poses to a given source portrait image, and choose one satisfactory image from the results as the target. Third, we extract portrait foregrounds from both source and target images. Then, the system extracts the semantic information, such as faces, eyes, eyebrows, lips, teeth, etc., from the extracted foreground of the source using image matting algorithms. After that, we perform color transfer between corresponding parts with the same semantic information. We get the final transferred result by seamlessly compositing different parts together using alpha blending. Experimental results show that our semantics-driven approach can generate better color transfer results for portraits than previous methods and provide users a new means to retouch their portraits.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. An X, Pellacini F (2010) User-controllable color transfer. Comp Graph Forum 29(2):263–271

    Article  Google Scholar 

  2. Chang Y, Saito S, Uchikawa K, Nakajima M (2005) Example-based color stylization of images. ACM Trans Appl Percept 2(3):322–345

    Article  Google Scholar 

  3. Chang Y, Saito S, Nakajima M (2007) Example-based color transformation of image and video using basic color categories. Trans Img Proc 16(2):329–336

    Article  MathSciNet  Google Scholar 

  4. Chia AY-S, Zhuo S, Gupta R K, Tai Y-W, Cho S-Y, Tan P, Lin S (2011) Semantic colorization with internet images. ACM Trans Graph 30 (6):156:1–156:8

    Article  Google Scholar 

  5. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  6. Dong W, Bao G, Zhang X, Paul J-C (2010) Fast local color transfer via dominant colors mapping. In: ACM SIGGRAPH ASIA 2010 sketches, SA ’10. ACM, New York, pp 46:1–46:2

    Google Scholar 

  7. Face++, 2015

  8. Fuzhang W, Dong W, Kong Y, Mei X, Paul J-C, Zhang X (2013) Content-based colour transfer. Comp Graph Forum 32(1):190–203

    Article  Google Scholar 

  9. Gastal ESL, Oliveira MM (2010) Shared sampling for real-time alpha matting. Comp Graph Forum 29(2):575–584

    Article  Google Scholar 

  10. Guan Y, Chen W, Liang X, Ding Z, Peng Q (2006) Easy matting - a stroke based approach for continuous image matting. Comp Graph Forum 25(3):567–576

    Article  Google Scholar 

  11. HaCohen Y, Shechtman E, Goldman DB, Lischinski D (2011) Non-rigid dense correspondence with applications for image enhancement. ACM Trans Graph 30 (4):70:1–70:10

    Article  Google Scholar 

  12. Huang H, Li X, Zhao H, Nie G, Zhongyi H, Xiao L (2014) Manifold-preserving image colorization with nonlocal estimation. Multimed Tools Appl:1–14

  13. Li X, Zhao H, Huang H, Zhongyi H, Xiao L (2015) Interactive image recoloring by combining global and local optimization. Multimed Tools Appl:1–13

  14. Liu S, Sun H, Zhang X (2012) Selective color transferring via ellipsoid color mixture map. J Vis Comun Image Represent 23(1):173–181

    Article  Google Scholar 

  15. Luan Q, Wen F, Xu Y-Q (2007) Color transfer brush. IEEE Computer Society, Washington, pp 465–468

    Google Scholar 

  16. Murray N, Skaff S, Marchesotti L, Perronnin F (2011) Towards automatic concept transfer. In: Proceedings of the ACM SIGGRAPH/Eurographics symposium on non-photorealistic animation and rendering, NPAR ’11. ACM, New York, pp 167–176

    Chapter  Google Scholar 

  17. Neumann L, Neumann A (2005) Color style transfer techniques using hue, lightness and saturation histogram matching. In: Proceedings of the 1st Eurographics conference on computational aesthetics in graphics, visualization and imaging, Computational Aesthetics’05. Eurographics Association, Aire-la-Ville, Switzerland, pp 111–122

    Google Scholar 

  18. Pérez P, Gangnet M, Blake A (July 2003) Poisson image editing. ACM Trans Graph 22(3):313–318

    Article  Google Scholar 

  19. Pitié F, Kokaram AC, Dahyot R (2007) Automated colour grading using colour distribution transfer. Comput Vis Image Underst 107(1–2):123–137

    Article  Google Scholar 

  20. Pouli T, Reinhard E (2011) Progressive color transfer for images of arbitrary dynamic range. Comp Grap 35(1):67–80. Extended Papers from Non-Photorealistic Animation and Rendering (NPAR) 2010

    Article  Google Scholar 

  21. Reinhard E, Ashikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Comput Graph Appl 21(5):34–41

    Article  Google Scholar 

  22. Tai Y-W, Jia J, Tang C-K (2005) Local color transfer via probabilistic segmentation by expectation-maximization, vol 1. IEEE Computer Society, Washington, pp 747–754

    Google Scholar 

  23. Wang B, Yizhou Y, Wong T-T, Chen C, Xu Y-Q (2010) Data-driven image color theme enhancement. ACM Trans Graph 29(6):146:1–146:10

    Google Scholar 

  24. Wen C-L, Hsieh C-H, Chen B-Y, Ouhyoung M (2008) Example-based multiple local color transfer by strokes. Comp Graph Forum 27(7):1765–1772

    Article  Google Scholar 

  25. Xiao X, Ma L (2009) Gradient-preserving color transfer. Comp Graph Forum 28(7):1879–1886

    Article  Google Scholar 

  26. Xue S, Wang J, Tong X, Dai Q, Guo B (2008) Image-based material weathering. Comp Graph Forum 27(2):617–626

    Article  Google Scholar 

  27. Yang C-K, Peng L-K (2008) Automatic mood-transferring between color images. IEEE Comput Graph Appl 28(2):52–61

    Article  MathSciNet  Google Scholar 

  28. Zhao Y, Jin X, Xu Y, Zhao H, Ai M, Zhou K (2015) Parallel style-aware image cloning for artworks. IEEE Trans Vis Comput Graph 21(2):229–240

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank the anonymous reviewers for their constructive comments. Xiaogang Jin was supported by the National Natural Science Foundation of China (61472351). Hanli Zhao was supported by Zhejiang Provincial Natural Science Foundation of China (LY15F020019).

Author information

Authors and Affiliations

  1. State Key Lab of CAD&CG, Zhejiang University, Hangzhou, 310058, China

    Yue Yang & Xiaogang Jin

  2. Intelligent Information Systems Institute, Wenzhou University, Wenzhou, China

    Hanli Zhao

  3. Bournemouth University, Dorset, UK

    Lihua You

  4. Fudan University, Shanghai, China

    Renlong Tu

  5. Xi’an University of Technology, Xi’an, China

    Xueyi Wu

Authors
  1. Yue Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanli Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lihua You
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Renlong Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xueyi Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaogang Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaogang Jin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Y., Zhao, H., You, L. et al. Semantic portrait color transfer with internet images. Multimed Tools Appl 76, 523–541 (2017). https://doi.org/10.1007/s11042-015-3063-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3063-x

Keywords

Search

Navigation