Skip to main content

Interactive image recoloring by combining global and local optimization


We propose a novel interactive image recoloring method by combining global and local optimization. Our approach assumes that each pixel is a linear transform of its neighbors which can be in spatial or feature space. Corresponding, a new framework for combining global and local energy optimization is designed and derived. By taking advantage of global and local color propagation, our approach requires only a few user scribbles to produce the high-quality results. We show various experimental results and comparisons on image recoloring. Compared with the state-of-the-art methods, our approach produces higher-quality results with only a small amount of user interaction than those only consider local propagation or global propagation approaches.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10


  1. 1.

    An X, Pellacini F (2008) Appprop: All-pairs appearance-space edit propagation. ACM Trans Graph 27(3):40:1–40:9

    Article  Google Scholar 

  2. 2.

    Beigpour S, van de Weijer J (2011) Object recoloring based on intrinsic image estimation. In: Proceedings of the 2011 International Conference on Computer Vision, ICCV ’11, pp 327–334, Washington, DC, USA. IEEE Computer Society

  3. 3.

    Bhat P, Lawrence Zitnick C, Cohen M, Curless B (2010) Gradientshop: A gradient-domain optimization framework for image and video filtering. ACM Trans Graph 29(2):10:1–10:14

    Article  Google Scholar 

  4. 4.

    Chen Q, Li D, Tang C-K (2013) Knn matting. IEEE Trans Pattern Anal Mach Intell 35(9):2175–2188

    Article  Google Scholar 

  5. 5.

    Chen X, Zou D, Zhao Q, Tan P (2012) Manifold preserving edit propagation. ACM Trans Graph 31(6):132:1–132:7

    Google Scholar 

  6. 6.

    Chen X, Zou D, Zhou SZ, Zhao Q, Tan P (2013) Image matting with local and nonlocal smooth priors. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’13, pp 1902–1907, Washington, DC, USA. IEEE Computer Society

  7. 7.

    Cohen-Or D, Sorkine O, Gal R, Leyvand T, Xu Y-Q (2006) Color harmonization. ACM Trans Graph 25(3):624–630

    Article  Google Scholar 

  8. 8.

    Criminisi A, Sharp T, Rother C, P’erez P (2010) Geodesic image and video editing. ACM Trans Graph 29(5):134:1–134:15

    Article  Google Scholar 

  9. 9.

    Fattal R (2009) Edge-avoiding wavelets and their applications. ACM Trans Graph 28(3):22:1–22:10

    Article  Google Scholar 

  10. 10.

    Farbman Z, Fattal R, Lischinski D (2010) Diffusion maps for edge-aware image editing. ACM Trans Graph 29(6):145:1–145:10

    Article  Google Scholar 

  11. 11.

    He K, Sun J, Tang X (2010) Guided image filtering. In: Proceedings of the 11th European Conference on Computer Vision: Part I, ECCV’10, pp 1–14. Springer-Verlag, Berlin, Heidelberg

  12. 12.

    Hsu E, Mertens T, Paris S, Avidan S, Durand F (2008) Light mixture estimation for spatially varying white balance. ACM Trans Graph 27(3):70:1–70:7

    Article  Google Scholar 

  13. 13.

    Huang H, Li X, Zhao H, Nie G, Zhongyi H, Xiao L (2014) Manifold-preserving image colorization with nonlocal estimation. Multimedia Tools and Applications:1–14

  14. 14.

    Krishnan D, Szeliski R (2011) Multigrid and multilevel preconditioners for computational photography. ACM Trans Graph 30(6):177:1–177:10

    Article  Google Scholar 

  15. 15.

    Lee P, Wu Y (2011) Nonlocal matting. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’11, pp 2193–2200, Washington, DC, USA. IEEE Computer Society

  16. 16.

    Levin A, Lischinski D, Weiss Y (2004) Colorization using optimization. ACM Trans Graph 23(3):689–694

    Article  Google Scholar 

  17. 17.

    Levin A, Rav-Acha A, Lischinski D (2008) Spectral matting. IEEE Trans Pattern Anal Mach Intell 30(10):1699–1712

    Article  Google Scholar 

  18. 18.

    Lin S, Ritchie D, Fisher M, Hanrahan P (2013) Probabilistic color-by-numbers: Suggesting pattern colorizations using factor graphs. ACM Trans Graph 32(4):37:1–37:12

    MATH  Google Scholar 

  19. 19.

    Liu X, Wan L, Yingge Q, Wong T-T, Lin S, Leung C-S, Heng P-A (2008) Intrinsic colorization. ACM Trans Graph 27(5):152:1–152:9

    Article  Google Scholar 

  20. 20.

    Musialski P, Cui M, Ye J, Razdan A, Wonka P (2013) A framework for interactive image color editing. Vis Comput 29(11):1173–1186

    Article  Google Scholar 

  21. 21.

    Olonetsky I, Avidan S (2012) Treecann - k-d tree coherence approximate nearest neighbor algorithm. In: Proceedings of the 12th European Conference on Computer Vision - Volume Part IV, ECCV’12, pp 602–615. Springer-Verlag, Berlin, Heidelberg

  22. 22.

    Sheng B, Sun H, Magnor M, Li P (2014) Video colorization using parallel optimization in feature space. IEEE Transactions on Circuits and Systems for Video Technology 24(3):407–417

    Article  Google Scholar 

  23. 23.

    Sheng B, Sun H, Chen S, Liu X, Enhua W (2011) Colorization using the rotation-invariant feature space. IEEE Comput Graph Appl 31(2):24–35

    Article  Google Scholar 

  24. 24.

    Seo S, Park Y, Ostromoukhov V (2013) Image recoloring using linear template mapping. Multimedia Tools Appl 64(2):293–308

    Article  Google Scholar 

  25. 25.

    Wang B, Yizhou Y, Wong T-T, Chen C, Xu Y-Q (2010) Data-driven image color theme enhancement. ACM Trans Graph 29(6):146:1–146:10

    Google Scholar 

  26. 26.

    Yatziv L, Sapiro G (2006) Fast image and video colorization using chrominance blending. Trans Img Proc 15(5):1120–1129

    Article  Google Scholar 

  27. 27.

    Yingge Q, Wong T-T, Heng P-A (2006) Manga colorization. ACM Trans Graph 25(3):1214–1220

    Article  Google Scholar 

Download references


This work was supported by the National Natural Science Foundation of China (Grant No. 61100146), the Zhejiang Provincial Natural Science Foundation of China (Grant Nos. LQ14F020006, LQ12F02010, LY12F02015 and LY12F02014), and the Science and Technology Plan Program of Wenzhou, China (Grant Nos. G20130017 and No. S20100053).

Author information



Corresponding author

Correspondence to Xujie Li.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Li, X., Zhao, H., Huang, H. et al. Interactive image recoloring by combining global and local optimization. Multimed Tools Appl 75, 6431–6443 (2016).

Download citation


  • Image recoloring
  • Global optimization
  • Local optimization
  • Color propagation