Advertisement

The Visual Computer

, Volume 29, Issue 11, pp 1173–1186 | Cite as

A framework for interactive image color editing

  • Przemyslaw Musialski
  • Ming Cui
  • Jieping Ye
  • Anshuman Razdan
  • Peter Wonka
Original Article

Abstract

We propose a new method for interactive image color replacement that creates smooth and naturally looking results with minimal user interaction. Our system expects as input a source image and rawly scribbled target color values and generates high quality results in interactive rates. To achieve this goal we introduce an algorithm that preserves pairwise distances of the signatures in the original image and simultaneously maps the color to the user defined target values. We propose efficient sub-sampling in order to reduce the computational load and adapt semi-supervised locally linear embedding to optimize the constraints in one objective function. We show the application of the algorithm on typical photographs and compare the results to other color replacement methods.

Keywords

Image processing Computational photography Color manipulation Interactive image editing Recoloring 

Notes

Acknowledgements

This research was financially supported by Science Foundation Arizona, US Navy, and NSF. We would like tom thank Tom Ang (Fig. 14) and Norman Koren (Figs. 1, 5) for the permission to use their outstanding photographs.
Fig. 14

Result of our recoloring method. In each row, from left to right: original, user input in form of strokes, our output. All original images in this figure are copyrighted by Tom Ang (http://www.tomang.com/). Best seen in the electronic version in close-up

Fig. 15

Our method can also be used to custom black-white conversion and it also allows selective conversion of spatial regions. Best seen in the electronic version in close-up

References

  1. 1.
    ADOBE Inc.: Photoshop. http://www.adobe.com/products/photoshop.html (2012)
  2. 2.
    An, X., Pellacini, F.: AppProp: all-pairs appearance-space edit propagation. ACM Trans. Graph. 27(3), 1 (2008) CrossRefGoogle Scholar
  3. 3.
    An, X., Pellacini, F.: User-controllable color transfer. Comput. Graph. Forum 29(2), 263–271 (2010) CrossRefGoogle Scholar
  4. 4.
    Carroll, R., Ramamoorthi, R., Agrawala, M.: Illumination decomposition for material recoloring with consistent interreflections. ACM Trans. Graph. 30(4), 1 (2011) CrossRefGoogle Scholar
  5. 5.
    Chang, Y., Saito, S., Nakajima, M.: Example-based color transformation of image and video using basic color categories. IEEE Trans. Image Process. 16(2), 329–336 (2007) MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chang, Y., Saito, S., Uchikawa, K., Nakajima, M.: Example-based color stylization of images. ACM Trans. Appl. Percept. 2(3), 322–345 (2005) CrossRefGoogle Scholar
  7. 7.
    Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. 26(3), 103 (2007) CrossRefGoogle Scholar
  8. 8.
    Chia, A.Y.-S., Zhuo, S., Gupta, R.K., Tai, Y.-W., Cho, S.-Y., Tan, P., Lin, S.: Semantic colorization with Internet images. ACM Trans. Graph. 30(6), 1 (2011) CrossRefGoogle Scholar
  9. 9.
    Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., Xu, Y.-Q.: Color harmonization. ACM Trans. Graph. 25(3), 624 (2006) CrossRefGoogle Scholar
  10. 10.
    Farbman, Z., Fattal, R., Lischinski, D.: Diffusion maps for edge-aware image editing. ACM Trans. Graph. 29(6), 1 (2010) CrossRefGoogle Scholar
  11. 11.
    Fowlkes, C., Belongie, S., Chung, F., Malik, J.: Spectral grouping using the Nyström method. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 214–225 (2004) CrossRefGoogle Scholar
  12. 12.
    Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. ACM Trans. Graph. 23(3), 689 (2004) CrossRefGoogle Scholar
  13. 13.
    Li, M.-T., Huang, M.-L., Wang, C.-M.: Example-based color alternation for images. In: 2010 2nd International Conference on Computer Engineering and Technology, pp. V7-316–V7-320. IEEE Press, New York (2010) Google Scholar
  14. 14.
    Li, Y., Ju, T., Hu, S.-M.: Instant propagation of sparse edits on images and videos. Comput. Graph. Forum 29(7), 2049–2054 (2010) CrossRefGoogle Scholar
  15. 15.
    Lischinski, D., Farbman, Z., Uyttendaele, M., Szeliski, R.: Interactive local adjustment of tonal values. ACM Trans. Graph. 25(3), 646 (2006) CrossRefGoogle Scholar
  16. 16.
    Liu, X., Wan, L., Qu, Y., Wong, T.-T., Lin, S., Leung, C.-S., Heng, P.-A.: Intrinsic colorization. ACM Trans. Graph. 27(5), 1 (2008) CrossRefzbMATHGoogle Scholar
  17. 17.
    Luan, Q., Wen, F., Xu, Y.-Q.: Color transfer brush. In: 15th Pacific Conference on Computer Graphics and Applications (PG’07), October 2007, pp. 465–468. IEEE Press, New York (2007) CrossRefGoogle Scholar
  18. 18.
    Mount, D.M., Arya, S.: ANN: a library for approximate nearest neighbor searching. http://www.cs.umd.edu/~mount/ANN/ (Jan. 2010)
  19. 19.
    Nadler, B., Lafon, S., Coifman, R.R., Kevrekidis, I.G.: Diffusion maps, spectral clustering and eigenfunctions of Fokker–Planck operators. Adv. Neural Inf. Process. Syst. 18(1), 955–962 (2005) Google Scholar
  20. 20.
    Nagel, D.: Color Replacement in Photoshop CS. http://www.digitalmediadesigner.com/2004/01_jan/tutorials/pscs-cr040129.htm (2004)
  21. 21.
    Pellacini, F., Lawrence, J.: AppWand. ACM Trans. Graph. 26(3), 54 (2007) CrossRefGoogle Scholar
  22. 22.
    Pitie, F., Kokaram, A.: The linear Monge–Kantorovitch linear colour mapping for example-based colour transfer. In: 4th European Conference on Visual Media Production, 2007, IETCVMP, pp. 1–9 (2007) Google Scholar
  23. 23.
    Pitie, F., Kokaram, A., Dahyot, R.: Automated colour grading using colour distribution transfer. Comput. Vis. Image Underst. 107(1–2), 123–137 (2007) CrossRefGoogle Scholar
  24. 24.
    Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(4), 34–41 (2001) CrossRefGoogle Scholar
  25. 25.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000) CrossRefGoogle Scholar
  26. 26.
    Saul, L.K., Roweis, S.T.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. J. Mach. Learn. Res. 4(2), 119–155 (2004) MathSciNetzbMATHGoogle Scholar
  27. 27.
    Shapira, L., Shamir, A., Cohen-Or, D.: Image appearance exploration by model-based navigation. Comput. Graph. Forum 28(2), 629–638 (2009) CrossRefGoogle Scholar
  28. 28.
    Stone, M.: A Field Guide to Digital Color. AK Peters/CRC Press, Boca Raton (2003) Google Scholar
  29. 29.
    Tai, Y.-W., Jia, J., Tang, C.-K.: Local color transfer via probabilistic segmentation by expectation-maximization. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), pp. 747–754. IEEE Press, New York (2005) Google Scholar
  30. 30.
    Tai, Y.-W., Jia, J., Tang, C.-K.: Soft color segmentation and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1520–1537 (2007) CrossRefGoogle Scholar
  31. 31.
    Wang, B., Yu, Y., Wong, T.-T., Chen, C., Xu, Y.-Q.: Data-driven image color theme enhancement. ACM Trans. Graph. 29(6), 1 (2010) zbMATHGoogle Scholar
  32. 32.
    Wang, B., Yu, Y., Xu, Y.-Q.: Example-based image color and tone style enhancement. ACM Trans. Graph. 30(4), 1 (2011) MathSciNetGoogle Scholar
  33. 33.
    Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, June 2007, pp. 1–8. IEEE Press, New York (2007) CrossRefGoogle Scholar
  34. 34.
    Welsh, T., Ashikhmin, M., Mueller, K.: Transferring color to greyscale images. ACM Trans. Graph. 21(3), 277 (2002) CrossRefGoogle Scholar
  35. 35.
    Wen, C.-L., Hsieh, C.-H., Chen, B.-Y., Ouhyoung, M.: Example-based multiple local color transfer by strokes. Comput. Graph. Forum 27(7), 1765–1772 (2008) CrossRefGoogle Scholar
  36. 36.
    Xiao, X., Ma, L.: Color transfer in correlated color space. In: Virtual Reality Continuum and Its Applications, p. 305 (2006). doi: 10.1145/1128923.1128974 Google Scholar
  37. 37.
    Xiao, X., Ma, L.: Gradient-preserving color transfer. Comput. Graph. Forum 28(7), 1879–1886 (2009) CrossRefGoogle Scholar
  38. 38.
    Xu, K., Li, Y., Ju, T., Hu, S.-M., Liu, T.-Q.: Efficient affinity-based edit propagation using K-D tree. ACM Trans. Graph. 28(5), 1 (2009) Google Scholar
  39. 39.
    Yang, C.-K., Peng, L.-K.: Automatic mood-transferring between color images. IEEE Comput. Graph. Appl. 28(2), 52–61 (2008) CrossRefGoogle Scholar
  40. 40.
    Yatziv, L., Sapiro, G.: Fast image and video colorization using chrominance blending. IEEE Trans. Image Process. 15(5), 1120–1129 (2006) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Przemyslaw Musialski
    • 1
    • 2
  • Ming Cui
    • 2
  • Jieping Ye
    • 2
  • Anshuman Razdan
    • 2
  • Peter Wonka
    • 2
    • 3
  1. 1.Vienna University of TechnologyViennaAustria
  2. 2.Arizona State UniversityTempeUSA
  3. 3.King Abdullah University of Science and TechnologyThuwalSaudi Arabia

Personalised recommendations