Skip to main content
SpringerLink
Log in

Image and Video Colorization Using Vector-Valued Reproducing Kernel Hilbert Spaces

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Motivated by the setting of reproducing kernel Hilbert space (RKHS) and its extensions considered in machine learning, we propose an RKHS framework for image and video colorization. We review and study RKHS especially in vectorial cases and provide various extensions for colorization problems. Theory as well as a practical algorithm is proposed with a number of numerical experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68, 337–404 (1950)

    Article  MathSciNet  MATH  Google Scholar 

  2. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)

    Article  MATH  Google Scholar 

  3. Berlinet, A., Thomas-Agnan, C.: Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer, Berlin (2004)

    MATH  Google Scholar 

  4. Bertalmio, M., Sapiro, G., Caselles, V., Balleste, C.: Image inpainting. In: Proceedings of SIGGRAPH 2000, New Orleans (2000)

  5. Blomgren, P., Chan, T.F.: Color TV: Total variation methods for restoration of vector-valued images. IEEE Trans. Image Process. 7(3), 304–309 (1998)

    Article  Google Scholar 

  6. Bochko, V., Parkkinen, J.: A spectral color analysis and colorization technique. IEEE Comput. Graph. Appl. 26, 74–82 (2006)

    Article  Google Scholar 

  7. Buades, A., Coll, B., Lisani, J.-L., Sbert, C.: Conditional image diffusion. Inverse Probl. Imaging 1(4), 593–608 (2007)

    MathSciNet  MATH  Google Scholar 

  8. Buades, A., Coll, B., Morel, J.-M.: A review of image denoising algorithms, with a new one. SIAM Multiscale Model. Simul. 15(3), 490–530 (2005)

    Article  MathSciNet  Google Scholar 

  9. Burns, G.: Museum of Broadcast Communications: Encyclopedia of Television. World Wide Web electronic publication (1997)

  10. Caponnetto, A., Pontil, M., Micchelli, C., Ying, Y.: Universal multi-task kernels. J. Mach. Learn. Res. 9, 1615–1646 (2008)

    MathSciNet  Google Scholar 

  11. Caponnetto, A., De Vito, E.: Optimal rates for regularized least-squares algorithm. Found. Comput. Math. 7(3), 331–368 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  12. Carmel, C., De Vito, E., Toigo, A., Umanità, V.: Vector valued reproducing kernel Hilbert spaces and universality. Anal. Appl. 8(1), 19–61 (2010)

    Article  Google Scholar 

  13. Carmeli, C., De Vito, E., Toigo, A.: Vector valued reproducing kernel Hilbert spaces of integrable functions and Mercer theorem. Anal. Appl. 4, 377–408 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  14. Chan, T.F., Kang, S.H., Shen, J.: Total variation denoising and enhancement of color images based on the CB and HSV color models. J. Vis. Commun. Image Represent. 12(4), 422–435 (2001)

    Article  Google Scholar 

  15. Chen, T., Wang, Y., Schillings, V., Meinel, C.: Grayscale image matting and colorization. In: Proceedings of Asian Conference on Computer Vision (ACCV) (2004)

  16. Chung, F.: Spectral Graph Theory. American Mathematical Society, Providence (1997)

    MATH  Google Scholar 

  17. Coifman, R.R., Lafon, S.: Geometric harmonics: a novel tool for multiscale out-of-sample extension of empirical functions. Appl. Comput. Harmon. Anal. 21, 31–52 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  18. Cucker, F., Smale, S.: On the mathematical foundations of learning. Bull. Am. Math. Soc. 39(1), 1–49 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  19. Drew, M., Finlayson, G.: Realistic colorization via the structure tensor. In: International Conference on Image Processing, ICIP08 (2008)

  20. Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Mathematics and Its Applications, vol. 375. Springer, Berlin (1996)

    MATH  Google Scholar 

  21. Fonseca, I., Leoni, G., Maggi, F., Morini, M.: Exact reconstruction of damaged color images using a total variation model. Preprint (2010)

  22. Fornasier, M.: Nonlinear projection digital image inpainting and restoration methods. J. Math. Imaging Vis. 24(3), 359–373 (2006)

    Article  MathSciNet  Google Scholar 

  23. Fornasier, M., March, R.: Restoration of color images by vector valued BV functions and variational calculus. SIAM J. Appl. Math. 68(2), 437–460 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  24. Gilboa, G., Osher, S.: Nonlocal linear image regularization and supervised segmentation. SIAM Multiscale Modeling and Simulation (MMS) 6(2) (2007)

  25. Gonzalez, R., Wood, R.: Digital Image Processing. Addison-Wesley, Reading (1992)

    Google Scholar 

  26. Horiuchi, T.: Estimation of color for gray-level image by probabilistic relaxation. In: Proc. IEEE Int. Conf. Pattern Recognition, pp. 867–870 (2002)

  27. Horiuchi, T., Hirano, S.: Colorization algorithm for grayscale image by propagating seed pixels. In: Proc. IEEE Int. Conf. Pattern Recognition, pp. 457–460 (2003)

  28. Irony, R., Cohen-Or, D., Lischinski, D.: Colorization by example. In: Proceedings of Eurographics Symposium on Rendering, pp. 201–210 (2005)

  29. Jones, F.: Lebesgue Integration on Euclidean Space. Jones and Bartlett, Boston (2001), revised edition

    Google Scholar 

  30. Kang, S.H., March, R.: Variational models for image colorization via chromaticity and brightness decomposition. IEEE Trans. Image Process. 16(9), 2251–2261 (2007)

    Article  MathSciNet  Google Scholar 

  31. Kimmel, R., Sochen, N.: Orientation diffusion or how to comb a porcupine? J. Vis. Commun. Image Represent. 13, 238–248 (2001)

    Article  Google Scholar 

  32. Kindermann, S., Osher, S., Jones, P.: Deblurring and denoising of images by nonlocal functionals. SIAM Multiscale Model. Simul. 4(4), 2005

  33. Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. ACM Trans. Graph. 23(3), 689–694 (2004)

    Article  Google Scholar 

  34. Lezoray, O., Ta, V.T., Elmoataz, A.: Nonlocal graph regularization for image colorization. In: Proceedings of 19th International Conference on Pattern Recognition, pp. 1–4 (2008)

  35. Liu, B., Liu, M., Wang, G.: Colorization based on image manifold learning. In: IEEE Region 10 Conference TENCON, pp. 1–3 (2006)

  36. Micchelli, C.A., Pontil, M.: On leaning vector-valued functions. Neural Comput. 17, 177–204 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  37. Minh, H.Q., Niyogi, P., Yao, Y.: Mercer’s theorem, feature maps, and smoothing. In: Proceedings of 19th Annual Conference on Learning Theory, Pittsburg, June 2006. Springer, Berlin (2006)

    Google Scholar 

  38. Pan, Z., Dong, Z., Zhang, M.: A new algorithm for adding color to video or animation clips. In: WSCG, pp. 515–520 (2004)

  39. Perona, P.: Orientation diffusion. IEEE Trans. Image Process. 7(3), 457–467 (1998)

    Article  Google Scholar 

  40. Qiu, G., Guan, J.: Color by linear neighborhood embedding. In: IEEE International Conference on Image Processing, vol. 3, pp. 988–991 (2005)

  41. Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Trans. Comput. Graph. Appl. 21, 34–41 (2002)

    Article  Google Scholar 

  42. Saitoh, S.: Integral Transforms, Reproducing Kernels and Their Applications. Pitman Research Notes in Mathematics Series, vol. 369. Longman, Harlow (1997)

    MATH  Google Scholar 

  43. Sapiro, G.: Inpainting the colors. In: ICIP 2005. IEEE International Conference on Image Processing, vol. 2, pp. 698–701 (2005)

  44. Schoenberg, I.J.: Metric spaces and positive definite functions. Trans. Am. Math. Soc. 44, 522–536 (1938)

    Article  MathSciNet  MATH  Google Scholar 

  45. Schökopf, B., Smola, A.: Learning with kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)

    Google Scholar 

  46. Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  47. Smale, S., Zhou, D.X.: Learning theory estimates via integral operators and their approximations. Constr. Approx. 26, 153–172 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  48. Stein, E.M.: Singular Integrals and Differentiability Properties of Functions. Princeton University Press, Princeton (1970)

    MATH  Google Scholar 

  49. Sýkora, D., Buriánek, J., Žáta, J.: Unsupervised colorization of black-and-white cartoons. In: Proceedings of the 3rd International Symposium on Non-Photorealistic Animation and Rendering, pp. 121–127. ACM, New York (2004)

    Chapter  Google Scholar 

  50. Tai, Y.W., Jia, J., Tang, C.K.: Soft color segmentation and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1520–1537 (2007)

    Article  Google Scholar 

  51. Tang, B., Sapiro, G., Caselles, V.: Color image enhancement via chromaticity diffusion. IEEE Trans. Image Process. 10, 701–707 (2001)

    Article  MATH  Google Scholar 

  52. Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)

    MATH  Google Scholar 

  53. Wahba, G.: Spline interpolation and smoothing on the sphere. SIAM J. Sci. Stat. Comput. 2(1), 5–16 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  54. Wahba, G.: Spline Models for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 59. SIAM, Philadelphia (1990)

    MATH  Google Scholar 

  55. Welsh, T., Ashikhmin, M., Mueller, K.: Transferring color to greyscale images. In: SIGGRAPH Conference Proceedings, p. 21 (2002)

  56. Yatziv, L., Sapiro, G.: Fast image and video colorization using chrominance blending. IEEE Trans. Image Process. 15(5), 1120–1129 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Humboldt University of Berlin, Berlin, Germany

    Minh Ha Quang

  2. School of Mathematics, Georgia Institute of Technology, Atlanta, GA, USA

    Sung Ha Kang

  3. Department of Mathematics, Yale University, New Haven, CT, USA

    Triet M. Le

Authors
  1. Minh Ha Quang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sung Ha Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Triet M. Le
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Minh Ha Quang.

Additional information

The collaboration began at Hausdorff Research Institute for Mathematics, Bonn, Germany, via support of the Junior Program in Analysis. This work is partially supported by DFG:GZ WI 1515/2-1, NSF:DMS-0908517, NSF: DMS-0809270, and ONR N000140910108.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ha Quang, M., Kang, S.H. & Le, T.M. Image and Video Colorization Using Vector-Valued Reproducing Kernel Hilbert Spaces. J Math Imaging Vis 37, 49–65 (2010). https://doi.org/10.1007/s10851-010-0192-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-010-0192-8

Keywords

Search

Navigation