Abstract
Automatic image colorization is an old problem in image processing that has regained interest in the recent years with the emergence of deep-learning approaches with dramatic results. A careful examination shows that these methods often suffer from the so-called “color halos” or “color bleeding” effect: some colors are not well localized and may cross shape edges. This phenomenon is caused by the non-alignment of edges in the luminance and chrominance maps. We address this problem by regularizing the output of an efficient image colorization method with deep image prior and coupled total variation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Antic, J., Howard, J., Manor, U.: DeCrappification, DeOldification, and super resolution. Fast.ai course (2019). https://www.fast.ai/posts/2019-05-03-decrappify.html
Arbelot, B., Vergne, R., Hurtut, T., Thollot, J.: Local texture-based color transfer and colorization. Comput. Graph. 62, 15–27 (2017)
Batard, T., Haro, G., Ballester, C.: DIP-VBTV: a color image restoration model combining a deep image prior and a vector bundle total variation. SIAM J. Imag. Sci. 14(4), 1816–1847 (2021)
Deshpande, A., Lu, J., Yeh, M.C., Chong, M., Forsyth, D.: Learning diverse image colorization. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2877–2885 (2017)
Deshpande, A., Rock, J., Forsyth, D.: Learning large-scale automatic image colorization. In: Proceedings of the International Conference on Computer Vision (ICCV), pp. 567–575 (2015)
Huang, Z., Zhao, N., Liao, J.: Unicolor: a unified framework for multi-modal colorization with transformer. ACM Trans. Graph. (Proc. SIGGRAPH2022) 41(6) (2022)
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans. Graph. (Proc. SIGGRAPH’16) 35(4), 1–11 (2016)
Kim, E., Lee, S., Park, J., Choi, S., Seo, C., Choo, J.: Deep edge-aware interactive colorization against color-bleeding effects. In: Proceedings of the International Conference on Computer Vision (ICCV), pp. 14667–14676 (2021)
Kim, G., et al.: Bigcolor: colorization using a generative color prior for natural images. In: Avidan, S., Brostow, G., Cisse, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision— ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol. 13667, pp. 350–366. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20071-7_21
Larsson, G., Maire, M., Shakhnarovich, G.: Learning representations for automatic colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 577–593. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_35
Liu, J., Sun, Y., Xu, X., Kamilov, U.: Image restoration using total variation regularized deep image prior. In: Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7715–7719 (2019)
Mouzon, T., Pierre, F., Berger, M.-O.: Joint CNN and variational model for fully-automatic image colorization. In: Lellmann, J., Burger, M., Modersitzki, J. (eds.) SSVM 2019. LNCS, vol. 11603, pp. 535–546. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22368-7_42
Pan, X., Zhan, X., Dai, B., Lin, D., Loy, C., Luo, P.: Exploiting deep generative prior for versatile image restoration and manipulation. IEEE Trans. Pattern Anal. Mach. Intell. 44(11), 7474–7489 (2022)
Pierre, F., Aujol, J.F.: Recent approaches for image colorization. In: Chen, K., Schönlieb, C.B., Tai, X.C., Younces, L. (eds.) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging: Mathematical Imaging and Vision. Springer (2021). https://doi.org/10.1007/978-3-030-03009-4_55-1
Pierre, F., Aujol, J.F., Bugeau, A., Papadakis, N., Ta, V.T.: Luminance-chrominance model for image colorization. SIAM J. Imag. Sci. 8(1), 536–563 (2015)
Salmona, A., Bouza, L., Delon, J.: DeOldify: a review and implementation of an automatic colorization method. Image Process. Line 12, 347–368 (2022)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Vitoria, P., Raad, L., Ballester, C.: ChromaGAN: adversarial picture colorization with semantic class distribution. In: Proceedings of the Winter Conference on Applications of Computer Vision, pp. 2445–2454 (2020)
Xia, M., Hu, W., Wong, T.T., Wang, J.: Disentangled image colorization via global anchors. ACM Trans. Graph. (Proc. SIGGRAPH2022) 41(6) (2022)
Zhang, R., et al.: Real-time user-guided image colorization with learned deep priors. ACM Trans. Graph. (Proc. SIGGRAPH2017) 36(4) (2017)
Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 649–666. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_40
Acknowledgments
This research was funded, in whole or in part, by l’Agence Nationale de la Recherche (ANR), project ANR-21-0008-01. For the purpose of open access, the authors have applied a CC-BY public copyright licence to any Author Accepted Manuscript (AAM) version arising from this submission.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Agazzotti, G., Pierre, F., Sur, F. (2023). Deep Image Prior Regularized by Coupled Total Variation for Image Colorization. In: Calatroni, L., Donatelli, M., Morigi, S., Prato, M., Santacesaria, M. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2023. Lecture Notes in Computer Science, vol 14009. Springer, Cham. https://doi.org/10.1007/978-3-031-31975-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-31975-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31974-7
Online ISBN: 978-3-031-31975-4
eBook Packages: Computer ScienceComputer Science (R0)