Skip to main content
SpringerLink
Log in

Image color rendering based on frequency channel attention GAN

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In recent years, channel attention mechanism has greatly improved the performance of computer vision-oriented network models. But the simple superposition of modules inevitably increases the complexity of the model. In order to improve the performance and reduce the complexity of the model, a novel frequency channel attention GAN is proposed and applied to image color rendering. Firstly, global average pooling is a special case of discrete cosine transform. In order to better capture the rich input mode information, we extend global mean pooling to the frequency domain to obtain the frequency channel attention mechanism. Secondly, the frequency channel attention mechanism is combined with U-Net network to represent all the feature information of the image. The effectiveness of channel attention GAN in frequency domain was verified by using DIV2K dataset and COCO dataset. Finally, compared with pix2pix, CycleGAN, and HCEGAN models, PSNR increased by 2.660 dB, 2.595 dB and 1.430 dB, and SSIM increased by 7.943%, 6.790% and 2.436%. Experimental results show that our method not only improves the image rendering effect and quality, but also enhances the model stability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Afifi, M., Brubaker, M.A., Brown, M.S.: Histogan: Controlling colors of gan-generated and real images via color histograms. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7941–7950 (2021)

  2. Allegra, D., Furnari, G., Gargano, S., et al.: A method to improve the color rendering accuracy in cultural heritage: preliminary results. In: Journal of Physics: Conference Series, p. 012057. IOP Publishing (2022)

  3. Bahng, H., Yoo, S., Cho, W., et al.: Coloring with words: guiding image colorization through text-based palette generation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 431–447 (2018)

  4. Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27, 2672–2680 (2014)

    Google Scholar 

  5. Hong’an, L., Min, Z., Zhuoming, D., et al.: Interactive image color editing method based on block feature. Infrared Laser Eng. 48(12), 293–298 (2019)

    Article  Google Scholar 

  6. Hong’an, L., Qiaoxue, Z., Wenjing, Y., et al.: Image super-resolution reconstruction for secure data transmission in Internet of Things environment. Math. Biosci. Eng. 18(5), 6652–6671 (2021)

  7. Isola, P., Zhu, J.Y., Zhou, T., et al.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision And Pattern Recognition, pp. 1125–1134 (2017)

  8. Kim, A.S., Cheng, W.C., Beams, R., et al.: Color rendering in medical extended-reality applications. J. Digit. Imaging 34, 16–26 (2021)

    Article  Google Scholar 

  9. Kumar, M., Weissenborn, D., Kalchbrenner, N.: Colorization transformer. arXiv:2102.04432 (2017)

  10. Lee, J., Kim, E., Lee, Y., et al.: Reference-based sketch image colorization using augmented-self reference and dense semantic correspondence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5801–5810 (2020)

  11. Li, J., Han, Y., Zhang, M., et al.: Multi-scale residual network model combined with global average pooling for action recognition. Multimed. Tools Appl. 1–19 (2022c)

  12. Li, J., Liu, K., Hu, Y., et al.: Eres-UNet++: Liver CT image segmentation based on high-efficiency channel attention and Res-UNet++. Comput. Biol. Med. 106501 (2022c)

  13. Li, B., Lai, Y.K., John, M., et al.: Automatic example-based image colorization using location-aware cross-scale matching. IEEE Trans. Image Process. 28(9), 4606–4619 (2019)

    Article  MathSciNet  Google Scholar 

  14. Li, H., Zhang, M., Yu, Z., et al.: An Improved pix2pix Model Based on Gabor Filter for Robust Color Image Rendering, pp. 86–101. AIMS Press, Springfield (2022)

    Google Scholar 

  15. Li, J., Han, Y., Zhang, M., et al.: Multi-scale residual network model combined with global average pooling for action recognition. Multimed. Tools Appl. 81(1), 1375–1393 (2022)

    Article  Google Scholar 

  16. Li, H., Zhang, M., Chen, D., et al.: Image color rendering based on hinge-cross-entropy GAN in internet of medical things. CMES-Comput. Model. Eng. Sci. 135(1), 779–794 (2023)

    Google Scholar 

  17. Liang, W., Ding, D., Wei, G.: An improved DualGAN for near-infrared image colorization. Infrared Phys. Technol. 116, 103764 (2021)

    Article  Google Scholar 

  18. Liang, Y., Lee, D., Li, Y., et al.: Unpaired medical image colorization using generative adversarial network. Multimed. Tools Appl. 81(19), 26669–26683 (2022)

    Article  Google Scholar 

  19. Liu, Y., Peng, S., Liu, L., et al.: Neural rays for occlusion-aware image-based rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7824–7833 (2022)

  20. Mirza, M., Osindero, S.: Conditional generative adversarial nets. Comput. Sci. 2672–2680 (2014)

  21. Oza, U., Pipara, A., Mandal, S., et al.: Automatic image colorization using ensemble of deep convolutional neural networks. In: 2022 IEEE Region 10 Symposium (TENSYMP), pp. 1–6. IEEE (2022)

  22. Ren, W., Pan, J., Zhang, H., et al.: Single image dehazing via multi-scale convolutional neural networks with holistic edges. Int. J. Comput. Vis. 128(1), 240–259 (2020)

    Article  Google Scholar 

  23. Sagar, A.: Dmsanet: dual multi scale attention network. In: International Conference on Image Analysis and Processing, pp. 633–645. Springer (2022)

  24. Wan, Z., Zhang, B., Chen, D., et al.: Bringing old photos back to life. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2747–2757 (2020)

  25. Wan-bo, Y., Xiang-xiang, W., Da-qing, W.: Face image recognition based on basis function iteration of discrete cosine transform. J. Graph. 41(1), 91–95 (2020)

    Google Scholar 

  26. Woo, S., Park, J., Lee, J.Y., et al.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  27. Wu, Y., Wang, X., Li, Y., et al.: Towards vivid and diverse image colorization with generative color prior. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14377–14386 (2021)

  28. Wu, Y., Wang, G., Wang, Z., et al.: Triplet attention fusion module: a concise and efficient channel attention module for medical image segmentation. Biomed. Signal Process. Control 82, 104515 (2023)

    Article  Google Scholar 

  29. Xuan, D.: Design of 3D animation color rendering system based on image enhancement algorithm and machine learning. Soft Comput. 1–10 (2023)

  30. Yuan, M., Simo-Serra, E.: Line art colorization with concatenated spatial attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3946–3950 (2021)

  31. Žeger, I., Grgic, S., Vuković, J., et al.: Grayscale image colorization methods: overview and evaluation. IEEE Access (2021)

  32. Zhang, X., Wang, T., Wang, J., et al.: Pyramid channel-based feature attention network for image dehazing. Comput. Vis. Image Underst. 197, 103003 (2020)

    Article  Google Scholar 

  33. Zhu, J.Y., Park, T., Isola, P., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1–18 (2017)

Download references

Acknowledgements

This work was partly supported by the Natural Science Basis Research Plan in Shaanxi Province of China under Grant 2023-JC-YB-517 and the Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University under Grant VRLAB2023B08, and the high-level talent introduction project of Shaanxi Technical College of Finance & Economics under Grant 2022KY01. All of the authors declare that there is no conflict of interest regarding the publication of this article and would like to thank the anonymous referees for their valuable comments and suggestions.

Author information

Author notes

  1. D. Wang, M. Zhang and J. Liu have contributed equally to this work.

Authors and Affiliations

  1. College of Computer Science and Technology, Xi’an University of Science and Technology, Xi’an, 710054, China

    Hong-an Li, Diao Wang & Min Zhang

  2. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, 100191, China

    Hong-an Li

  3. Xi’an Xiangteng Microelectronics Technology Co., Ltd, Xi’an, 710018, China

    Min Zhang

  4. Shaanxi Technical College of Finance and Economics, Xianyang, 712099, China

    Jun Liu

Authors
  1. Hong-an Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Diao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

These authors contributed equally to this work.

Corresponding author

Correspondence to Diao Wang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Ha., Wang, D., Zhang, M. et al. Image color rendering based on frequency channel attention GAN. SIViP 18, 3179–3186 (2024). https://doi.org/10.1007/s11760-023-02980-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02980-7

Keywords

Search

Navigation