Skip to main content
SpringerLink
Log in

Deep learning algorithm in ancient relics image colour restoration technology

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In order to restore the original colours of ancient relics more accurately and to reduce the burden of manual restoration, we developed a novel colour-restoration technique based on the DenseNet algorithm, which was used in a case study involving restoration of Dunhuang mural images and is based on deep learning. In recent years, deep learning-based methods have been an important direction for research into virtual restoration of image colours. Typical, damaged murals were generated from 60 mural datasets as input for the system, and these were enhanced by DenseNet, based on the interactive, digital mural-restoration system. We compared execution time, peak signal-to-noise ratio and structural similarities to evaluate DenseNet, SegNet, Deeplab and ResNet algorithms. In terms of time efficiency, the DenseNet algorithm was 44.62% faster than the SegNet algorithm. Regarding structural similarity (SSIM) values for the four models, DenseNet was the lowest: 1.289% lower than SegNet, 2.442% lower than Deeplab v3 and 1.288% lower than ResNet. In terms of the overall comparison, the repair effect for DenseNet was the best. Our method is highly reliable for mural restoration and not only saves time but also produces better virtual restoration results than other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

All data generated or analysed during this study are included in this published article [and its supplementary information files].

References

  1. Ancora D, Bassi A (2020) Deconvolved image restoration from auto-correlations. IEEE Trans Image Process 30:1332–1341

    MathSciNet  Google Scholar 

  2. Azad R, Asadi-Aghbolaghi M, Fathy M, Escalera S (2020) Attention deeplabv3+: multi-level context attention mechanism for skin lesion segmentation. In European conference on computer vision, Glasgow, UK, pp. 251-266

  3. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Google Scholar 

  4. Belfkih S, Montesinos P (2002) Color image restoration. In conference on colour in graphics, imaging, and vision, Poitiers, France, Vol. 2002, no. 1, pp. 416-419

  5. Bescos J, Altamirano JH, Santisteban A, Santamaria J (1988) Digital restoration models for color imaging. Appl Opt 27(2):419–424

    Google Scholar 

  6. Bhargavi K, Jyothi S (2014) A survey on threshold based segmentation technique in image processing. Int J Innov Res Dev 3(12):234–239

    Google Scholar 

  7. Ĉadík M (2008) Perceptual evaluation of color-to-grayscale image conversions. Comput Graph Forum, Oxford, UK 27(7):1745–1754

    Google Scholar 

  8. Caraffa L, Tarel JP (2013) Markov random field model for single image defogging. In 2013 IEEE intelligent vehicles symposium (IV), Gold Coast, QLD, Australia, pp. 994-999

  9. Chen Z, Shen J, Roth P (2013) Single image defogging algorithm based on Dark Channel priority. J Multimed 8(4):432–438

    Google Scholar 

  10. Daschiel H, Datcu M (2005) Information mining in remote sensing image archives: system evaluation. IEEE Trans Geosci Remote Sensing 43(01):188–199

    MATH  Google Scholar 

  11. Deng X, Dragotti PL (2020) Deep convolutional neural network for multi-modal image restoration and fusion. IEEE Trans Pattern Anal Mach Intell 43(10):3333–3348

    Google Scholar 

  12. Dolz J, Gopinath K, Yuan J, Lombaert H, Desrosiers C, Ayed IB (2018) HyperDense-net: a hyper-densely connected CNN for multi-modal image segmentation. IEEE Trans Med Imaging 38(5):1116–1126

    Google Scholar 

  13. Ewees AA, Abd Elaziz M, Al-Qaness MA, Khalil HA, Kim S (2020) Improved artificial bee colony using sine-cosine algorithm for multi-level thresholding image segmentation. IEEE Access 8:26304–26315

    Google Scholar 

  14. Fathima E, Anithaa S (2018) An innovative of clustering algorithm for image segmentation using standard deviation, and PSNR. Int J Pure Appl Math 119(12):12415–12422

    Google Scholar 

  15. Gevrekci M, Gunturk BK, Altunbasak Y (2007) POCS-based restoration of Bayer-sampled image sequences. In 2007 IEEE international conference on acoustics, speech and signal processing-ICASSP'07, Honolulu, Hawaii, USA, Vol. 1, pp. I-753

  16. Gibson KB, Nguyen TQ (2013) An analysis of single image defogging methods using a color ellipsoid framework. Eurasip J Image Video Process 1:1–14

    Google Scholar 

  17. Gottapu RD, Dagli CH (2018) DenseNet for anatomical brain segmentation. Procedia Comput Sci 140:179–185

    Google Scholar 

  18. Gu E, Wang J, Xu D, Chen C (2001) Perceptually based approach to color quantization. Proc. of international symposium on multispectral image processing and pattern recognition. ISMIPPR'2001, 4552, 292-297

  19. Haseyama M, Kumagai M, Kitajima H (1999) A genetic algorithm based image segmentation for image analysis. In 1999 IEEE international conference on acoustics, speech, and signal processing. Proceedings (cat. No. 99CH36258), Phoenix, AZ, USA, Vol. 6, pp. 3445-3448

  20. Hastings E, Guha R, Stanley KO (2007) Neat particles: Design, representation, and animation of particle system effects. In 2007 IEEE Symposium on Computational Intelligence and Games, Honolulu, HI, USA, pp. 154–160

  21. Hoekstra D (2010) Fresco: intangible heritage as a Fresco: intangible heritage as a key to unlocking the links between the conservation of biological and cultural diversity in Alamos. Int J Intang Herit 5(6):61–71

    Google Scholar 

  22. Huang L, Xia Y (2020) Joint blur kernel estimation and CNN for blind image restoration. Neurocomputing 396:324–345

    Google Scholar 

  23. Jia S, Zhang Y (2018) Saliency-based deep convolutional neural network for no-reference image quality assessment. Multimed Tools Appl 77(12):14859–14872

    Google Scholar 

  24. Jiang J, Zhuo G, Wang Z (2013) Research of Tibet mural digital images in painting using TV model. Electron Des Eng 2(22):177–179

    Google Scholar 

  25. Jiang B, Woodell GA, Jobson DJ (2015) Novel multi-scale retinex with color restoration on graphics processing unit. J Real-Time Image Proc 14(2):527–253

    Google Scholar 

  26. Jin Z, Iqbal MZ, Bobkov D, Zou W, Li X, Steinbach E (2019) A flexible deep CNN framework for image restoration. IEEE Trans Multimedia 22(4):1055–1068

    Google Scholar 

  27. Kallel F, Hamida AB (2017) A new adaptive gamma correction based algorithm using DWT-SVD for non-contrast CT image enhancement. IEEE Trans Nanobioscience 16(8):666–675

    Google Scholar 

  28. Kamran SA, Sabbir AS (2018) Efficient yet deep convolutional neural networks for semantic segmentation. In 2018 international symposium on advanced intelligent informatics (SAIN), Yogyakarta, Indonesia, pp. 123-130

  29. Kekre HB, Thepade SD (2008) Color traits transfer to grayscale images. In 2008 first international conference on emerging trends in engineering and technology, Nagpur, India, pp. 82-85

  30. Li QQ, Wang H, Zou Q (2018) An algorithm for mural restoration based on sparse representation model. Geomatics and Information Science of Wuhan University, 237

  31. Li C, Tang S, Yan J, Zhou T (2020) Low-light image enhancement via pair of complementary gamma functions by fusion. IEEE Access 8:169887–169896

    Google Scholar 

  32. Lin KY, Wu JH, Xu LH (2015) A survey on color image segmentation techniques. Chinese J Image Graph 01:1–10

    Google Scholar 

  33. Liu P, Zhang H, Zhang K, Lin L, Zuo W (2018) Multi-level wavelet-CNN for image restoration. In proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 773-782

  34. McAuliffe MJ, Lalonde FM, McGarry D, Gandler W, Csaky K, Trus BL (2001) Medical image processing, analysis and visualization in clinical research. In proceedings 14th IEEE symposium on computer-based medical systems, Bethesda, MD, USA, pp. 381-386

  35. Pavićević A (2010) Dead men walking: corpses, relics and icons as cultural/political remnants. Bulgarian Ethnology 1-2:51–61

    Google Scholar 

  36. Prasad S, Kumar P, Sinha KP (2015) Grayscale to color map transformation for efficient image analysis on low processing devices. Adv Intell Syst Comput 320(01):9–18

    Google Scholar 

  37. Schutzius TM, Bayer IS, Jursich GM, Das A, Megaridis CM (2012) Superhydrophobic–superhydrophilic binary micropatterns by localized thermal treatment of polyhedral oligomeric silsesquioxane (POSS)–silica films. Nanoscale 4(17):5378–5385

    Google Scholar 

  38. Shen C, Liu L, Zhu L, Kang J, Wang N, Shao L (2020) High-throughput in situ root image segmentation based on the improved DeepLabv3+ method. Front Plant Sci 11:576791. https://doi.org/10.3389/fpls.2020.576791

  39. Song G, Wang H (2021) Artificial intelligence-assisted Fresco restoration with multiscale line drawing generation. Complexity 4:1–12

    Google Scholar 

  40. Steffens CR, Messias LR, Drews-Jr PJ, Botelho SSDC (2020) CNN based image restoration. J Intell Robot Syst 99(3):609–627

    Google Scholar 

  41. Suhr JK, Jung HG, Li G, Kim J (2010) Mixture of Gaussians-based background subtraction for Bayer-pattern image sequences. IEEE Trans Circuits Syst Video Technol 21(3):365–370

    Google Scholar 

  42. Tang Z, Zhao G, Ouyang T (2021) Two-phase deep learning model for short-term wind direction forecasting. Renew Energy 173:1005–1016. https://doi.org/10.1016/j.renene.2021.04.041

    Article  Google Scholar 

  43. Tian B (2018) Application of special effects art in 3D animation design. J Shandong Inst Commerce Technol 2(2):25–29

    Google Scholar 

  44. Wu Z, Kumar N (2020) Multi-organ nuclei segmentation with fully convolutional DenseNet. IEEE Trans Med Imaging 28(2):136–140

    Google Scholar 

  45. Yadav V, Pavlick RA, Meckler SM, Sen A (2014) Triggered detection and deposition: toward the repair of microcracks. Chem Mater 26(15):4647–4652

    Google Scholar 

  46. Yan X, Hu S, Mao Y, Ye Y, Yu H (2021) Deep multi-view learning methods: a review. Neurocomputing 448:106–129

    Google Scholar 

  47. Yan X, Shi K, Ye Y, Yu H (2022) Deep correlation mining for multi-task image clustering. Expert Syst Appl 187:115973

    Google Scholar 

  48. Yang S, Gao T, Wang J, Deng B, Lansdell B, Linares-Barranco B (2021) Efficient spike-driven learning with dendritic event-based processing. Front Neurosci 15:97

    Google Scholar 

  49. Yang S, Wang J, Deng B, Azghadi MR, Linares-Barranco B (2021) Neuromorphic context-dependent learning framework with fault-tolerant spike routing. IEEE Trans Neural Netw Learn Syst PP:1–15

    Google Scholar 

  50. Yang S, Wang J, Zhang N, Deng B, Pang Y, Azghadi MR (2021) CerebelluMorphic: large-scale neuromorphic model and architecture for supervised motor learning. IEEE Trans Neural Netw Learn Syst33(9):4398–4412. https://doi.org/10.1109/TNNLS.2021.3057070

  51. Yang S, Wang J, Hao X, Li H, Wei X, Deng B, Loparo KA (2021) BiCoSS: toward large-scale cognition brain with multigranular neuromorphic architecture. IEEE Trans Neural Netws Learn Syst 33(7):2801–2815. https://doi.org/10.1109/TNNLS.2020.3045492

  52. Yin WY (2020) Research on virtual color restoration of mural image based on convolutional Neural network. Lanzhou University of Technology, 1–52

  53. Yue Y, Li X, Zhao H, Wang H (2020) Image Segmentation Method of Crop Diseases Based on Improved Segnet Neural Network. In 2020 IEEE International Conference on Mechatronics and Automation (ICMA), Beijing, China, pp. 1986-1991

  54. Zamir SW, Arora A, Khan S, Hayat M, Khan FS, Yang MH, Shao L (2021) Multi-stage progressive image restoration. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14821-14831

  55. Zhang Y, Aydın TO (2021, May) Deep HDR estimation with generative detail reconstruction. Comput Graph Forum 40(2):179–190

    Google Scholar 

  56. Zhang S, Zeng P, Luo X, Zheng H (2012) Multi-scale Retinex with color restoration and detail compensation. J Xi'an Jiaotong Univ 78(5):45–49

    Google Scholar 

  57. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2020) Residual dense network for image restoration. IEEE Trans Pattern Anal Mach Intell 43(7):2480–2495

    Google Scholar 

  58. Zhang G, Raina A, Cagan J, McComb C (2021) A cautionary tale about the impact of AI on human design teams. Des Stud 72:100990

    Google Scholar 

  59. Zhao H, Xiao C, Yu J, Dai Y (2014) Retinex algorithm for night color image enhancement based on WLS. J Beijing Univ Technol 40(3):404–410

    Google Scholar 

  60. Zhu X, Suk HI, Lee SW, Shen D (2015) Subspace regularized sparse multitask learning for multiclass neurodegenerative disease identification. IEEE Trans Biomed Eng 63(3):607–618

    Google Scholar 

Download references

Acknowledgements

The authors acknowledge the support by Fujian Provincial Key Laboratory of Data-Intensive Computing, Fujian University Laboratory of Intelligent Computing and Information Processing, and Fujian Provincial Big Data Research Institute of Intelligent Manufacturing.

Funding statement

The authors received no specific funding for this study.

Author information

Authors and Affiliations

  1. Xiamen Academy of Arts and Design, Fuzhou University, Xiamen, 361021, China

    Wanni Xu

  2. Academy of Arts and Design, Nanchang Institute of Technology, Nanchang, 330044, China

    Wanni Xu & Youlei Fu

  3. School of Creative Arts, Jiangxi Tellhow Animation College, Nanchang, 330200, China

    Wanni Xu

  4. Fine Art and Design College, Quanzhou Normal University, Quanzhou, 362000, China

    Youlei Fu

Authors
  1. Wanni Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Youlei Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wanni Xu.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Ethical compliance

There is no ethics approval required for this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A

figure a
figure b

Appendix B

figure c
figure d
figure e

Appendix C

figure f

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, W., Fu, Y. Deep learning algorithm in ancient relics image colour restoration technology. Multimed Tools Appl 82, 23119–23150 (2023). https://doi.org/10.1007/s11042-022-14108-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14108-z

Keywords

Search

Navigation