Abstract
A clear stele image of ancient Chinese calligraphy pieces is very useful for studying ancient Chinese calligraphy. However, due to hundreds of or even thousands of years of natural or artificial damage on stele, images of ancient Chinese stele calligraphy works usually suffer from a large amount of image noise, and which usually leads to a poor visibility. To address this problem, in this paper, we propose a de-noising method based on L0 gradient minimization and guided filter. It consists of two main operations in sequence: First, L0 gradient minimization is utilized to obtain a random-noise free map, and then the random-noise free map is used as a guided image, and convoluted with its corresponding original noised stele image by a guided filter to obtain an edge preserved random-noise free image. Finally, the eight-connection region-based de-noising technique is followed to remove ant-like isolated blocks. Experiments demonstrate that the proposed method is superior to several recent published stele image de-noising techniques in terms of preserving the character structures.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.1 Introduction
Ancient Chinese calligraphy works are very useful for people to learn and study Chinese history and culture [1]. From these works, we can enjoy the beauty of ancient Chinese culture and learn about how the ancient Chinese lived; furthermore, we can learn about where Chinese history and culture come from and where it will go. Of these calligraphy works, most of them used steles as their carriers. However, because of hundreds of or even thousands of years of natural or artificial damage on steles, images of ancient Chinese stele calligraphy works usually suffer from poor visibility with a large amount of noise, and the structure of characters in regions of the stele images is too spoiled by noise to see, as shown in Fig. 1. Hence, it is highly desirable to remove noise from ancient Chinese stele calligraphy images.
However, to the best of our knowledge, though there are many works on image de-noising [2,3,4], few works have focused on ancient Chinese calligraphy images. For this reason, a de-noising method for ancient Chinese calligraphy works on steles based on L0 gradient minimization and guided filter is proposed in this paper. We first utilize L0 gradient minimization to smooth the noised stele image. The goal of this step is to obtain random-noise free maps. Then, we take the random-noise free map as a guided image and convolute it with its corresponding original noised stele image via a guided filter. The goal of this step is to obtain an edge preserved random-noise free image. Finally, we used the eight-connection region-based de-noising technique as a tool to remove isolated blocks. Experiments show that our method is superior to several recent published stele image de-noising techniques in preserving characters’ structures.
Our contributions in this work are summarized as follows:
-
1)
We put forward a L0 gradient minimization guided image filter for random-noise removal. It consists of two operation steps: First, to smooth a noised stele image by using L0 gradient minimization technique. Then, to compute an edge-preserved noise free image by convoluting the smoothed image with the input noised stele image.
-
2)
We employ the eight-connected domain technique for ant-like noise removal. The effectiveness of the proposed method is demonstrated by experimental results.
The remainder of this paper is structured as follows: Section 2 simply introduces the related works. Section 3 describes the proposed method in detail. Section 4 presents all the experimental results. Section 5 concludes the paper.
2 Related work
Interests regarding image preprocessing, including image de-noising [5] and image enhancement [6], especially on ancient Chinese calligraphy image enhancement [1,2,3,4, 7,8,9,10] have seen increasing in recent years; for instance, Zheng et.al. [1] presented a de-noising method for stele images using guided filter on the L channel. Wang et al. [7] used a threshold method to remove noise from a Chinese calligraphy image. In [3], an adaptive shock filter based on anisotropic diffusion is proposed. In [8], the Laplacian pyramid is combined with the fractional Fourier domain to extract the structures of characters with rotation angles at different scales. In [9], an anisotropic diffusion filter with the width information of different strokes was proposed for noise removal. In [10], structure characteristics of Chinese character strokes and run-length statistics were used for stele de-noising. In [11], an integration of multiple de-noising filters was proposed for removing block and line noise from stele images. In [12], KSVD dictionary learning was used for de-noising as well as for preserving the character structures.
Although there are many works on stele image processing, research on stele image de-nosing are still very limited. Solutions for ancient Chinese calligraphy image de-noising are still opened.
Recently, a piece wise filtering model termed L0 gradient minimization has been employed for image de-noising. By optimizing an energy function based on prior gradient sparsity, the L0 gradient minimization model shows a decent ability to perform image de-noising and a strong ability to preserve sharp features [13] in respect to traditional image de-noising method, such as Gaussian filter and shock filter. Similar works are also found in [14, 15].
Motivated by the aforementioned research, in this paper, we employ L0 gradient minimization as well as a guided filter for ancient stele calligraphy image de-noising.
3 Proposed method
3.1 Overview
Figure 2 shows the framework of our presented method. Firstly, a stele image is smoothed using L0 gradient minimization. The goal of this step is to obtain a random-noise free map. This random-noise free map is then convoluted with its corresponding original noised stele image to obtain an edge preserved random-noise free image. Next, a connection region based de-noising technique is used to remove isolated blocks. Details of each step are described as follows.
3.2 Random-noise free map computation with L0 gradient minimization
The goal of the computation in this section is to obtain a random-noise free map. For this purpose, there are several filters that can be chosen, such as the Gaussian filter, shock filter, and L0 gradient minimization [13]. Results with these filters were nearly identical. However, with respect to Gaussian filter and shock filter, the L0 gradient minimization model shows a decent ability to perform image de-noising and a strong ability to preserve sharp features by optimizing an energy function based on prior gradient sparsity [13]. For what was mentioned above, in this section, we select the L0 gradient minimization as the tools for random-noise free map computation. Firstly, we decompose the input image into two parts using L0 gradient minimization: a base image and detailed image. The L0 norm of the gradient specifies the number of non-zero gradients [13] and drives the minimization to produce an output that is comparable to input around strong edges (as shown in Fig. 2 by the color blocks), which is based on the previous gradient.
Let I(x,y) be an input image and B(x,y) be its corresponding base image, and D(x,y) be its corresponding detail image. The gradients of I(x,y) are denoted as ∇I(x, y). Formally, we define the energy function of L0 gradient minimization as:
where γ controls the level of detail in the final image and ∇I(x, y) represents the number of non-zero gradient and is described as:
where # denotes the pixel number and ∇ indexes the operator difference.
The gradient L0 norm is usually computed at positions containing sharp edges because it denotes the numbers of non-zero gradients. In term of this view, we multiply a binary map Edge(x,y) on the original regularization term ∇I(x, y) to speed up the process.
where Edge(x,y) denotes a binary map. Any technique that can obtain sharp edges can be used to get the binary map, such as applying the differential of Gaussian (DOG) filter on I(x,y) with a threshold, or applying the Laplacian of Gaussian filter on I(x,y) with a threshold. In this paper, we apply the differential of Gaussian (DOG) filter on I(x,y) with a threshold to get the binary map. The threshold value is determined based on experiments.
To easily calculate the L0 norm of gradient via Eq. (1), two dependent variables αx(x, y) and αy(x, y) are introduced, corresponding to\( \frac{\partial B\left(x,y\right)}{\partial x} \) and\( \frac{\partial B\left(x,y\right)}{\partial y} \), respectively. Equation (1) is reconstructed as:
where β is used to regulate the difference between the dependent variables and its corresponding gradients. To address this minimization, an alternating minimization strategy is used, viz. We keep one of the variable sets unchanged to obtain another variable set. In our work, we keep αx(x, y) and αy(x, y) unchanged and use the below cost function to first obtain B(x,y):
Then, we keep the evaluated B(x,y) unchanged and through the following cost function obtain αx(x, y) and αy(x, y)
where we model the L0 norm of gradient as:
where H(| αx(x, y)| + | αy(x, y)| ) is a binary function returning 1 when ∣αx(x, y) ∣ + ∣ αy(x, y) ∣ ≠ 0; otherwise, it returns 0. By alternatively computing (5, 6), we obtain the final base image B(x,y) and its corresponding detail image D(x,y). Because all of the random-noise remains in the detail image D(x,y), we take the final base image B(x,y) as the random-noise free map. Figure 3 shows an example of random-noise removal results with the L0 gradient minimization operation. It is observed that the L0 gradient minimization smoothed image (Fig. 3b) contains little noise compared with the original image (Fig. 3a).
3.3 Recover of the Chinese character structure using a guided filter
After the L0 gradient minimization operation, we obtain a random-noise-free image, as shown in Fig. 3b. However, we find that some stroke edge details are removed as random noise by the L0 minimization operation, as indicated by the yellow box in Fig. 3. The guided filter is an edge-preserving filter [16]. Its goal is to smooth input images by calculating the content of the guidance image. Many studies shows that edges in an image after using guided filter will change differently. For step edges, it is still step edges after using guided filter, but their ranges become smaller, which means that the step edges become smoother after guided filter; For ridge edges, if the ridge edges with small size are unaffected by the other edges, their variances are close to 0, then the ridge edges will disappear and tend to the background; Valley edges will become larger than the input. From what was mentioned above, we can see that the guided filter has well preserving ability on image edges. Therefore, it can be used for image texture recover.
For what was mentioned above, to recover the over-smoothed stroke edge details, guided filter is employed in this section.
Formally, given a guidance image I and an input image Iin, the corresponding guided filtering output image\( {I}_{\mathrm{out}}^{\mathrm{guided}} \) is defined as:
where ωk is a window centered at the pixel x, αk and bk are constants in ωk, respectively. These two parameters are determined as:
where μk indexes the average value of input image Iin in window ωk , and \( {\upsigma}_k^2 \) is the variance of input image Iin in window ωk .nω is the number of pixels in window ωk. \( {\overline{p}}_k \) represents the mean of the guided image in window ωk. ε represents the regularization parameter which is used to determine the changing of pixels’ intensity. The initial value of parameter ε in Eq. (9) is determined by referring [16], and its finial optimized value is determined by experiment.
In this work, for the use of guided filter, we employ the random-noise-free map obtained in above section as a guided image and convolute it with the input noised stele image to recover the Chinese character structure in a stele image. Figure 4 shows some of the results of recovering the Chinese character structure using the guided filter. As observed from the figure, the incomplete character strokes (as show in Fig. 4b) caused by the over-smoothing of the L0 gradient minimization operation are recovered well; these strokes are indicated by the yellow boxes in Fig. 4c.
3.4 Ant-like blocks removal using the eight-connected region technique
From Fig. 4, we also find that although most of the noise in the input image is removed after L0 gradient minimization filtering, there remains a few isolated ant-like blocks in the smoothed images, as denoted by the red circle in Fig. 4c.
One simple relationship between pixels is connectivity. Based on the connectivity of pixels, we can easily determine a pixel is an image component or image noise. To do this, there are two connected region techniques can be used: the four-connected region technique and the eight-connected region technique [10]. Due to the four-connected region technique considered as neighbors, only the four pixels that share an edge (not a corner) with the pixel in question: (x + 1,y), (x − 1,y), (x,y + 1), and (x,y − 1); pixels on the diagonal could not be taken as an edge though they are serve as an effective insulator between two pixel set, and as a result, this creates undesirable topological anomalies, whereas the eight-connected region technique consider a pixel as connected not only pixels on the same row or column, but also the diagonal pixels. For this reason, also inspired by the work described [10], we utilize the eight-connected region technique [10] in this section to address the ant-like blocks problem. The processing to remove ant-like blocks using eight-connected region technique consists of the following three basic steps:
-
1)
Compute the connected components using an eight-connected domain-based search technique;
-
2)
Calculate each component’s area first. Then, sort all of connected components in terms of their area from large to small. Many studies index that if the area value of a block is smaller than the area value of 2/3 of the array of permutations, then the block could be taken as an isolated noise component and could be removed from an image. For this reason, we take the area value of 2/3 of the array of permutations as the value of threshold T;
-
3)
Remove all of isolated components when the area is smaller than the value of threshold T.
Figure 5 shows partial results of the ant-like block removal via the eight-connected region technique. It is observed that almost all of the ant-like blocks are removed from the input stele image, and the characters in the image are highlighted and the structures of the strokes have been preserved perfectly (as shown in Fig. 5d) compared with that of the input stele image.
4 Experimental results and discussions
4.1 Experimental setting
To the best of our knowledge, there is no standard ancient Chinese stele calligraphy image set available for benchmarking currently. Therefore, we collected 100 stele calligraphy images from the released image set by [1]. We also generated 50 synthetically noised stele images using Adobe Photoshop CS6 to quantitatively evaluate the proposed method.
For comparison, several state of the art image de-noising methods, including Zheng’s method [1], Zhang’s method [10], L0 gradient minimization smoothing [13], and block-matching 3D filtering (BM3D) [17], as well as methods in our two prior works [11, 12] are employed in our experiments. The results of these methods are produced in MatLab with authors suggested parameter settings. All experiments are performed on a PC with 4G RAM and a 2.60 GHz Intel Dual Core processor using MatLab R2015.
4.2 Qualitative results
Figure 6 shows partial processing results with our method; the top row shows the original images, and the bottom row shows our processing results. From Fig. 6, it is observed that our method can remove most of the random-noise and ant-like noise without spoiling the characters.
In Fig. 7, we compared our proposed method with Zheng’s method [1]. It can be observed that Zheng’s method failed to remove larger block noise from the images (Fig. 7b). However, the proposed method worked well for removing all types of noise (as shown in Fig. 7c).
In Fig. 8, we compared our method with Zhang’s method [10]. It can be seen that both the two methods worked well in removing image noise, whereas our method obtained more smooth edges (as shown in Fig. 8c), compared with the results processed by Zhang’s method [10] where a small amount of “flash” remained in the edges (as shown in Fig. 8b).
Figure 9 compares the proposed method with the L0 gradient minimization algorithm [13] and the block-matching 3D filtering (BM3D) algorithm [17]. It is observed that our proposed method retains more details than the other two methods and does not spoil the character structures.
In Fig. 10, we compare the proposed method in this paper with our two prior works, including an integrated de-noising method [11] and the KSVD de-noising method [12]. It is observed that the three methods were able to take out of most image noise from the input noisy images, whereas it should be noticed that the proposed method in this paper obtains the best results with clearer images.
4.3 Quantitative evaluation
In this section, we quantitative evaluate the performance of our method on synthetic and real noised stele images, respectively.
For evaluation on the synthetic noised stele calligraphy images, peak signal-to-noise ratio (PSNR) and Structure Similarity Index (SSIM) [18] are employed as evaluation measures because of available ground truth, where the larger the values are, the better the results are. Referred to [18], the definition formulation of PSNR and SSIM are described as Eq. (11, 12), respectively.
where I is a noise-free image, K is its noisy approximation, and MSE is the mean squared error (MSE) of I and K.
where μI is the average of I, μK is the average of K, \( {\sigma}_I^2 \) is the variance of I, \( {\sigma}_K^2 \)is the variance of K, σI, K is the covariance of I and K, c1 and c2 are two variables to stabilize the division with weak denominator.
Table 1 shows the evaluation result. From Table 1, as we can observe that the values of SNR and SSIM for our method in this work are 32.834 and 0.9952, that for Zheng’s method [1] are 26.801 and 0.9841, that for Zhang’s method [10] are 29.010 and 0.9858, that for L0 [13] are 27.062 and 0.9742, that for BM3D [17] are 27.960 and 0.9801, that for our prior work [11] are 32.690 and 0.9882, and that for our prior work [12] are 32.783 and 0.9916. These results clearly indexes that our proposed method in this work achieves the best de-noising performance of all evaluated method in this experiment in terms of both of PSNR and SSIM.
For evaluation on real noisy stele images, we employ the overall quality index [19] as evaluation measures because of unavailable ground truth. A larger overall quality index value indexes a better enhanced visibility. Table 2 shows the evaluation results in this experiment. From Table 2, it is observed that the proposed method obtained an average overall quality index value of 0.7056, which is much higher than that of other methods, including 0.6795 for Zheng’s method [1], 0.6865 for Zhang’s method [10], 0.6538 for L0 gradient minimization [13], 0.6895 for our prior work [11], and 0.6905 for our prior work [12]. This result indicates that our proposed method in this work remarkably improves the visibility of images in respect to other methods.
5 Conclusions
We put forward a de-noising method based on L0 gradient minimization and guided filtering method for ancient Chinese stele calligraphy image de-noising in this paper. Experiments by comparing our presented method with several state of the art de-noising methods show the superior of our method for stele image de-noising.
In our next work, we will focus on how to restore spoiled character structures for the reason of preserving the beauty of art work.
References
X. Zheng, Q. Miao, Z. Shi, Y. Fan, W. Shui, A new artistic information extraction method with multi channels and guided filters for calligraphy works. Multimedia Tools & Applications 75(14), 8719–8744 (2015). https://doi.org/10.1007/s11042-015-2788-x
J. Chen, X. Kang, Y. Liu, Z. Wang, Median filtering forensics based on convolutional neural networks. IEEE Signal Processing Letters 22(11), 1849–1853 (2015). https://doi.org/10.1109/LSP.2015.2438008
K. Guemri, F. Drira, in Proc. of 6th International Conference of Soft Computing and Pattern Recognition, Tunis, Tunisia. Adaptative shock filter for image characters enhancement and denoising (2015), pp. 279–283
H. Li, C.Y. Suen, A novel non-local means image denoising method based on grey theory. Pattern Recogn. 49(1), 237–248 (2016). https://doi.org/10.1016/j.patcog.2015.05.028
Y. B, An improved infrared image processing method based on adaptive threshold denoising. EURASIP Journal on Image and Video Processing 2019, 5 (2019) https://doi.org/10.1186/s13640-018-0401-8
Z. Shi, M. Zhu, B. Guo, M. Zhao, Nighttime low illumination image enhancement with single image using bright/dark channel prior. EURASIP Journal on Image and Video Processing 2018, 13 (2018) https://doi.org/10.1186/s13640-018-0251-4
S.Z. Wang, H.J. Lee, in Proc. of 6th International Conference on Document Analysis and Recognition, Seattle, WA, USA. Dual-binarization and anisotropic diffusion of Chinese characters in calligraphy documents (2001), pp. 271–275
H.T. Nguyen, N. Linh-Trung, in Proc. of 10th International Conference on Information Sciences Signal Processing and their Applications, Kuala Lumpur, Malaysia. The Laplacian pyramid with rational scaling factors and application on image denoising (2010), pp. 468–471
T. Koga, E. Uchino, N. Suetake, G. Hashimoto, T. Hiro, M. Matsuzaki, in Proc. of the 7th Conference on Circuits, Systems, Electronics, Control and Signal Processing, Tenerife, Canary Islands, Spain. Edge-preserved smoothing method with special reference to intravascular ultrasound image using anisotropic diffusion filter controlled by weighted separability measure (2008), pp. 87–92
J. Zhang, J. Yu, G. Mao, and X. Ye, Denoising of Chinese calligraphy tablet images based on run-length statistics and structure characteristic of character strokes, Journal of Zhejiang University SCIENCE A, vol.7, no.7, pp.1178–1186, July. 2006,https://doi.org/10.1631/jzus.2006.A
Z. Shi, B. Xu, X. Zheng, M. Zhao, An integrated method for ancient Chinese tablet images de-noising based on the assemble of image smooth and noise reduction. Multimedia Tools & Applications 75(19), 12245–12261 (2016). https://doi.org/10.1007/s11042-016-3421-3
Z. Shi, B. Xu, X. Zheng, and M. Zhao, A KSVD based Chinese character structure preserved denoising method” Multimedia Tools & Applications, vol.76, no.13, pp.14921U˝ 14936, 2017,https://doi.org/10.1007/s11042-016-4284-3
L. Xu, C. Lu, Y. Xu, J. Jia, Image smoothing via L0 gradient minimization. ACM Trans. Graph. 30(6), 1U˝ 12 (2011). https://doi.org/10.1145/2024156.2024208
Z. Jin, G. Zhou, D. Gao, Y. Zhang, EEG classification using sparse Bayesian extreme learning machine for brain–computer interface. Neural Comput. & Applic. (2018). https://doi.org/10.1007/s00521-018-3735-3
Y. Zhang, C. Nam, G. Zhou, J. Jin, X. Wang, A. Cichocki, in IEEE Transactions on Cybernetics. Temporally constrained sparse group spatial patterns for motor imagery BCI (2018), pp. 1–11. https://doi.org/10.1109/TCYB.2018.2841847
K. He, J. Sun, X. Tang, in Computer Vision- ECCV 2010. ECCV 2010, ed. by K. Daniilidis, P. Maragos, N. Paragios. Guided image filtering (Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 2010), pp. 1–14
C.A. Metzler, A. Maleki, R.G. Baraniuk, in Proc. of 2015 IEEE International Conference on Image Processing, Quebec City, QC, Canada. BM3D-AMP: a new image recovery algorithm based on BM3D denoising (2015), pp. 3116–3120
A. Hore, D. Ziou, in Proc.of 20th International Conference on Pattern Recognition, Istanbul, Turkey. Image quality metrics: PSNR vs. SSIM (2010), pp. 2366–2369
Z. Wang, A.C. Bovik, Universal image quality index. IEEE Signal Processing Letters 9(3), 81–84 (2002)
Acknowledgements
The authors thank the editor and anonymous reviewers for their helpful comments and valuable suggestions.
Funding
Not applicable.
Availability of data and materials
Please contact author for data requests.
Author information
Authors and Affiliations
Contributions
All authors take part in the discussion of the work described in this paper. All authors read and approved the final manuscript.
Corresponding authors
Ethics declarations
Authors’ information
Feihang Ge received the B.E. degree and the M.S. degree in computer science and technology from Xi’an University of Technology in 2002 and 2005, respectively. He is currently studying for his Ph. D Degree at Aichi Prefectural University, Aichi, Japan. His research interests include intelligent image processing and computer vision. E-mail: fhge2018@163.com
Lifeng He received the B.E. degree from Northwest Institute of Light Industry, Shaanxi, China, in 1982, the second B.E. degree from Xian Jiaotong University, Shaanxi, in 1986, and the M.S. and the Ph.D. degrees in artificial intelligence and computer science from Nagoya Institute of Technology, Aichi, Japan, in 1994 and 1997, respectively. He is currently a Professor at Aichi Prefectural University, Aichi, and a Guest Professor at Shaanxi University of Science and Technology, Shaanxi. From September 2006 to May 2007, he was a Research Associate at The University of Chicago. His research interests include intelligent image processing, computer vision, medical image processing. E-mail: helifeng@ist.aichi-pu.ac.jp
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Ge, F., He, L. A de-noising method based on L0 gradient minimization and guided filter for ancient Chinese calligraphy works on steles. J Image Video Proc. 2019, 32 (2019). https://doi.org/10.1186/s13640-019-0423-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13640-019-0423-x