Abstract
Smoothing images, especially with rich texture, is an important problem in computer vision. Obtaining an ideal result is difficult due to complexity, irregularity, and anisotropicity of the texture. Besides, some properties are shared by the texture and the structure in an image. It is a hard compromise to retain structure and simultaneously remove texture. To create an ideal algorithm for image smoothing, we face three problems. For images with rich textures, the smoothing effect should be enhanced. We should overcome inconsistency of smoothing results in different parts of the image. It is necessary to create a method to evaluate the smoothing effect. We apply texture pre-removal based on global sparse decomposition with a variable smoothing parameter to solve the first two problems. A parametric surface constructed by an improved Bessel method is used to determine the smoothing parameter. Three evaluation measures: edge integrity rate, texture removal rate, and gradient value distribution are proposed to cope with the third problem. We use the alternating direction method of multipliers to complete the whole algorithm and obtain the results. Experiments show that our algorithm is better than existing algorithms both visually and quantitatively. We also demonstrate our method’s ability in other applications such as clip-art compression artifact removal and content-aware image manipulation.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Sun, Y. J.; Schaefer, S.; Wang, W. P. Image structure retrieval via L minimization. IEEE Transactions on Visualization and Computer Graphics Vol. 24, No. 7, 2129–2139, 2018.
Tomasi, C.; Manduchi, R. Bilateral filtering for gray and color images. In: Proceedings of the International Conference on Computer Vision, 839–846, 1998.
Chen, J. W.; Paris, S.; Durand, F. Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics Vol. 26, No. 3, 103, 2007.
Weiss, B. Fast median and bilateral filtering. ACM Transactions on Graphics Vol. 25, No. 3, 519–526, 2006.
Cho, H.; Lee, H.; Kang, H.; Lee, S. Bilateral texture filtering. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 128, 2014.
Bao, L. C.; Song, Y. B.; Yang, Q. X.; Yuan, H.; Wang, G. Tree filtering: Efficient structure-preserving smoothing with a minimum spanning tree. IEEE Transactions on Image Processing Vol. 23, No. 2, 555–569, 2014.
Perona, P.; Malik, J. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 12, No. 7, 629–639, 1990.
He, K. M.; Sun, J.; Tang, X. O. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 6, 1397–1409, 2013.
Subr, K.; Soler, C.; Durand, F. Edge-preserving multiscale image decomposition based on local extrema. In: Proceedings of the ACM SIGGRAPH Asia 2009 Papers, Article No. 147, 2009.
Farbman, Z.; Fattal, R.; Lischinski, D.; Szeliski, R. Edge-preserving decompositions for multi-scale tone and detail manipulation. In: Proceedings of the ACM SIGGRAPH 2008 Papers, Article No. 67, 2008.
Lindeberg, T. Scale-space theory: A basic tool for analyzing structures at different scales. Journal of Applied Statistics Vol. 21, Nos. 1–2, 225–270, 1994.
Mikolajczyk, K.; Schmid, C. An affine invariant interest point detector. In: Proceedings of the 7th European Conference on Computer Vision-Part I, 128–142, 2002.
Cai, B. L.; Xing, X. F.; Xu, X. M. Edge/structure preserving smoothing via relativity-of-Gaussian. In: Proceedings of the IEEE International Conference on Image Processing, 250–254, 2017.
Rudin, L. I.; Osher, S.; Fatemi, E. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena Vol. 60, Nos. 1–4, 259–268, 1992.
Xu, L.; Lu, C. W.; Xu, Y.; Jia, J. Y. Image smoothing via L gradient minimization. ACM Transactions on Graphics Vol. 30, No. 6, https://doi.org/10.1145/2070781.2024208, 2011.
Ono, S. L gradient projection. IEEE Transactions on Image Processing Vol. 26, No. 4, 1554–1564, 2017.
Ono, S. Edge-preserving filtering by projection onto L gradient constraint. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 1492–1496, 2017.
Xu, L.; Yan, Q.; Xia, Y.; Jia, J. Y. Structure extraction from texture via relative total variation. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 139, 2012.
Chen, Q. F.; Xu, J.; Koltun, V. Fast image processing with fully-convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, 2516–2525, 2017.
Xu, L.; Ren, J.; Yan, Q.; Liao, R.; Jia, J. Deep edge-aware filters. In: Proceedings of the 32nd International Conference on Machine Learning, Vol. 37, 1669–1678, 2015.
Farbman, Z.; Fattal, R.; Lischinski, D.; Szeliski, R. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics Vol. 27, No. 3, https://doi.org/10.1145/1360612.1360666, 2008.
Kim, Y.; Ham, B.; Do, M. N.; Sohn, K. Structure-texture image decomposition using deep variational priors. IEEE Transactions on Image Processing Vol. 28, No. 6, 2692–2704, 2019.
Zhao, M.; Zhang, H. X.; de Sun, J. A novel image retrieval method based on multi-trend structure descriptor. Journal of Visual Communication and Image Representation Vol. 38, 73–81, 2016.
Zhang, F.; Li, J. J.; Liu, P. Q.; Fan, H. Computing knots by quadratic and cubic polynomial curves. Computational Visual Media Vol. 6, No. 4, 417–430, 2020.
Liu, X. X.; Zhang, Y. F.; Bao, F. X.; Shao, K.; Sun, Z. Y.; Zhang, C. M. Kernel-blending connection approximated by a neural network for image classification. Computational Visual Media Vol. 6, No. 4, 467–476, 2020.
Wang, Y. L.; Yang, J. F.; Yin, W. T.; Zhang, Y. A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences Vol. 1, No. 3, 248–272, 2008.
Gu, S. H.; Xie, Q.; Meng, D. Y.; Zuo, W. M.; Feng, X. C.; Zhang, L. Weighted nuclear norm minimization and its applications to low level vision. International Journal of Computer Vision Vol. 121, No. 2, 183–208, 2017.
Eckstein, J.; Bertsekas, D. P. On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming Vol. 55, Nos. 1–3, 293–318, 1992.
Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends®in Machine Learning Vol. 3, No. 1, 1–122, 2011.
Zhang, R.; Kwok, J. Asynchronous distributed ADMM for consensus optimization. In: Proceedings of the 31st International Conference on International Conference on Machine Learning, Vol. 32, II-1701–II-1709 2014.
Sun, D. L.; Févotte, C. Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 6201–6205, 2014.
Zhang, M. L.; Desrosiers, C. High-quality image restoration using low-rank patch regularization and global structure sparsity. IEEE Transactions on Image Processing Vol. 28, No. 2, 868–879, 2019.
Ma, G. H.; Zhang, M. L.; Li, X. M.; Zhang, C. M. Image smoothing based on image decomposition and sparse high frequency gradient. Journal of Computer Science and Technology Vol. 33, No. 3, 502–510, 2018.
Zhu, F. D.; Liang, Z. T.; Jia, X. X.; Zhang, L.; Yu, Y. Z. A benchmark for edge-preserving image smoothing. IEEE Transactions on Image Processing Vol. 28, No. 7, 3556–3570, 2019.
Jian, M. W.; Zhang, W. Y.; Yu, H.; Cui, C. R.; Nie, X. S.; Zhang, H. X.; Yin, Y. Saliency detection based on directional patches extraction and principal local color contrast. Journal of Visual Communication and Image Representation Vol. 57, 1–11, 2018.
Acknowledgements
This work was supported by NSFC Joint Fund with Zhejiang Integration of Informatization and Industrialization under Key Project (U1609218).
Author information
Authors and Affiliations
Corresponding author
Additional information
Xiang Ma is currently a Ph.D. student in the School of Software, Shandong University, Jinan. He received his master degree from Shandong University in 2020. His research interests include image processing and information visualization.
Xuemei Li received her master and doctor degrees from Shandong University in 2004 and 2010, respectively. She is currently an associate professor in the School of Software, Shandong University, and a member of the GDIV Lab. She is engaged in research on geometric modeling, CAGD, image processing, and information visualization.
Yuanfeng Zhou received his master and Ph.D. degrees from the School of Software, Shandong University, in 2005 and 2009, respectively. He held a post-doctoral position with the Graphics Group, Department of Computer Science, University of Hong Kong, from 2009 to 2011. He is currently a professor with the School of Software, Shandong University, where he is also a member of the GDIV Lab. His current research interests include computer graphics, information visualization, and image processing.
Caiming Zhang is a professor and doctoral supervisor of the Software College at Shandong University. He is now also the dean of the Digital Media Research Institute at Shandong University of Finance and Economics. He received his B.S. and M.E. degrees in computer science from Shandong University in 1982 and 1984, respectively, and his Dr.Eng. degree in computer science from the Tokyo Institute of Technology, Japan, in 1994. From 1997 to 2000, he held a visiting position at the University of Kentucky, USA. His research interests include CAGD, CG, information visualization, and medical image processing.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Ma, X., Li, X., Zhou, Y. et al. Image smoothing based on global sparsity decomposition and a variable parameter. Comp. Visual Media 7, 483–497 (2021). https://doi.org/10.1007/s41095-021-0220-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-021-0220-1