Abstract
Optimization-based Image smoothing is routinely formulated as the game between a smoothing energy and an appearance preservation energy. Achieving adequate smoothing is a fundamental goal of these Image smoothing algorithms. We show that partially “erasing” the appearance preservation facilitate adequate Image smoothing. In this paper, we call this manipulation as Erasing Appearance Preservation (EAP). We conduct an user study, allowing users to indicate the “erasing” positions by drawing scribbles interactively, to verify the correctness and effectiveness of EAP. We observe the characteristics of human-indicated “erasing” positions, and then formulate a simple and effective 0-1 knapsack to automatically synthesize the “erasing” positions. We test our synthesized erasing positions in a majority of Image smoothing methods. Experimental results and large-scale perceptual human judgments show that the EAP solution tends to encourage the pattern separation or elimination capabilities of Image smoothing algorithms. We further study the performance of the EAP solution in many image decomposition problems to decompose textures, shadows, and the challenging specular reflections. We also present examinations of diversiform image manipulation applications like texture removal, retexturing, intrinsic decomposition, layer extraction, recoloring, material manipulation, etc. Due to the widespread applicability of Image smoothing, the EAP is also likely to be used in more image editing applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bach, F., et al.: Structured sparsity through convex optimization. Stat. Sci. 27(4), 450–468 (2012)
Bao, L., Song, Y., Yang, Q., Yuan, H., Wang, G.: Tree filtering efficient structure preserving smoothing with a minimum spanning tree. IEEE Trans. Image Process. (2014)
Barrow, H.G., Tenenbaum, J.M.: Recovering intrinsic scene characteristics from images. In: Hanson, A., Riseman, E. (eds.) Computer Vision Systems, pp. 3–26. Academic Press (1978)
Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Graph. 33(4), 1–12 (2014)
Bi, S., Han, X., Yu, Y.: An L1 image transform for edge preserving smoothing and scene level intrinsic decomposition. ACM Trans. Graph. 34(4), 1–12 (2015)
Bousseau, A., Paris, S., Durand, F.: User-assisted intrinsic images. ACM Trans. Graph. (2009)
Buzug, M, T.: Computed Tomography. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-39408-2
Carroll, R., Ramamoorthi, R., Agrawala, M.: Illumination decomposition for material recoloring with consistent interreflections. ACM Trans. Graph. (2011)
Champandard, A.J.: Semantic style transfer and turning two-bit doodles into fine artworks. CoRR abs/1603.01768 (2016)
Chen, Q., Li, D., Tang, C.K.: KNN matting. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2175–2188 (2013)
Cho, H., Lee, H., Kang, H., Lee, S.: Bilateral texture filtering. ACM Trans. Graph. 33(4), 1–8 (2014)
Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)
Fan, Q., Yang, J., Hua, G., Chen, B., Wipf, D.: Revisiting deep intrinsic image decompositions. In: CVPR (2018)
Fan, Q., Yang, J., Wipf, D., Chen, B., Tong, X.: Image smoothing via unsupervised learning. ACM Trans. Graph. 37(6), 1–14 (2018)
Galić, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H.P.: Image compression with anisotropic diffusion. J. Math. Imaging Vis. 31(2–3), 255–269 (2008)
Garces, E., Munoz, A., Lopez-Moreno, J., Gutierrez, D.: Intrinsic images by clustering. In: Computer Graphics Forum (2012)
Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T.: Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: ICCV (2019)
He, K., Sun, J., Tang, X.: Guied image filtering. TPAMI 35(6), 1397–1409 (2013)
Hoeltgen, L., Setzer, S., Weickert, J.: An optimal control approach to find sparse data for laplace interpolation. In: Heyden, A., Kahl, F., Olsson, C., Oskarsson, M., Tai, X.-C. (eds.) EMMCVPR 2013. LNCS, vol. 8081, pp. 151–164. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40395-8_12
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. (Proc. of SIGGRAPH 2017) 36(4), 107:1–107:14 (2017)
Kovacs, B., Bell, S., Snavely, N., Bala, K.: Shading annotations in the wild. In: CVPR (2017)
Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. In: ACM SIGGRAPH 2004 Papers, SIGGRAPH 2004, pp. 689–694. Association for Computing Machinery, New York (2004)
Li, Z., Snavely, N.: CGIntrinsics: better intrinsic image decomposition through physically-based rendering. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 381–399. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_23
Li, Z., Snavely, N.: Learning intrinsic image decomposition from watching the world. In: CVPR (2018)
Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. In: The European Conference on Computer Vision (ECCV) (2018)
Min, D., Choi, S., Lu, J., Ham, B., Sohn, K., Do, M.N.: Fast global image smoothing based on weighted least squares. IEEE Trans. Image Process. 23(12), 5638–5653 (2014)
Prasath, V.S., Vorotnikov, D., Pelapur, R., Jose, S., Seetharaman, G., Palaniappan, K.: Multiscale Tikhonovtotal variation image restoration using spatially varying edge coherence exponent. IEEE Trans. Image Process. 24(12), 5220–5235 (2015)
Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D: Nonlinear Phenomena 60(1–4), 259–268 (1992)
Serra, M., Penacchio, O., Benavente, R., Vanrell, M.: Names and shades of color for intrinsic image estimation. In: CVPR (2012)
Shen, J., Yang, X., Jia, Y., Li, X.: Intrinsic images using optimization. In: CVPR (2011)
Tomasi, C.: Bilateral filtering for gray and color images. In: ICCV (1998)
Holland, P.W., Welsch, R.E.: Robust regression using iteratively reweighted leastsquares. Commun. Stat. Theory Methods 6(9), 813–827 (1977)
Xu, L., Lu, C., Xu, Y., Jia, J.: Image smoothing via L0 gradient minimization. ACM Trans. Graph. (2011)
Xu, L., Yan, Q., Xia, Y., Jia, J.: Structure extraction from texture via relative total variation. ACM Trans. Graph. 31(6), 1–10 (2012)
Yang, J., Zhang, Y., Yin, W.: An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise. SIAM J. Sci. Comput. 31(4), 2842–2865 (2009)
Yin, H., Gong, Y., Qiu, G.: Side window filtering. In: CVPR (2019)
Zhao, Q., Tan, P., Dai, Q., Shen, L., Wu, E., Lin, S.: A closed-form solution to retinex with nonlocal texture constraints. TPAMI 34(7), 1437–1444 (2012)
Zhou, H., Yu, X., Jacobs, D.W.: Glosh: global-local spherical harmonics for intrinsic image decomposition. In: ICCV (2019)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, L., Li, C., Ji, Y., Liu, C., Wong, Tt. (2020). Erasing Appearance Preservation in Optimization-Based Smoothing. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12351. Springer, Cham. https://doi.org/10.1007/978-3-030-58539-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-58539-6_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58538-9
Online ISBN: 978-3-030-58539-6
eBook Packages: Computer ScienceComputer Science (R0)