Abstract
Gradient-domain compositing is an essential tool in computer vision and its applications, e.g., seamless cloning, panorama stitching, shadow removal, scene completion and reshuffling. While easy to implement, these gradient-domain techniques often generate bleeding artifacts where the composited image regions do not match. One option is to modify the region boundary to minimize such mismatches. However, this option may not always be sufficient or applicable, e.g., the user or algorithm may not allow the selection to be altered. We propose a new approach to gradient-domain compositing that is robust to inaccuracies and prevents color bleeding without changing the boundary location. Our approach improves standard gradient-domain compositing in two ways. First, we define the boundary gradients such that the produced gradient field is nearly integrable. Second, we control the integration process to concentrate residuals where they are less conspicuous. We show that our approach can be formulated as a standard least-squares problem that can be solved with a sparse linear system akin to the classical Poisson equation. We demonstrate results on a variety of scenes. The visual quality and run-time complexity compares favorably to other approaches.
Chapter PDF
References
Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. on Graphics 22 (2003)
Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D.H., Cohen, M.F.: Interactive digital photomontage. ACM Trans. on Graphics 23 (2004)
Georgiev, T.: Covariant derivatives and vision. In: European Conf. on Computer Vision (2006)
Jia, J., Sun, J., Tang, C.K., Shum, H.Y.: Drag-and-drop pasting. ACM Trans. on Graphics 25 (2006)
Levin, A., Zomet, A., Peleg, S., Weiss, Y.: Seamless image stitching in the gradient domain. In: European Conf. on Computer Vision (2006)
Agarwala, A.: Efficient gradient-domain compositing using quadtrees. ACM Trans. on Graphics 26 (2007)
Sivic, J., Kaneva, B., Torralba, A., Avidan, S., Freeman, W.T.: Creating and exploring a large photorealistic virtual space. In: IEEE Workshop on Internet Vision (2008)
Whyte, O., Sivic, J., Zisserman, A.: Get out of my picture! Internet-based inpainting. In: British Machine Vision Conf. (2009)
Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. International Journal of Computer Vision (2009)
Hays, J., Efros, A.A.: Scene completion using millions of photographs. ACM Trans. on Graphics 26 (2007)
Cho, T.S., Avidan, S., Freeman, W.T.: The patch transform. IEEE Trans. on Pattern Analysis and Machine Intelligence (2010)
Tappen, M.F., Adelson, E.H., Freeman, W.T.: Recovering intrinsic images from a single image. IEEE Trans. on Pattern Analysis and Machine Intelligence 27 (2005)
Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. IEEE Trans. on Pattern Analysis and Machine Intelligence 28 (2006)
Agrawal, A., Raskar, R., Chellappa, R.: What is the range of surface reconstructions from a gradient field? In: European Conf. on Computer Vision (2006)
Bhat, P., Zitnick, C.L., Cohen, M., Curless, B.: Gradientshop: A gradient-domain optimization framework for image and video filtering. ACM Trans. on Graphics 28 (2009)
Lalonde, J.F., Hoiem, D., Efros, A., Rother, C., Winn, J., Criminisi, A.: Photo clip art. ACM Trans. on Graphics 26 (2007)
Farbman, Z., Hoffer, G., Lipman, Y., Cohen-Or, D., Fattal, R., Lischinski, D.: Coordinates for instant image cloning. ACM Trans. on Graphics 28 (2009)
Reddy, D., Agrawal, A., Chellappa, R.: Enforcing integrability by error correction using L-1 minimization. In: Computer Vision and Pattern Recognition (2009)
Drettakis, G., Bonneel, N., Dachsbacher, C., Lefebvre, S., Schwarz, M., Viaud-Delmon, I.: An interactive perceptual rendering pipeline using contrast and spatial masking. Rendering Techniques (2007)
Ramanarayanan, G., Ferwerda, J., Walter, B., Bala, K.: Visual equivalence: Towards a new standard for image fidelity. ACM Trans. on Graphics 26 (2007)
Vangorp, P., Laurijssen, J., Dutré, P.: The influence of shape on the perception of material reflectance. ACM Trans. on Graphics 26 (2007)
Ramanarayanan, G., Bala, K., Ferwerda, J.: Perception of complex aggregates. ACM Trans. on Graphics 27 (2008)
Su, S., Durand, F., Agrawala, M.: De-emphasis of distracting image regions using texture power maps. In: ICCV Workshop on Texture Analysis and Synthesis (2005)
Bae, S., Paris, S., Durand, F.: Two-scale tone management for photographic look. ACM Trans. on Graphics 25 (2006)
Aubert, G., Kornprobst, P.: Mathematical problems in image processing: Partial Differential Equations and the Calculus of Variations. Applied Mathematical Sciences, vol. 147. Springer, Heidelberg (2002)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE Trans. on Image Processing 13 (2004)
Farbman, Z., Fattal, R., Lischinski, D., Szeliski, R.: Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. on Graphics 27 (2008)
Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Analysis Machine Intelligence 12 (1990)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tao, M.W., Johnson, M.K., Paris, S. (2010). Error-Tolerant Image Compositing. In: Daniilidis, K., Maragos, P., Paragios, N. (eds) Computer Vision – ECCV 2010. ECCV 2010. Lecture Notes in Computer Science, vol 6311. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15549-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-15549-9_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15548-2
Online ISBN: 978-3-642-15549-9
eBook Packages: Computer ScienceComputer Science (R0)