Error-Tolerant Image Compositing

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6311)


Gradient-domain compositing is an essential tool in computer vision and its applications, e.g., seamless cloning, panorama stitching, shadow removal, scene completion and reshuffling. While easy to implement, these gradient-domain techniques often generate bleeding artifacts where the composited image regions do not match. One option is to modify the region boundary to minimize such mismatches. However, this option may not always be sufficient or applicable, e.g., the user or algorithm may not allow the selection to be altered. We propose a new approach to gradient-domain compositing that is robust to inaccuracies and prevents color bleeding without changing the boundary location. Our approach improves standard gradient-domain compositing in two ways. First, we define the boundary gradients such that the produced gradient field is nearly integrable. Second, we control the integration process to concentrate residuals where they are less conspicuous. We show that our approach can be formulated as a standard least-squares problem that can be solved with a sparse linear system akin to the classical Poisson equation. We demonstrate results on a variety of scenes. The visual quality and run-time complexity compares favorably to other approaches.


Poisson Equation Texture Region Visual Masking Sparse Linear System Shadow Removal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. on Graphics 22 (2003)Google Scholar
  2. 2.
    Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D.H., Cohen, M.F.: Interactive digital photomontage. ACM Trans. on Graphics 23 (2004)Google Scholar
  3. 3.
    Georgiev, T.: Covariant derivatives and vision. In: European Conf. on Computer Vision (2006)Google Scholar
  4. 4.
    Jia, J., Sun, J., Tang, C.K., Shum, H.Y.: Drag-and-drop pasting. ACM Trans. on Graphics 25 (2006)Google Scholar
  5. 5.
    Levin, A., Zomet, A., Peleg, S., Weiss, Y.: Seamless image stitching in the gradient domain. In: European Conf. on Computer Vision (2006)Google Scholar
  6. 6.
    Agarwala, A.: Efficient gradient-domain compositing using quadtrees. ACM Trans. on Graphics 26 (2007)Google Scholar
  7. 7.
    Sivic, J., Kaneva, B., Torralba, A., Avidan, S., Freeman, W.T.: Creating and exploring a large photorealistic virtual space. In: IEEE Workshop on Internet Vision (2008)Google Scholar
  8. 8.
    Whyte, O., Sivic, J., Zisserman, A.: Get out of my picture! Internet-based inpainting. In: British Machine Vision Conf. (2009)Google Scholar
  9. 9.
    Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. International Journal of Computer Vision (2009)Google Scholar
  10. 10.
    Hays, J., Efros, A.A.: Scene completion using millions of photographs. ACM Trans. on Graphics 26 (2007)Google Scholar
  11. 11.
    Cho, T.S., Avidan, S., Freeman, W.T.: The patch transform. IEEE Trans. on Pattern Analysis and Machine Intelligence (2010)Google Scholar
  12. 12.
    Tappen, M.F., Adelson, E.H., Freeman, W.T.: Recovering intrinsic images from a single image. IEEE Trans. on Pattern Analysis and Machine Intelligence 27 (2005)Google Scholar
  13. 13.
    Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. IEEE Trans. on Pattern Analysis and Machine Intelligence 28 (2006)Google Scholar
  14. 14.
    Agrawal, A., Raskar, R., Chellappa, R.: What is the range of surface reconstructions from a gradient field? In: European Conf. on Computer Vision (2006)Google Scholar
  15. 15.
    Bhat, P., Zitnick, C.L., Cohen, M., Curless, B.: Gradientshop: A gradient-domain optimization framework for image and video filtering. ACM Trans. on Graphics 28 (2009)Google Scholar
  16. 16.
    Lalonde, J.F., Hoiem, D., Efros, A., Rother, C., Winn, J., Criminisi, A.: Photo clip art. ACM Trans. on Graphics 26 (2007)Google Scholar
  17. 17.
    Farbman, Z., Hoffer, G., Lipman, Y., Cohen-Or, D., Fattal, R., Lischinski, D.: Coordinates for instant image cloning. ACM Trans. on Graphics 28 (2009)Google Scholar
  18. 18.
    Reddy, D., Agrawal, A., Chellappa, R.: Enforcing integrability by error correction using L-1 minimization. In: Computer Vision and Pattern Recognition (2009)Google Scholar
  19. 19.
    Drettakis, G., Bonneel, N., Dachsbacher, C., Lefebvre, S., Schwarz, M., Viaud-Delmon, I.: An interactive perceptual rendering pipeline using contrast and spatial masking. Rendering Techniques (2007)Google Scholar
  20. 20.
    Ramanarayanan, G., Ferwerda, J., Walter, B., Bala, K.: Visual equivalence: Towards a new standard for image fidelity. ACM Trans. on Graphics 26 (2007)Google Scholar
  21. 21.
    Vangorp, P., Laurijssen, J., Dutré, P.: The influence of shape on the perception of material reflectance. ACM Trans. on Graphics 26 (2007)Google Scholar
  22. 22.
    Ramanarayanan, G., Bala, K., Ferwerda, J.: Perception of complex aggregates. ACM Trans. on Graphics 27 (2008)Google Scholar
  23. 23.
    Su, S., Durand, F., Agrawala, M.: De-emphasis of distracting image regions using texture power maps. In: ICCV Workshop on Texture Analysis and Synthesis (2005)Google Scholar
  24. 24.
    Bae, S., Paris, S., Durand, F.: Two-scale tone management for photographic look. ACM Trans. on Graphics 25 (2006)Google Scholar
  25. 25.
    Aubert, G., Kornprobst, P.: Mathematical problems in image processing: Partial Differential Equations and the Calculus of Variations. Applied Mathematical Sciences, vol. 147. Springer, Heidelberg (2002)zbMATHGoogle Scholar
  26. 26.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE Trans. on Image Processing 13 (2004)Google Scholar
  27. 27.
    Farbman, Z., Fattal, R., Lischinski, D., Szeliski, R.: Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. on Graphics 27 (2008)Google Scholar
  28. 28.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Analysis Machine Intelligence 12 (1990)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.University of CaliforniaBerkeley
  2. 2.Massachusetts Institute of Technology 
  3. 3.Adobe Systems, Inc 

Personalised recommendations