International Journal of Computer Vision

, Volume 103, Issue 2, pp 178–189 | Cite as

Error-Tolerant Image Compositing

  • Michael W. Tao
  • Micah K. Johnson
  • Sylvain Paris
Article

Abstract

Gradient-domain compositing is an essential tool in computer vision and its applications, e.g., seamless cloning, panorama stitching, shadow removal, scene completion and reshuffling. While easy to implement, these gradient-domain techniques often generate bleeding artifacts where the composited image regions do not match. One option is to modify the region boundary to minimize such mismatches. However, this option may not always be sufficient or applicable, e.g., the user or algorithm may not allow the selection to be altered. We propose a new approach to gradient-domain compositing that is robust to inaccuracies and prevents color bleeding without changing the boundary location. Our approach improves standard gradient-domain compositing in two ways. First, we define the boundary gradients such that the produced gradient field is nearly integrable. Second, we control the integration process to concentrate residuals where they are less conspicuous. We show that our approach can be formulated as a standard least-squares problem that can be solved with a sparse linear system akin to the classical Poisson equation. We demonstrate results on a variety of scenes. The visual quality and run-time complexity compares favorably to other approaches.

Keywords

Gradient-domain compositing Visual masking 

Supplementary material

11263_2012_579_MOESM1_ESM.zip (13.3 mb)
(ZIP 13.3 MB)

References

  1. Agarwala, A. (2007). Efficient gradient-domain compositing using quadtrees. In ACM transactions on graphics: Vol. 26. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  2. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D. H., & Cohen, M. F. (2004). Interactive digital photomontage. In ACM transactions on graphics: Vol. 23. Proceedings of the ACM SIGGRAPH conference (pp. 294–302). Google Scholar
  3. Agrawal, A., Raskar, R., & Chellappa, R. (2006). What is the range of surface reconstructions from a gradient field? In Proceedings of the European conference on computer vision. Google Scholar
  4. Aubert, G., & Kornprobst, P. (2002). Applied mathematical sciences: Vol. 147. Mathematical problems in image processing: partial differential equations and the calculus of variations. Berlin: Springer. MATHGoogle Scholar
  5. Bae, S., Paris, S., & Durand, F. (2006). Two-scale tone management for photographic look. In ACM transactions on graphics: Vol. 25. Proceedings of the ACM SIGGRAPH conference (pp. 637–645). Google Scholar
  6. Bhat, P., Zitnick, C. L., Cohen, M., & Curless, B. (2009). Gradientshop: a gradient-domain optimization framework for image and video filtering. In ACM transactions on graphics. Google Scholar
  7. Cho, T. S., Avidan, S., & Freeman, W. T. (2010). The patch transform. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1489–1501. CrossRefGoogle Scholar
  8. Drettakis, G., Bonneel, N., Dachsbacher, C., Lefebvre, S., Schwarz, M., & Viaud-Delmon, I. (2007). An interactive perceptual rendering pipeline using contrast and spatial masking. Rendering Techniques. Google Scholar
  9. Farbman, Z., Fattal, R., Lischinski, D., & Szeliski, R. (2008). Edge-preserving decompositions for multi-scale tone and detail manipulation. In ACM transactions on graphics: Vol. 27. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  10. Farbman, Z., Hoffer, G., Lipman, Y., Cohen-Or, D., Fattal, R., & Lischinski, D. (2009). Coordinates for instant image cloning. In ACM transactions on graphics: Vol. 28. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  11. Finlayson, G. D., Hordley, S. D., Lu, C., & Drew, M. S. (2006). On the removal of shadows from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 59–68. CrossRefGoogle Scholar
  12. Finlayson, G. D., Drew, M. S., & Lu, C. (2009). Entropy minimization for shadow removal. International Journal of Computer Vision, 85(1), 35–57. CrossRefGoogle Scholar
  13. Georgiev, T. (2006). Covariant derivatives and vision. In Proceedings of the European conference on computer vision. Google Scholar
  14. Hays, J., & Efros, A. A. (2007). Scene completion using millions of photographs. In ACM transactions on graphics: Vol. 26. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  15. Jia, J., Sun, J., Tang, C. K., & Shum, H. Y. (2006). Drag-and-drop pasting. In ACM transactions on graphics: Vol. 25. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  16. Lalonde, J. F., Hoiem, D., Efros, A., Rother, C., Winn, J., & Criminisi, A. (2007). Photo clip art. In ACM transactions on graphics: Vol. 26. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  17. Levin, A., Zomet, A., Peleg, S., & Weiss, Y. (2006). Seamless image stitching in the gradient domain. In Proceedings of the European conference on computer vision. Google Scholar
  18. Perona, P., & Malik, J. (1990). Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 629–639. CrossRefGoogle Scholar
  19. Prez, P., Gangnet, M., & Blake, A. (2003). Poisson image editing. In ACM transactions on graphics: Vol. 22. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  20. Ramanarayanan, G., Ferwerda, J., Walter, B., & Bala, K. (2007). Visual equivalence: towards a new standard for image fidelity. In ACM transactions on graphics: Vol. 26. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  21. Ramanarayanan, G., Bala, K., & Ferwerda, J. (2008). Perception of complex aggregates. In ACM transactions on graphics: Vol. 27. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  22. Reddy, D., Agrawal, A., & Chellappa, R. (2009). Enforcing integrability by error correction using L-1 minimization. In Proceedings of the conference on computer vision and pattern recognition. Google Scholar
  23. Sivic, J., Kaneva, B., Torralba, A., Avidan, S., & Freeman, W. T. (2008). Creating and exploring a large photorealistic virtual space. In Proceedings of the IEEE workshop on internet vision. Google Scholar
  24. Su, S., Durand, F., & Agrawala, M. (2005). De-emphasis of distracting image regions using texture power maps. In Proceedings of the ICCV workshop on texture analysis and synthesis. Google Scholar
  25. Tappen, M. F., Adelson, E. H., & Freeman, W. T. (2005). Recovering intrinsic images from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 1459–1472. CrossRefGoogle Scholar
  26. Vangorp, P., Laurijssen, J., & Dutr, P. (2007). The influence of shape on the perception of material reflectance. In ACM transactions on graphics: Vol. 26. Proceedings of the ACM SIGGRAPH conference. Google Scholar
  27. Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(14), 600–612. CrossRefGoogle Scholar
  28. Whyte, O., Sivic, J., & Zisserman, A. (2009). Get out of my picture! Internet-based inpainting. In Proceedings of the British machine vision conference. Google Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  • Michael W. Tao
    • 1
  • Micah K. Johnson
    • 2
  • Sylvain Paris
    • 3
  1. 1.University of CaliforniaBerkeleyUSA
  2. 2.Massachusetts Institute of TechnologyCambridgeUSA
  3. 3.Adobe Systems, Inc.CambridgeUSA

Personalised recommendations