Advertisement

The Visual Computer

, Volume 29, Issue 3, pp 217–229 | Cite as

Object cloning using constrained mean value interpolation

  • Hui Du
  • Xiaogang JinEmail author
Original Article

Abstract

We present an object-cloning algorithm using constrained mean value interpolation over images. Different from previous methods, we first define the guidance vector field as a weighted gradient of the source and destination images in the gradient domain, which produces a similar Laplace equation to that of the original Poisson method but with different boundary conditions. Then, we use mean value coordinates to solve the new Laplace equation approximately instead of solving a large linear system. The cloned result matches well with the destination image on the luma and preserves the chroma of the source object as much as possible. Our cloned results are visually pleasing without smudging and discoloration artifacts. We also extend our approach to video object cloning. Experimental results demonstrate the effectiveness of our object cloning algorithm.

Keywords

Object cloning Mean value coordinates Alpha matting Laplace equation 

Notes

Acknowledgements

Xiaogang Jin was supported by Zhejiang Provincial Natural Science Foundation of China (Grant No. Z1110154), NSFC-MSRA Joint Funding (Grant No. 60970159), and the National Natural Science Foundation of China (Grant No. 60933007). Hui Du was supported by Zhejiang Provincial Science and Technology Planning Project (Grant No. 2010C31090) and Scientific Research Fund of Zhejiang Provincial Education Department (Grant No. Y201016221).

Supplementary material

371_2012_722_MOESM1_ESM.pdf (3.1 mb)
(PDF 3.1 MB)

(AVI 8.8 MB)

References

  1. 1.
    Agarwala, A.: Efficient gradient-domain compositing using quadtrees. ACM Trans. Graph. 26(3) (2007) Google Scholar
  2. 2.
    Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. In: Proceedings of IEEE CVPR (2006) Google Scholar
  3. 3.
    Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008) CrossRefGoogle Scholar
  4. 4.
    Rhemann, C., Rother, C., Kohli, P., Gelautz, M.: A spatially varying PSF-based prior for alpha matting. In: Proceedings of IEEE CVPR (2010) Google Scholar
  5. 5.
    C. Corporation: Knockout User Guide (2002) Google Scholar
  6. 6.
    Guo, D., Sim, T.: Color me right-seamless image compositing. In: 13th Intl. Conf. on Computer Analysis of Images and Patterns (CAIP) (2009) Google Scholar
  7. 7.
    Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. Comput. Graph. Forum 29(2), 575–584 (2010) CrossRefGoogle Scholar
  8. 8.
    Floater, M.S., Kós, G., Reimers, M.: Mean value coordinates in 3D. Comput. Aided Geom. Des. 22(7), 623–631 (2005) zbMATHCrossRefGoogle Scholar
  9. 9.
    Wang, H.-C., Ramesh, R., Narendra, A.: Seamless video editing. In: ICPR’04: Proceedings of the Pattern Recognition, 17th International Conference on (ICPR’04), vol. 3, pp. 858–861 (2004) Google Scholar
  10. 10.
    Bolz, J., Farmer, I., Grinspun, E., Schröder, P.: Sparse matrix solvers on the GPU: conjugate gradients and multigrid. ACM Trans. Graph. 22, 917–924 (2003) CrossRefGoogle Scholar
  11. 11.
    Jia, J., Sun, J., Tang, C.-K., Shum, H.-Y.: Drag-and-drop pasting. ACM Trans. Graph. 25(3), 631–637 (2006) CrossRefGoogle Scholar
  12. 12.
    Manson, J., Schaefer, S.: Moving least squares coordinates. Comput. Graph. Forum 29(5), 1517–1524 (2010) CrossRefGoogle Scholar
  13. 13.
    Mccann, J., Pollard, N.S.: Real-time gradient domain painting. ACM Trans. Graph. 27(3) (2008) Google Scholar
  14. 14.
    Sun, J., Jia, J., Tang, C.-K., Shum, H.-Y.: Poisson matting. ACM Trans. Graph. 23(3) (2004) Google Scholar
  15. 15.
    Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. In: Proceedings of IEEE CVPR, pp. 1–8 (2007) Google Scholar
  16. 16.
    Lalonde, J.-F., Hoiem, D., Efros, A.A., Rother, C., Winn, J., Criminisi, A.: Photo clip art. ACM Trans. Graph. 26(3), 3 (2007). (SIGGRAPH 2007) CrossRefGoogle Scholar
  17. 17.
    Hormann, K., Sukumar, N.: Maximum entropy coordinates for arbitrary polytopes. Comput. Graph. Forum 27(5), 1513–1520 (2008) CrossRefGoogle Scholar
  18. 18.
    Sunkavalli, K., Micah, J., Matusik, W., Pfister, H.: Multi-scale image harmonization. ACM Trans. Graph. 29(4) (2010) Google Scholar
  19. 19.
    He, K.-M., Sun, J., Tang, X.: Fast matting using large kernel matting Laplacian matrices. In: Proceedings of IEEE CVPR, pp. 2165–2172 (2010) Google Scholar
  20. 20.
    Grady, L., Schiwietz, T., Aharon, S., Westermann, R.: Random walks for interactive alpha-matting. In: Proceedings of the Fifth IASTED International Conference on Visualization, Imaging and Image Processing, pp. 423–429 (2005) Google Scholar
  21. 21.
    Levin, A., Rav-Acha, A., Lischinski, D.: Spectral Matting (2007) Google Scholar
  22. 22.
    Bratkova, M., Boulos, S., Shirley, P.: oRGB: a practical opponent color space for computer graphics. IEEE Comput. Graph. Appl. 29(1), 42–55 (2009) CrossRefGoogle Scholar
  23. 23.
    Ding, M., Tong, R.-F.: Content-aware copying and pasting in images. Vis. Comput. 26(6–8), 721–729 (2010) CrossRefGoogle Scholar
  24. 24.
    Kazhdan, M., Hoppe, H.: Streaming multigrid for gradient-domain operations on large images. ACM Trans. Graph. 27(3) (2008) Google Scholar
  25. 25.
    Tao, M.W., Johnson, M.K., Paris, S.: Error-tolerant image compositing. In: European Conference on Computer Vision (2010) Google Scholar
  26. 26.
    Floater, M.S.: Mean value coordinates. Comput. Aided Geom. Des. 20(1), 19–27 (2003) MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003) CrossRefGoogle Scholar
  28. 28.
    Szeliski, R.: Locally adapted hierarchical basis preconditioning. ACM Trans. Graph. 25, 1135–1143 (2006) CrossRefGoogle Scholar
  29. 29.
    Wang, R., Chen, W.-F., Pan, M.-H., Bao, H.-J.: Harmonic coordinates for real-time image cloning. J. Zhejiang Univ. Sci. 11, 690–698 (2010) Google Scholar
  30. 30.
    Ruzon, M.A., Tomasi, C.: Alpha estimation in natural images. In: Proceedings of IEEE CVPR, pp. 18–25 (2000) Google Scholar
  31. 31.
    Jeschke, S., Cline, D., Wonka, P.: A GPU Laplacian solver for diffusion curves and Poisson image editing. ACM Trans. Graph. 28(5) (2009) Google Scholar
  32. 32.
    Lee, S.-Y., Lee, I.-K.: Improved coordinate-based image and video cloning algorithm. In: ACM SIGGRAPH Asia Posters (2009) Google Scholar
  33. 33.
    Chen, T., Cheng, M.-M., Tan, P., Shamir, A., Hu, S.-M.: Sketch2photo:internet image montage. ACM Transactions on Graphics 28(5), 124:1–124:10 (2009) Google Scholar
  34. 34.
    Ju, T., Schaefer, S., Joe, W.: Mean value coordinates for closed triangular meshes. ACM Trans. Graph. 24(3), 561–566 (2005) CrossRefGoogle Scholar
  35. 35.
    Bai, X., Sapiro, G.: A geodesic framework for fast interactive image and video segmentation and matting. In: International Conference on Computer Vision (2007) Google Scholar
  36. 36.
    Bai, X., Wang, J., Simons, D., Sapiro, G.: Video snapcut: robust video object cutout using localized classifiers. ACM Trans. Graph. 28(3), 70:1–70:11 (2009) CrossRefGoogle Scholar
  37. 37.
    Guan, Y., Chen, W., Liang, X., Ding, Z., Peng, Q.-S.: Easy matting—a stroke based approach for continuous image matting. Comput. Graph. Forum 25(3), 567–576 (2006) CrossRefGoogle Scholar
  38. 38.
    Mishima, Y.: Soft edge chroma-key generation based upon hexoctahedral color space. US Patent 5,355,174 (1994) Google Scholar
  39. 39.
    Zhang, Y., Tong, R.-F.: Environment-sensitive cloning in images. Vis. Comput. 27, 739–748 (2011) CrossRefGoogle Scholar
  40. 40.
    Zheng, Y.-J., Kambhamettu, C.: Learning based digital matting. In: International Conference on Computer Vision (2009) Google Scholar
  41. 41.
    Chuang, Y.-Y., Curless, B., Salesin, D.H., Szeliski, R.: A Bayesian approach to digital matting. In: Proceedings of IEEE CVPR, vol. 2, pp. 264–271 (2001) Google Scholar
  42. 42.
    Farbman, Z., Hoffer, G., Lipman, Y., Cohen-Or, D., Lischinski, D.: Coordinates for instant image cloning. ACM Trans. Graph. 28(3) (2009) Google Scholar
  43. 43.
    Xie, Z.-F., Shen, Y., Ma, L.-Z., Chen, Z.-H.: Seamless video composition using optimized mean-value cloning. Vis. Comput. 26(6–8), 1123–1134 (2010) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  1. 1.State Key Lab of CAD&CGZhejiang UniversityHangzhouChina
  2. 2.Zhejiang University of Media and CommunicationsHangzhouChina

Personalised recommendations