Skip to main content
SpringerLink
Log in

Depth-aware image vectorization and editing

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Image vectorization is one of the primary means of creating vector graphics. The quality of a vectorized image depends crucially on extracting accurate features from input raster images. However, correct object edges can be difficult to detect when color gradients are weak. We present an image vectorization technique that operates on a color image augmented with a depth map and uses both color and depth edges to define vectorized paths. We output a vectorized result as a diffusion curve image. The information extracted from the depth map allows us more flexibility in the manipulation of the diffusion curves, in particular permitting high-level object segmentation. Our experimental results demonstrate that this method achieves high reconstruction quality and provides greater control in the organization and editing of vectorized images than existing work based on diffusion curves.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. (TOG) 28(3), 24 (2009)

    Article  Google Scholar 

  2. Boyé, S., Barla, P., Guennebaud, G.: A vectorial solver for free-form vector gradients. ACM Trans. Graph. (TOG) 31(6), 173 (2012)

    Article  Google Scholar 

  3. Cheng, M.M.: Curve structure extraction for cartoon images. In: 5th Joint Conference on Harmonious Human Machine Environment, 13–25 (2009)

  4. Finch, M., Snyder, J., Hoppe, H.: Freeform vector graphics with controlled thin-plate splines. ACM Trans. Graph. (TOG) 30(6), 166 (2011)

    Article  Google Scholar 

  5. Ge, L., Liang, H., Yuan, J., Thalmann, D.: Robust 3D hand pose estimation in single depth images: from single-view CNN to multi-view CNNs. In: IEEE Conference on Computer Vision and Pattern Recognition, 3593–3601 (2016)

  6. Gupta, S., Arbelaez, P., Malik, J.: Perceptual organization and recognition of indoor scenes from RGB-D images. In: IEEE Conference on Computer Vision and Pattern Recognition, 564–571 (2013)

  7. Hou, F., Sun, Q., Fang, Z., Liu, Y.J., Hu, S.M., Hao, A., Qin, H., He, Y.: Poisson vector graphics (pvg). In: IEEE Transactions on Visualization and Computer Graphics (2018)

  8. Ilbery, P., Kendall, L., Concolato, C., McCosker, M.: Biharmonic diffusion curve images from boundary elements. ACM Trans. Graph. (TOG) 32(6), 219 (2013)

    Article  Google Scholar 

  9. Jeschke, S.: Generalized diffusion curves: an improved vector representation for smooth-shaded images. Comput. Graph. Forum 35(2), 71–79 (2016)

    Article  Google Scholar 

  10. Jeschke, S., Cline, D., Wonka, P.: A gpu laplacian solver for diffusion curves and poisson image editing. ACM Trans. Graph. (TOG) 28(5), 116 (2009)

    Google Scholar 

  11. Jeschke, S., Cline, D., Wonka, P.: Estimating color and texture parameters for vector graphics. Comput. Graph. Forum 30(2), 523–532 (2011)

    Article  Google Scholar 

  12. Lai, Y.K., Hu, S.M., Martin, R.R.: Automatic and topology-preserving gradient mesh generation for image vectorization. ACM Trans. Graph. (TOG) 28(3), 85 (2009)

    Article  Google Scholar 

  13. Liao, Z., Hoppe, H., Forsyth, D., Yu, Y.: A subdivision-based representation for vector image editing. IEEE Trans. Vis. Comput. Graph. 18(11), 1858–1867 (2012)

    Article  Google Scholar 

  14. Lindeberg, T.: Edge detection and ridge detection with automatic scale selection. Int. J. Comput. Vis. 30(2), 117–156 (1998)

    Article  Google Scholar 

  15. Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: 10th IEEE international symposium on Mixed and augmented reality, 127–136. IEEE (2011)

  16. Orzan, A., Bousseau, A., Barla, P., Thollot, J.: Structure-preserving manipulation of photographs. In: 5th International Symposium on Non-photorealistic Animation and Rendering, 103–110. ACM (2007)

  17. Orzan, A., Bousseau, A., Winnemöller, H., Barla, P., Thollot, J., Salesin, D.: Diffusion curves: a vector representation for smooth-shaded images. ACM Trans. Graph. (TOG) 27(3), 92 (2008)

    Article  Google Scholar 

  18. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990)

    Article  Google Scholar 

  19. Price, B., Barrett, W.: Object-based vectorization for interactive image editing. Vis. Comput. 22(9–11), 661–670 (2006)

    Article  Google Scholar 

  20. Rother, C., Kolmogorov, V., Blake, A.: Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. (TOG) 23(3), 309–314 (2004)

    Article  Google Scholar 

  21. Ruderman, D.L., Bialek, W.: Statistics of natural images: scaling in the woods. Phys. Rev. Lett. 73(6), 814–817 (1994)

    Article  Google Scholar 

  22. Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., Westling, P.: High-resolution stereo datasets with subpixel-accurate ground truth. Ger. Conf. Pattern Recognit. 8753, 31–42 (2014)

    MathSciNet  Google Scholar 

  23. Schneider, P.J.: An algorithm for automatically fitting digitized curves. In: Graphics Gems I, 1st edn., pp. 612–626. Academic Press Professional, Cambridge, Massachusetts, USA (1990)

  24. Shen, J., Wang, D., Li, X.: Depth-aware image seam carving. IEEE Trans. Cybern. 43(5), 1453–1461 (2013)

    Article  Google Scholar 

  25. Song, S., Xiao, J.: Sliding shapes for 3d object detection in depth images. In: European Conference on Computer Vision, 634–651. Springer (2014)

  26. Song, S., Xiao, J.: Deep sliding shapes for amodal 3D object detection in RGB-D images. In: IEEE Conference on Computer Vision and Pattern Recognition, 808–816 (2016)

  27. Sun, D., Sudderth, E.B., Pfister, H.: Layered RGBD scene flow estimation. In: IEEE Conference on Computer Vision and Pattern Recognition, 548–556 (2015)

  28. Sun, T., Thamjaroenporn, P., Zheng, C.: Fast multipole representation of diffusion curves and points. ACM Trans. Graph. (TOG) 33(4), 53 (2014)

    Article  MATH  Google Scholar 

  29. Wang, C., Zhu, J., Guo, Y., Wang, W.: Video vectorization via tetrahedral remeshing. IEEE Trans. Image Process. 26(4), 1833–1844 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  30. Wang, T.C., Srikanth, M., Ramamoorthi, R.: Depth from semi-calibrated stereo and defocus. In: IEEE Conference on Computer Vision and Pattern Recognition, 3717–3726 (2016)

  31. Xia, T., Liao, B., Yu, Y.: Patch-based image vectorization with automatic curvilinear feature alignment. ACM Trans. Graph. (TOG) 28(5), 115 (2009)

    Article  Google Scholar 

  32. Xie, G., Sun, X., Tong, X., Nowrouzezahrai, D.: Hierarchical diffusion curves for accurate automatic image vectorization. ACM Trans. Graph. (TOG) 33(6), 230 (2014)

    Article  Google Scholar 

  33. Yang, J., Ye, X., Li, K., Hou, C., Wang, Y.: Color-guided depth recovery from RGB-D data using an adaptive autoregressive model. IEEE Trans. Image Process. 23(8), 3443–3458 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  34. Ye, X., Yang, J., Huang, H., Hou, C., Wang, Y.: Computational multi-view imaging with Kinect. IEEE Trans. Broadcast. 60(3), 540–554 (2014)

    Article  Google Scholar 

  35. Zhao, S., Durand, F., Zheng, C.: Inverse diffusion curves using shape optimization. IEEE Trans. Visual. Comput. Graph. 24(7), 2153–2166 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by Zhejiang Provincial Natural Science Foundation of China (No. LY19F020027), the National Natural Science Foundation of China (No. 61402410), the Key Research and Development Program of Zhejiang Province (No. 2018C03055), and the National Natural Science Foundation of China (No. 61732015).

Author information

Authors and Affiliations

  1. Zhejiang University of Technology, Hangzhou, China

    Shufang Lu, Xuefeng Ding, Fei Gao & Jiazhou Chen

  2. University of Victoria, Victoria, BC, Canada

    Wei Jiang

  3. Zhejiang University, Hangzhou, China

    Xiaogang Jin

  4. University of Waterloo, Waterloo, ON, Canada

    Craig S. Kaplan

Authors
  1. Shufang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuefeng Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Craig S. Kaplan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaogang Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jiazhou Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiazhou Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, S., Jiang, W., Ding, X. et al. Depth-aware image vectorization and editing. Vis Comput 35, 1027–1039 (2019). https://doi.org/10.1007/s00371-019-01671-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01671-0

Keywords

Search

Navigation