Skip to main content
SpringerLink
Log in

A Hierarchical Level Set Approach to for RGBD Image Matting

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11295))

Included in the following conference series:

Abstract

This paper presents a novel method for the image matting of RGBD data, using a Hierarchical Level Set. The approach has four main steps. First of all, the color and depth channel is preprocessed. Noise is eliminated by using a Directional Joint Bilateral Filter and holes are removed from the depth map. Secondly, color cues and depth cues are integrated to segment the image using a Hierarchical Level Set Framework. After this, the segmentation of the color and depth cues is used to generate a trimap. Finally, an extended alpha matting approach is used to obtain the final matting result, with the color image, depth image and trimap serving as input. Experiments using complex natural images demonstrate that the proposed RGBD matting approach is able to generate good matting results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Change history

  • 31 January 2019

    In the original version of the book, the following belated corrections have been incorporated:

References

  1. Smith, A.R., Blinn, J.F.: Blue screen matting. In: International Conference on Computer Graphics and Interactive Techniques, pp. 259–268 (1996)

    Google Scholar 

  2. Naqvi, S.S., Browne, W.N., Hollitt, C.: Salient object detection via spectral matting. Pattern Recogn. 51, 209–224 (2016)

    Article  Google Scholar 

  3. Chuang, Y., Curless, B., Salesin, D., Szeliski, R.: A Bayesian approach to digital matting, vol. 2, pp. 264–271 (2001)

    Google Scholar 

  4. Sun, J., Jia, J., Tang, C., Shum, H.: Poisson matting. In: International Conference on Computer Graphics and Interactive Techniques (2004)

    Article  Google Scholar 

  5. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008)

    Article  Google Scholar 

  6. Cho, J., Ziegler, R., Gross, M.H., Lee, K.H.: Improving alpha matte with depth information. IEICE Electron. Express 6(22), 1602–1607 (2009)

    Article  Google Scholar 

  7. Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. In: Computer Graphics Forum, vol. 29, no. 2, pp. 575–584 (2010)

    Article  Google Scholar 

  8. Pollefeys, M., Aksoy, Y., Aydin, T.O.: Designing effective inter-pixel information flow for natural image matting (2017)

    Google Scholar 

  9. Crabb, R., Tracey, C., Puranik, A., Davis, J.: Real-time foreground segmentation via range and color imaging, pp. 1–5 (2008)

    Google Scholar 

  10. Wang, O., Finger, J., Yang, Q., Davis, J., Yang, R.: Automatic natural video matting with depth. In: Pacific Conference on Computer Graphics and Applications, pp. 469–472 (2007)

    Google Scholar 

  11. Pitie, F., Kokaram, A.: Matting with a depth map. In: IEEE International Conference on Image Processing, pp. 21–24 (2010)

    Google Scholar 

  12. Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)

    Article  MathSciNet  Google Scholar 

  13. Xu, L., Sun, W., Au, O.C., et al.: Adaptive depth map assisted matting in 3D video. In: IEEE International Conference on Multimedia and Expo, pp. 1–6 (2011)

    Google Scholar 

  14. Lee, S.W., Seo, Y.H., Yang, H.S.: Efficient foreground extraction using RGB-D imaging. Kluwer Academic Publishers (2016)

    Google Scholar 

  15. Wang, L., Gong, M., Zhang, C., Yang, R., Zhang, C., Yang, Y.H.: Automatic real-time video matting using time-of-flight camera and multichannel poisson equations. Int. J. Comput. Vision 97(1), 104–121 (2012)

    Article  Google Scholar 

  16. Ge, L., Ju, R., Ren, T., Wu, G.: Interactive RGB-D image segmentation using hierarchical graph cut and geodesic distance. In: Ho, Y.-S., Sang, J., Ro, Y.M., Kim, J., Wu, F. (eds.) PCM 2015. LNCS, vol. 9314, pp. 114–124. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24075-6_12

    Chapter  Google Scholar 

  17. Lu, T., Li, S.: Image matting with color and depth information, pp. 3787–3790 (2012)

    Google Scholar 

  18. Memar, S., Jin, K., Boufama, B.: Object detection using active contour model with depth clue. In: Kamel, M., Campilho, A. (eds.) ICIAR 2013. LNCS, vol. 7950, pp. 640–647. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39094-4_73

    Chapter  Google Scholar 

  19. Hao, W., Zheng, S., Guo, C., Xie, Y.: Level set contour extraction based on data-adaptive Gaussian smoother, pp. 11–15 (2012)

    Google Scholar 

  20. Hu, P., Shuai, B., Liu, J., Wang, G.: Deep level sets for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)

    Google Scholar 

  21. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    Article  Google Scholar 

  22. Zanuttigh, P., Marin, G., Dal Mutto, C., Dominio, F., Minto, L., Cortelazzo, G.M.: Time-of-Flight and Structured Light Depth Cameras. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30973-6

    Book  Google Scholar 

  23. Chen, L., Lin, H., Li, S.: Depth image enhancement for kinect using region growing and bilateral filter. In: International Conference on Pattern Recognition, pp. 3070-3073 (2013)

    Google Scholar 

  24. Le, A.V., Jung, S., Won, C.S.: Directional joint bilateral filter for depth images. Sensors 14(7), 11362–11378 (2014)

    Article  Google Scholar 

  25. Jung, S.: Enhancement of image and depth map using adaptive joint trilateral filter. IEEE Trans. Circuits Syst. Video Technol. 23(2), 269–280 (2013)

    Article  Google Scholar 

  26. Li, C., Xu, C., Gui, C., Fox, M.D.: Level set evolution without re-initialization: a new variational formulation. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 430–436 (2005)

    Google Scholar 

  27. Ju, R., Liu, Y., Ren, T., Ge, L., Wu, G.: Depth-aware salient object detection using anisotropic center-surround difference. Signal Process. Image Commun. 38(C), 115–126 (2015)

    Article  Google Scholar 

  28. Leens, J., Piérard, S., Barnich, O., Van Droogenbroeck, M., Wagner, J.-M.: Combining color, depth, and motion for video segmentation. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 104–113. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04667-4_11

    Chapter  Google Scholar 

  29. Varnousfaderani, E.S., Rajan, D.: Weighted color and texture sample selection for image matting. IEEE Trans. Image Process. 22(11), 4260–4270 (2013)

    Article  MathSciNet  Google Scholar 

  30. Li, C., Wang, P., Zhu, X., Pi, H.: Three-layer graph framework with the sumD feature for alpha matting. Comput. Vis. Image Underst. 162, 34–45 (2017)

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 61502060), National Natural Science Foundation of China (No. 61701051).

Author information

Authors and Affiliations

  1. College of Computer Science, Chongqing University, NO. 174, Shazheng Street, Shapingba Street, Chongqing, China

    Wenliang Zeng & Ji Liu

Authors
  1. Wenliang Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ji Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenliang Zeng .

Editor information

Editors and Affiliations

  1. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Ioannis Kompatsiaris

  2. EURECOM, Sophia Antipolis, France

    Benoit Huet

  3. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Vasileios Mezaris

  4. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  5. National Chiao Tung University, Hsinchu, Taiwan

    Wen-Huang Cheng

  6. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Stefanos Vrochidis

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zeng, W., Liu, J. (2019). A Hierarchical Level Set Approach to for RGBD Image Matting. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, WH., Vrochidis, S. (eds) MultiMedia Modeling. MMM 2019. Lecture Notes in Computer Science(), vol 11295. Springer, Cham. https://doi.org/10.1007/978-3-030-05710-7_52

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-05710-7_52

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-05709-1

  • Online ISBN: 978-3-030-05710-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation