Skip to main content
SpringerLink
Log in

Unsupervised Segmentation of RGB-D Images

  • Conference paper
  • First Online:
Computer Vision -- ACCV 2014 (ACCV 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9005))

Included in the following conference series:

  • 2624 Accesses

Abstract

While unsupervised segmentation of RGB images has never led to results comparable to supervised segmentation methods, a surprising message of this paper is that unsupervised image segmentation of RGB-D images yields comparable results to supervised segmentation. We propose an unsupervised segmentation algorithm that is carefully crafted to balance the contribution of color and depth features in RGB-D images. The segmentation problem is then formulated as solving the Maximum Weight Independence Set (MWIS) problem. Given superpixels obtained from different layers of a hierarchical segmentation, the saliency of each superpixel is estimated based on balanced combination of features originating from depth, gray level intensity, and texture information. We want to stress four advantages of our method: (1) Its output is a single scale segmentation into meaningful segments of a RGB-D image; (2) The output segmentation contains large as well as small segments correctly representing the objects located in a given scene; (3) Our method does not need any prior knowledge from ground truth images, as is the case for every supervised image segmentation; (4) The computational time is much less than supervised methods. The experimental results show that our unsupervised segmentation method yields comparable results to the recently proposed, supervised segmentation methods [1, 2] on challenging NYU Depth dataset v2.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from rgbd images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  2. Gupta, S., Arbelaez, P., Malik, J.: Perceptual organization and recognition of indoor scenes from RGB-D images. In: CVPR (2013)

    Google Scholar 

  3. Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. ACM Trans. Graph. 23, 689–694 (2004)

    Article  Google Scholar 

  4. Brice, C., Fennema, C.: Scene analysis using regions. Artif. Intell. 1(3), 205–226 (1970)

    Article  Google Scholar 

  5. Horowitz, S., Pavlidis, T.: Picture segmentation by a tree traversal algorithm. JACM 23, 368–388 (1976)

    Article  MATH  Google Scholar 

  6. Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. IJCV 59, 167–181 (2004)

    Article  Google Scholar 

  7. Shi, J., Malik, J.: Normalized cuts and image segmentation. PAMI 22, 888–905 (2000)

    Article  Google Scholar 

  8. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. PAMI 33(5), 898–916 (2011)

    Article  Google Scholar 

  9. Brendel, W., Todorovic, S.: Segmentation as maximum weight independent set. In: NIPS (2010)

    Google Scholar 

  10. Comaniciu, D.: Mean shift: a robust approach toward feature space analysis. PAMI 24(5), 603–619 (2002)

    Article  Google Scholar 

  11. Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view RGB-D object dataset. In: ICRA (2011)

    Google Scholar 

  12. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: CVPR (2011)

    Google Scholar 

  13. Hoiem, D., Efros, A., Hebert, M.: Recovering occlusion boundaries from an image. IJCV 91(3), 328–346 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  14. Strom, J., Richardson, A., Olson, E.: Graph-based segmentation for colored 3d laser point clouds. In: IROS (2010)

    Google Scholar 

  15. Unnikrishnan, R., Pantofaru, C., Hebert, M.: A measure for objective evaluation of image segmentation algorithms. In: CVPRW (2005)

    Google Scholar 

  16. Meila, M.: Comparing clusterings by the variation of information. In: Schölkopf, B., Warmuth, M.K. (eds.) Learning Theory and Kernel Machines. LNCS, vol. 2777, pp. 173–187. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  17. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV (2001)

    Google Scholar 

  18. Freixenet, J., Muñoz, X., Raba, D., Martí, J., Cufí, X.: Yet another survey on image segmentation: region and boundary information integration. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part III. LNCS, vol. 2352, pp. 408–422. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  19. Hoiem, D., Efros, A., Hebert, M.: Geometric context from a single image. In: ICCV (2005)

    Google Scholar 

  20. Hedau, V., Hoiem, D., Forsyth, D.: Recovering the spatial layout of cluttered rooms. In: ICCV (2009)

    Google Scholar 

  21. Hedau, V., Hoiem, D., Forsyth, D.: Thinking inside the box: using appearance models and context based on room geometry. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 224–237. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  22. Lee, D., Hebert, M., Kanade, T.: Geometric reasoning for single image structure recovery. In: CVPR (2009)

    Google Scholar 

  23. Lee, D., Gupta, A., Hebert, M., Kanade, T.: Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces. In: NIPS (2010)

    Google Scholar 

  24. Guan, L., Yu, T., Tu, P., Lim, S.: Simultaneous image segmentation and 3D plane fitting for RGB-D sensors - an iterative framework. In: CVPRW (2012)

    Google Scholar 

  25. Erdogan, C., Paluri, M., Dellaert, F.: Planar segmentation of RGBD images using fast linear fitting and Markov chain monte carlo. In: CRV (2012)

    Google Scholar 

  26. Taylor, C., Cowley, A.: Segmentation and analysis of RGB-D data. In: RSS (2011)

    Google Scholar 

  27. Taylor, C.J., Cowley, A.: Parsing indoor scenes using RGB-D imagery. In: Robotics (2013)

    Google Scholar 

  28. Ion, A., Carreira, J., Sminchisescu, C.: Image segmentation by figure-ground composition into maximal cliques. In: ICCV (2011)

    Google Scholar 

  29. Cour, T., Benezit, F., Shi, J.: Spectral segmentation with multiscale graph decomposition. In: CVPR (2005)

    Google Scholar 

  30. Meyer, F.: Color image segmentation. In: Image Processing and its Applications (1992)

    Google Scholar 

  31. Ren, X., Bo, L., Fox, D.: RGB-(D) scene labeling: features and algorithms. In: CVPR (2012)

    Google Scholar 

  32. Varma, M., Zisserman, A.: A statistical approach to texture classification from single images. IJCV 62(2), 61–81 (2005)

    Article  Google Scholar 

  33. Deng, Y., Manjunath, B., Shin, H.: Color image segmentation. In: CVPR (1999)

    Google Scholar 

  34. Khoshelham, K.: Accuracy analysis of kinect depth data. In: ISPRS Workshop (2011)

    Google Scholar 

  35. Lang, C., Nguyen, T.V., Katti, H., Yadati, K., Kankanhalli, M., Yan, S.: Depth matters: influence of depth cues on visual saliency. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 101–115. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  36. Lempitsky, V., Vedaldi, A., Zisserman, A.: Pylon model for semantic segmentation. In: NIPS (2011)

    Google Scholar 

Download references

Acknowledgements

This work was in part supported by NSF under Grants IIS-1302164 and OIA-1027897.

Author information

Authors and Affiliations

  1. Department of Computer and Information Sciences, Temple University, Philadelphia, USA

    Zhuo Deng & Longin Jan Latecki

Authors
  1. Zhuo Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Longin Jan Latecki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhuo Deng .

Editor information

Editors and Affiliations

  1. Technische Universität München, Garching, Bayern, Germany

    Daniel Cremers

  2. University of Adelaide, Adelaide, South Australia, Australia

    Ian Reid

  3. Keio University, Yokohama, Kanagawa, Japan

    Hideo Saito

  4. University of California at Merced, Merced, California, USA

    Ming-Hsuan Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Deng, Z., Latecki, L.J. (2015). Unsupervised Segmentation of RGB-D Images. In: Cremers, D., Reid, I., Saito, H., Yang, MH. (eds) Computer Vision -- ACCV 2014. ACCV 2014. Lecture Notes in Computer Science(), vol 9005. Springer, Cham. https://doi.org/10.1007/978-3-319-16811-1_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-16811-1_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-16810-4

  • Online ISBN: 978-3-319-16811-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation