The Visual Computer

, Volume 32, Issue 5, pp 601–609 | Cite as

Automatic segmentation of point clouds from multi-view reconstruction using graph-cut

  • Rongjiang PanEmail author
  • Gabriel Taubin
Original Article


In multi-view reconstruction systems, the recovered point cloud often consists of numerous unwanted background points. We propose a graph-cut based method for automatically segmenting point clouds from multi-view reconstruction. Based on the observation that the object of interest is likely to be central to the intended multi-view images, our method requires no user interaction except two roughly estimated parameters of objects covering in the central area of images. The proposed segmentation process is carried out in two steps: first, we build a weighted graph whose nodes represent points and edges that connect each point to its k-nearest neighbors. The potentials of each point being object and background are estimated according to distances between its projections in images and the corresponding image centers. The pairwise potentials between each point and its neighbors are computed according to their positions, colors and normals. Graph-cut optimization is then used to find the initial binary segmentation of object and background points. Second, to refine the initial segmentation, Gaussian mixture models (GMMs) are created from the color and density features of points in object and background classes, respectively. The potentials of each point being object and background are re-calculated based on the learned GMMs. The graph is updated and the segmentation of point clouds is improved by graph-cut optimization. The second step is iterated until convergence. Our method requires no manual labeling points and employs available information of point clouds from multi-view systems. We test the approach on real-world data generated by multi-view reconstruction systems.


Graph-cut Point clouds Segmentation Multi-view reconstruction Fixation constraints 



This work was supported by State Scholarship Fund of China, Program of Science and Technology Development of Shandong Province (2014GGX101016), the Fundamental Research Funds of Shandong University (2014JC003).


  1. 1.
  2. 2.
    Wu, C.: VisualSFM: a visual structure from motion system.
  3. 3.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring image collections in 3D. ACM Trans. Graph. (Proceedings of SIGGRAPH 2006) 25(3), 835–846 (2006)CrossRefGoogle Scholar
  4. 4.
    Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. IEEE Trans. Pattern Anal. Mach. Intell 32(8), 1362–1376 (2010)CrossRefGoogle Scholar
  5. 5.
    Gay, E., Galor, K., Cooper, D.B., Willis, A., Kimia, B.B., Karumuri, S., Taubin, G., Doutre, W., Sanders, D., Liu, S. : REVEAL Intermediate report. In: Proceedings of CVPR Workshop on Applications of Computer Vision in Archaeology (ACVA’10), June 2010Google Scholar
  6. 6.
    Calakli, F., Taubin, G.: SSD-CSR: Smooth signed distance colored surface reconstruction. In: Dill, J., Earnshaw, R., Kasik, D., Vince, J., Wong, P.-C. (eds.) State-of-the-Art Volume on Computer Graphics, Visualization, Visual Analytics, VR and HCI Dedicated to the memory of Jim Thomas (2012)Google Scholar
  7. 7.
    Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. ICCV 1, 105–112 (2001)Google Scholar
  8. 8.
    Rother, C., Kolmogorov, V., Blake, A.: GrabCut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)CrossRefGoogle Scholar
  9. 9.
    Golovinskiy, A., Funkhouser, T.: Min-Cut based segmentation of point clouds. IEEE Workshop on Search in 3D and Video (S3DV) at ICCV September 2009 (2009)Google Scholar
  10. 10.
    Johnson-Roberson, M., Bohg, J., Björkman, M., Kragic, D.: Attention-based active 3D point cloud segmentation. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1165–1170, 18–22 Oct. 2010 (2010)Google Scholar
  11. 11.
    Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., Kang, S.B.: Image-based plant modeling. ACM Trans. Graph. 25(3), 599–604 (2006)CrossRefGoogle Scholar
  12. 12.
    Sedlacek, D., Zara, J.: Graph cut based point-cloud segmentation for polygonal reconstruction, Adv. Visual Comput. pp. 218–227 (2009)Google Scholar
  13. 13.
    Campbell, N.D.F., Vogiatzis, G., Hernandez, C., Cipolla, R.: Automatic 3d object segmentation in multiple views using volumetric graph-cuts. Image Vis. Comput. 28(1), 14–25 (2010)CrossRefGoogle Scholar
  14. 14.
    Mishra, A.K., Aloimonos, Y., Cheong, L.F., Kassim, A.A.: Active visual segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 639–653 (2012)CrossRefGoogle Scholar
  15. 15.
    Campbell, N., Vogiatzis, G., Hernandez, C., Cipolla, R.: Automatic object segmentation from calibrated images. Visual Media Production (CVMP) (2011)Google Scholar
  16. 16.
    Djelouah, A., Franco, J.S., Boyer, E., Clerc, F.L., Pérez, P.: Multi-view object segmentation in space and time. ICCV 2013-IEEE International Conference on Computer Vision, Syndey, Australia. pp. 2640–2647, Dec 2013 (2013)Google Scholar
  17. 17.
    Arya, S., Mount, D.M., Netanyahu, N.S., Silverman, R., Wu, A.Y.: An optimal algorithm for approximate nearest neighbor searching fixed dimensions. J. ACM 45(6), 891–923 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)CrossRefzbMATHGoogle Scholar
  19. 19.
    Cignoni, P., Corsini, M., Ranzuglia, G. : MeshLab: an openSource 3D mesh processing system, ERCIM News, No.73, pp. 45–46,, Apr, 2008 (2008)
  20. 20.
    Kowdle, A., Sinha, S.N., Szeliski, R.: Multiple View Object Cosegmentation using Appearance and Stereo Cues, in Proceedings of the 12th European Conference on Computer Vision (ECCV), Springer, Berlin 8 October 2012 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.School of Computer Science and TechnologyShandong UniversityJinanChina
  2. 2.School of EngineeringBrown UniversityProvidenceUSA
  3. 3.Engineering Research Center of Digital Media TechnologyMinistry of Education of PRCJinanChina

Personalised recommendations