Efficient Single-View 3D Co-segmentation Using Shape Similarity and Spatial Part Relations

  • Nikita Araslanov
  • Seongyong Koo
  • Juergen Gall
  • Sven Behnke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9796)


The practical use of the latest methods for supervised 3D shape co-segmentation is limited by the requirement of diverse training data and a watertight mesh representation. Driven by practical considerations, we assume only one reference shape to be available and the query shape to be provided as a partially visible point cloud. We propose a novel co-segmentation approach that constructs a part-based object representation comprised of shape appearance models of individual parts and isometric spatial relations between the parts. The partial query shape is pre-segmented using planar cuts, and the segments accompanied by the learned representation induce a compact Conditional Random Field (CRF). CRF inference is performed efficiently by \(A^*\)-search with global optimality guarantees. A comparative evaluation with two baselines on partial views generated from the Labelled Princeton Segmentation Benchmark and point clouds recorded with an RGB-D sensor demonstrate superiority of the proposed approach both in accuracy and efficiency.


Manifold Covariance Agglomeration 



This work was supported by German Research Foundation (DFG) under grant BE 2556/12 ALROMA in priority programme SPP 1527 Autonomous Learning.


  1. 1.
    Alhashim, I., Xu, K., Zhuang, Y., Cao, J., Simari, P., Zhang, H.: Deformation-driven topology-varying 3D shape correspondence. TOG 34(6), 236 (2015)CrossRefGoogle Scholar
  2. 2.
    Bergtholdt, M., Kappes, J., Schmidt, S., Schnörr, C.: A study of parts-based object class detection using complete graphs. IJCV 87(1–2), 93–117 (2010)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bronstein, A.M., Bronstein, M.M., Kimmel, R., Mahmoudi, M., Sapiro, G.: A Gromov-Hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching. IJCV 89(2–3), 266–286 (2010)CrossRefGoogle Scholar
  4. 4.
    Bronstein, M.M., Bronstein, A.M.: Shape recognition with spectral distances. PAMI 33(5), 1065–1071 (2010)CrossRefGoogle Scholar
  5. 5.
    Chatfield, K., Lempitsky, V.S., Vedaldi, A., Zisserman, A.: The devil is in the details: an evaluation of recent feature encoding methods. In: BMVC (2011)Google Scholar
  6. 6.
    Chen, X., Golovinskiy, A., Funkhouser, T.: A benchmark for 3D mesh segmentation. ACM TOG 28, 73 (2009)Google Scholar
  7. 7.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. PAMI 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  8. 8.
    Golovinskiy, A., Funkhouser, T.: Consistent segmentation of 3D models. Comput. Graph. 33(3), 262–269 (2009)CrossRefGoogle Scholar
  9. 9.
    Hu, R., Fan, L., Liu, L.: Co-segmentation of 3D shapes via subspace clustering. CGF 31, 1703–1713 (2012)Google Scholar
  10. 10.
    Huang, Q., Koltun, V., Guibas, L.: Joint shape segmentation with linear programming. ACM TOG 30, 125 (2011)Google Scholar
  11. 11.
    Huang, Q., Wang, F., Guibas, L.: Functional map networks for analyzing and exploring large shape collections. TOG 33(4), 36 (2014)Google Scholar
  12. 12.
    Jaakkola, T., Haussler, D., et al.: Exploiting generative models in discriminative classifiers. In: NIPS, pp. 487–493 (1999)Google Scholar
  13. 13.
    van Kaick, O., Tagliasacchi, A., Sidi, O., Zhang, H., Cohen-Or, D., Wolf, L., Hamarneh, G.: Prior knowledge for part correspondence. CGF 30, 553–562 (2011)Google Scholar
  14. 14.
    van Kaick, O., Xu, K., Zhang, H., Wang, Y., Sun, S., Shamir, A., Cohen-Or, D.: Co-hierarchical analysis of shape structures. TOG 32(4), 69 (2013)MATHGoogle Scholar
  15. 15.
    Kalogerakis, E., Hertzmann, A., Singh, K.: Learning 3D mesh segmentation and labeling. ACM TOG 29, 102 (2010)CrossRefGoogle Scholar
  16. 16.
    Kim, V.G., Li, W., Mitra, N.J., Chaudhuri, S., DiVerdi, S., Funkhouser, T.: Learning part-based templates from large collections of 3D shapes. TOG 32(4), 70 (2013)MATHGoogle Scholar
  17. 17.
    Liang, J., Lai, R., Wong, T.W., Zhao, H.: Geometric understanding of point clouds using Laplace-Beltrami operator. In: CVPR, pp. 214–221 (2012)Google Scholar
  18. 18.
    Liu, Y., Prabhakaran, B., Guo, X.: Point-based manifold harmonics. VCG 18(10), 1693–1703 (2012)Google Scholar
  19. 19.
    Marton, Z.C., Rusu, R.B., Beetz, M.: On fast surface reconstruction methods for large and noisy datasets. In: ICRA, Kobe, Japan, 12–17 May 2009Google Scholar
  20. 20.
    Meng, M., Xia, J., Luo, J., He, Y.: Unsupervised co-segmentation for 3D shapes using iterative multi-label optimization. CAD 45(2), 312–320 (2013)MathSciNetGoogle Scholar
  21. 21.
    Mitra, N.J., Wand, M., Zhang, H., Cohen-Or, D., Kim, V., Huang, Q.X.: Structure-aware shape processing. In: Eurographics Report. ACM (2014)Google Scholar
  22. 22.
    Ohbuchi, R., Osada, K., Furuya, T., Banno, T.: Salient local visual features for shape-based 3D model retrieval. In: SMA, pp. 93–102. IEEE (2008)Google Scholar
  23. 23.
    Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. TOG 31(4), 30 (2012)CrossRefGoogle Scholar
  24. 24.
    Ovsjanikov, M., Bronstein, A.M., Bronstein, M.M., Guibas, L.J.: Shape Google: a computer vision approach to isometry invariant shape retrieval. In: ICCV Workshops, pp. 320–327. IEEE (2009)Google Scholar
  25. 25.
    Ovsjanikov, M., Li, W., Guibas, L., Mitra, N.J.: Exploration of continuous variability in collections of 3D shapes. TOG 30(4), 33 (2011)CrossRefGoogle Scholar
  26. 26.
    Perronnin, F., Sánchez, J., Mensink, T.: Improving the fisher kernel for large-scale image classification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 143–156. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  27. 27.
    Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3D registration. In: ICRA, pp. 3212–3217. IEEE (2009)Google Scholar
  28. 28.
    Sánchez, J., Perronnin, F., Mensink, T., Verbeek, J.: Image classification with the fisher vector: theory and practice. IJCV 105(3), 222–245 (2013)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Schnitman, Y., Caspi, Y., Cohen-Or, D., Lischinski, D.: Inducing semantic segmentation from an example. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 373–384. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  30. 30.
    Schoeler, M., Papon, J., Wörgötter, F.: Constrained planar cuts-object partitioning for point clouds. In: CVPR, pp. 5207–5215 (2015)Google Scholar
  31. 31.
    Shapira, L., Shalom, S., Shamir, A., Cohen-Or, D., Zhang, H.: Contextual part analogies in 3D objects. IJCV 89(2), 309–326 (2010)CrossRefGoogle Scholar
  32. 32.
    Shapira, L., Shamir, A., Cohen-Or, D.: Consistent mesh partitioning and skeletonisation using the shape diameter function. VC 24(4), 249–259 (2008)Google Scholar
  33. 33.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: Textonboost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. IJCV 81(1), 2–23 (2009)CrossRefGoogle Scholar
  34. 34.
    Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., Cohen-Or, D.: Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM TOG 30, 126 (2011)CrossRefGoogle Scholar
  35. 35.
    Toldo, R., Castellani, U., Fusiello, A.: Visual vocabulary signature for 3D object retrieval and partial matching. In: Proceedings of the 3DOR Conference, pp. 21–28 (2009)Google Scholar
  36. 36.
    Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 356–369. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  37. 37.
    Toshev, A., Shi, J., Daniilidis, K.: Image matching via saliency region correspondences. In: CVPR, pp. 1–8 (2007)Google Scholar
  38. 38.
    Van Kaick, O., Zhang, H., Hamarneh, G., Cohen-Or, D.: A survey on shape correspondence. CGF 30, 1681–1707 (2011)Google Scholar
  39. 39.
    Wainwright, M.J., Jaakkola, T.S., Willsky, A.S.: Map estimation via agreement on trees: message-passing and linear programming. IEEE Trans. Inf. Theor. 51(11), 3697–3717 (2005)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Wang, F., Huang, Q., Ovsjanikov, M., Guibas, L.J.: Unsupervised multi-class joint image segmentation. In: CVPR, pp. 3142–3149 (2014)Google Scholar
  41. 41.
    Wang, Y., Asafi, S., van Kaick, O., Zhang, H., Cohen-Or, D., Chen, B.: Active co-analysis of a set of shapes. TOG 31(6), 165 (2012)CrossRefGoogle Scholar
  42. 42.
    Xu, K., Li, H., Zhang, H., Cohen-Or, D., Xiong, Y., Cheng, Z.Q.: Style-content separation by anisotropic part scales. TOG 29(6), 184 (2010)CrossRefGoogle Scholar
  43. 43.
    Yedidia, J.S., Freeman, W.T., Weiss, Y.: Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Trans. Inf. Theor. 51(7), 2282–2312 (2005)MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Zhang, H., Sheffer, A., Cohen-Or, D., Zhou, Q., Van Kaick, O., Tagliasacchi, A.: Deformation-driven shape correspondence. CGF 27, 1431–1439 (2008)Google Scholar
  45. 45.
    Zheng, Y., Cohen-Or, D., Averkiou, M., Mitra, N.J.: Recurring part arrangements in shape collections. CGF 33, 115–124 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Nikita Araslanov
    • 1
  • Seongyong Koo
    • 1
  • Juergen Gall
    • 1
  • Sven Behnke
    • 1
  1. 1.Computer Science InstituteUniversity of BonnBonnGermany

Personalised recommendations