The Visual Computer

, Volume 30, Issue 11, pp 1293–1308 | Cite as

A benchmark of simulated range images for partial shape retrieval

  • Ivan SipiranEmail author
  • Rafael Meruane
  • Benjamin Bustos
  • Tobias Schreck
  • Bo Li
  • Yijuan Lu
  • Henry Johan
Original Article


In this paper, we address the evaluation of algorithms for partial shape retrieval using a large-scale simulated benchmark of partial views which are used as queries. Since the scanning of real objects is a time-consuming task, we create a simulation that generates a set of views from a target model and at different levels of complexity (amount of missing data). In total, our benchmark contains 7,200 partial views. Furthermore, we propose the use of weighted effectiveness measures based on the complexity of a query. With these characteristics, we aim at jointly evaluating the effectiveness, efficiency and robustness of existing algorithms. As a result of our evaluation, we found that a combination of methods provides the best effectiveness, mainly due to the complementary information that they deliver. The obtained results open new questions regarding the difficulty of the partial shape retrieval problem. As a consequence, potential future directions are also identified.


Partial shape retrieval Performance evaluation Benchmarking 



The work of Ivan Sipiran and Tobias Schreck was supported by EC FP7 STREP Project PRESIOUS, Grant No. 600533. Benjamin Bustos has been partially funded by FONDECYT (Chile) Project 1140783. This work of Bo Li and Yijuan Lu has been supported by the Army Research Office grant W911NF-12-1-0057, Texas State University Research Enhancement Program (REP), and NSF CRI 1305302 to Yijuan Lu. Henry Johan is supported by Fraunhofer IDM@NTU, which is funded by the National Research Foundation (NRF) and managed through the multi-agency Interactive & Digital Media Programme Office (IDMPO) hosted by the Media Development Authority of Singapore (MDA).


  1. 1.
    Beecks, C., Uysal, M.S., Seidl, T.: Signature quadratic form distances for content-based similarity. In: Proceedings of the 17th ACM International Conference on Multimedia, MM’09, pp. 697–700. ACM (2009)Google Scholar
  2. 2.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)CrossRefGoogle Scholar
  3. 3.
    Biasotti, S., Bai, X., Bustos, B., Cerri, A., Giorgi, D., Li, L., Mortara, M., Sipiran, I., Zhang, S., Spagnuolo, M.: SHREC’12 Track: Stability on Abstract Shapes. In: Proc. Eurographics Workshop on 3D Object, pp. 101–107. Eurographics Association (2012)Google Scholar
  4. 4.
    Bronstein, A., Bronstein, M., Guibas, L., Ovsjanikov, M.: Shape Google: geometric words and expressions for invariant shape retrieval. ACM Trans. Comput. Graph. 30(1), 1–20 (2011)Google Scholar
  5. 5.
    Browatzki, B., Fischer, J., Graf, B., Bulthoff, H., Wallraven, C.: Going into depth: evaluating 2D and 3D cues for object classification on a new, large-scale object dataset. IEEE ICCV Workshops 2011, 1189–1195 (2011)Google Scholar
  6. 6.
    Bustos, B., Sipiran, I.: 3D shape matching for retrieval and recognition. In: Pears, N., Liu, Y., Bunting, P. (eds.) 3D Imaging, Analysis and Applications, pp. 265–308. Springer, London (2012)CrossRefGoogle Scholar
  7. 7.
    Dutagaci, H., Godil, A., Axenopoulos, A., Daras, P., Furuya, T., Ohbuchi, R.: SHREC’09 Track: Querying with Partial Models. In: Proc. Eurographics Workshop on 3D Object Retrieval, pp. 69–76. Eurographics Association (2009)Google Scholar
  8. 8.
    Dutagaci, H., Godil, A., Cheung, C.P., Furuya, T., Hillenbrand, U., Ohbuchi, R.: SHREC’10 Track: Range Scan Retrieval. In: Proc. Eurographics Workshop on 3D Object Retrieval, pp. 109–115. Eurographics Association (2010)Google Scholar
  9. 9.
    Frome, A., Huber, D., Kolluri, R., Bulow, T., Malik, J.: Recognizing objects in range data using regional point descriptors. In: Proceedings of the European Conference on Computer Vision (ECCV) (2004)Google Scholar
  10. 10.
    Johnson, A.: Spin-images: a representation for 3-D surface matching. Ph.D. thesis, Robotics Institute, Carnegie Mellon University, Pittsburgh (1997)Google Scholar
  11. 11.
    Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view RGB-D object dataset. In: ICRA, pp. 1817–1824. IEEE (2011)Google Scholar
  12. 12.
    Leow, W.K., Li, R.: The analysis and applications of adaptive-binning color histograms. Comput. Vis. Image Underst. 94, 67–91 (2004)CrossRefGoogle Scholar
  13. 13.
    Li, B., Johan, H.: View context: a 3D model feature for retrieval. In: Boll, S. et al. (eds.): MMM 2010, LNCS, pp. 185–195. Springer, Heidelberg (2010) (5916)Google Scholar
  14. 14.
    Li, B., Johan, H.: Sketch-based 3D model retrieval by incorporating 2D–3D alignment. Multimed. Tools Appl. 65(3), 363–385 (2013)Google Scholar
  15. 15.
    Li, B., Lu, Y., Johan, H.: Sketch-based 3D model retrieval by viewpoint entropy-based adaptive view clustering. In: Proc. Eurographics Workshop on 3D Object Retrieval, pp. 49–56. Eurographics Association (2013)Google Scholar
  16. 16.
    Li, B., Schreck, T., Godil, A., Alexa, M., Boubekeur, T., Bustos, B., Chen, J., Eitz, M., Furuya, T., Hildebrand, K., Huang, S., Johan, H., Kuijper, A., Ohbuchi, R., Richter, R., Saavedra, J.M., Scherer, M., Yanagimachi, T., Yoon, G.J., Yoon, S.M.: SHREC’12 track: sketch-based 3D shape retrieval. In: Proc. Eurographics Workshop on 3D Object Retrieval, pp. 109–118. Eurographics Association (2012) Google Scholar
  17. 17.
    Liu, Y., Zha, H., Qin, H.: Shape topics: a compact representation and new algorithms for 3d partial shape retrieval. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2006)Google Scholar
  18. 18.
    Osada, R., Funkhouser, T., Chazelle, B., Dobkin, D.: Shape distributions. ACM Trans. Gr. 21(4), 807–832 (2002)CrossRefGoogle Scholar
  19. 19.
    Rusu, R., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3D registration. In: Robotics and Automation, 2009. ICRA ’09. IEEE International Conference, pp. 3212–3217 (2009)Google Scholar
  20. 20.
    Rusu, R., Holzbach, A., Blodow, N., Beetz, M.: Fast geometric point labeling using conditional random fields. In: Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference, pp. 7–12 (2009)Google Scholar
  21. 21.
    Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: International Conference on Robotics and Automation. Shanghai (2011)Google Scholar
  22. 22.
    Sipiran, I., Bustos, B.: Harris 3D: a robust extension of the Harris operator for interest point detection on 3D meshes. Vis. Comput. 27(11), 963–976 (2011)CrossRefGoogle Scholar
  23. 23.
    Sipiran, I., Bustos, B., Schreck, T.: Data-aware 3D partitioning for generic shape retrieval. Comput. Gr. 37(5), 460–472 (2013)CrossRefGoogle Scholar
  24. 24.
    Sipiran, I., Meruane, R., Bustos, B., Schreck, T., Li, B., Lu, Y., Johan, H.: SHREC’13 Track: Large-Scale Partial Shape Retrieval Using Simulated Range Images. In: Proc. Eurographics Workshop on 3D Object Retrieval, pp. 81–88. Eurographics Association (2013)Google Scholar
  25. 25.
    Takahashi, S., Fujishiro, I., Takeshima, Y., Nishita, T.: A feature-driven approach to locating optimal viewpoints for volume visualization. In: IEEE Visualization, p. 63. IEEE Computer Society (2005)Google Scholar
  26. 26.
    Veltkamp, R., Ter Haar, F.: SHREC 2007 3D retrieval contest. Technical report. Department of Information and Computing Science (2007)Google Scholar
  27. 27.
    Vranic, D.V.: DESIRE: a composite 3D-shape descriptor. In: Proceedings of the IEEE International Conference on Multimedia and Expo (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Ivan Sipiran
    • 1
    Email author
  • Rafael Meruane
    • 2
  • Benjamin Bustos
    • 2
  • Tobias Schreck
    • 1
  • Bo Li
    • 3
  • Yijuan Lu
    • 3
  • Henry Johan
    • 4
  1. 1.Department of Computer and Information SciencesUniversity of KonstanzKonstanzGermany
  2. 2.Department of Computer ScienceUniversity of ChileSantiagoChile
  3. 3.Department of Computer ScienceTexas State UniversitySan MarcosUSA
  4. 4.Fraunhofer IDM@NTUSingaporeSingapore

Personalised recommendations