Skip to main content
SpringerLink
Log in

View selection for sketch-based 3D model retrieval using visual part shape description

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Hand drawings are the imprints of shapes in human’s mind. How a human expresses a shape is a consequence of how he or she visualizes it. A query-by-sketch 3D object retrieval application is closely tied to this concept from two aspects. First, describing sketches must involve elements in a figure that matter most to a human. Second, the representative 2D projection of the target 3D objects should be limited to “the canonical views” from a human cognition perspective. We advocate for these two rules by presenting a new approach for sketch-based 3D object retrieval that describes a 2D shape by the visual protruding parts of its silhouette. Furthermore, we present a list of candidate 2D projections that represent the canonical views of a 3D object. The general rule is that humans would visually avoid part occlusion and symmetry. We quantify the extent of part occlusion of the projected silhouettes of 3D objects by skeletal length computations. Sorting the projected views in the decreasing order of skeletal lengths gives access to a subset of the best representative views. We experimentally show how views that cause misinterpretation and mismatching can be detected according to the part occlusion criteria. We also propose criteria for locating side, off axis, or asymmetric views.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Al-Naymat, G., Chawla, S., Taheri, J.: Sparsedtw: a novel approach to speed up dynamic time warping. In: Proceedings of the eighth Australasian data mining conference—vol. 101, AusDM ’09, pp. 117–127. Australian Computer Society, Inc., Darlinghurst, Australia, Australia. http://dl.acm.org/citation.cfm?id=2449360.2449384 (2009)

  2. Aono, M., Iwabuchi, H.: 3d shape retrieval from a 2d image as query. In: Signal & information processing association annual summit and conference (APSIPA ASC) 2012, vol. 3 (2012)

  3. Bertamini, M., Wagemans, J.: Processing convexity and concavity along a 2-d contour: figureground, structural shape, and attention. Psychon. Bull. Rev. 20(2), 191–207 (2013). doi:10.3758/s13423-012-0347-2

    Article  Google Scholar 

  4. Blanz, V., Tarr, M.J., Bülthoff, H.H., Vetter, T.: What object attributes determine canonical views? Percept. Lond. 28(5), 575–600 (1999)

    Article  Google Scholar 

  5. Catmull, E., Clark, J.: Recursively generated b-spline surfaces on arbitrary topological meshes. Computer-aided Des. 10(6), 350–355 (1978)

    Article  Google Scholar 

  6. Chaouch, M., Verroust-Blondet, A.: Alignment of 3d models. Graph. Models 71(2), 63–76 (2009)

    Article  Google Scholar 

  7. Cohen, E.H., Singh, M.: Geometric determinants of shape segmentation: tests using segment identification. Vis. Res. 47(22), 2825–2840 (2007). doi:10.1016/j.visres.2007.06.021. http://www.sciencedirect.com/science/article/pii/S0042698907002696

  8. De Winter, J., Wagemans, J.: The awakening of attneave’s sleeping cat: identification of everyday objects on the basis of straight-line versions of outlines. Perception 37, 245–270 (2008). doi:10.1068/p5429

    Article  Google Scholar 

  9. DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., Santella, A.: Suggestive contours for conveying shape. ACM Trans. Graph. 22(3), 848–855 (2003). doi:10.1145/882262.882354

    Article  Google Scholar 

  10. Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. 31(4), 44:1–44:10 (2012). doi:10.1145/2185520.2185540

  11. Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., Alexa, M.: Sketch-based shape retrieval. ACM Trans. Graph. 31(4), 31:1–31:10 (2012). doi:10.1145/2185520.2185527

  12. Furuya, T., Ohbuchi, R.: Ranking on cross-domain manifold for sketch-based 3d model retrieval. In: international conference on cyberworlds (CW), pp. 274–281 (2013). doi: 10.1109/CW.2013.60

  13. Hoffman, D.D., Singh, M.: Salience of visual parts. Cognition 63(1), 29–78 (1997). doi:10.1016/S0010-0277(96)00791-3. http://www.sciencedirect.com/science/article/pii/S0010027796007913

  14. Judd, T., Durand, F., Adelson, E.: Apparent ridges for line drawing. ACM Trans. Graph. 26(3) (2007). doi:10.1145/1276377.1276401

  15. Lemire, D.: Faster retrieval with a two-pass dynamic-time-warping lower bound. Pattern Recogn. 42(9), 2169–2180 (2009). doi:10.1016/j.patcog.2008.11.030

    Article  MATH  Google Scholar 

  16. Li, B., Johan, H.: Sketch-based 3d model retrieval by incorporating 2d-3d alignment. Multimed. Tools Appl. 61(1) (2012) (Online first version)

  17. Li, B., Lu, Y., Fares, R.: Semantic sketch-based 3d model retrieval. In: Multimedia and expo workshops (ICMEW), 2013 IEEE international conference on, pp. 1–4. IEEE (2013)

  18. Li, B., Lu, Y., Godil, A., Schreck, T., Aono, M., Johan, H., Saavedra, J.M., Tashiro, S.: Shrec’13 track: large scale sketch-based 3d shape retrieval. In: Proceedings of the Sixth Eurographics Workshop on 3D Object Retrieval, 3DOR ’13, pp. 89–96. Eurographics Association, Aire-la-Ville, Switzerland, Switzerland (2013). doi:10.2312/3DOR/3DOR13/089-096

  19. Li, B., Lu, Y., Godil, A., Schreck, T., Bustos, B., Ferreira, A., Furuya, T., Fonseca, M.J., Johan, H., Matsuda, T., Ohbuchi, R., Pascoal, P.B., Saavedra, J.M.: A comparison of methods for sketch-based 3d shape retrieval. Comput. Vis. Image Underst. 119, 57–80 (2014). doi:10.1016/j.cviu.2013.11.008. http://www.sciencedirect.com/science/article/pii/S1077314213002282

  20. Li, B., Lu, Y., Johan, H.: Sketch-based 3d model retrieval by viewpoint entropy-based adaptive view clustering. In: Proceedings of the sixth Eurographics workshop on 3D object retrieval, 3DOR ’13, pp. 49–56. Eurographics Association, Aire-la-Ville, Switzerland, Switzerland (2013). doi:10.2312/3DOR/3DOR13/049-056

  21. Li, B., Lu, Y., Li, C., Godil, A., Schreck, T., Aono, M., Burtscher, M., Fu, H., Furuya, T., Johan, H., et al.: Extended large scale sketch-based 3d shape retrieval. In: Eurographics workshop on 3D object retrieval, pp. 121–130. The Eurographics Association (2014)

  22. Li, B., Schreck, T., Godil, A., Alexa, M., Boubekeur, T., Bustos, B., Chen, J., Eitz, M., Furuya, T., Hildebrand, K., Huang, S., Johan, H., Kuijper, A., Ohbuchi, R., Richter, R., Saavedra, J.M., Scherer, M., Yanagimachi, T., Yoon, G.J., Yoon, S.M.: Shrec’12 track: sketch-based 3d shape retrieval. In: 3DOR, pp. 109–118 (2012)

  23. Mezuman, E., Weiss, Y.: Learning about canonical views from internet image collections. In: Proceedings of the Neural Information Processing Systems Conference, pp. 719–727 (2012). https://papers.nips.cc/paper/4827-learning-about-canonical-views-from-internetimagecollections

  24. Napoléon, T., Sahbi, H.: From 2d silhouettes to 3d object retrieval: contributions and benchmarking. J. Image Video Process. 2010, 1:1–1:22 (2010). doi:10.1155/2010/367181

  25. Neri, P.: Wholes and subparts in visual processing of human agency. Proc. R. Soc. B Biol. Sci. 276(1658), 861–869 (2009). doi:10.1098/rspb.2008.1363. http://rspb.royalsocietypublishing.org/content/276/1658/861.abstract

  26. Ohbuchi, R., Furuya, T.: Scale-weighted dense bag of visual features for 3d model retrieval from a partial view 3d model. In: IEEE ICCV 2009 workshop on search in 3D and video (S3DV) pp. 63–70 (2009)

  27. Palmer, S., Rosch, E., Chase, P.: Canonical perspective and the perception of objects. Atten. Perform. IX, 135–151 (1981)

  28. Prasad, L.: Rectification of the chordal axis transform skeleton and criteria for shape decomposition. Image Vis. Comput. 25(10), 1557–1571 (2007). doi:10.1016/j.imavis.2006.06.025. http://www.sciencedirect.com/science/article/pii/S026288560600309X. (Discrete Geometry for Computer Imagery 2005)

  29. Saavedra, J., Bustos, B., Scherer, M., Schreck, T.: Stela: sketch-based 3d model retrieval using a structure-based local approach. In: Proc. ACM international conference on multimedia retrieval (ICMR’11), pp. 26:1–26:8. ACM (2011)

  30. Saavedra, J.M., Bustos, B., Schreck, T., Yoon, S.M., Scherer, M.: Sketch-based 3d model retrieval using keyshapes for global and local representation. In: Proceedings of the 5th eurographics conference on 3D object retrieval, EG 3DOR, pp. 47–50 (2012). doi:10.2312/3DOR/3DOR12/047-050

  31. Salvador, S., Chan, P.: Toward accurate dynamic time warping in linear time and space. Intell. Data Anal. 11(5), 561–580 (2007). http://dl.acm.org/citation.cfm?id=1367985.1367993

  32. Sebastian, T.B., Klein, P.N., Kimia, B.B.: Recognition of shapes by editing their shock graphs. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 550–571 (2004). doi:10.1109/TPAMI.2004.1273924

    Article  Google Scholar 

  33. Shao, T., Xu, W., Yin, K., Wang, J., Zhou, K., Guo, B.: Discriminative sketch-based 3d model retrieval via robust shape matching. Comput. Graph. Forum 30(7), 2011–2020 (2011)

    Article  Google Scholar 

  34. Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton shape benchmark. In: Shape Modeling Applications, 2004. Proceedings, pp. 167–178 (2004). doi:10.1109/SMI.2004.1314504

  35. Vintsyuk, T.: Speech discrimination by dynamic programming. Cybern. Syst. Anal. 4(1), 52–57 (1968)

    Article  MathSciNet  Google Scholar 

  36. Wang, X., Mueen, A., Ding, H., Trajcevski, G., Scheuermann, P., Keogh, E.: Experimental comparison of representation methods and distance measures for time series data. Data Min. Knowl. Discov. 26(2), 275–309 (2013). doi:10.1007/s10618-012-0250-5

    Article  MathSciNet  Google Scholar 

  37. Yasseen, Z., Verroust-Blondet, A., Nasri, A.: Sketch-based 3D object retrieval using two views and a visual part alignment. In: Pratikakis, I., Spagnuolo, M., Theoharis, T., Gool, L.V., Veltkamp, R. (eds.) 3DOR 2015—Eurographics workshop on 3D object retrieval, p. 8. Zurich, Switzerland (2015). doi:10.2312/3dor.20151053. https://hal.inria.fr/hal-01184954

  38. Yasseen, Z., Verroust-Blondet, A., Nasri, A.: Shape matching by part alignment using extended chordal axis transform. Pattern Recognit. 57, 115–135 (2016). doi:10.1016/j.patcog.2016.03.022. http://www.sciencedirect.com/science/article/pii/S0031320316300073

  39. Yoon, S.M., Scherer, M., Schreck, T., Kuijper, A.: Sketch-based 3d model retrieval using diffusion tensor fields of suggestive contours. In: Proceedings of the international conference on Multimedia, MM ’10, pp. 193–200. ACM, New York, NY, USA (2010). doi:10.1145/1873951.1873961

  40. Zhao, L., Liang, S., Jia, J., Wei, Y.: Learning best views of 3d shapes from sketch contour. Vis. Comput. 31(6), 765–774 (2015). doi:10.1007/s00371-015-1091-1

  41. Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. Adv. Neural Inf. Process. Syst. 16(16), 321–328 (2004)

    Google Scholar 

  42. Zou, C., Wang, C., Wen, Y., Zhang, L., Liu, J.: Viewpoint-aware representation for sketch-based 3d model retrieval. Signal Process. Lett. IEEE 21(8), 966–970 (2014)

    Article  Google Scholar 

Download references

Acknowledgments

We thank the Computer Science Department of the American University of Beirut for offering lab space and machines to perform the extensive tests presented in this paper. Particular thanks are due for Mr. Mustapha (Mike) Hamam, the systems analyst of the department.

Author information

Authors and Affiliations

  1. Inria Paris, 2 rue Simone Iff CS 42112, 75589, Paris Cedex 12, France

    Zahraa Yasseen & Anne Verroust-Blondet

  2. National Council for Scientific Research, CNRS, 59, Zahia Salmane street, Jnah, P.O. Box 11-8281, Beirut, Lebanon

    Ahmad Nasri

Authors
  1. Zahraa Yasseen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anne Verroust-Blondet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmad Nasri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zahraa Yasseen.

Additional information

This work is supported by the Lebanese National Council for Scientific Research through a Ph.d. Scholarship given to Zahraa Yasseen and a research grant awarded to Ahmad Nasri.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yasseen, Z., Verroust-Blondet, A. & Nasri, A. View selection for sketch-based 3D model retrieval using visual part shape description. Vis Comput 33, 565–583 (2017). https://doi.org/10.1007/s00371-016-1328-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-016-1328-7

Keywords

Search

Navigation