Advertisement

The Visual Computer

, Volume 33, Issue 5, pp 565–583 | Cite as

View selection for sketch-based 3D model retrieval using visual part shape description

  • Zahraa YasseenEmail author
  • Anne Verroust-Blondet
  • Ahmad Nasri
Original Article

Abstract

Hand drawings are the imprints of shapes in human’s mind. How a human expresses a shape is a consequence of how he or she visualizes it. A query-by-sketch 3D object retrieval application is closely tied to this concept from two aspects. First, describing sketches must involve elements in a figure that matter most to a human. Second, the representative 2D projection of the target 3D objects should be limited to “the canonical views” from a human cognition perspective. We advocate for these two rules by presenting a new approach for sketch-based 3D object retrieval that describes a 2D shape by the visual protruding parts of its silhouette. Furthermore, we present a list of candidate 2D projections that represent the canonical views of a 3D object. The general rule is that humans would visually avoid part occlusion and symmetry. We quantify the extent of part occlusion of the projected silhouettes of 3D objects by skeletal length computations. Sorting the projected views in the decreasing order of skeletal lengths gives access to a subset of the best representative views. We experimentally show how views that cause misinterpretation and mismatching can be detected according to the part occlusion criteria. We also propose criteria for locating side, off axis, or asymmetric views.

Keywords

Sketch-based 3D object retrieval 2D Shape description Best view selection Symmetry estimation Side view 

Notes

Acknowledgments

We thank the Computer Science Department of the American University of Beirut for offering lab space and machines to perform the extensive tests presented in this paper. Particular thanks are due for Mr. Mustapha (Mike) Hamam, the systems analyst of the department.

References

  1. 1.
    Al-Naymat, G., Chawla, S., Taheri, J.: Sparsedtw: a novel approach to speed up dynamic time warping. In: Proceedings of the eighth Australasian data mining conference—vol. 101, AusDM ’09, pp. 117–127. Australian Computer Society, Inc., Darlinghurst, Australia, Australia. http://dl.acm.org/citation.cfm?id=2449360.2449384 (2009)
  2. 2.
    Aono, M., Iwabuchi, H.: 3d shape retrieval from a 2d image as query. In: Signal & information processing association annual summit and conference (APSIPA ASC) 2012, vol. 3 (2012)Google Scholar
  3. 3.
    Bertamini, M., Wagemans, J.: Processing convexity and concavity along a 2-d contour: figureground, structural shape, and attention. Psychon. Bull. Rev. 20(2), 191–207 (2013). doi: 10.3758/s13423-012-0347-2 CrossRefGoogle Scholar
  4. 4.
    Blanz, V., Tarr, M.J., Bülthoff, H.H., Vetter, T.: What object attributes determine canonical views? Percept. Lond. 28(5), 575–600 (1999)CrossRefGoogle Scholar
  5. 5.
    Catmull, E., Clark, J.: Recursively generated b-spline surfaces on arbitrary topological meshes. Computer-aided Des. 10(6), 350–355 (1978)CrossRefGoogle Scholar
  6. 6.
    Chaouch, M., Verroust-Blondet, A.: Alignment of 3d models. Graph. Models 71(2), 63–76 (2009)CrossRefGoogle Scholar
  7. 7.
    Cohen, E.H., Singh, M.: Geometric determinants of shape segmentation: tests using segment identification. Vis. Res. 47(22), 2825–2840 (2007). doi: 10.1016/j.visres.2007.06.021. http://www.sciencedirect.com/science/article/pii/S0042698907002696
  8. 8.
    De Winter, J., Wagemans, J.: The awakening of attneave’s sleeping cat: identification of everyday objects on the basis of straight-line versions of outlines. Perception 37, 245–270 (2008). doi: 10.1068/p5429 CrossRefGoogle Scholar
  9. 9.
    DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., Santella, A.: Suggestive contours for conveying shape. ACM Trans. Graph. 22(3), 848–855 (2003). doi: 10.1145/882262.882354 CrossRefGoogle Scholar
  10. 10.
    Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. 31(4), 44:1–44:10 (2012). doi: 10.1145/2185520.2185540
  11. 11.
    Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., Alexa, M.: Sketch-based shape retrieval. ACM Trans. Graph. 31(4), 31:1–31:10 (2012). doi: 10.1145/2185520.2185527
  12. 12.
    Furuya, T., Ohbuchi, R.: Ranking on cross-domain manifold for sketch-based 3d model retrieval. In: international conference on cyberworlds (CW), pp. 274–281 (2013). doi:  10.1109/CW.2013.60
  13. 13.
    Hoffman, D.D., Singh, M.: Salience of visual parts. Cognition 63(1), 29–78 (1997). doi: 10.1016/S0010-0277(96)00791-3. http://www.sciencedirect.com/science/article/pii/S0010027796007913
  14. 14.
    Judd, T., Durand, F., Adelson, E.: Apparent ridges for line drawing. ACM Trans. Graph. 26(3) (2007). doi: 10.1145/1276377.1276401
  15. 15.
    Lemire, D.: Faster retrieval with a two-pass dynamic-time-warping lower bound. Pattern Recogn. 42(9), 2169–2180 (2009). doi: 10.1016/j.patcog.2008.11.030 CrossRefzbMATHGoogle Scholar
  16. 16.
    Li, B., Johan, H.: Sketch-based 3d model retrieval by incorporating 2d-3d alignment. Multimed. Tools Appl. 61(1) (2012) (Online first version)Google Scholar
  17. 17.
    Li, B., Lu, Y., Fares, R.: Semantic sketch-based 3d model retrieval. In: Multimedia and expo workshops (ICMEW), 2013 IEEE international conference on, pp. 1–4. IEEE (2013)Google Scholar
  18. 18.
    Li, B., Lu, Y., Godil, A., Schreck, T., Aono, M., Johan, H., Saavedra, J.M., Tashiro, S.: Shrec’13 track: large scale sketch-based 3d shape retrieval. In: Proceedings of the Sixth Eurographics Workshop on 3D Object Retrieval, 3DOR ’13, pp. 89–96. Eurographics Association, Aire-la-Ville, Switzerland, Switzerland (2013). doi: 10.2312/3DOR/3DOR13/089-096
  19. 19.
    Li, B., Lu, Y., Godil, A., Schreck, T., Bustos, B., Ferreira, A., Furuya, T., Fonseca, M.J., Johan, H., Matsuda, T., Ohbuchi, R., Pascoal, P.B., Saavedra, J.M.: A comparison of methods for sketch-based 3d shape retrieval. Comput. Vis. Image Underst. 119, 57–80 (2014). doi: 10.1016/j.cviu.2013.11.008. http://www.sciencedirect.com/science/article/pii/S1077314213002282
  20. 20.
    Li, B., Lu, Y., Johan, H.: Sketch-based 3d model retrieval by viewpoint entropy-based adaptive view clustering. In: Proceedings of the sixth Eurographics workshop on 3D object retrieval, 3DOR ’13, pp. 49–56. Eurographics Association, Aire-la-Ville, Switzerland, Switzerland (2013). doi: 10.2312/3DOR/3DOR13/049-056
  21. 21.
    Li, B., Lu, Y., Li, C., Godil, A., Schreck, T., Aono, M., Burtscher, M., Fu, H., Furuya, T., Johan, H., et al.: Extended large scale sketch-based 3d shape retrieval. In: Eurographics workshop on 3D object retrieval, pp. 121–130. The Eurographics Association (2014)Google Scholar
  22. 22.
    Li, B., Schreck, T., Godil, A., Alexa, M., Boubekeur, T., Bustos, B., Chen, J., Eitz, M., Furuya, T., Hildebrand, K., Huang, S., Johan, H., Kuijper, A., Ohbuchi, R., Richter, R., Saavedra, J.M., Scherer, M., Yanagimachi, T., Yoon, G.J., Yoon, S.M.: Shrec’12 track: sketch-based 3d shape retrieval. In: 3DOR, pp. 109–118 (2012)Google Scholar
  23. 23.
    Mezuman, E., Weiss, Y.: Learning about canonical views from internet image collections. In: Proceedings of the Neural Information Processing Systems Conference, pp. 719–727 (2012). https://papers.nips.cc/paper/4827-learning-about-canonical-views-from-internetimagecollections
  24. 24.
    Napoléon, T., Sahbi, H.: From 2d silhouettes to 3d object retrieval: contributions and benchmarking. J. Image Video Process. 2010, 1:1–1:22 (2010). doi: 10.1155/2010/367181
  25. 25.
    Neri, P.: Wholes and subparts in visual processing of human agency. Proc. R. Soc. B Biol. Sci. 276(1658), 861–869 (2009). doi: 10.1098/rspb.2008.1363. http://rspb.royalsocietypublishing.org/content/276/1658/861.abstract
  26. 26.
    Ohbuchi, R., Furuya, T.: Scale-weighted dense bag of visual features for 3d model retrieval from a partial view 3d model. In: IEEE ICCV 2009 workshop on search in 3D and video (S3DV) pp. 63–70 (2009)Google Scholar
  27. 27.
    Palmer, S., Rosch, E., Chase, P.: Canonical perspective and the perception of objects. Atten. Perform. IX, 135–151 (1981)Google Scholar
  28. 28.
    Prasad, L.: Rectification of the chordal axis transform skeleton and criteria for shape decomposition. Image Vis. Comput. 25(10), 1557–1571 (2007). doi: 10.1016/j.imavis.2006.06.025. http://www.sciencedirect.com/science/article/pii/S026288560600309X. (Discrete Geometry for Computer Imagery 2005)
  29. 29.
    Saavedra, J., Bustos, B., Scherer, M., Schreck, T.: Stela: sketch-based 3d model retrieval using a structure-based local approach. In: Proc. ACM international conference on multimedia retrieval (ICMR’11), pp. 26:1–26:8. ACM (2011)Google Scholar
  30. 30.
    Saavedra, J.M., Bustos, B., Schreck, T., Yoon, S.M., Scherer, M.: Sketch-based 3d model retrieval using keyshapes for global and local representation. In: Proceedings of the 5th eurographics conference on 3D object retrieval, EG 3DOR, pp. 47–50 (2012). doi: 10.2312/3DOR/3DOR12/047-050
  31. 31.
    Salvador, S., Chan, P.: Toward accurate dynamic time warping in linear time and space. Intell. Data Anal. 11(5), 561–580 (2007). http://dl.acm.org/citation.cfm?id=1367985.1367993
  32. 32.
    Sebastian, T.B., Klein, P.N., Kimia, B.B.: Recognition of shapes by editing their shock graphs. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 550–571 (2004). doi: 10.1109/TPAMI.2004.1273924 CrossRefGoogle Scholar
  33. 33.
    Shao, T., Xu, W., Yin, K., Wang, J., Zhou, K., Guo, B.: Discriminative sketch-based 3d model retrieval via robust shape matching. Comput. Graph. Forum 30(7), 2011–2020 (2011)CrossRefGoogle Scholar
  34. 34.
    Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton shape benchmark. In: Shape Modeling Applications, 2004. Proceedings, pp. 167–178 (2004). doi: 10.1109/SMI.2004.1314504
  35. 35.
    Vintsyuk, T.: Speech discrimination by dynamic programming. Cybern. Syst. Anal. 4(1), 52–57 (1968)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Wang, X., Mueen, A., Ding, H., Trajcevski, G., Scheuermann, P., Keogh, E.: Experimental comparison of representation methods and distance measures for time series data. Data Min. Knowl. Discov. 26(2), 275–309 (2013). doi: 10.1007/s10618-012-0250-5 MathSciNetCrossRefGoogle Scholar
  37. 37.
    Yasseen, Z., Verroust-Blondet, A., Nasri, A.: Sketch-based 3D object retrieval using two views and a visual part alignment. In: Pratikakis, I., Spagnuolo, M., Theoharis, T., Gool, L.V., Veltkamp, R. (eds.) 3DOR 2015—Eurographics workshop on 3D object retrieval, p. 8. Zurich, Switzerland (2015). doi: 10.2312/3dor.20151053. https://hal.inria.fr/hal-01184954
  38. 38.
    Yasseen, Z., Verroust-Blondet, A., Nasri, A.: Shape matching by part alignment using extended chordal axis transform. Pattern Recognit. 57, 115–135 (2016). doi: 10.1016/j.patcog.2016.03.022. http://www.sciencedirect.com/science/article/pii/S0031320316300073
  39. 39.
    Yoon, S.M., Scherer, M., Schreck, T., Kuijper, A.: Sketch-based 3d model retrieval using diffusion tensor fields of suggestive contours. In: Proceedings of the international conference on Multimedia, MM ’10, pp. 193–200. ACM, New York, NY, USA (2010). doi: 10.1145/1873951.1873961
  40. 40.
    Zhao, L., Liang, S., Jia, J., Wei, Y.: Learning best views of 3d shapes from sketch contour. Vis. Comput. 31(6), 765–774 (2015). doi: 10.1007/s00371-015-1091-1
  41. 41.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. Adv. Neural Inf. Process. Syst. 16(16), 321–328 (2004)Google Scholar
  42. 42.
    Zou, C., Wang, C., Wen, Y., Zhang, L., Liu, J.: Viewpoint-aware representation for sketch-based 3d model retrieval. Signal Process. Lett. IEEE 21(8), 966–970 (2014)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Inria ParisParis Cedex 12France
  2. 2.National Council for Scientific Research, CNRSBeirutLebanon

Personalised recommendations