The Visual Computer

, Volume 31, Issue 6–8, pp 765–774 | Cite as

Learning best views of 3D shapes from sketch contour

  • Long Zhao
  • Shuang Liang
  • Jinyuan Jia
  • Yichen Wei
Original Article


In this paper, we introduce a novel learning-based approach to automatically select the best views of 3D shapes using a new prior. We think that a viewpoint of the 3D shape is reasonable if a human usually draws the shape from it. Hand-drawn sketches collected from relevant datasets are used to model this concept. We reveal the connection between sketches and viewpoints by taking context information of their contours into account. Furthermore, a learning framework is proposed to generalize this connection which aims to learn an automatic best view selector for different kinds of 3D shapes. Experiments on the Princeton Shape Benchmark dataset are conducted to demonstrate the superiority of our approach. The results show that compared with other state-of-the-art methods, our approach is not only robust but also efficient when applied to shape retrieval tasks.


Best view selection Sketch-based modeling Context similarity Bag-of-features 



This research work was supported by the National Science Foundation of China (No. 61272276, 61305091), the National Twelfth Five-Year Plan Major Science and Technology Project of China (No. 2012BAC11B01-04-03), Special Research Fund of Higher Colleges Doctorate (No. 20130072110035), the Fundamental Research Funds for the Central Universities (No. 2100219038), and Shanghai Pujiang Program (No. 13PJ1408200).


  1. 1.
    Bay, H., Tuytelaars, T., Gool, L.: Surf: speeded up robust features. In: European Conference on Computer Vision (ECCV) (2006)Google Scholar
  2. 2.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell 24(5), 603–619 (2002)CrossRefGoogle Scholar
  3. 3.
    DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., Santella, A.: Suggestive contours for conveying shape. In: SIGGRAPH, pp. 848–855 (2003)Google Scholar
  4. 4.
    Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., Alexa, M.: Sketch-based shape retrieval. ACM Trans. Graph. (Proc. SIGGRAPH) 31(4), 31:1–31:10 (2012)Google Scholar
  5. 5.
    Endres, I., Hoiem, D.: Category-independent object proposals with diverse ranking. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 222–234 (2014)CrossRefGoogle Scholar
  6. 6.
    Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)zbMATHGoogle Scholar
  7. 7.
    Feldman, J.A., Feldman, C.M., Falk, G., Grape, C., Pearlman, J., Sobel, I., Tenenbaum, J.M.: The stanford hand-eye project. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 521–526 (1969)Google Scholar
  8. 8.
    Fisher, M., Hanrahan, P.: Context-based search for 3D models. ACM Trans. Graph. 29(6), 182 (2010)CrossRefGoogle Scholar
  9. 9.
    Giorgi, D., Mortara, M., Spagnuolo, M.: 3D shape retrieval based on best view selection. In: ACM workshop on 3D object retrieval (2010)Google Scholar
  10. 10.
    Laga, H.: Data-driven approach for automatic orientation of 3D shapes. Vis. Comput. 27(11), 977–989 (2011)CrossRefGoogle Scholar
  11. 11.
    Laga, H., Mortara, M., Spagnuolo, M.: Geometry and context for semantic correspondences and functionality recognition in manmade 3D shapes. ACM Trans. Graph. 32(5), 150-1–150-16 (2013)Google Scholar
  12. 12.
    Laga, H., Nakajima, M.: Supervised learning of salient 2D views of 3D models. J. Soc. Art Sci. 7(4), 124–131 (2008)CrossRefGoogle Scholar
  13. 13.
    Lee, C.H., Varshney, A., Jacobs, D.: Mesh Saliency. In: SIGGRAPH (2005)Google Scholar
  14. 14.
    Li, B., Lu, Y., Godil, A., et al.: SHREC’13 track: large scale sketch-based 3D shape retrieval. In: Eurographics Workshop on 3D Object Retrieval (3DOR), pp. 89–96 (2013)Google Scholar
  15. 15.
    Li, B., Lua, Y., Li, C., et al.: A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries. Comput. Vis. Image Underst. 131, 1–27 (2015)CrossRefGoogle Scholar
  16. 16.
    Liang, S., Zhao, L., Wei, Y., Jia, J.: Sketch-based retrieval using content-aware hashing. In: Pacific-Rim Conference on Multimedia (PCM), pp. 133–142 (2014)Google Scholar
  17. 17.
    Liu, H., Zhang, L., Huang, H.: Web-image driven best views of 3D shapes. Vis. Comput. 28(3), 279–287 (2012)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Liu, Y.J., Luo, X., Joneja, A., Ma, C.X., Fu, X.L., Song, D.: User-adaptive sketch-based 3-D CAD model retrieval. IEEE Trans. Autom. Sci. Eng. 10(3), 783–795 (2013)CrossRefGoogle Scholar
  19. 19.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 42(3), 145–175 (2001)CrossRefGoogle Scholar
  20. 20.
    Ma, C., Yang, X., Zhang, C., Ruan, X., Yang, M.H.: Sketch retrieval via dense stroke features. In: British Machine Vision Conference (BMVC) (2013)Google Scholar
  21. 21.
    Malisiewicz, T., Efros, A.A.: Beyond categories: the visual memex model for reasoning about object relationships. In: Annual Conference on Neural Information Processing Systems (NIPS) (2009)Google Scholar
  22. 22.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)CrossRefGoogle Scholar
  23. 23.
    Page, D., Koschan, A., Sukumar, S., Roui-Abidi, B., Abidi, M.: Shape analysis algorithm based on information theory. In: International Conference on Image Processing (ICIP) (2003)Google Scholar
  24. 24.
    Park, H.S., Jun, C.H.: A simple and fast algorithm for K-medoids clustering. Expert Syst. Appl. 36(2), 3336–3341 (2009)CrossRefGoogle Scholar
  25. 25.
    Shao, T., Xu, W., Yin, K., Wang, J., Zhou, K., Guo, B.: Discriminative sketch-based 3D model retrieval via robust shape matching. In: Pacific Graphics (PG) (2011)Google Scholar
  26. 26.
    Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton Shape Benchmark. In: Shape Modeling International (SMI) (2004)Google Scholar
  27. 27.
    Shtrom, E., Leifman, G., Tal, A.: Saliency detection in large point sets. In: International Conference on Computer Vision (ICCV) (2013)Google Scholar
  28. 28.
    Sivic, J., Zisserman, A.: Video google: a text retrieval approach to object matching in videos. In: International Conference on Computer Vision (ICCV), pp. 1470–1477 (2003)Google Scholar
  29. 29.
    Vázquez, P.P., Feixas, M., Sbert, M., Heidrich, W.: Viewpoint selection using viewpoint entropy. In: 6th International Fall Workshop Vision, Modeling and Visualization (2001)Google Scholar
  30. 30.
    Wang, F., Lin, L., Tang, M.: A new sketch-based 3D model retrieval approach by using global and local features. Graph. Models 76(3), 128–139 (2014)CrossRefGoogle Scholar
  31. 31.
    Zhang, T.Y., Suen, C.Y.: A fast parallel algorithm for thinning digital patterns. Commun. ACM 27(3), 236–239 (1984)CrossRefGoogle Scholar
  32. 32.
    Zhao, S., Ooi, W.T., Carlier, A., Morin, G., Charvillat, V.: Bandwidth adaptation for 3D mesh preview streaming. ACM Trans. Multimed. Comput. Commun. Appl. 10(1), 13-1–13-20 (2014)Google Scholar
  33. 33.
    Zitnick, C.L., Dollár, P.: Edge boxes: locating object proposals from edges. In: European Conference on Computer Vision (ECCV) (2014)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Long Zhao
    • 1
  • Shuang Liang
    • 1
  • Jinyuan Jia
    • 1
  • Yichen Wei
    • 2
  1. 1.Tongji UniversityShanghaiChina
  2. 2.Microsoft ResearchBeijingChina

Personalised recommendations