Advertisement

Frontiers of Computer Science

, Volume 12, Issue 5, pp 1000–1012 | Cite as

Local features and manifold ranking coupled method for sketch-based 3D model retrieval

  • Xiaohui Tan
  • Yachun Fan
  • Ruiliang Guo
Research Article
  • 18 Downloads

Abstract

3D model retrieval can benefit many downstream virtual reality applications. In this paper, we propose a new sketch-based 3D model retrieval framework by coupling local features and manifold ranking. At technical fronts, we exploit spatial pyramids based local structures to facilitate the efficient construction of feature descriptors. Meanwhile, we propose an improved manifold ranking method, wherein all the categories between arbitrary model pairs will be taken into account. Since the smooth and detail-preserving line drawings of 3D model are important for sketch-based 3D model retrieval, the Difference of Gaussians (DoG) method is employed to extract the line drawings over the projected depth images of 3D model, and Bezier Curve is then adopted to further optimize the extracted line drawing. On that basis, we develop a 3D model retrieval engine to verify our method. We have conducted extensive experiments over various public benchmarks, and have made comprehensive comparisons with some state-of-the-art 3D retrieval methods. All the evaluation results based on the widely-used indicators prove the superiority of our method in accuracy, reliability, robustness, and versatility.

Keywords

sketch-based retrieval 3D model manifold ranking line drawing local features 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

The authors would like to thank Zhang Dongdong for his great help in experiments. This work was supported by the National Natural Science Foundation of China (Grant No. 61602324), the Scientific Research Project of Beijing Educational Committeen (KM201710028018), the open funding project of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (BUAA-VR-17KF-12) and Beijing Advanced Innovation Center for Imaging Technology (BAICIT-2016004).

Supplementary material

11704_2017_6595_MOESM1_ESM.ppt (410 kb)
Supplementary material, approximately 408 KB.

References

  1. 1.
    Zeleznik R C, Herndon K P, Hughes J F. SKETCH: an interface for sketching 3D scenes. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. 1996, 163–170Google Scholar
  2. 2.
    Gao Y, Wang M, Tao D C, Ji R R, Dai Q H. 3-D Object retrieval and recognition with hypergraph analysis. IEEE Transactions on Image Processing, 2012, 21(9): 4290–4303MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Walther D B, Chai B, Caddigan E, Beck DM, Li F F. Simple line drawings suffice for functional MRI decoding of natural scene categories. Proceedings of the National Academy of Sciences, 2011, 108(23): 9661–9666CrossRefGoogle Scholar
  4. 4.
    Eitz M, Hays J, Alexa M. How do humans sketch objects? ACM Transactions on Graphics, 2012, 31(4): 44Google Scholar
  5. 5.
    Funkhouser T, Min P, Kazhdan M, Chen J, Halderman A, Dobkin D, Jacobs D. A search engine for 3D models. ACM Transactions on Graphics, 2003, 22(1): 83–105CrossRefGoogle Scholar
  6. 6.
    Li B, Lu Y J, Ghumman A, Strylowski B, Gutierrez M, Sadiq S, Forster S, Feola N, Bugerin T. 3D Sketch-Based 3D model retrieval. In: Proceedings of the 5th ACM International Conference on Multimedia Retrieval. 2015, 555–558CrossRefGoogle Scholar
  7. 7.
    Liu X L, Huang L, Deng C, Tao D C. Query-Adaptive hash code ranking for large-scale multi-view visual search. IEEE Transactions on Image Processing, 2016, 25(10): 4514–4524MathSciNetCrossRefGoogle Scholar
  8. 8.
    Liu X L, Du B W, Deng C, Liu M, Lang B. Structure sensitive hashing with adaptive product quantization. IEEE transactions on cybernetics, 2016, 46(10): 2252–2264CrossRefGoogle Scholar
  9. 9.
    DeCarlo D, Finkelstein A, Rusinkiewicz S, Santella. A Suggestive contours for conveying shape. ACM Transactions on Graphics, 2003, 22(3): 848–855CrossRefGoogle Scholar
  10. 10.
    Saavedra J M, Bustos B, Schreck T, Yoon S M, Scherer M. Sketchbased 3D model retrieval using keyshapes for global and local representation. In: Proceedings of the 5th Eurographics Conference on 3D Object Retrieval. 2012, 47–50Google Scholar
  11. 11.
    Yoon S, Scherer M, Schreck T, Kuijper A. A Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In: Proceedings of the 18th ACM International Conference on Multimedia. 2010, 193–200Google Scholar
  12. 12.
    Yoon SM, Kuijper A. Sketch-based 3D model retrieval using compressive sensing classification. Electronics Letters, 2011, 47(21): 1181–1183CrossRefGoogle Scholar
  13. 13.
    Judd T, Durand F, Adelson E. Apparent ridges for line drawing. ACM Transactions on Graphics, 2007, 26(3): 19CrossRefGoogle Scholar
  14. 14.
    Eitz M, Richter R, Boubekeur T, Hildebrand K Alexa M. Sketch-based shape retrieval. ACM Transactions on Graphics, 2012, 31(4): 1–10Google Scholar
  15. 15.
    Li B, Schreck T, Godil A, Alexa M, Boubekeur T, Bustos B, Chen J, Eitz M, Furuya T, Hildebrand K, Huang S, Johan H, Kuijper A, Ohbuchi R, Richter R, Saavedra J M, Scherer M, Yanagimachi T, Yoon G J, Yoon S M. Track: sketch-based 3D shape retrieval. In: Proceedings of the 5th Eurographics Conference on 3D Object Retrieval. 2012, 109–118Google Scholar
  16. 16.
    Shao T J, Xu W W, Yin K K, Wang J D, Zhou K, Guo B N. Discriminative sketch-based 3D model retrieval via robust shape matching. In: Proceedings of Comput Graph Forum. 2011Google Scholar
  17. 17.
    Li B, Lu Y, Li C, Godil A, Schreck T, Aono M, Burtscher M, Fu H, Furuya T, Johan H, Liu J, Ohbuchi R, Tatsuma A, Zou C. SHREC’14 track: extended large scale sketch-based 3D shape retrieval. In: Proceedings of Eurographics Workshop on 3D Object Retrieval. 2014Google Scholar
  18. 18.
    Li B, Lu Y, Godil A, Schreck T, Aono M, Johan H, Saavedra J M, Tashiro S. SHREC’13 track: scale sketch-based 3D shape retrieval. In: Proceedings of the 6th Eurographics Workshop on 3D Object Retrival. 2013, 89–96Google Scholar
  19. 19.
    Yoon S M, Yoon G, Schreck T. User-drawn sketch-based 3D object retrieval using sparse coding. Multimedia Tools and Applications. 2015, 74(13): 4707–4722CrossRefGoogle Scholar
  20. 20.
    Hu R, Wang T H, Collomosse J. A bag-of-regions approach to sketchbased image retrieval. In: Proceedings of the 18th IEEE International Conference on Image Processing. 2011, 3661–3664Google Scholar
  21. 21.
    Li B, Johan H. Sketch-based 3D model retrieval by incorporating 2D-3D alignment. Multimedia Tools and Applications, 2013, 65(3): 363–385CrossRefGoogle Scholar
  22. 22.
    Li B, Johan H. A 3D model feature for retrieval. In: Proceedings of MMM. 2010, 185–195Google Scholar
  23. 23.
    Belongie S, Malik J, Puzicha J. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4): 509–522CrossRefGoogle Scholar
  24. 24.
    Ohbuchi R, Furuya T. Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model. In: Proceedings of the 12th IEEE International Conference on Computer Vision. 2009, 63–70Google Scholar
  25. 25.
    Lowe D G. Distinctive Image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110MathSciNetCrossRefGoogle Scholar
  26. 26.
    Wang F, Lin L F, Tang M. A new sketch-based 3D model retrieval approach by using global and local features. Graphical Models, 2014, 76(3): 128–139CrossRefGoogle Scholar
  27. 27.
    Wang F, Le K, Li Y. Sketch-based 3D shape retrieval using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1875–1883Google Scholar
  28. 28.
    Zhu F, Xie J, Fang Y. Learning cross-domain neural networks for sketch-based 3D shape retrieval. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016.Google Scholar
  29. 29.
    He J R, Li M J, Zhang H J, Tong H H, Zhang C S. Manifold-ranking based image retrieval. In: Proceedings of the 12th Annual ACM International Conference on Multimedia. 2004, 9–16CrossRefGoogle Scholar
  30. 30.
    Tong H H, He J R, Li M J, Ma W Y, Zhang H J, Zhang C S. Manifoldranking- based keyword propagation for image retrieval. EURASIP Journal on Applied Signal Processing, 2006, 2006: 190Google Scholar
  31. 31.
    Guan Z Y, Bu J J, Mei Q Z, Chen C, Wang C. Personalized tag recommendation using graph-based ranking on multi-type interrelated objects. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2009, 540–547Google Scholar
  32. 32.
    Agarwal S. Ranking on on graph data. In: Proceedings of the 23rd Internatioal Conference on Machine Learning. 2006, 25–32Google Scholar
  33. 33.
    Bu J J, Tan S L, Chen C, Wang C, Wu H, Zhang L J, He X F. Music recommendation by unified hypergraph: combining social media information and music content. In: Proceedings of the 18th ACM International Conference on Multimedia. 2010, 391–400Google Scholar
  34. 34.
    Furuya T, Ohbuchi R. Similarity metric learning for sketch-based 3D object retrieval. Multimedia Tools and Applications, 2015, 74(23): 10367–10392CrossRefGoogle Scholar
  35. 35.
    Liu X L, Lang B, Xu Y, Cheng B. Feature grouping and local soft match for mobile visual search. Pattern Recognition Letters, 2012, 33(3): 239–246CrossRefGoogle Scholar
  36. 36.
    Yang Y1, Nie F P, Xu D B, Luo J, Zhuang Y T, Pan Y H. Multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(4): 723–742CrossRefGoogle Scholar
  37. 37.
    Liu X L, Deng C, Lang B, Tao D C, Li X L. Query-Adaptive reciprocal hash tables for nearest neighbor search. IEEE Transactions on Image Processing, 2015, 25(2): 907–919MathSciNetCrossRefGoogle Scholar

Copyright information

© Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.College of Information EngineeringCapital Normal UniversityBeijingChina
  2. 2.College of Information Science and TechnologyBeijing Normal UniversityBeijingChina
  3. 3.School of FashionBeijing Institute of Fashion TechnologyBeijingChina

Personalised recommendations