Skip to main content
SpringerLink
Log in

Super-vector coding features extracted from both depth buffer and view-normal-angle images for part-based 3D shape retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We have witnessed 3D shape models abundant in many application fields including 3D CAD/CAM, augmented/mixed reality (AR/MR), and entertainment. Creating 3D shape models from scratch is still very expensive. Efficient and accurate methods for shape retrieval is essential for 3D shape models to be reused. To retrieve similar 3D shape models, one must provide an arbitrary 3D shape as a query. Most of the research on 3D shape retrieval has been conducted with a “whole” shape as a query (aka whole-to-whole shape retrieval), while a “part” shape (aka part-to-whole shape retrieval) is more practically requested as a query especially by mechanical engineering with 3D CAD/CAM applications. A “part” shape is naturally constructed by a 3D range scanner as an input device. In this paper, we focus on the efficient method for part-to-whole shape retrieval where the “part” shape is assumed to be given by a 3D range scanner. Specifically, we propose a Super-Vector coding feature with SURF local features extracted from the View-Normal-Angle image, or the image synthesized by taking account of the angle between the view vector and the surface normal vector, together with the depth-buffered image, for part-to-whole shape retrieval. In addition, we propose a weighted whole-to-whole re-ranking method taking advantage of global information based on the result of part-to-whole shape retrieval. Through experiments we demonstrate that our proposed method outperforms the previous methods with or without re-ranking.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Alcantarilla PF, Bartoli A, Davison AJ (2012) KAZE features. In: Proceedings of the 12th European Conference on Computer Vision - Volume Part VI, ECCV’12. Springer-Verlag, Berlin, Heidelberg, pp 214–227. doi:10.1007/978-3-642-33783-3_16

    Google Scholar 

  2. Arandjelović R, Zisserman A (2012) Three things everyone should know to improve object retrieval. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2911–2918. doi:10.1109/CVPR.2012.6248018

  3. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359. doi:10.1016/j.cviu.2007.09.014

    Article  Google Scholar 

  4. Besl PJ, McKay ND (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256. doi:10.1109/34.121791

    Article  Google Scholar 

  5. Chatfield K, Lempitsky V, Vedaldi A, Zisserman A (2011) The devil is in the details: an evaluation of recent feature encoding methods. In: Proceedings of the british machine vision conference. BMVA press, pp 76.1–76.12. doi:10.5244/C.25.76

  6. Chen Y, Medioni G (1992) Object modelling by registration of multiple range images. Image Vision Comput 10(3):145–155. doi:10.1016/0262-8856(92)90066-C

    Article  Google Scholar 

  7. Chen D-Y, Tian X-P, Shen Y-T, Ouhyoung M (2003) On visual similarity based 3D model retrieval. Computer Graphics Forum. Blackwell Publishers, Inc and the Eurographics Association. ISSN 1467–8659. doi:10.1111/1467-8659.00669

  8. Darom T, Keller Y (2012) Scale-invariant features for 3-d mesh models. IEEE Trans Image Process 21(5):2758–2769. doi:10.1109/TIP.2012.2183142

    Article  MathSciNet  Google Scholar 

  9. Dutagaci H, Godil A, Cheung CP, Furuya T, Hillenbrand U, Ohbuchi R (2010) SHREC’10 Track: Range Scan Retrieval. In: Daoudi M, Schreck T (eds) Eurographics Workshop on 3D Object Retrieval. The Eurographics Association. doi:10.2312/3DOR/3DOR10/109-115

  10. Feng J, Wang Y, Chang SF (2016) 3D shape retrieval using a single depth image from low-cost sensors. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp 1–9. doi:10.1109/WACV.2016.7477652

  11. Furuya T, Kurabe S, Ohbuchi R (2015) Randomized sub-volume partitioning for part-based 3D model retrieval. In: Proceedings of the 2015 Eurographics Workshop on 3D Object Retrieval, 3DOR. Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, pp 15–22. doi:10.2312/3dor.20151050

    Google Scholar 

  12. Furuya T, Ohbuchi R (2015) Similarity metric learning for sketch-based 3d object retrieval. Multimed Tools Appl 74(23):10,367–10,392. doi:10.1007/s11042-014-2171-3

    Article  Google Scholar 

  13. Godil A, Dutagaci H, Bustos B, Choi S, Dong S, Furuya T, Li H, Link N, Moriyama A, Meruane R, Ohbuchi R, Paulus D, Schreck T, Seib V, Sipiran I, Yin H, Zhang C (2015) Range Scans based 3D Shape Retrieval. In: Pratikakis I, Spagnuolo M, Theoharis T, Gool LV, Veltkamp R (eds) Eurographics Workshop on 3D Object Retrieval. The Eurographics Association. doi:10.2312/3dor.20151069

  14. Guo Y, Sohel F, Bennamoun M, Lu M, Wan J (2013) Rotational projection statistics for 3D local surface description and object recognition. Int J Comput Vis 105(1):63–86. doi:10.1007/s11263-013-0627-y

    Article  MathSciNet  MATH  Google Scholar 

  15. Jancsary J, Nowozin S, Sharp T, Rother C (2012) Regression tree fields – an efficient, non-parametric approach to image labeling problems. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp 2376–2383. doi:10.1109/CVPR.2012.6247950

  16. Jégou H, Douze M, Schmid C, Pérez P (2010) Aggregating local descriptors into a compact image representation. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3304–3311. doi:10.1109/CVPR.2010.5540039

  17. Li B, Johan H (2013) Sketch-based 3D model retrieval by incorporating 2D-3D alignment. Multimed Tools Appl 65(3):363–385. doi:10.1007/s11042-012-1009-0

    Article  Google Scholar 

  18. Li B, Lu Y, Li C, Godil A, Schreck T, Aono M, Burtscher M, Chen Q, Chowdhury NK, Fang B, Fu H, Furuya T, Li H, Liu J, Johan H, Kosaka R, Koyanagi H, Ohbuchi R, Tatsuma A, Wan Y, Zhang C, Zou C (2015) A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries. Comp Vision Image Underst 131:1–27. doi:10.1016/j.cviu.2014.10.006. http://www.sciencedirect.com/science/article/pii/S1077314214002100. Special section: Large Scale Data-Driven Evaluation in Computer Vision

    Article  Google Scholar 

  19. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110. doi:10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  20. Ohbuchi R, Furuya T (2009) Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model. In: 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), pp 63–70. doi:10.1109/ICCVW.2009.5457716

  21. Osada R, Funkhouser T, Chazelle B, Dobkin D (2002) Shape distributions. ACM Trans Graph 21(4):807–832. doi:10.1145/571647.571648

    Article  MathSciNet  MATH  Google Scholar 

  22. Papadakis P, Pratikakis I, Theoharis T, Perantonis S (2010) Panorama: A 3d shape descriptor based on panoramic views for unsupervised 3D object retrieval. Int J Comput Vis 89(2):177–192. doi:10.1007/s11263-009-0281-6

    Article  Google Scholar 

  23. Perronnin F, Sánchez J, Mensink T (2010) Improving the fisher kernel for large-scale image classification. In: Proceedings of the 11th European Conference on Computer Vision: Part IV, ECCV’10. Springer-Verlag, Berlin, pp 143–156. http://dl.acm.org/citation.cfm?id=1888089.1888101

    Google Scholar 

  24. Pratikakis I, Savelonas M, Arnaoutoglou F, Ioannakis G, Koutsoudis A, Theoharis T, Tran MT, Nguyen VT, Pham VK, Nguyen HD, Le HA, Tran BH, To Q, Truong MB, Phan T, Nguyen MD, Than TA, Mac KN, Do M, Duong AD, Furuya T, Ohbuchi R, Aono M, Tashiro S, Pickup D, Sun X, Rosin P, Martin R (2016) Partial Shape Queries for 3D Object Retrieval. In: Ferreira A, Giachetti A, Giorgi D (eds) Eurographics Workshop on 3D Object Retrieval. The Eurographics Association. doi:10.2312/3dor.20161091

  25. Quan L, Tang K (2015) Polynomial local shape descriptor on interest points for 3D part-in-whole matching. Comput Aided Des 59:119–139. doi:10.1016/j.cad.2014.09.005. http://www.sciencedirect.com/science/article/pii/S0010448514002280

    Article  Google Scholar 

  26. Rusu RB, Blodow N, Beetz M (2009) Fast point feature histograms (FPFH) for 3D registration. In: IEEE International Conference on Robotics and Automation, 2009. ICRA ’09, pp 3212–3217. doi:10.1109/ROBOT.2009.5152473

  27. Savelonas M, Pratikakis I, Sfikas K (2016) Fisher encoding of differential fast point feature histograms for partial 3D object retrieval. Pattern Recogn 55(C):114–124. doi:10.1016/j.patcog.2016.02.003

    Article  Google Scholar 

  28. Serra G, Grana C, Manfredi M, Cucchiara R (2015) GOLD: Gaussians of local descriptors for image representation. Comput Vis Image Underst 134:22–32. doi:10.1016/j.cviu.2015.01.005. http://www.sciencedirect.com/science/article/pii/S1077314215000193 . Image Understanding for Real-world Distributed Video Networks

    Article  Google Scholar 

  29. Sfikas K, Pratikakis I, Koutsoudis A, Savelonas M, Theoharis T (2016) Partial matching of 3D cultural heritage objects using panoramic views. Multimed Tools Appl 75(7):3693–3707. doi:10.1007/s11042-014-2069-0

    Article  Google Scholar 

  30. Shen X, Lin Z, Brandt J, Avidan S, Wu Y (2012) Object retrieval and localization with spatially-constrained similarity measure and k-nn re-ranking. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3013–3020. doi:10.1109/CVPR.2012.6248031

  31. Shilane P, Min P, Kazhdan M, Funkhouser T (2004) The Princeton shape benchmark. In: Shape modeling international

  32. Sipiran I, Meruane R, Bustos B, Schreck T, Johan H, Li B, Lu Y (2013) Shrec’13 track: Large-scale partial shape retrieval using simulated range images. In: Proceedings of the Sixth Eurographics Workshop on 3D Object Retrieval, 3DOR ’13. Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, pp 81–88. doi:10.2312/3DOR/3DOR13/081-088

    Google Scholar 

  33. Su H, Maji S, Kalogerakis E, Learned-Miller E (2015) Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV ’15. IEEE Computer Society, Washington, DC, pp 945–953. doi:10.1109/ICCV.2015.114

    Chapter  Google Scholar 

  34. Tao S, Huang Z, Ma L, Guo S, Wang S, Xie Y (2013) Partial retrieval of CAD models based on local surface region decomposition. Computer-Aided Design 45 (11):1239 – 1252. doi:10.1016/j.cad.2013.05.008. http://www.sciencedirect.com/science/article/pii/S0010448513000857

    Article  Google Scholar 

  35. Tatsuma A, Aono M (2009) Multi-fourier spectra descriptor and augmentation with spectral clustering for 3D shape retrieval. Vis Comput 25(8):785–804. ISSN 1432–2315. doi:10.1007/s00371-008-0304-2

    Article  Google Scholar 

  36. Vedaldi A, Fulkerson B (2008) VLFeat: An open and portable library of computer vision algorithms. http://www.vlfeat.org/

  37. Wang M, Li H, Tao D, Lu K, Wu X (2012) Multimodal graph-based reranking for web image search. IEEE Trans Image Process 21 (11):4649–4661. doi:10.1109/TIP.2012.2207397

    Article  MathSciNet  Google Scholar 

  38. Wang Y, Feng J, Wu Z, Wang J, Chang SF (2014) From Low-Cost Depth Sensors to CAD: Cross-Domain 3D Shape Retrieval via Regression Tree Fields. Springer International Publishing, Cham, pp 489–504. doi:10.1007/978-3-319-10590-1_32

    Google Scholar 

  39. Yang J, Li H, Jia Y (2016) Go-icp: A globally optimal solution to 3d icp point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(11):2241–2254. doi:10.1109/TPAMI.2015.2513405

    Article  Google Scholar 

  40. Yang X, Koknar-Tezel S, Latecki LJ (2009) Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp 357–364. doi:10.1109/CVPR.2009.5206844

  41. Zhou D, Weston J, Gretton A, Bousquet O, Schölkopf B. (2004) Ranking on data manifolds. In: Thrun S, Saul LK, Schölkopf B (eds) Advances in Neural Information Processing Systems. MIT Press, vol 16, pp 169–176. http://papers.nips.cc/paper/2447-ranking-on-data-manifolds.pdf

  42. Zhou X, Yu K, Zhang T, Huang TS (2010) Image classification using super-vector coding of local image descriptors. In: Proceedings of the 11th European Conference on Computer Vision: Part V, ECCV’10. Springer-Verlag, Berlin, pp 141–154. http://dl.acm.org/citation.cfm?id=1888150.1888162

    Google Scholar 

  43. Zhu CZ, Jégou H., Satoh S (2013) Query-adaptive asymmetrical dissimilarities for visual object retrieval. In: 2013 IEEE International Conference on Computer Vision, pp 1705–1712. doi:10.1109/ICCV.2013.214

Download references

Acknowledgements

This work was supported by Kayamori Foundation of Information Science Advancement, Toukai Foundation for Technology, and JSPS KAKENHI Grant Numbers JP26280038, JP15K15992. We are indebted to Dr. Michalis Savelonas for providing the evaluation scripts and SHREC 2013 ground truth data.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Toyohashi University of Technology, Toyohashi, Japan

    Shoki Tashiro, Atsushi Tatsuma & Masaki Aono

Authors
  1. Shoki Tashiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Atsushi Tatsuma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masaki Aono
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shoki Tashiro.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tashiro, S., Tatsuma, A. & Aono, M. Super-vector coding features extracted from both depth buffer and view-normal-angle images for part-based 3D shape retrieval. Multimed Tools Appl 76, 22059–22076 (2017). https://doi.org/10.1007/s11042-017-4801-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4801-z

Keywords

Search

Navigation