Skip to main content
SpringerLink
Log in

3D object retrieval based on multi-view convolutional neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, 3D objects have been widely designed and applied in various technical applications. In this paper, we propose a novel 3D model retrieval method based on Multi-View Convolutional Neural Networks (MVCNN). By integrating visual information from multiple views, we construct a composite CNN structure to generate single terse descriptor with powerful discrimination for individual 3D object. Our method can benefit from the hidden relevance of visual information in deep structure. Instead of computing similarities between each pair of view-feature, we only need to measure the comparability of two object once, which brings high efficiency. Moreover, this method can avoid camera constraint when capturing multi-view representation. Extensive experiments on NTU and ITI datasets can support the superiority of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ansary TF, Daoudi M, Philippe J (2007) Vandeborre: A bayesian 3-d search engine using adaptive views clustering. IEEE Trans Multimed 9(1):78–88

    Article  Google Scholar 

  2. Bai S, Bai X, Liu W, Roli F (2015) Neural shape codes for 3d model retrieval. Pattern Recogn Lett 65(C):15–21

    Article  Google Scholar 

  3. Chen N, Xiao HD (2013) Perceptual audio hashing algorithm based on zernike moment and maximum-likelihood watermark detection. Digit Signal Process 23 (4):1216–1227

    Article  MathSciNet  Google Scholar 

  4. Chen D-Y, Tian X-P, Shen Y-T, Ming O (2003) On visual similarity based 3d model retrieval. Comput Graph Forum 22(3):223–232

    Article  Google Scholar 

  5. Cheng Y, Zhao X, Huang K, Tan T (2014) Semi-supervised learning for rgb-d object recognition. In: 2014 22nd International Conference on Pattern Recognition (ICPR), pp 2377–2382

  6. Cover T., Hart P. (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27

    Article  MATH  Google Scholar 

  7. Daras P, Axenopoulos A (2010) A 3d shape retrieval framework supporting multimodal queries. Int J Comput Vis 89(2):229–247

    Article  Google Scholar 

  8. Gao Y, Tang J, Li H, Dai Q, Zhang N (2010) View-based 3d model retrieval with probabilistic graph model. Neurocomputing 73(10-12):1900–1905

    Article  Google Scholar 

  9. Gao Y, Wang M, Zha ZJ, Qi T, Dai Q, Zhang N (2011) Less is more Efficient 3-d object retrieval with query view selection. IEEE Trans Multimed 13 (5):1007–1018

    Article  Google Scholar 

  10. Gao X, Wang Q, Li X, Tao D, Zhang K (2011) Zernike-moment-based image super resolution. IEEE Trans Image Process A Publ IEEE Signal Process Soc 20(10):2738–2747

    Article  MathSciNet  Google Scholar 

  11. Gao Y, Tang J, Hong R, Yan S, Dai Q, Zhang N, Chua TS (2012) Camera constraint-free view-based 3-d object retrieval. IEEE Trans Image Process Publ IEEE Signal Process Soc 21(4):2269–2281

    Article  MathSciNet  Google Scholar 

  12. Gao Z, Yang Y, Zhai L, Jin N (2016) A four-sector conductance method for measuring and characterizing low-velocity oil-water two-phase flows. IEEE Transactions on Instrumentation and Measurement:1–8

  13. Guo Y, Lin S, Su Z, Luo X, Wang R, Kang Y (2016) A 3d model perceptual feature metric based on global height field. Visual Computer:1–14

  14. Hao T, Zheng Z, Wang B, Zhang Y, Liu Y, Geng X, Sun J (2014) The protein-protein interaction network of eyestalk, y-organ and hepatopancreas in chinese mitten crab eriocheir sinensis. BMC Syst Biol 8(1):417–422

    Article  Google Scholar 

  15. Hao T, Yu AL, Peng W, Wang B, Sun JS (2016) Cross domain mitotic cell recognition. Neurocomputing 195(C):6–12

    Article  Google Scholar 

  16. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25(2):2012

    Google Scholar 

  17. Leng B, Qin Z (2008) A powerful relevance feedback mechanism for content-based 3d model retrieval. Multimed Tools Appl 40(1):135–150

    Article  Google Scholar 

  18. Leng B, Xiong Z (2011) Modelseek: an effective 3d model retrieval system. Multimed Tools Appl 51(3):935–962

    Article  Google Scholar 

  19. Liu Q (2012) A survey of recent view-based 3d model retrieval methods. Computer Science

  20. Liu AA, Nie W, Gao Y, Su Y (2016) Multi-modal clique-graph matching for view-based 3d model retrieval. IEEE Trans Image Process 25(5):1–1

    Article  MathSciNet  Google Scholar 

  21. Liu AA, Su YT, Jia PP, Gao Z, Hao T, Yang ZX (2015) Multipe/single-view human action recognition via part-induced multitask structural learning. IEEE Trans Cybern 45(6):1194–1208

    Article  Google Scholar 

  22. Liu AA, Su YT, Nie WZ, Kankanhalli M (2016) Hierarchical clustering multi-task learning for joint human action grouping and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence:1–1

  23. Liu AA, Xu N, Nie WZ, Su YT (2016) Benchmarking a multimodal and multiview and interactive dataset for human action recognition. IEEE Transactions on Cybernetics:1–14

  24. Lu K, Ji R, Tang J, Gao Y (2014) Learning-based bipartite graph matching for view-based 3d model retrieval. IEEE Trans Image Process Publ IEEE Signal Process Soc 23(10):4553–63

    Article  MathSciNet  Google Scholar 

  25. Nie L, Wang M, Zha ZJ, Chua TS (2012) Oracle in image search A content-based approach to performance prediction. Acm Trans Inf Syst 30(2):1–23

    Article  Google Scholar 

  26. Nie L, Wang M, Gao Y, Zha ZJ, Chua TS (2013) Beyond text qa: Multimedia answer generation by harvesting web information. IEEE Trans Multimed 15(2):426–441

    Article  Google Scholar 

  27. Nie L, Zhang L, Yi Y, Wang M, Hong R, Chua TS (2015) Beyond doctors: Future health prediction from multimedia and multimodal observations. In: ACM International Conference on Multimedia, pp 591– 600

  28. Nie WZ, An AL, Gao Z, Su YT (2015) Clique-graph matching by preserving global and local structure. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 4503–4510

  29. Nie W, Li X, Liu AA, Su Y (2015) 3d object retrieval based on spatial+lda model. Multimed Tools Appl:1–14

  30. Nie W, Cao Q, Liu AA, Su Y (2015) Convolutional deep learning for 3d object retrieval. Multimedia Systems:1–8

  31. Nie W, Liu AA, Wang Z, Su Y (2016) Effective 3d object detection based on detector and tracker. Neurocomputing

  32. Osada R, Funkhouser T, Chazelle B, Dobkin D (2002) Shape distributions. Acm Trans Graph 21(4):807–832

    Article  MathSciNet  MATH  Google Scholar 

  33. Paquet E, Rioux M (1999) Nefertiti: A query by content system for three-dimensional model and image databases management. Image Vis Comput 17 (2):157–166

    Article  Google Scholar 

  34. Shih JL, Lee CH, Wang JT (2007) A new 3d model retrieval approach based on the elevation descriptor. Pattern Recogn 40(1):283–295

    Article  MATH  Google Scholar 

  35. Simonyan K, Parkhi O, Vedaldi A, Zisserman A (2013) Fisher vector faces in the wild. In: British Machine Vision Conference, pp 8.1–8.11

  36. Su H, Maji S, Kalogerakis E, Learnedmiller E (2015) Multi-view convolutional neural networks for 3d shape recognition. Computer Science:945–953

  37. Tong H, Wei P, Qian W, Wang B, Sun J (2016) Reconstruction and application of protein–protein interaction network. Int J Mol Sci 7(6)

  38. Wang XF, Geng GH, Zhang F (2012) 3d model relevance feedback retrieval algorithm based on range image. Appl Res Comput 29(6):2350–2353

    Google Scholar 

  39. Wang B, Ning Q, Hao T, Yu A, Sun J (2015) Reconstruction and analysis of a genome-scale metabolic model for eriocheir sinensis eyestalks. Mol Biosyst 12(1):246–252

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic Information Engineering, Tianjin University, Tianjin, China

    Xi-Xi Li & Qun Cao

  2. China Electronics Standardization Institute, Beijing, China

    Sha Wei

Authors
  1. Xi-Xi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qun Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sha Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sha Wei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, XX., Cao, Q. & Wei, S. 3D object retrieval based on multi-view convolutional neural networks. Multimed Tools Appl 76, 20111–20124 (2017). https://doi.org/10.1007/s11042-016-4250-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4250-0

Keywords

Search

Navigation