Skip to main content
SpringerLink
Log in

A unified framework for cross-modality 3D model retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Currently, there exists diverse modalities of 3D models, such as the view set representation of 3D models, the virtual 3D model designed by CAD tools, and the 2.5D model captured by Kinect sensors. To realize flexible access to 3D models in different modalities, this paper proposes the unified framework for cross-modality 3D model retrieval. First, we develop a toolbox with OpenGL to convert 3D models in multiple modalities into the view set representation of 3D models, which are usually represented by a set of characteristic views. Then, we extract discriminative visual feature for multi-view representation. These visual features can be utilized to construct the graphical model to represent the structural characteristics of individual 3D model. Finally, we leverage the graph matching algorithm for similarity measure between pairwise 3D models. We evaluate this unified framework on several well-known 3D model datasets. The comparison experiments demonstrate that this unified framework can achieve competing performances against the state of the arts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ankerst M, Kastenmüller G, Kriegel H-P, Seidl T (1999) 3d shape histograms for similarity search and classification in spatial databases. In: SSD, pp 207–226

  2. Ansary T F, Daoudi M, Vandeborre J-P (2007) A bayesian 3-d search engine using adaptive views clustering. IEEE Trans Multimedia 9(1):78–88

    Article  Google Scholar 

  3. Chen D-Y, Tian X-P, Shen Y-T, Ouhyoung M (2003) On visual similarity based 3d model retrieval. Comput Graph Forum 22(3):223–232

    Article  Google Scholar 

  4. Chen F, Ji R, Cao L (2016) Multimodal learning for view-based 3d object classification. Neurocomputing

  5. Daras P, Axenopoulos A (2010) A 3d shape retrieval framework supporting multimodal queries. Int J Comput Vis 89(2–3):229–247

    Article  Google Scholar 

  6. Gao Y, Dai Q (2014) View-based 3d object retrieval: challenges and approaches. IEEE MultiMed 21(3):52–57

    Article  Google Scholar 

  7. Gao Y, Dai Q, Wang M, Zhang N (2011) 3d model retrieval using weighted bipartite graph matching. Signal Process Image Commun 26(1):39–47

    Article  Google Scholar 

  8. Gao Y, Dai Q, Zhang N (2010) 3d model comparison using spatial structure circular descriptor. Pattern Recogn 43(3):1142–1151

    Article  MATH  Google Scholar 

  9. Gao Y, Tang J, Hong R, Yan S, Dai Q, Zhang N, Chua T-S (2012) Camera constraint-free view-based 3-d object retrieval. IEEE Trans Image Process 21 (4):2269–2281

    Article  MathSciNet  Google Scholar 

  10. Gao Z, Zhang H, Xu G P, Xue Y B (2015) Multi-perspective and multi-modality joint representation and recognition model for 3d action recognition. Neurocomputing 151:554–564

    Article  Google Scholar 

  11. Gao Z, Zhang H, Xu G P, Xue Y B, Hauptmann AG (2015) Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition. Signal Process 112:83–97

    Article  Google Scholar 

  12. Gao Z, Zhang L, Chen M, Hauptmann A G, Zhang H, Cai A-N (2014) Enhanced and hierarchical structure algorithm for data imbalance problem in semantic extraction under massive video dataset. Multimedia Tools Appl 68(3):641–657

    Article  Google Scholar 

  13. Gao Z, Zhang Y, Zhang H, Xue Y B, Xu G P (2016) Multi-dimensional human action recognition model based on image set and group sparisty. Neurocomputing 215:138–149

    Article  Google Scholar 

  14. Guo Y, Sohel F, Bennamoun M, Wan J, Lu M (2015) A novel local surface feature for 3d object recognition under clutter and occlusion. Inf Sci 293:196–213

    Article  Google Scholar 

  15. Hao T, Yu A-L, Peng W, Wang B, Sun J (2016) Cross domain mitotic cell recognition. Neurocomputing 195(C):6–12

    Article  Google Scholar 

  16. Hu M-C, Chen C-W, Cheng W-H, Chang C-H, Lai J-H, Wu J-L (2015) Real-time human movement retrieval and assessment with kinect sensor. IEEE Trans Cybern 45(4):742–753

    Article  Google Scholar 

  17. Liu A, Li K, Kanade T (2012) A semi-markov model for mitosis segmentation in time-lapse phase contrast microscopy image sequences of stem cell populations. IEEE Trans Med Imaging 31(2):359–369

    Article  Google Scholar 

  18. Liu A, Nie W, Su Y, Ma L, Hao T, Yang Z (2015) Coupled hidden conditional random fields for RGB-D human action recognition. Signal Process 112:74–82

    Article  Google Scholar 

  19. Liu A, Nie W, Gao Y, Su Y (2016) Multi-modal clique-graph matching for view-based 3d model retrieval. IEEE Trans Image Process 25(5):2103–2116

    Article  MathSciNet  Google Scholar 

  20. Liu A, Su Y, Jia P-P, Gao Z, Hao T, Yang Z (2015) Multipe/single-view human action recognition via part-induced multitask structural learning. IEEE Trans Cybern 45(6):1194–1208

    Article  Google Scholar 

  21. Liu A-A, Su Y-T, Nie W-Z, Kankanhalli M (2016) Hierarchical clustering multi-task learning for joint human action grouping and recognition. IEEE Trans Pattern Anal Mach Intell 1–1

  22. Liu A, Wang Z, Nie W, Su Y (2015) Graph-based characteristic view set extraction and matching for 3d model retrieval. Inf Sci 320:429–442

    Article  Google Scholar 

  23. Liu A, Xu N, Nie W, Su Y, Wong Y, Kankanhalli M (2016) Benchmarking a multimodal and multiview and interactive dataset for human action recognition. IEEE Trans Cybern 0(0):1–1

    Google Scholar 

  24. Lu F, Sato I, Sato Y (2015) Uncalibrated photometric stereo based on elevation angle recovery from brdf symmetry of isotropic materials. Proc IEEE Conf Comput Vis Pattern Recognit 168–176

  25. Matsushita Y, Sato I, Okabe T, Sato Y (2015) From intensity profile to surface normal: photometric stereo for unknown light sources and isotropic reflectances. IEEE Trans Pattern Anal Mach Intell 37(10):1999–2012

    Article  Google Scholar 

  26. Nie W, Liu A, Gao Z, Su Y (2015) Clique-graph matching by preserving global & local structure. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, June 7–12, 2015, pp 4503–4510

  27. Nie W, Liu A, Su Y (2016) 3d object retrieval based on sparse coding in weak supervision. J Vis Commun Image Represent 37:40–45

    Article  Google Scholar 

  28. Nie L, Zhang L, Yang Y, Wang M, Hong R, Chua T-S (2015) Beyond doctors: future health prediction from multimedia and multimodal observations. In: Proceedings of the 23rd annual ACM conference on multimedia conference, MM ’15, Brisbane, Australia, October 26–30, 2015, pp 591–600

  29. Ning B, Chen Y, Liu H, Zhang S (2016) Cooling capacity improvement for a radiant ceiling panel with uniform surface temperature distribution. Build Environ 102:64–72

    Article  Google Scholar 

  30. Semenza J C, Hardwick K G, Dean N, Pelham H R (1990) Erd2, a yeast gene required for the receptor-mediated retrieval of luminal er proteins from the secretory pathway. Cell 61(61):1349–57

    Article  Google Scholar 

  31. Shih J L, Lee C H, Wang J T (2007) A new 3d model retrieval approach based on the elevation descriptor. Pattern Recogn 40(1):283–295

    Article  MATH  Google Scholar 

  32. Vandeborre J-P, Couillet V, Daoudi M (2002) A practical approach for 3d model indexing by combining local and global invariants. In: 3DPVT, pp 644–647

  33. Vinayak, Murugappan S, Liu H, Ramani K (2013) Shape-it-up: hand gesture based creative expression of 3d shapes using intelligent generalized cylinders. Comput Aided Des 45(2):277–287

    Article  Google Scholar 

  34. Wang X, Nie W (2015) 3d model retrieval with weighted locality-constrained group sparse coding. Neurocomputing 151:620–625

    Article  Google Scholar 

  35. Xu Q, Liu Y, Li X, Yang Z, Wang J, Sbert M, Scopigno R (2014) Browsing and exploration of video sequences: a new scheme for key frame extraction and 3d visualization using entropy based jensen divergence. Inf Sci 278:736–756

    Article  MathSciNet  Google Scholar 

  36. Yue M (2015) Hand fine-motion recognition based on 3d mesh mosift feature descriptor. Neurocomputing 151:574–582

    Article  Google Scholar 

  37. Zhang Y, Jiang F, Rho S, Liu S, Zhao D, Ji R (2016) 3d object retrieval with multi-feature collaboration and bipartite graph matching. Neurocomputing 195:40–49

    Article  Google Scholar 

  38. Zhao S, Chen L, Yao H, Zhang Y, Sun X (2015) Strategy for dynamic 3d depth data matching towards robust action retrieval. Neurocomputing 151:533–543

    Article  Google Scholar 

  39. Zhou J L, Zhou M Q, Geng G H (2015) 3d model retrieval based on distance classification histogram. In: Applied mechanics and materials, vol 733. Trans Tech Publ, pp 931–934

  40. Zou H, Da F, Wang Z (2015) A novel 3d face feature based on geometry image vertical shape information. Optik - Int J Light Electron Opt 126(9–10):898–902

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by National High-Tech Research and Development Program of China (863 programs, 2012AA10A401 and 2012AA092205), Grants of the Major State Basic Research Development Program of China (973 programs, 2012CB114405), National Natural Science Foundation of China (21106095), National Key Technology R&D Program (2011BAD13B07 and 2011BAD13B04), Tianjin Research Program of Application Foundation and Advanced Technology (15JCYBJC30700), Project of introducing one thousand high level talents in three years, Foundation of Introducing Talents to Tianjin Normal University(5RL123), “131” Innovative Talents cultivation of Tianjin, Academic Innovation Foundation of Tianjin Normal University (52XC1403).

Author information

Authors and Affiliations

  1. Tianjin Key Laboratory of Animal and Plant Resistance/College of Life Sciences, Tianjin Normal University, Tianjin, 300387, China

    Tong Hao, Qian Wang, Dan Wu & Jin-Sheng Sun

  2. Tianjin Aquatic Animal Infectious Disease Control and Prevention Center, Tianjin, 300221, China

    Jin-Sheng Sun

Authors
  1. Tong Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jin-Sheng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin-Sheng Sun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hao, T., Wang, Q., Wu, D. et al. A unified framework for cross-modality 3D model retrieval. Multimed Tools Appl 76, 20217–20230 (2017). https://doi.org/10.1007/s11042-017-4417-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4417-3

Keywords

Search

Navigation