Skip to main content

Advertisement

SpringerLink
Log in

REBOR: A new sketch-based 3d object retrieval framework using retina inspired features

  • 1154T: Content-Based Multimedia Indexing in the era of Artificial Intelligence
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the rapid development of data science and modeling engineering, the capacity of cyberspace has significantly expanded which enables the online storage of increasing number of 3D models. Hence, the development of effective and efficient approaches to search 3D models is becoming increasingly important and urgent. In this paper, we propose a new sketch-based 3D retrieval framework named REBOR under the inspiration of retina which is not only consistent with human perception sensitivity but also simplifies the requirement of retrieval query by enabling hand-drawn sketch. The feature extraction process incorporates human visual system by simulating the ganglion perceptive mechanism in retina. Support Vector Machine is used to classify the query sketches and is further optimized by means of an global optimization algorithm so as to acquire optimal results automatically. Experiments are done on the database generated by ourselves with 15 categories of 3D objects, and the results indicate the effectiveness of REBOR in terms of retrieval accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Alexandre A, Raphael O, Pierre V (2012) Freak: Fast retina keypoint. In: IEEE conference on computer vision and pattern recognition, https://doi.org/10.1109/CVPR.2012.6247715

  2. Calonder M, Lepetit V, Strecha C et al (2010) Brief: binary robust independent elementary features. In: European conference on computer vision, https://doi.org/10.1007/978-3-642-15561-1_56

  3. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297. https://doi.org/10.1007/BF00994018

    MATH  Google Scholar 

  4. Das GP, Vance PJ, Kerr D, Coleman SA, McGinnity TM, Liu JK (2019) Computational modelling of salamander retinal ganglion cells using machine learning approaches. Neurocomputing 325:101–112. https://doi.org/10.1016/j.neucom.2018.10.004

    Article  Google Scholar 

  5. Deng J, Dong W, Socher R, Li L, Li K, Feifei L (2009) Imagenet: A large-scale hierarchical image database. Comput Vision Pattern Recognit 248–255

  6. Duan L, Li W, Tsang IW, Xu D (2011) Improving web image search by bag-based reranking. IEEE Trans Image Process 20(11):3280–3290. https://doi.org/10.1109/TIP.2011.2159227

    Article  MathSciNet  Google Scholar 

  7. Gao R, Asano SM, Upadhyayula S et al (2019) Cortical column and whole-brain imaging with molecular contrast and nanoscale resolution. Science 363:245–246. https://doi.org/10.1126/science.aau8302

    Google Scholar 

  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Comput Vision Pattern Recognit 770–778

  9. Herbert B, Tinne T, Van GL (2006) Surf: Speeded up robust features. In: European Conference on Computer Vision, https://doi.org/10.1007/11744023_32

  10. Itti Laurent, Koch C (2001) Computational modelling of visual attention. Nat Rev Neurosci 2:194–203. https://doi.org/10.1038/35058500

    Article  Google Scholar 

  11. Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (abc) algorithm. J Global Optim 39:459–471. https://doi.org/10.1007/s10898-007-9149-x

    Article  MathSciNet  Google Scholar 

  12. Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: State of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19. https://doi.org/10.1145/1126004.1126005

    Article  Google Scholar 

  13. Li B, Johan H (2013) 3d model retrieval using hybrid features and class information. Multimed Tools Appl 62(3):821–846. https://doi.org/10.1007/s11042-011-0873-3

    Article  Google Scholar 

  14. Li Y, Lei H, Lin S, Luo G (2018a) A new sketch-based 3d model retrieval method by using composite features. Multimed Tools Appl 77 (2):2921–2944. https://doi.org/10.1007/s11042-017-4446-y

    Article  Google Scholar 

  15. Li B, Lu Y, Johan H, Fares R (2017) Sketch-based 3d model retrieval utilizing adaptive view clustering and semantic information. Multimed Tools Appl 76(24):26603–26631. https://doi.org/10.1007/s11042-016-4187-3

    Article  Google Scholar 

  16. Li Y, Miao Z, Wang J, Zhang Y (2018b) Nonlinear embedding neural codes for visual instance retrieval. Neurocomputing 275:1275–1281. https://doi.org/10.1016/j.neucom.2017.09.072

    Article  Google Scholar 

  17. Li Y, Wang S, Tian Q, et al. (2015) A survey of recent advances in visual feature detection. Neurocomputing 149:736–751. https://doi.org/10.1016/j.neucom.2014.08.003

    Article  Google Scholar 

  18. Lowe DG (1999) Object recognition from local scale-invariant features. In: iccv IEEE Computer Society, https://doi.org/10.1109/ICCV.1999.790410

  19. Mair E, Hager GD, Burschka D et al (2010) Adaptive and generic corner detection based on the accelerated segment test. Europ Conference on Comput Vision 1:183–196. https://doi.org/10.1007/978-3-642-155-9_14

    Article  Google Scholar 

  20. Martin JH, Yahata BD, Hundley JM, Mayer JA, Schaedler TA, Pollock TM (2017) 3d printing of high-strength aluminium alloys. Nature 549 (7672):365–369. https://doi.org/10.1038/nature23894

    Article  Google Scholar 

  21. Medathati NVK, Neumann H, Masson GS, Kornprobst P (2016) Bio-inspired computer vision: Towards a synergistic approach of artificial and biological vision. Comput Vision Image Understand 150:1–30

    Article  Google Scholar 

  22. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. Comput Vision Pattern Recognit 4510–4520

  23. Stefan L, Margarita C, Roland SY (2011) Brisk: Binary robust invariant scalable keypoints. In: International conference on computer vision, https://doi.org/10.1109/ICCV.2011.6126542

  24. Tangelder JWH, Veltkamp RC (2008) A survey of content based 3d shape retrieval methods. Multimed Tools Appl 39(3):441–471. https://doi.org/10.1007/s11042-007-0181-0

    Article  Google Scholar 

  25. Ungerleider Sabine KG (2000) Mechanisms of visual attention in the human cortex. Ann Rev Neurosci 23:315–41. https://doi.org/10.1146/annurev.neuro.23.1.315

    Article  Google Scholar 

  26. Vanrullen R, Thorpe SJ (2002) Surfing a spike wave down the ventral stream. Vis Res 42(23):2593–2615

    Article  Google Scholar 

  27. Wang D, Wang B, Zhao S, Yao H, Liu H (2017) View-based 3d object retrieval with discriminative views. Neurocomputing 252:58–66. https://doi.org/10.1016/j.neucom.2016.06.095

    Article  Google Scholar 

  28. Weng D, Wang Y, Gong M, Tao D, Wei H, Huang D (2015) Derf: Distinctive efficient robust features from the biological modeling of the p ganglion cells. IEEE Trans Image Process 24(8):2287–2302

    Article  MathSciNet  Google Scholar 

  29. Wu L, Jin R, Jain AK (2013) Tag completion for image retrieval. IEEE Trans Pattern Anal Machine Intell 35(3):716–727. https://doi.org/10.1109/TPAMI.2012.124

    Article  Google Scholar 

  30. Yang Y, Li B, Li P, Liu Q (2018) A two-stage clustering based 3d visual saliency model for dynamic scenarios. IEEE Trans Multimed 21 (4):809–820. https://doi.org/10.1109/tmm.2018.2867742

    Article  Google Scholar 

  31. Yao Q, Cosme JGL, Xu T, Miszuk JM, Picciani PHS, Fong H, Sun H (2017) Three dimensional electrospun pcl/pla blend nanofibrous scaffolds with significantly improved stem cells osteogenic differentiation and cranial bone formation. Biomaterials 115:115–127. https://doi.org/10.1016/j.biomaterials.2016.11.018

    Article  Google Scholar 

  32. Zhao X, Gregor R, Mavrids P, Schreck T (2017) Sketch-based 3d object retrieval with skeleton line views - initial results and research problems. In: EG workshop on 3D object retrieval

  33. Zhao X, Mavrids P, Schreck T (2019) A high-performance approach for enhancingthe clarityof hand-drawn sketch images. Basic Sci J Textile Univ 31(2):237–260

    Google Scholar 

Download references

Acknowledgements

This work is sponsored by the National Natural Science Foundation of China under Grant No.61806160 and Shaanxi Association for Science and Technology of Colleges and Universities Youth Talent Development Program, No. 20190112 and the Youth Innovation Team of Shaanxi Universities and Shaanxi Province Technical Innovation Foundation(2020CGXNG-012).

Author information

Authors and Affiliations

  1. School of Computer Science, Xi’an Polytechnic University, Xi’an, Shaanxi, China

    Xin Shi, Huijuan Chen & Xueqing Zhao

  2. Shaanxi Key Laboratory of Clothing Intelligence, School of Computer Science, Xi’an Polytechnic University, Xi’an, 710048, China

    Xin Shi & Xueqing Zhao

  3. National and Local Joint Engineering Research Center for Advanced Networking and Intelligent Information Service, Xi’an Polytechnic University, Xi’an, 710048, China

    Xueqing Zhao

  4. National Engineering Laboratory for Video Technology, School of Electronics Engineering and Computer Science, Peking University, Beijing, 100871, China

    Xueqing Zhao

Authors
  1. Xin Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huijuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xueqing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xueqing Zhao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, X., Chen, H. & Zhao, X. REBOR: A new sketch-based 3d object retrieval framework using retina inspired features. Multimed Tools Appl 80, 23297–23311 (2021). https://doi.org/10.1007/s11042-021-10618-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10618-4

Keywords

Search

Navigation