Skip to main content

3D Object Classification, Visual Search from RGB-D Data

  • Conference paper
  • First Online:
Applied Mathematics and Computational Mechanics for Smart Applications

Abstract

In this chapter, we consider the problem of creating a system for processing 3D models obtained using RGB-D sensors for the purpose of semiautomatic selection and classification of objects and their auto-completion based on visual search. We have proposed several heuristic preprocessing algorithms for selecting an object of interest on a scan that contains noise and extraneous objects. To implement the visual search algorithm, we obtained a modification of the ray casting 3D-shape feature extraction algorithm. To solve the classification problem, the possibility of using deep learning architectures based on convolution mechanisms on graphs is investigated. The information about the object class obtained during the classification stage is used for faster and more accurate auto-completion. The resulting system has been tested on real data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhang, K., Hao, M., Wang, J., de Silva, C.W., Fu, C.: Linked dynamic graph CNN: learning on point cloud via linking hierarchical features. CoRR ArXiv Preprint, arXiv:1904.10014 [cs.CV] (2019)

  2. Vranic, D.V.: 3D model retrieval. Ph. D. Dissertation, University of Leipzig (2004)

    Google Scholar 

  3. Weinmann, M., Jutzi, B., Hinz, S., Mallet, C.: Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers. ISPRS J. Photogramm. Remote Sens. 105, 286–304 (2015)

    Article  Google Scholar 

  4. Zhang, J., Zhao, X., Chen, Z., Lu, Z.: A review of deep learning-based semantic segmentation for point cloud. IEEE Access 7, 179118–179133 (2019)

    Article  Google Scholar 

  5. Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3D ShapeNets: a deep representation for volumetric shapes. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1912–1920. IEEE Computer Society Press, Boston, MA, USA (2015)

    Google Scholar 

  6. Armeni. I., Sener, O., Zamir, A.R., Jiang, H., Brilakis, I., Fischer, M., Savarese, S.: 3D semantic parsing of large-scale indoor spaces. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognit., pp. 1534–1543. IEEE Computer Society Press, Las Vegas, NV, USA (2016)

    Google Scholar 

  7. Yi, L., Shao, L., Savva, M., Huang, H., Zhou, Y., Wang, Q., Graham, B., Engelcke, M., Klokov, R., Lempitsky, V., Gan, Y., Wang, P., Liu, K., Yu, F., Shui, P., Hu, B., Zhang, Y., Li, Y., Bu, R., Sun, M., Wu, W., Jeong, M., Choi, J., Kim, C., Geetchandra, A., Murthy, N., Ramu, B., Manda, B., Ramanathan, M., Kumar, G., Preetham, P., Srivastava, S., Bhugra, S., Lall, B., Haene, C., Tulsiani, S., Malik, J., Lafer, J., Jones, R., Li, S., Lu, J., Jin, S., Yu, J., Huang, Q., Kalogerakis, E., Savarese, S., Hanrahan, P., Funkhouser, T., Su, H., Guibas, L.: Large-scale 3D shape reconstruction and segmentation from ShapeNet Core55. CoRR ArXiv Preprint, arXiv:1710.06104 [cs.CV] (2017)

  8. Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., Nießner, M.: ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2432–2443. IEEE Computer Society Press, Honolulu, HI, USA (2017)

    Google Scholar 

  9. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph Cnn for learning on point clouds. ACM Trans. Graph. 38(5), 146.1–146.13 (2019)

    Google Scholar 

  10. Reingold, O.: Undirected connectivity in log-space. J. ACM. 55, 1–24 (2008)

    Article  MathSciNet  Google Scholar 

  11. Cohen, M.B., Lee, Y.T., Miller, G., Pachocki, J., Sidford, A.: Geometric median in nearly linear time. In: The 48th Annual ACM SIGACT Symposium on Theory of Computing—STOC 2016, pp. 9–21. ACM Press, New York, New York, USA (2016)

    Google Scholar 

  12. GitHub page of our project. https://github.com/phygitalism/blender_scan_filter. Last accessed 2020/06/26

  13. Zhou, Q.-Y., Park, J., Koltun, V.: Open3D: A modern library for 3D data processing. CoRR ArXiv Preprint, arXiv:1801.09847 [cs.CV] (2018)

  14. GitHub page of LDGCNN. https://github.com/KuangenZhang/ldgcnn. Last accessed 2020/06/26

  15. Kelleher, J.D., Namee, B.M., D’Arcy, A.: Fundamentals of Machine Learning for Predictive Data Analytics Algorithms: Worked examples, and Case Studies. The MIT Press, London (2015)

    MATH  Google Scholar 

  16. Novotni, M., Klein, R.: 3D Zernike descriptors for content based shape retrieval. In: Symposium on Solid Modeling and Applications, pp. 216–225. Washington, USA (2003)

    Google Scholar 

  17. Ede, J.D., Wenninger, M.J.: Spherical models. Math. Gaz. 65(431), 65 (1981)

    Article  Google Scholar 

  18. Toussaint, G.T.: Solving geometric problems with the rotating calipers. In: IEEE MELECON’83, p. A10. Athens, Greece (1983)

    Google Scholar 

  19. Vranic, D.V.: An improvement of ray-based shape descriptor. In: 8th Leipziger Informatics, pp. 55–58. Leipzig, Germany (2000)

    Google Scholar 

  20. Moller, T., Trumbore, B.: Fast, minimum storage ray/triangle intersection. ACM J. Graph. Tools. 2(1), 21–28 (1997)

    Article  Google Scholar 

  21. Gkioxari, G., Malik, J., Johnson, J.: Mesh R-CNN. In: IEEE International Conference on Computer Vision, pp. 9785–9795. Seoul, Korea (2019)

    Google Scholar 

  22. Fisher, M., Savva, M., Hanrahan, P.: Characterizing structural relationships in scenes using graph kernels. ACM Trans. Graph. 30(4), 34.1–34.11 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Vadim L. Kondarattsev .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kondarattsev, V.L., Kryuchkov, A.Y., Chumak, R.M. (2021). 3D Object Classification, Visual Search from RGB-D Data. In: Jain, L.C., Favorskaya, M.N., Nikitin, I.S., Reviznikov, D.L. (eds) Applied Mathematics and Computational Mechanics for Smart Applications. Smart Innovation, Systems and Technologies, vol 217. Springer, Singapore. https://doi.org/10.1007/978-981-33-4826-4_24

Download citation

  • DOI: https://doi.org/10.1007/978-981-33-4826-4_24

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-33-4825-7

  • Online ISBN: 978-981-33-4826-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics