Skip to main content
SpringerLink
Log in

Object Recognition Methods in a Built Environment

  • Chapter
  • First Online:
DigiTwin: An Approach for Production Process Optimization in a Built Environment

Part of the book series: Springer Series in Advanced Manufacturing ((SSAM))

Abstract

Recognition of an object from a point cloud, image or video is an important task in computer vision which plays a crucial role in many real-world applications. The challenges involved in object recognition, aiming at locating object instances from a large number of predefined categories in collections (images, video or, model library), are multi-model, multi-pose, complicated background, occlusion, and depth variations. In the past few years numerous methods were developed to tackle these challenges and have reported remarkable progress for 3D objects. However, suitable methods of object recognition are needed to achieve added value in built environment. Suitable acquisition methods are also necessary to compensate the impact of darkness, dirt, and occlusion. This chapter provides a comprehensive overview of the recent advances in 3D object recognition of indoor objects using Convolutional Neural Networks (CNN). Methodology for object recognition, approaches for point cloud generation, and test bases are presented. The comparison of main recognition methods based on methods of geometric shape descriptor and supervised learning and their strengths and weakness are also included. The focus lies on the specific requirements and constrains in an industrial environment like tight assembly, light, dirt, occlusion, or incomplete data sets. Finally, a recommendation for use of existing CNN framework for implementation of an automatic object recognition procedure is given.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vision 128(2):261–318. https://doi.org/10.1007/s11263-019-01247-4

    Article  Google Scholar 

  2. Wognum N, Bil C, Elgh F, Peruzzini M, Stjepandić J, Verhagen WJC (2019) Transdisciplinary systems engineering: implications, challenges and research agenda. Int J Agile Syst Manage 12(1):58–89. https://doi.org/10.1504/IJASM.2019.098728

  3. Ríos J, Mas Morate F, Oliva M, Hernández JC (2016) Framework to support the aircraft digital counterpart concept with an industrial design view. Int J Agile Syst Manage 9(3):212–231. https://doi.org/10.1504/IJASM.2016.079934

  4. Quintana B, Prieto SA, Adán A, Vázquez AS (2016) Semantic scan planning for indoor structural elements of buildings. Adv Eng Inform 30:643–659. https://doi.org/10.1016/j.aei.2016.08.003

    Article  Google Scholar 

  5. Zutshi A, Grilo A (2019) The emergence of digital platforms a conceptual platform architecture and impact on industrial engineering. Comput Ind Eng 136:546–555. https://doi.org/10.1016/j.cie.2019.07.027

    Article  Google Scholar 

  6. Bondar S, Salem B, Stjepandić J (2018) Indoor object reconstruction based on acquisition by low-cost devices. Adv Transdisciplinary Eng 7(2018):113–122. https://doi.org/10.3233/978-1-61499-898-3-113

    Article  Google Scholar 

  7. Geng Z, Bidanda B (2017) Review of reverse engineering systems—current state of the art. Virtual Phys Prototyping 12(2):161–172. https://doi.org/10.1080/17452759.2017.1302787

    Article  Google Scholar 

  8. Poza-Lujan J-L, Posadas-Yagüe J-L, Simó-Ten J-E, Blanes F (2020) Distributed architecture to integrate sensor information: object recognition for smart cities. Sensors 20:112. https://doi.org/10.3390/s20010112

  9. Carvalho LE, von Wangenheim A (2019) 3D object recognition and classification: a systematic literature review. Pattern Anal Appl 22:1243–1292. https://doi.org/10.1007/s10044-019-00804-4

    Article  MathSciNet  Google Scholar 

  10. Ma Z, Liu S (2018) A review of 3D reconstruction techniques in civil engineering and their applications. Adv Eng Inform 37(2018):163–174. https://doi.org/10.1016/j.aei.2018.05.005

    Article  Google Scholar 

  11. Sommer M, Stjepandić J, Stobrawa S, von Soden M (2019) Automatic generation of digital twin based on scanning and object recognition. Adv Transdisciplinary Eng 10:645–654. https://doi.org/10.3233/ATDE190174

    Article  Google Scholar 

  12. Sommer M, Stjepandić J, Stobrawa S, von Soden M (2020) Automated generation of a digital twin of a manufacturing system by using scan and convolutional neural networks. Adv Transdisciplinary Eng 12:363–372. https://doi.org/10.3233/ATDE200095

    Article  Google Scholar 

  13. Bondar S, Ruppert C, Stjepandić J (2014) Ensuring data quality beyond change management in virtual enterprise. Int J Agile Syst Manage 7(3/4):304–323. https://doi.org/10.1504/IJASM.2014.065346

  14. Ostrosi E, Stjepandić J, Fukuda S, Kurth M (2014) Modularity: new trends for product platform strategy support in concurrent engineering. Adv Transdisciplinary Eng 1:414–423. https://doi.org/10.3233/978-1-61499-440-4-414

    Article  Google Scholar 

  15. Sommer M, Stjepandić J, Stobrawa S, von Soden M (2020) Improvement of factory planning by automated generation of a Digital Twin. Adv Transdisciplinary Eng 12:645–654. https://doi.org/10.3233/ATDE190174

    Article  Google Scholar 

  16. Singh RD, Mittal A, Bhatia RK (2019) 3D convolutional neural network for object recognition: a review. Multimedia Tools Appl 78:15951–15995. https://doi.org/10.1007/s11042-018-6912-6

    Article  Google Scholar 

  17. Rangel JC, Martínez-Gómez J, Romero-González C, García-Varea I, Cazorla M (2018) Semi-supervised 3D object recognition through CNN labelling. Appl Soft Comput J 65:603–613. https://doi.org/10.1016/j.asoc.2018.02.005

  18. Xiao Y, Ma Y, Zhou M, Zhang J (2018) Deep multi-scale learning on point sets for 3D object recognition. Commun Comput Inform Sci 875:341–348. https://doi.org/10.1007/978-981-13-1702-6_34

    Article  Google Scholar 

  19. Fathi H, Dai F, Lourakis M (2015) Automated as-built 3D reconstruction of civil infrastructure using computer vision: achievements, opportunities, and challenges. Adv Eng Inform 29(2):149–161. https://doi.org/10.1016/j.aei.2015.01.012

    Article  Google Scholar 

  20. Liu Li, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vision 128:261–318. https://doi.org/10.1007/s11263-019-01247-4

    Article  Google Scholar 

  21. Zhao X, Ilieş HT (2017) Learned 3D shape descriptors for classifying 3D point cloud models. Comput Aided Des Appl 14(4):507–515. https://doi.org/10.1080/16864360.2016.1257192

    Article  Google Scholar 

  22. Ghodrati H, Hamza B (2017) Nonrigid 3D shape retrieval using deep auto-encoders. Appl Intell 47:44–61. https://doi.org/10.1007/s10489-016-0880-1

  23. Kamble K, Kulkarni H, Patil J, Sukhatankar S (2018) Object recognition through smartphone using deep learning techniques. In: 2nd international conference on soft computing systems, ICSCS 2018, vol 837. Communications in Computer and Information Science, pp 242–249. https://doi.org/10.1007/978-981-13-1936-5_27

  24. Liu D, Cui Y, Chen Y, Zhang J, Fan B (2020) Video object detection for autonomous driving: Motion-aid feature calibration. Neurocomputing 409:1–11. https://doi.org/10.1016/j.neucom.2020.05.027

    Article  Google Scholar 

  25. Ding X, Luo Y, Li Q, Cheng Y, Cai G, Munnoch R, Xue D, Qingying Yu, Zheng X, Wang B (2018) Prior knowledge-based deep learning method for indoor object recognition. Syst Sci Control Eng 6(1):249–257. https://doi.org/10.1080/21642583.2018.1482477

    Article  Google Scholar 

  26. Wu Z, Song S, Khosla A, Yu F (2015) 3D ShapeNets: a deep representation for volumetric shapes. In: IEEE conference on computer vision and pattern recognition, pp 1912–1920. https://doi.org/10.1109/CVPR.2015.7298801

  27. Salem B, Stjepandić J, Stobrawa S (2019) Assessment of methods for industrial indoor object recognition. Adv Transdisciplinary Eng 10:390–399. https://doi.org/10.3233/ATDE190145

    Article  Google Scholar 

  28. Princeton Modelnet Repository. https://modelnet.cs.princeton.edu/#. Accessed 30 Sep 2020

  29. Chang AX, Funkhouser T, Guibas L, Hanrahan P, Huang Q, Li Z, Savarese S, Savva M, Song S, Su H, Xiao J, Yi L, Yu F (2020) ShapeNet: an information-rich 3D model repository. https://arxiv.org/abs/1512.03012. Accessed 30 Sep 2020

  30. Quadros A (2013) Representing 3D shape in sparse range images for urban object classification. Ph.D. thesis, The University of Sydney

    Google Scholar 

  31. Lehtola VV, Kaartinen H, Nüchter A, Kaijaluoto R, Kukko A, Litkey P, Honkavaara E, Rosnell T, Vaaja MT, Virtanen J-P, Kurkela M, El Issaoui A, Zhu L, Jaakkola A, Hyyppä J (2017) Comparison of the selected state-of-the-art 3D indoor scanning and point cloud generation methods. Remote Sensing 9:796. https://doi.org/10.3390/rs9080796

    Article  Google Scholar 

  32. Rebolj D, Pučko Z, Čuš Babič N, Bizjak M, Mongus D (2017) Point cloud quality requirements for Scan-vs-BIM based automated construction progress monitoring. Autom Construct 84:323–334. https://doi.org/10.1016/j.autcon.2017.09.021

  33. Dominguez M, Dhamdhere R, Petkar A, Jain S, Sah S, Ptucha R (2018) General-purpose deep point cloud feature extractor. In: 2018 IEEE winter conference on applications of computer vision, pp 1972–1981. https://doi.org/10.1109/WACV.2018.00218

  34. Nguyen A, Le B (2013) 3D point cloud segmentation: a survey. In: 2013 IEEE 6th international conference on robotics, automation and mechatronics (RAM). https://doi.org/10.1109/RAM.2013.6758588

  35. Zhang Le, Sun J, Zheng Q (2018) 3D point cloud recognition based on a multi-view convolutional neural network. Sensors 18:3681. https://doi.org/10.3390/s18113681

    Article  Google Scholar 

  36. Liu X, Lu Y, Wu T, Yuan T (2018) An improved local descriptor based object recognition in cluttered 3D point clouds. Int J Comput Commun Control 13(2):221–234. https://doi.org/10.15837/ijccc.2018.2.3010

    Article  Google Scholar 

  37. Zhang W, Su S, Wang B, Hong Q, Sun L (2020) Local k-NNs pattern in omni-direction graph convolution neural network for 3D point clouds. Neurocomputing 413:487–498. https://doi.org/10.1016/j.neucom.2020.06.095

    Article  Google Scholar 

  38. Wang K, Kim M-K (2019) Applications of 3D point cloud data in the construction industry: a fifteen-year review from 2004 to 2018. Adv Eng Inform 39:306–319. https://doi.org/10.1016/j.aei.2019.02.007

  39. Sommer M, Stjepandić J, Stobrawa S, von Soden M (2021) Automated generation of a digital twin in manufacturing for a built environment using scan and object detection. J Indus Inform Integr XX (in press)

    Google Scholar 

  40. Pătrăucean V, Armeni I, Nahangi M, Yeung J, Brilakis I, Haas C (2015) State of research in automatic as-built modelling. Adv Eng Inform 29(2):162–171. https://doi.org/10.1016/j.aei.2015.01.001

    Article  Google Scholar 

  41. Chen J, Fang Y, Cho Y (2018) Performance evaluation of 3D descriptors for object recognition in construction applications. Autom Construct 86:44–52. https://doi.org/10.1016/j.autcon.2017.10.033

  42. Brock A, Lim T, Ritchie JM, Weston N (2016) Generative and discriminative voxel modeling with convolutional neural networks. 3D Deep Learning Workshop at NIPS 2016. http://3ddl.cs.princeton.edu/2016/papers/Brock_et_al.pdf. Accessed 12 June 2019

  43. Qi CR, Su H, Mo K, Guibas LJ (2017) PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21–26 July 2017

    Google Scholar 

  44. Maturana D, Scherer S (2015) VoxNet: a 3D convolutional neural network for real-time object recognition. In: IEEE international conference on intelligent robots and systems, 2015-December, 7353481, pp 922–928. https://doi.org/10.1109/IROS.2015.7353481

  45. Zhou Y, Tuzel O (2018) VoxelNet: end-to-end learning for point cloud based 3D object detection. In: The IEEE conference on computer vision and pattern recognition (CVPR), pp 4490–4499. https://doi.org/10.1109/CVPR.2018.00472

  46. Zhi S, Liu Y, Li X, Guo Y (2017) LightNet: a lightweight 3D convolutional neural network for real-time 3D object recognition. In: Eurographics workshop on 3D object retrieval, pp 9–16. https://doi.org/10.2312/3dor.20171046

  47. Hegde V, Zadeh R (2019) FusionNet: 3D object classification using multiple data representations. In: 3D deep learning workshop at NIPS 2016. https://stanford.edu/~rezab/papers/fusionnet.pdf

  48. Wang C, Cheng M, Sohel F, Bennamoun M, Li J (2019) NormalNet: a voxel-based CNN for 3D object classification and retrieval. Neurocomputing 323:139–147. https://doi.org/10.1016/j.neucom.2018.09.075

    Article  Google Scholar 

  49. Czerniawski T (2020) Fernanda Leite, (2020) Automated digital modeling of existing buildings: a review of visual object recognition methods. Autom Construct 113:103131. https://doi.org/10.1016/j.autcon.2020.103131

    Article  Google Scholar 

  50. Garcia-Garcia A, Garcia-Rodriguez J, Orts-Escolano S, Oprea S, Gomez-Donoso F, Cazorla M (2017) A study of the effect of noise and occlusion on the accuracy of convolutional neural networks applied to 3D object recognition. Comput Vision Image Understand 164:124–134. https://doi.org/10.1016/j.cviu.2017.06.006

  51. Ye Y, Chen H, Zhang C, Hao X, Zhang Z (2020) SARPNET: shape attention regional proposal network for liDAR-based 3D object detection. Neurocomputing 379:53–63. https://doi.org/10.1016/j.neucom.2019.09.086

    Article  Google Scholar 

  52. Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS (2016) Deep learning for visual understanding: a review. Neurocomputing 187:27–48. https://doi.org/10.1016/j.neucom.2015.09.116

    Article  Google Scholar 

  53. Kuhn O, Liese H, Stjepandić J (2011) Methodology for knowledge-based engineering template update. In: Cavallucci D, Guio R, Cascini G (eds) Building innovation pipelines through computer-aided innovation. Springer-Verlag, Berlin Heidelberg, pp 178–191. https://doi.org/10.1007/978-3-642-22182-8_14

  54. Dekhtiar J, Durupt A, Bricogne M, Eynard B, Rowson H, Kiritsis D (2018) Deep learning for big data applications in CAD and PLM—Research review, opportunities and case study. Comput Ind 100:227–243. https://doi.org/10.1016/j.compind.2018.04.005

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. PROSTEP AG, Dolivostr. 11, 64293, Darmstadt, Germany

    Josip Stjepandić

  2. isb innovative software businesses GmbH, Otto-Lilienthal-Strasse 2, 88046, Friedrichshafen, Germany

    Markus Sommer

Authors
  1. Josip Stjepandić
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Markus Sommer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Josip Stjepandić .

Editor information

Editors and Affiliations

  1. PROSTEP AG, Darmstadt, Germany

    Josip Stjepandić

  2. isb innovative software businesses GmbH, Friedrichshafen, Germany

    Markus Sommer

  3. Institute of Production Engineering and Machine Tools, Leibnitz University of Hannover, Garbsen, Niedersachsen, Germany

    Berend Denkena

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Stjepandić, J., Sommer, M. (2022). Object Recognition Methods in a Built Environment. In: Stjepandić, J., Sommer, M., Denkena, B. (eds) DigiTwin: An Approach for Production Process Optimization in a Built Environment. Springer Series in Advanced Manufacturing. Springer, Cham. https://doi.org/10.1007/978-3-030-77539-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77539-1_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77538-4

  • Online ISBN: 978-3-030-77539-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation