Skip to main content
SpringerLink
Log in

Method for clustering and identification of objects in laser scanning point clouds using dynamic logic

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Today there is a gap between a presence of manifold new equipment on the market which provides streams of various digital data about the environment, in particular in the form of laser scanning point clouds, and the lack of adequate efficient methods and software for information extraction from such data. A solution to the problem of bridging this gap on the basis of neural modeling field theory and dynamic logic (DL) is proposed. We present a DL-based method of extracting and analyzing information in hybrid point clouds, which include not only spatial coordinates and intensity, but also the color of each point from multiple sources including terrestrial, mobile, and airborne laser scanning data. The proposed method is significant for creating a fundamental theoretical basis for new application algorithms and software for many new applications, including building information modeling and “smart city” environment. The proposed method is fairly new to solving various problems related to extracting semantically rich information from a nontraditional type of digital data, especially hybrid point clouds created from laser scanning. This method will allow to significantly expand the existing boundaries of knowledge in the field of extraction and analysis of information from various digital data, because neural modeling field theory and DL can improve the performance of relevant calculations and close the existing gap in analysis of digital images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Diagram 1

Similar content being viewed by others

Data and code availability

The results in this work are applicable. The original experiment data are not provided.

References

  1. Kim Y, Nguyen CHP, Choi Y (2020) Automatic pipe and elbow recognition from three-dimensional point cloud model of industrial plant piping system using convolutional neural network-based primitive classification. Autom Constr 116:103236. https://doi.org/10.1016/j.autcon.2020.103236

    Article  Google Scholar 

  2. Xia S, Chen D, Wang R, Li J, Zhang X (2020) Geometric primitives in LiDAR point clouds: a review. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 13:685–707. https://doi.org/10.1109/JSTARS.2020.2969119

    Article  Google Scholar 

  3. Trujillo-Jiménez MA, Navarro P, Pazos B, Morales L, Ramallo V, Paschetta C, De Azevedo S, Ruderman A, Pérez O, Delrieux C, Gonzalez-José R (2020) body2vec: 3D point cloud reconstruction for precise anthropometry with handheld devices. Journal of. Imaging 6(9):94. https://doi.org/10.3390/jimaging6090094

    Article  Google Scholar 

  4. Volk R, Luu TH, Mueller-Roemer JS, Sevilmis N, Schultmann F (2018) Deconstruction project planning of existing buildings based on automated acquisition and reconstruction of building information. Autom Constr 91:226–245. https://doi.org/10.1016/j.autcon.2018.03.017

    Article  Google Scholar 

  5. Hoiem D, Savarese S (2011) Representations and techniques for 3D object recognition and scene interpretation. Synthesis Lectures on Artificial Intelligence and Machine Learning 5(5):1–169. https://doi.org/10.2200/S00370ED1V01Y201107AIM015

    Article  Google Scholar 

  6. Opitz R, Limp WF (2015) Recent developments in high-density survey and measurement (HDSM) for archaeology: implications for practice and theory. Annu Rev Anthropol 44:347–364. https://doi.org/10.1146/annurev-anthro-102214-013845

    Article  Google Scholar 

  7. Chen X, Kundu K, Zhu Y, Ma H, Fidler S, Urtasun R (2017) 3d object proposals using stereo imagery for accurate object class detection. IEEE Trans Pattern Anal Mach Intell 40(5):1259–1272. https://doi.org/10.1109/TPAMI.2017.2706685

    Article  Google Scholar 

  8. Zhang Z, Li J, Guo Y, Li X, Lin Y, Xiao G, Wang C (2019) Robust procedural model fitting with a new geometric similarity estimator. Pattern Recogn 85:120–131. https://doi.org/10.1016/j.patcog.2018.07.027

    Article  Google Scholar 

  9. Liu W, Sun J, Li W, Hu T, Wang P (2019) Deep learning on point clouds and its application: a survey. Sensors 19(19):4188. https://doi.org/10.3390/s19194188

    Article  Google Scholar 

  10. Hashimoto M, Akizuki S, Takei S (2017) A survey and technology trends of 3D features for object recognition. Electronics and Communication in Japan 100(11):31–42. https://doi.org/10.1002/ecj.11992

    Article  Google Scholar 

  11. Santoso F, Garratt MA, Pickering MR, Asikuzzaman M (2015) 3D mapping for visualization of rigid structures: A review and comparative study. IEEE Sensors J 16(6):1484–1507. https://doi.org/10.1109/JSEN.2015.2498940

    Article  Google Scholar 

  12. Reiterer A, Wäschle K, Störk D, Leydecker A, Gitzen N (2020) Fully automated segmentation of 2D and 3D mobile mapping data for reliable modeling of surface structures using deep learning. Remote Sens 12(16):2530. https://doi.org/10.3390/RS12162530

    Article  Google Scholar 

  13. Chen S, Tan D (2018) A SA-ANN-based modeling method for human cognition mechanism and the PSACO cognition algorithm. Complexity 2018:6264124–6264121. https://doi.org/10.1155/2018/6264124

    Article  MathSciNet  Google Scholar 

  14. Gregory Trafton J, Harrison AM (2011) Embodied spatial cognition. Top Cogn Sci 3(4):686–706. https://doi.org/10.1111/j.1756-8765.2011.01158.x

    Article  Google Scholar 

  15. Qi C, Li M, Wang Q, Zhang H, Xing J, Gao Z, Zhang H (2018) Facial expressions recognition based on cognition and mapped binary patterns. IEEE Access 6:18795–18803. https://doi.org/10.1109/ACCESS.2018.2816044

    Article  Google Scholar 

  16. Cui C, Lin P, Nie X, Yin Y, Zhu Q (2017) Hybrid textual-visual relevance learning for content-based image retrieval. J Vis Commun Image Represent 48:367–374. https://doi.org/10.1016/j.jvcir.2017.03.011

    Article  Google Scholar 

  17. Perlovsky LI (2016) Physics of the mind. Front Syst Neurosci 10:84. https://doi.org/10.3389/fnsys.2016.00084

    Article  Google Scholar 

  18. Perlovsky LI (2006) Toward physics of the mind: concepts, emotions, consciousness, and symbols. Phys Life Rev 3(1):23–55. https://doi.org/10.1016/j.plrev.2005.11.003

    Article  MathSciNet  Google Scholar 

  19. Perlovsky LI (2007) Neural networks, fuzzy models and dynamic logic. Studies in Fuzziness and Soft Computing 209:363–386. https://doi.org/10.1007/978-3-540-37522-7_17

    Article  Google Scholar 

  20. Fontanari JF, Tikhanoff V, Cangelosi A, Ilin R, Perlovsky LI (2009) Cross-situational learning of object–word mapping using Neural Modeling Fields. Neural Netw 22(5-6):579–585. https://doi.org/10.1016/j.neunet.2009.06.010

    Article  Google Scholar 

  21. Kovalerchuk B, Perlovsky L, Wheeler G (2012) Modelling phenomena and dynamic logic of phenomena. Journal of Applied Non-Classical Logics 22(1-2):53–82. https://doi.org/10.1080/11663081.2012.682439

    Article  MathSciNet  MATH  Google Scholar 

  22. Schoeller F, Perlovsky L, Arseniev D (2018) Physics of mind: experimental confirmations of theoretical predictions. Phys Life Rev 25:45–68. https://doi.org/10.1016/j.plrev.2017.11.021

    Article  Google Scholar 

  23. Dantcheva A, Elia P, Ross A (2015) What else does your biometric data reveal? A survey on soft biometrics. IEEE Transactions on Information Forensics and Security 11(3):441–467. https://doi.org/10.1109/TIFS.2015.2480381

    Article  Google Scholar 

  24. Singh M, Singh R, Ross A (2019) A comprehensive overview of biometric fusion. Information Fusion 52:187–205. https://doi.org/10.1016/j.inffus.2018.12.003

    Article  Google Scholar 

  25. Drozdowski P, Rathgeb C, Busch C (2019) Computational workload in biometric identification systems: an overview. IET Biometrics 8(6):351–368. https://doi.org/10.1049/iet-bmt.2019.0076

    Article  Google Scholar 

  26. Hu Z (2013) Intelligent road sign inventory (IRSI) with image recognition and attribute computation from video log. Computer-Aided Civil and Infrastructure Engineering 28(2):130–145. https://doi.org/10.1111/j.1467-8667.2012.00768.x

    Article  Google Scholar 

  27. Li F, Lehtomäki M, Elberink SO, Vosselman G, Kukko A, Puttonen E, Chen Y, Hyyppä J (2019) Semantic segmentation of road furniture in mobile laser scanning data. ISPRS J Photogramm Remote Sens 154:98–113. https://doi.org/10.1016/j.isprsjprs.2019.06.001

    Article  Google Scholar 

  28. Balali V, Golparvar-Fard M (2016) Evaluation of multiclass traffic sign detection and classification methods for US roadway asset inventory management. J Comput Civ Eng 30(2):04015022. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000491

    Article  Google Scholar 

  29. Spencer JBF, Hoskere V, Narazaki Y (2019) Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering 5:199–222. https://doi.org/10.1016/j.eng.2018.11.030

    Article  Google Scholar 

  30. Che E, Jung J, Olsen MJ (2019) Object recognition, segmentation, and classification of mobile laser scanning point clouds: a state of the art review. Sensors 19(4):810. https://doi.org/10.3390/s19040810

    Article  Google Scholar 

  31. Balado J, Díaz-Vilariño L, Arias P, Soilán M (2017) Automatic building accessibility diagnosis from point clouds. Autom Constr 82:103–111. https://doi.org/10.1016/j.autcon.2017.06.026

    Article  Google Scholar 

  32. Hussain R, Zeadally S (2018) Autonomous cars: research results, issues, and future challenges. IEEE Communications Surveys & Tutorials 21(2):1275–1313. https://doi.org/10.1109/COMST.2018.2869360

    Article  Google Scholar 

  33. Xue JR, Fang JW, Zhang P (2018) A survey of scene understanding by event reasoning in autonomous driving. Int J Autom Comput 15(3):249–266. https://doi.org/10.1007/s11633-018-1126-y

    Article  Google Scholar 

  34. Belmonte LM, Morales R, Fernández-Caballero A (2019) Computer vision in autonomous unmanned aerial vehicles—a systematic mapping study. Appl Sci 9(15):3196. https://doi.org/10.3390/app9153196

    Article  Google Scholar 

  35. Al-Kaff A, Martin D, Garcia F, de la Escalera A, Armingol JM (2018) Survey of computer vision algorithms and applications for unmanned aerial vehicles. Expert Syst Appl 92:447–463. https://doi.org/10.1016/j.eswa.2017.09.033

    Article  Google Scholar 

  36. Loquercio A, Maqueda AI, Del-Blanco CR, Scaramuzza D (2018) Dronet: learning to fly by driving. IEEE Robotics and Automation Letters 3(2):1088–1095. https://doi.org/10.1109/LRA.2018.2795643

    Article  Google Scholar 

  37. Rebelo C, Rodrigues AM, Tenedório JA (2019) Dasymetric mapping using UAV high resolution 3D data within urban areas. Remote Sens 11(14):1716. https://doi.org/10.3390/rs11141716

    Article  Google Scholar 

  38. Gruen A, Huang X, Qin R, Du T, Fang W, Boavida J, Oliveira A (2013) Joint processing of UAV imagery and terrestrial mobile mapping system data for very high resolution city modeling. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives 40(1W2):175–182. https://doi.org/10.5194/isprsarchives-XL-1-W2-175-2013

    Article  Google Scholar 

  39. Nex F, Remondino F (2014) UAV for 3D mapping applications: a review. Applied geomatics 6(1):1–15. https://doi.org/10.1007/s12518-013-0120-x

    Article  Google Scholar 

  40. Rothermel M, Haala N, Wenzel K, Bulatov D (2014) Fast and robust generation of semantic urban terrain models from UAV video streams. In: 2014 22nd International Conference on Pattern Recognition. IEEE, pp 592–597. https://doi.org/10.1109/ICPR.2014.112

  41. Yan WY, Shaker A, El-Ashmawy N (2015) Urban land cover classification using airborne LiDAR data: a review. Remote Sens Environ 158:295–310. https://doi.org/10.1016/j.rse.2014.11.001

    Article  Google Scholar 

  42. Guan H, Li J, Cao S, Yu Y (2016) Use of mobile LiDAR in road information inventory: a review. Int J Image Data Fusion 7(3):219–242. https://doi.org/10.1080/19479832.2016.1188860

    Article  Google Scholar 

  43. Wang Y, Chen Q, Zhu Q, Liu L, Li C, Zheng D (2019) A survey of mobile laser scanning applications and key techniques over urban areas. Remote Sens 11(13):1540. https://doi.org/10.3390/rs11131540

    Article  Google Scholar 

  44. Gharbi H, Bahroun S, Zagrouba E (2019) Key frame extraction for video summarization using local description and repeatability graph clustering. SIViP 13(3):507–515. https://doi.org/10.1007/s11760-018-1376-8

    Article  Google Scholar 

  45. Kong X, Ma K, Hou S, Shang D, Xia F (2019) Human interactive behavior: a bibliographic review. IEEE Access 7:4611–4628. https://doi.org/10.1109/ACCESS.2018.2887341

    Article  Google Scholar 

  46. Macchione F, Costabile P, Costanzo C, De Lorenzo G (2019) Extracting quantitative data from non-conventional information for the hydraulic reconstruction of past urban flood events. A case study. J Hydrol 576:443–465. https://doi.org/10.1016/j.jhydrol.2019.06.031

    Article  Google Scholar 

  47. Zhang G, Vela PA, Karasev P, Brilakis I (2015) A sparsity-inducing optimization-based algorithm for planar patches extraction from noisy point-cloud data. Computer-Aided Civil and Infrastructure Engineering 30(2):85–102. https://doi.org/10.1111/mice.12063

    Article  Google Scholar 

  48. Nguyen HL, Belton D, Helmholz P (2019) Planar surface detection for sparse and heterogeneous mobile laser scanning point clouds. ISPRS J Photogramm Remote Sens 151:141–161. https://doi.org/10.1016/j.isprsjprs.2019.03.006

    Article  Google Scholar 

  49. Gilani SAN, Awrangjeb M, Lu G (2016) An automatic building extraction and regularisation technique using lidar point cloud data and orthoimage. Remote Sens 8(3):258. https://doi.org/10.3390/rs8030258

    Article  Google Scholar 

  50. Czerniawski T, Leite F (2020) Automated digital modeling of existing buildings: a review of visual object recognition methods. Autom Constr 113:103131. https://doi.org/10.1016/j.autcon.2020.103131

    Article  Google Scholar 

  51. Zeng S, Chen J, Cho YK (2020) User exemplar-based building element retrieval from raw point clouds using deep point-level features. Autom Constr 114:103159. https://doi.org/10.1016/j.autcon.2020.103159

    Article  Google Scholar 

  52. Chen L, Wang Y, Siu MFF (2020) Detecting semantic regions of construction site images by transfer learning and saliency computation. Autom Constr 114:103185. https://doi.org/10.1016/j.autcon.2020.103185

    Article  Google Scholar 

  53. Chen J, Kira Z, Cho YK (2019) Deep learning approach to point cloud scene understanding for automated scan to 3D reconstruction. J Comput Civ Eng 33(4):04019027. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000842

    Article  Google Scholar 

  54. Badenko V, Fedotov A, Vinogradov K (2019) Hybrid algorithms of laser scanning point cloud for topological analysis. Advances in Intelligent Systems and Computing 797:223–234. https://doi.org/10.1007/978-981-13-1165-9_20

    Article  Google Scholar 

  55. Badenko V, Zotov D, Fedotov A (2018) Hybrid processing of laser scanning data. E3S Web of Conferences 33:01047. https://doi.org/10.1051/e3sconf/20183301047

    Article  Google Scholar 

  56. Badenko V, Fedotov A, Vinogradov K (2018) Algorithms of laser scanner data processing for ground surface reconstruction. Lect Notes Comput Sci 10961:397–411. https://doi.org/10.1007/978-3-319-95165-2_28

    Article  Google Scholar 

  57. Badenko V, Fedotov A, Zotov D, Lytkin S, Volgin D, Garg RD, Min L (2019) Scan-to-BIM methodology adapted for different application. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 42(5/W2):1–7. https://doi.org/10.5194/isprs-archives-XLII-5-W2-1-2019

    Article  Google Scholar 

  58. Perlovsky LI (2017) Physics of the mind, dynamic logic, and monotone Boolean functions. Studies in Computational Intelligence 683:193–231. https://doi.org/10.1007/978-3-319-51052-1_13

    Article  Google Scholar 

  59. Perlovsky LI, Deming R, Ilin R (2011) Emotional cognitive neural algorithms with engineering applications dynamic logic: From Vague to Crisp Studies. Comput Intell 371:1–204. https://doi.org/10.1007/978-3-642-22830-8

    Article  Google Scholar 

  60. Shi BQ, Liang J, Liu Q (2011) Adaptive simplification of point cloud using k-means clustering. Comput Aided Des 43(8):910–922. https://doi.org/10.1016/j.cad.2011.04.001

    Article  Google Scholar 

  61. Yang J, Rahardja S, Fränti P (2021) Mean-shift outlier detection and filtering. Pattern Recogn 115:107874. https://doi.org/10.1016/j.patcog.2021.107874

    Article  Google Scholar 

Download references

Funding

The research is partially funded by the Ministry of Science and Higher Education of the Russian Federation as part of World-class Research Center program: Advanced Digital Technologies (contract no. 075-15-2020-934 dated 17.11.2020)

Author information

Authors and Affiliations

  1. Peter the Great St. Petersburg Polytechnic University, Polytechnicheskaya 29, 195251, St. Petersburg, Russia

    Yevgeny Milanov, Vladimir Badenko & Vladimir Yadykin

  2. Northeastern University, Boston, MA, USA

    Leonid Perlovsky

Authors
  1. Yevgeny Milanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vladimir Badenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladimir Yadykin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Leonid Perlovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yevgeny Milanov: algorithm and software development and computer experiments, draft version editing; Vladimir Badenko: research supervising, final editing, and testing; Vladimir Yadykin: data obtaining for computer experiments, interpretation of results, algorithm development; Leonid Perlovsky: main idea, key algorithm, final editing. All authors have read the article, partially edited it, and agree with its content.

Corresponding author

Correspondence to Vladimir Badenko.

Ethics declarations

Ethics approval

We further confirm that any aspect of the work covered in this manuscript that has involved either experimental animals or human patients has been conducted with the ethical approval of all relevant bodies and that such approvals are acknowledged within the manuscript.

Consent to participate

We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us.

Consent for publication

We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing, we confirm that we have followed the regulations of our institutions concerning intellectual property.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Milanov, Y., Badenko, V., Yadykin, V. et al. Method for clustering and identification of objects in laser scanning point clouds using dynamic logic. Int J Adv Manuf Technol 117, 2309–2318 (2021). https://doi.org/10.1007/s00170-021-07286-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-021-07286-x

Keywords

Search

Navigation