Abstract
In the reconstruction of façade elements, the identification of specific object types remains challenging and is often circumvented by rectangularity assumptions or the use of bounding boxes. We propose a new approach for the reconstruction of 3D façade details. We combine mobile laser scanning (MLS) point clouds and a pre-defined 3D model library using a Bag of words (BoW) concept, which we augment by incorporating semi-global features. We conduct experiments on the models superimposed with random noise and on the TUM-FAÇADE dataset (Wysocki et al. , 2023). Our method demonstrates promising results, improving the conventional BoW approach. It holds the potential to be utilized for more realistic facade reconstruction without rectangularity assumptions, which can be used in applications such as testing automated driving functions or estimating façade solar potential.
This article was selected based on the results of a double-blind review of an extended abstract.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bamler R, Shi Y (2022) Estimation theory: 2—recap of basic propability theory -part 4. 13-5-2022, Chair of remote sensing technology technical university of Munich. Lecture of Estimation Theory
Biljecki F, Stoter J, Ledoux H, Zlatanova S, Çöltekin A (2015) Applications of 3D city models: state of the art review. ISPRS Int J Geo-Inf 4(4):2842–2889
Bronstein AM, Bronstein MM, Guibas LJ, Ovsjanikov M (2011) Shape google. ACM Trans Graph 30(1):1–20. https://doi.org/10.1145/1899404.1899405
Calonder M, Lepetit V, Ozuysal M, Trzcinski T, Strecha C, Fua P (2012) BRIEF: computing a local binary descriptor very fast. IEEE Trans Patt Anal Mach Intell 34(7):1281–1298. https://doi.org/10.1109/tpami.2011.222
Cha SH (2008) Taxonomy of nominal type histogram distance measures. In: Long C, Sohrab SH, Bognar G, Perlovsky L (eds) MATH’08: proceedings of the American conference on applied mathematics. World Scientific and Engineering Academy and Society, pp 325–330
Congalton RG, Green K (2019) Assessing the accuracy of remotely sensed data: principles and practices. Taylor and Francis, 3 edn
Csurka G, Dance CR, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Proceedings of the ECCV international workshop on statistical learning in computer vision (2004)
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer Vision and pattern recognition (CVPR’05). IEEE. https://doi.org/10.1109/cvpr.2005.177
Fan H, Kong G, Zhang C (2021) An interactive platform for low-cost 3d building modeling from VGI data using convolutional neural network. Big Earth Data 5(1):49–65. https://doi.org/10.1080/20964471.2021.1886391
Fröch T (2023) Project photogrammetry—reconstructing façade details using MLS point clouds and bag-of-words approach Tech rep, Technical University Munich. https://github.com/ThomasFroech/ReconstructingFacadeDetailsBoW
Grandini M, Bagli E, Visani G (2020) Metrics for multi-class classification: an overview. https://doi.org/10.48550/ARXIV.2008.05756
Hensel S, Goebbels S, Kada M (2019) Facade reconstruction for textured LOD2 CityGML models based on deep learning and mixed integer linear programming. ISPRS Annals Photogram Remote Sens Spatial Inf Sci IV-2/W5:37–44. https://doi.org/10.5194/isprs-annals-iv-2-w5-37-2019
Hoegner L, Gleixner G (2022) Automatic extraction of facades and windows from MLS point clouds using voxelspace and visibility analysis. Int Arch Photogram Remote Sens Spatial Inf Sci XLIII-B2-2022:387–394. https://doi.org/10.5194/isprs-archives-xliii-b2-2022-387-2022
Huang H, Michelini M, Schmitz M, Roth L, Mayer H (2020) LoD3 building reconstruction from multi-source images. Int Arch Photogram Remote Sens Spatial Inf Sci XLIII-B2-2020:427–434. https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-427-2020, https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2020/427/2020/
Kang Z, Yang J (2018) A probabilistic graphical model for the classification of mobile LiDAR point clouds. ISPRS J Photogram Remote Sens 143:108–123. https://doi.org/10.1016/j.isprsjprs.2018.04.018
Li J, Lee GH (2019) USIP: unsupervised stable interest point detection from 3D point clouds. In: Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pp 361–370
Lowe D (1999) Object recognition from local scale-invariant features. In: Proceedings of the seventh IEEE international conference on computer vision. IEEE. https://doi.org/10.1109/iccv.1999.790410
Mallick S (2016) Histogram of oriented gradients explained using OpenCV. learnopencv.com. https://learnopencv.com/histogram-of-oriented-gradients/
Memon SA, Akthar F, Mahmood T, Azeem M, Shaukat Z (2019) 3d shape retrieval using bag of word approaches. In: 2019 2nd international conference on computing, mathematics and engineering technologies (iCoMET). IEEE. https://doi.org/10.1109/icomet.2019.8673397
Nowak E, Jurie F, Triggs B (2008) Sampling strategies for bag-of-features image classification. In: Leonardis A, Bischof H, Prinz A (eds) European conference on computer vision 2006, Part IV, vol 3954. Lecture notes in computer science. Springer, Berlin Heidelberg, pp 490–503
Pantoja-Rosero BG, Achanta R, Kozinski M, Fua P, Perez-Cruz F, Beyer K (2022) Generating LoD3 building models from structure-from-motion and semantic segmentation. Autom Construct 141:104430
Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. https://doi.org/10.48550/ARXIV.1804.02767
Ripperda N (2010) Rekonstruktion von Fassadenstrukturen mittels formaler Grammatiken und Reversible Jump Markov Chain Monte Carlo Sampling. Ph.D. thesis, Leibnitz Universität Hannover, Hannover (2010), wissenschaftliche Arbeiten der Fachrichtung Geodäsie und Geoinformatik der Leibniz Universität Hannover
Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: An efficient alternative to SIFT or SURF. In: 2011 international conference on computer vision. IEEE. https://doi.org/10.1109/iccv.2011.6126544
Salton G, McGill MJ (1986) Introduction to modern information retrieval. McGraw-Hill Inc, New York, NY, USA
Sipiran I, Bustos B (2011) Harris 3d: a robust extension of the harris operator for interest point detection on 3d meshes. The Visual Comput 27(11):963–976. https://doi.org/10.1007/s00371-011-0610-y
Trimble Inc.: SketchUp 3d warehouse (2021). https://3dwarehouse.sketchup.com
Tuttas S, Stilla U (2013) Reconstruction of façades in point clouds from multi aspect oblique ALS. ISPRS Ann Photogram Remote Sens Spatial Inf Sci II-3/W3:91–96. https://doi.org/10.5194/isprsannals-II-3-W3-91-2013, http://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/II-3-W3/91/2013/
Wysocki O, Grilli E, Hoegner L, Stilla U (2023) Combining visibility analysis and deep learning for refinement of semantic 3d building models by conflict classification. ISPRS Ann Photogram Remote Sens Spatial Inf Sci X-4/W2:289–296. https://doi.org/10.5194/isprs-annals-x-4-w2-2022-289-2022
Wysocki O, Hoegner L, Stilla U (2023) TUM-FAÇADE: reviewing and enriching point cloud benchmarks for façade segmentation. arXiv preprint arXiv:2304.07140
Xu S, Fang T, Li D, Wang S (2010) Object classification of aerial images with bag-of-visual words. IEEE Geosci Remote Sens Lett 7(2):366–370. https://doi.org/10.1109/lgrs.2009.2035644
Zhu Q, Zhong Y, Zhao B, Xia GS, Zhang L (2016) Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery. IEEE Geosci Remote Sens Lett 13(6):747–751. https://doi.org/10.1109/lgrs.2015.2513443
Acknowledgements
This work was supported by the Bavarian State Ministry for Economic Affairs, Regional Development and Energy within the framework of the IuK Bayern project MoFa3D - Mobile Erfassung von Fassaden mittels 3D Punktwolken, Grant No. IUK643/001. Moreover, the work was conducted within the framework of the Leonhard Obermeyer Center at the Technical University of Munich (TUM). We gratefully acknowledge the Geoinformatics team at the TUM for the valuable insights and for providing the CityGML datasets.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Froech, T., Wysocki, O., Hoegner, L., Stilla, U. (2024). Reconstructing Façade Details Using MLS Point Clouds and Bag-of-Words Approach. In: Kolbe, T.H., Donaubauer, A., Beil, C. (eds) Recent Advances in 3D Geoinformation Science. 3DGeoInfo 2023. Lecture Notes in Geoinformation and Cartography. Springer, Cham. https://doi.org/10.1007/978-3-031-43699-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-43699-4_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43698-7
Online ISBN: 978-3-031-43699-4
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)