Abstract
Supervised Machine Learning systems such as Convolutional Neural Networks (CNNs) are known for their great need for labeled data. However, in case of geospatial data and especially in terms of Airborne Laserscanning (ALS) point clouds, labeled data is rather scarce, hindering the application of such systems. Therefore, we rely on Active Learning (AL) for significantly reducing necessary labels and we aim at gaining a deeper understanding on its working principle for ALS point clouds. Since the key element of AL is sampling of most informative points, we compare different basic sampling strategies and try to further improve them for geospatial data. While AL reduces total labeling effort, the basic issue of experts doing this labor- and therefore cost-intensive task remains. Therefore, we propose to outsource data annotation to the crowd. However, when employing crowdworkers, labeling errors are inevitable. As a remedy, we aim on selecting points, which are easier for interpretation and evaluate the robustness of AL to labeling errors. Applying these strategies for different classifiers, we estimate realistic segmentation results from crowdsourced data solely, only differing in Overall Accuracy by about 3% points compared to results based on completely labeled dataset, which is demonstrated for two different scenes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Dataset will be made publicly available in early 2021.
References
Argamon-Engelson, S., Dagan, I.: Committee-based sample selection for probabilistic classifiers. J. Artif. Intell. Res. 11, 335–360 (1999)
Becker, C., Häni, N., Rosinskaya, E., d’Angelo, E., Strecha, C.: Classification of aerial photogrammetric 3D point clouds. ISPRS Annals IV-1/W1, pp. 3–10 (2017)
Branson, S., et al.: Visual recognition with humans in the loop. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 438–451. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_32
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). https://doi.org/10.1023/A:1010933404324
Chehata, N., Guo, L., Mallet, C.: Airborne LiDAR feature selection for urban classification using random forests. ISPRS Arch. 38 (2009)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995). https://doi.org/10.1007/BF00994018
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.F.: ImageNet: a large-scale hierarchical image database. In: CVPR 2009, pp. 248–255 (2009)
Ertekin, S., Huang, J., Bottou, L., Giles, L.: Learning on the border: active learning in imbalanced data classification. In: CIKM 2007, pp. 127–136. ACM, New York (2007)
Gadiraju, U., Kawase, R., Siehndel, P., Fetahu, B.: Breaking bad: understanding behavior of crowd workers in categorization microtasks. In: HT 2015, pp. 33–38. ACM (2015)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: ICML 2016, vol. 48, pp. 1050–1059. PMLR, New York (2016)
Graham, B., Engelcke, M., van der Maaten, L.: 3D semantic segmentation with submanifold sparse convolutional networks. In: CVPR 2018, pp. 9224–9232 (2018)
Haala, N., Kölle, M., Cramer, M., Laupheimer, D., Mandlburger, G., Glira, P.: hybrid georeferencing, enhancement and classification of ultra-high resolution UAV LiDAR and image point clouds for monitoring applications. ISPRS Annals V-2-2020, pp. 727–734 (2020)
Hirth, M., Hoßfeld, T., Tran-Gia, P.: Anatomy of a crowdsourcing platform - using the example of Microworkers.com. In: IMIS 2011, pp. 322–329. IEEE Computer Society, Washington (2011)
Hui, Z., et al.: An active learning method for DEM extraction from airborne LiDAR point clouds. IEEE Access 7, 89366–89378 (2019)
Kellenberger, B., Marcos, D., Lobry, S., Tuia, D.: Half a percent of labels is enough: efficient animal detection in UAV imagery using deep CNNs and active learning. TRGS 57(12), 9524–9533 (2019)
Kirsch, A., van Amersfoort, J., Gal, Y.: BatchBALD: efficient and diverse batch acquisition for deep Bayesian active learning. In: NIPS 2019, pp. 7026–7037. Curran Associates, Inc. (2019)
Kölle, M., Walter, V., Schmohl, S., Soergel, U.: Hybrid acquisition of high quality training data for semantic segmentation of 3D point clouds using crowd-based active learning. ISPRS Annals V-2-2020, pp. 501–508 (2020)
Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical Report TR-2009, University of Toronto, Toronto (2009)
Li, N., Pfeifer, N.: Active learning to extend training data for large area airborne LiDAR classification. ISPRS Archives XLII-2/W13, pp. 1033–1037 (2019)
Lin, Y., Vosselman, G., Cao, Y., Yang, M.Y.: Efficient training of semantic point cloud segmentation via active learning. ISPRS Annals V-2-2020, pp. 243–250 (2020)
Lloyd, S.P.: Least squares quantization in PCM. IEEE Trans. Inf. Theory 28(2), 129–137 (1982)
Lockhart, J., Assefa, S., Balch, T., Veloso, M.: Some people aren’t worth listening to: periodically retraining classifiers with feedback from a team of end users. CoRR abs/2004.13152 (2020)
Luo, H., et al.: Semantic labeling of mobile lidar point clouds via active learning and higher order MRF. TGRS 56(7), 3631–3644 (2018)
Mackowiak, R., Lenz, P., Ghori, O., Diego, F., Lange, O., Rother, C.: CEREALS - cost-effective region-based active learning for semantic segmentation. In: BMVC 2018 (2018)
Niemeyer, J., Rottensteiner, F., Soergel, U.: Contextual classification of lidar data and building object detection in urban areas. ISPRS J. 87, 152–165 (2014)
Patra, S., Bruzzone, L.: A cluster-assumption based batch mode active learning technique. Pattern Recogn. Lett. 33(9), 1042–1048 (2012)
Schmohl, S., Sörgel, U.: Submanifold sparse convolutional networks for semantic segmentation of large-scale ALS point clouds. ISPRS Annals IV-2/W5, pp. 77–84 (2019)
Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison (2009)
Tuia, D., Ratle, F., Pacifici, F., Kanevski, M.F., Emery, W.J.: Active learning methods for remote sensing image classification. TGRS 47(7), 2218–2232 (2009)
Vaughan, J.W.: Making better use of the crowd: how crowdsourcing can advance machine learning research. J. Mach. Learn. Res. 18(193), 1–46 (2018)
Walter, V., Kölle, M., Yin, Y.: Evaluation and optimisation of crowd-based collection of trees from 3D point clouds. ISPRS Annals V-4-2020, pp. 49–56 (2020)
Walter, V., Soergel, U.: Implementation, results, and problems of paid crowd-based geospatial data collection. PFG 86, 187–197 (2018). https://doi.org/10.1007/s41064-018-0058-z
Weinmann, M., Jutzi, B., Hinz, S., Mallet, C.: Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers. ISPRS J. 105, 286–304 (2015)
Xu, Z., Akella, R., Zhang, Y.: Incorporating diversity and density in active learning for relevance feedback. In: Amati, G., Carpineto, C., Romano, G. (eds.) ECIR 2007. LNCS, vol. 4425, pp. 246–257. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-71496-5_24
Zhdanov, F.: Diverse mini-batch active learning. CoRR abs/1901.05954 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kölle, M., Walter, V., Schmohl, S., Soergel, U. (2021). Remembering Both the Machine and the Crowd When Sampling Points: Active Learning for Semantic Segmentation of ALS Point Clouds. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12667. Springer, Cham. https://doi.org/10.1007/978-3-030-68787-8_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-68787-8_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68786-1
Online ISBN: 978-3-030-68787-8
eBook Packages: Computer ScienceComputer Science (R0)
