Abstract
Road information plays a fundamental role in application fields such as map updating, traffic management, and road monitoring. Extracting road features from remote sensing images is a hot and frontier issue in the remote sensing field, and it is also one of the most challenging research topics. In view of this, this research systematically reviews the deep learning technology applied to road extraction in remote sensing images and summarizes the existing theories and methods. According to the different annotation types and learning methods, they can be divided into three methods: fully supervised, weakly supervised and unsupervised learning. Then, the datasets and performance evaluation metrics related to road extraction from remote sensing images are summarized, and on this basis, the effects of common road extraction methods are analysed. Finally, suggestions and prospects for the development of road extraction are proposed.
Zusammenfassung
Eine Übersicht über Methoden zur Straßenextraktion in Fernerkundungsbildern basierend auf Deep Learning. Straßeninformationen spielen eine grundlegende Rolle in Anwendungsbereichen wie der Kartenaktualisierung, dem Verkehrsmanagement und der Straßenüberwachung. Die Extraktion von Straßenmerkmalen aus Fernerkundungsbildern ist ein heißes und zukunftsweisendes Thema in der Fernerkundung. In Anbetracht dessen gibt diese Untersuchung einen systematischen Überblick über die Deep-Learning-Technologie, die für die Straßenextraktion in Fernerkundungsbildern eingesetzt wird, und fasst die vorhandenen Theorien und Methoden zusammen. Je nach Art der Lernmethoden können sie in drei Methoden unterteilt werden: vollständig überwachtes, schwach überwachtes und unbeaufsichtigtes Lernen. Anschließend werden die Datensätze und Leistungsbewertungsmetriken im Zusammenhang mit der Straßenextraktion aus Fernerkundungsbildern zusammengefasst, und auf dieser Grundlage werden die Auswirkungen gängiger Straßenextraktionsmethoden analysiert. Schließlich werden Vorschläge und Perspektiven für die Entwicklung der Straßenextraktion gegeben.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdollahi A, Pradhan B, Alamri A (2020a) VNet: an end-to-end fully convolutional neural network for road extraction from high-resolution remote sensing data. IEEE Access 8:179424–179436. https://doi.org/10.1109/ACCESS.2020.3026658
Abdollahi A, Pradhan B, Shukla N, Chakraborty S, Alamri A (2020b) Deep learning approaches applied to remote sensing datasets for road extraction: a state-of-the-art review. Remote Sens Basel 12(9):1444. https://doi.org/10.3390/rs12091444
Abdollahi A, Pradhan B, Alamri A (2021a) RoadVecNet: a new approach for simultaneous road network segmentation and vectorization from aerial and google earth imagery in a complex urban set-up. Gisci Remote Sens. https://doi.org/10.1080/15481603.2021.1972713
Abdollahi A, Pradhan B, Sharma G, Maulud KNA, Alamri A (2021b) Improving road semantic segmentation using generative adversarial network. IEEE Access 9:64381–64392. https://doi.org/10.1109/ACCESS.2021.3075951
Abdollahi A, Pradhan B, Shukla N (2021c) Road extraction from high-resolution orthophoto images using convolutional neural network. J Indian Soc Remote 49(3):569–583. https://doi.org/10.1007/s12524-020-01228-y
Abdollahi A, Pradhan B, Shukla N, Chakraborty S, Alamri A (2021d) Multi-object segmentation in complex urban scenes from high-resolution remote sensing data. Remote Sens Basel 13(18):3710. https://doi.org/10.3390/rs13183710
Abdullahi S, Pradhan B, Jebur MN (2015) GIS-based sustainable city compactness assessment using integration of MCDM, Bayes theorem and RADAR technology. Geocarto Int 30(4):365–387. https://doi.org/10.1080/10106049.2014.911967
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal 34(11):2274–2282. https://doi.org/10.1109/TPAMI.2012.120
Al-Sammaraie MF (2015) Contrast enhancement of roads images with foggy scenes based on histogram equalization. Paper presented at IEEE, Cambridge, UK
Alshaikhli T, Liu W, Maruyama Y (2019) Automated method of road extraction from aerial images using a deep convolutional neural network. Appl Sci 9(22):4825. https://doi.org/10.3390/app9224825
Alshehhi R, Marpu PR, Woon WL, Mura MD (2017) Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks. ISPRS J Photogramm Remote Sens 130:139–149. https://doi.org/10.1016/j.isprsjprs.2017.05.002
Arjovsky M, Bottou L (2017) Towards principled methods for training generative adversarial networks. arXiv preprint arXiv: 1701.04862
Arjovsky M, Chintala S, Bottou L (2017) Wasserstein GAN. arXiv preprint arXiv: 1701.07875
Audebert N, Le Saux B, Lefèvre S (2017) Segment-before-detect: vehicle detection and classification through semantic segmentation of aerial images. Remote Sens Basel 9(4):368. https://doi.org/10.3390/rs9040368
Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder–decoder architecture for image segmentation. IEEE Trans Pattern Anal 39(12):2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615
Bastani F, He S, Abbar S, Alizadeh M, Balakrishnan H, Chawla S, Madden S, Dewitt D. (2018). RoadTracer: automatic extraction of road networks from aerial images. Paper presented at IEEE, Salt Lake City, UT, USA, 18–23 June 2018. https://doi.org/10.1109/CVPR.2018.00496
Chaurasia A, Culurciello E (2017) LinkNet: exploiting encoder representations for efficient semantic segmentation. Paper presented at IEEE, St. Petersburg, FL, USA. https://doi.org/10.1109/VCIP.2017.8305148
Chen L, Papandreou G, Schroff F, Adam H (2017) Rethinking Atrous convolution for semantic image segmentation. arXiv preprint arXiv: 1706.05587
Chen Z, Fan W, Zhong B, Li J, Du J, Wang C (2020) Corse-to-fine road extraction based on local Dirichlet mixture models and multiscale-high-order deep learning. IEEE Trans Intell Transp 21(10):4283–4293. https://doi.org/10.1109/TITS.2019.2939536
Chen D, Zhong Y, Zheng Z, Ma A, Lu X (2021a) Urban road mapping based on an end-to-end road vectorization mapping network framework. Isprs J Photogramm 178:345–365. https://doi.org/10.1016/j.isprsjprs.2021.05.016
Chen Z, Wang C, Li J, Fan W, Du J, Zhong B (2021b) Adaboost-like end-to-end multiple lightweight U-nets for road extraction from optical remote sensing images. Int J Appl Earth Obs Geoinf 100:102341. https://doi.org/10.1016/j.jag.2021.102341
Chen Z, Wang C, Li J, Xie N, Han Y, Du J (2021c) Reconstruction bias U-Net for road extraction from optical remote sensing images. IEEE J STARS 14:2284–2294. https://doi.org/10.1109/JSTARS.2021.3053603
Cheng Z, Fu D (2020) Remote sensing image segmentation method based on HRNET. Paper presented at IEEE, Waikoloa, HI, USA. https://doi.org/10.1109/IGARSS39084.2020.9324289
Cheng J, Ding W, Ku X, Sun J (2012) Road extraction from high-resolution SAR images via automatic local detecting and human-guided global tracking. Int J Antenn Propag 2012:1–10. https://doi.org/10.1155/2012/989823
Cheng G, Wang Y, Xu S, Wang H, Xiang S, Pan C (2017) Automatic road detection and centerline extraction via cascaded end-to-end convolutional neural network. IEEE Trans Geosci Remote Sens 55(6):3322–3337. https://doi.org/10.1109/TGRS.2017.2669341
Cira C, Manso-Callejo M, Alcarria R, Fernández Pareja T, Bordel Sánchez B, Serradilla F (2021) Generative learning for postprocessing semantic segmentation predictions: a lightweight conditional generative adversarial network based on Pix2pix to improve the extraction of road surface areas. Land 10(10):79. https://doi.org/10.3390/land10010079
Clevert D, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (ELUs). arXiv preprint arXiv: 1511.07289
Costea D, Marcu A, Leordeanu M, Slusanschi E (2017) Creating roadmaps in aerial images with generative adversarial networks and smoothing-based optimization. Paper presented at IEEE, Venice, Italy. https://doi.org/10.1109/ICCVW.2017.246
Dai J, Du Y, Zhu T, Wang Y, Gao L (2019) Multiscale residual convolution neural network and sector descriptor-based road detection method. IEEE Access 7:173377–173392. https://doi.org/10.1109/ACCESS.2019.2956725
Dai J, Wang Y, Du Y, Zhu T, Xie S, Li C, Fang X (2020) Development and prospect of road extraction method for optical remote sensing image. Natl Remote Sens Bull 24(7):804–823
Demir I, Koperski K, Lindenbaum D, Pang G, Huang J, Basu S, Hughes F, Tuia D, Raskar R (2018) DeepGlobe 2018: a challenge to parse the earth through satellite images. Paper presented at IEEE, Salt Lake City, UT, USA. https://doi.org/10.1109/CVPRW.2018.00031
Diakogiannis FI, Waldner F, Caccetta P, Wu C (2020) ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data. Isprs J Photogramm 162:94–114. https://doi.org/10.1016/j.isprsjprs.2020.01.013
Ding L, Bruzzone L (2020) DiResNet: direction-aware residual network for road extraction in VHR remote sensing images. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2020.3034011
Eerapu KK, Ashwath B, Lal S, Dell Acqua F, Narasimha Dhan AV (2019) Dense refinement residual network for road extraction from aerial imagery data. IEEE Access 7:151764–151782. https://doi.org/10.1109/ACCESS.2019.2928882
Etten A, Lindenbaum D, Bacastow T (2018) SpaceNet: a remote sensing dataset and challenge series. arXiv preprint arXiv: 1807.01232
Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139. https://doi.org/10.1006/jcss.1997.1504
Frizzelle BG, Evenson KR, Rodriguez DA, Laraia BA (2009) The importance of accurate road data for spatial applications in public health: customizing a road network. Int J Health Geogr 8(1):24. https://doi.org/10.1186/1476-072X-8-24
Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. Paper presented at IEEE, Long Beach, CA, USA. https://doi.org/10.1109/CVPR.2019.00326
Gao L, Song W, Dai J, Chen Y (2019) Road extraction from high-resolution remote sensing imagery using refined deep residual convolutional neural network. Remote Sens Basel 11(5):552. https://doi.org/10.3390/rs11050552
Ge Z, Zhao Y, Wang J, Wang D, Si Q (2021) Deep feature-review transmit network of contour-enhanced road extraction from remote sensing images. IEEE Geosci Remote Sens Lett. https://doi.org/10.1109/LGRS.2021.3061764
Gong Z, Xu L, Tian Z, Bao J, Ming D (2020) Road network extraction and vectorization of remote sensing images based on deep learning. Paper presented at IEEE Chongqing, China. https://doi.org/10.1109/ITOEC49072.2020.9141903
Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Paper presented at the NIPS'14, Cambridge, MA, USA
Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville A (2017) Improved training of Wasserstein GANs. Paper presented at the NIPS'17, Red Hook, NY, USA. https://doi.org/10.5555/3295222.3295327
Hao S, Wang W, Salzmann M (2021) Geometry-aware deep recurrent neural networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 59(3):2448–2460. https://doi.org/10.1109/TGRS.2020.3005623
He K, Zhang X (2016) Identity mappings in deep residual networks. Paper presented at Cham. https://doi.org/10.1007/978-3-319-46493-0_38
He C, Liao Z, Yang F, Deng X, Liao M (2012) Road extraction from SAR imagery based on multiscale geometric analysis of detector responses. IEEE J STARS 5(5):1373–1382. https://doi.org/10.1109/JSTARS.2012.2219614
He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. Paper presented at IEEE, Santiago, Chile. https://doi.org/10.1109/ICCV.2015.123
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Paper presented at IEEE, Las Vegas, NV, USA. https://doi.org/10.1109/CVPR.2016.90
He H, Yang D, Wang S, Wang S, Li Y (2019) Road extraction by using atrous spatial pyramid pooling integrated encoder–decoder network and structural similarity loss. Remote Sens Basel 11(9):1015. https://doi.org/10.3390/rs11091015
He X, Li D, Li P, Hu S, Chen M, Tian Z, Zhou G (2020) Road extraction from high resolution remote sensing images based on EDRNet model. Comput Eng 1–11
Hinton GEAK (2011) Transforming auto-encoders. Paper presented at Berlin, Heidelberg
Hong S, Yeo D, Kwak S, Lee H, Han B (2017) Weakly supervised semantic segmentation using web-crawled videos. Paper presented at IEEE, Honolulu, HI, USA. https://doi.org/10.1109/CVPR.2017.239
Hong Z, Ming D, Zhou K, Guo Y, Lu T (2018) Road extraction from a high spatial resolution remote sensing image based on richer convolutional features. IEEE Access 6:46988–47000. https://doi.org/10.1109/ACCESS.2018.2867210
Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal 42(8):2011–2023. https://doi.org/10.1109/TPAMI.2019.2913372
Hu A, Chen S, Wu L, Xie Z, Qiu Q, Xu Y (2021) WSGAN: an improved generative adversarial network for remote sensing image road network extraction by weakly supervised processing. Remote Sens Basel 13(13):2506. https://doi.org/10.3390/rs13132506
Huang L, Yang Y, Deng Y, Yu Y (2015) DenseBox: unifying landmark localization with end to end object detection. Comput Sci
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. Paper presented at IEEE, Honolulu, HI, USA. https://doi.org/10.1109/CVPR.2017.243
Isola P, Zhu J, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. Paper presented at IEEE, Honolulu, HI, USA. https://doi.org/10.1109/CVPR.2017.632
Jin F, Wang L, Liu Z, Wang F, Jia G (2019) Double U-Uet remote sensing image road extraction method. J Geomat Sci Technol 36(4):377–381
Kahraman I, Turan M, Karaş IR (2015) Road detection from high satellite images using neural networks. Int J Model Optim 5:304–307. https://doi.org/10.7763/IJMO.2015.V5.47
Krähenbühl P, Koltun V (2011) Efficient inference in fully connected CRFs with Gaussian edge potentials. Paper presented at Granada, Spain, 0010-01-10
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Paper presented at the NIPS'12, Red Hook, NY, USA
Lalonde R, Bagci U (2018) Capsules for object segmentation. Paper presented at Amsterdam, The Netherlands, 0004-01-04
Lan M, Zhang Y, Zhang L, Du B (2020) Global context based automatic road segmentation via dilated convolutional neural network. Inf Sci 535:156–171. https://doi.org/10.1016/j.ins.2020.05.062
Lateef F, Ruichek Y (2019) Survey on semantic segmentation using deep learning techniques. Neurocomputing 338:321–348. https://doi.org/10.1016/j.neucom.2019.02.003
Lemarechal C, Fjørtoft R, Marthon P, Cubero-Castan E, Lopes A (1998) SAR image segmentation by morphological methods. In: Proceedings of SPIE—the international society for optical engineering, p 111–122. https://doi.org/10.1117/12.331343
Leordeanu M, Hebert M (2008) Smoothing-based optimization. Paper presented at IEEE, Anchorage, AK, USA. https://doi.org/10.1109/CVPR.2008.4587482
Li M, Stein A, Bijker W, Zhan Q (2016a) Region-based urban road extraction from VHR satellite images using binary partition tree. Int J Appl Earth Obs 44:217–225. https://doi.org/10.1016/j.jag.2015.09.005
Li P, Zang Y, Wang C, Li J, Cheng M, Luo L, Yu Y (2016b) Road network extraction via deep learning and line integral convolution. Paper presented at https://doi.org/10.1109/IGARSS.2016.7729408
Li Y, Guo L, Xu L, Wang X, Jin S (2018) Road recognition based on multi-scale convolutional network with multi-level feature fusion. Paper presented at Chengdu, China, 0005-01-05. https://doi.org/10.1117/12.2524175
Li Y, Guo L, Rao J, Xu L, Jin S (2019a) Road segmentation based on hybrid convolutional network for high-resolution visible remote sensing image. IEEE Geosci Remote Sens Lett 16(4):613–617. https://doi.org/10.1109/LGRS.2018.2878771
Li Y, Peng B, He L, Fan K, Li Z, Tong L (2019b) Road extraction from unmanned aerial vehicle remote sensing images based on improved neural networks. Sensors Basel 19(19):4115. https://doi.org/10.3390/s19194115
Li Y, Peng B, He L, Fan K, Tong L (2019c) Road segmentation of unmanned aerial vehicle remote sensing images using adversarial network with multiscale context aggregation. IEEE J STARS 12(7):2279–2287. https://doi.org/10.1109/JSTARS.2019.2909478
Li Y, Xu L, Rao J, Guo L, Yan Z, Jin S (2019d) A Y-Net deep learning method for road segmentation using high-resolution visible remote sensing images. Remote Sens Lett 10(4):381–390. https://doi.org/10.1080/2150704X.2018.1557791
Li D, Guo H, Zhang B, Zhao C, Lu J, Yu D (2020a) Double vision full convolution network for object extraction in remote sensing imagery. J Image Graph 25(3):535–545
Li D, He X, Li P, Tian Z, Zhou G (2020b) Road extraction network of remote sensing image based on SPUD-ResNet. Comput Eng Appl 1–10
Li X, Wang Y, Zhang L, Liu S, Mei J, Li Y (2020c) Topology-enhanced urban road extraction via a geographic feature-enhanced network. IEEE Trans Geosci Remote Sens 58(12):8819–8830. https://doi.org/10.1109/TGRS.2020.2991006
Lian R, Huang L (2020) DeepWindow: sliding window based on deep learning for road extraction from remote sensing images. IEEE J STARS 13:1905–1916. https://doi.org/10.1109/JSTARS.2020.2983788
Lian R, Huang L (2021) Weakly supervised road segmentation in high-resolution remote sensing images using point annotations. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2021.3059088
Lian R, Wang W, Mustafa N, Huang L (2020) Road extraction methods in high-resolution remote sensing images: a comprehensive review. IEEE J STARS 13:5489–5507. https://doi.org/10.1109/JSTARS.2020.3023549
Lin D, Dai J, Jia J, He K, Sun J (2016) ScribbleSup: scribble-supervised convolutional networks for semantic segmentation. Paper presented at IEEE, Las Vegas, NV, USA. https://doi.org/10.1109/CVPR.2016.344
Lin Y, Xu D, Wang N, Shi Z, Chen Q (2020) Road extraction from very-high-resolution remote sensing images via a nested SE-deeplab model. Remote Sens Basel 12(18):2985. https://doi.org/10.3390/rs12182985
Liu H, Wang X (2019) Remote sensing image segmentation model based on attention mechanism. Laser Optoelectron Progr 57(04):170–180
Liu B, Wu H, Wang Y, Liu W (2015) Main road extraction from ZY-3 grayscale imagery based on directional mathematical morphology and VGI prior knowledge in urban areas. PLoS ONE 10:e138071. https://doi.org/10.1371/journal.pone.0138071
Liu R, Lehman J, Molino P, Such FP, Frank E, Sergeev A, Yosinski J (2018) An intriguing failing of convolutional neural networks and the CoordConv solution. Paper presented at the NIPS'18, Red Hook, NY, USA
Liu R, Miao Q, Song J, Quan Y, Li Y, Xu P, Dai J (2019a) Multiscale road centerlines extraction from high-resolution aerial imagery. Neurocomputing 329:384–396. https://doi.org/10.1016/j.neucom.2018.10.036
Liu Y, Cheng M, Hu X, Bian J, Zhang L, Bai X, Tang J (2019b) Richer convolutional features for edge detection. IEEE Trans Pattern Anal 41(8):1939–1946. https://doi.org/10.1109/TPAMI.2018.2878849
Liu Y, Yao J, Lu X, Xia M, Wang X, Liu Y (2019c) RoadNet: learning to comprehensively analyze road networks in complex urban scenes from high-resolution remotely sensed images. IEEE Trans Geosci Remote Sens 57(4):2043–2056. https://doi.org/10.1109/TGRS.2018.2870871
Liu J, Lin H, Yang L, Xu B, Wen D (2020) Multi-element hierarchical attention capsule network for stock prediction. IEEE Access 8:143114–143123. https://doi.org/10.1109/ACCESS.2020.3014506
Lu X, Zhong Y, Zhao J (2019a) Multi-scale enhanced deep network for road detection. Paper presented at IEEE, Yokohama, Japan. https://doi.org/10.1109/IGARSS.2019.8899115
Lu X, Zhong Y, Zheng Z, Liu Y, Zhao J, Ma A, Yang J (2019b) Multi-scale and multi-task deep learning framework for automatic road extraction. IEEE Trans Geosci Remote Sens 57(11):9362–9377. https://doi.org/10.1109/TGRS.2019.2926397
Lu X, Zhong Y, Zheng Z, Zhao J, Zhang L (2020) Edge-reinforced convolutional neural network for road detection in very-high-resolution remote sensing imagery. Photogramm Eng Remote Sens 86:153–160
Luc P, Couprie C, Chintala S, Verbeek J (2016) Semantic segmentation using adversarial networks. arXiv preprint arXiv: 1611.08408
Ma T, Tan H, Li T, Wu Y, Liu Q (2020a) Road extraction method from GF-1 remote sensing images based on dilated convolution residual network with multi-scale feature fusion. Laser Optoelectron Progr 58:0228001
Ma X, Zhong H, Li Y, Ma J, Cui Z, Wang Y (2020b) Forecasting transportation network speed using deep capsule networks with nested LSTM models. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2020.2984813
Manandhar P, Marpu PR, Aung Z, Melgani F (2019) Towards automatic extraction and updating of VGI-based road networks using deep learning. Remote Sens Basel 11(9):1012. https://doi.org/10.3390/rs11091012
Marcu A, Leordeanu M (2016) Dual local-global contextual pathways for recognition in aerial imagery. arXiv preprint arXiv: 1605.05462
Maurya R, Gupta PR, Shukla AS (2011) Road extraction using K-means clustering and morphological operations. Paper presented at IEEE, Shimla, India
Milletari F, Navab N, Ahmadi S (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. Paper presented at IEEE, Stanford, CA, USA. https://doi.org/10.1109/3DV.2016.79
Mnih V (2013) Machine learning for aerial image labeling. PhD thesis, University of Toronto, Toronto, ON, Canada
Mnih V, Hinton GE (2010) Learning to detect roads in high-resolution aerial images. Paper presented at Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15567-3_16
Panboonyuen T, Jitkajornwanich K, Lawawirojwong S, Srestasathiern P, Vateekul P (2017) Road segmentation of remotely-sensed images using deep convolutional neural networks with landscape metrics and conditional random fields. Remote Sens Basel 9(7):680. https://doi.org/10.3390/rs9070680
Paoletti ME, Haut JM, Fernandez-Beltran R, Plaza J, Plaza A, Li J, Pla F (2019) Capsule networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 57(4):2145–2160. https://doi.org/10.1109/TGRS.2018.2871782
Perciano T, Tupin F Jr, Hirata R Jr, Cesar RM (2016) A two-level Markov random field for road network extraction and its application with optical, SAR, and multitemporal data. Int J Remote Sens 37(16):3584–3610. https://doi.org/10.1080/01431161.2016.1201227
Pinheiro P, Collobert R (2014) Recurrent convolutional neural networks for scene parsing. Paper presented at Beijing, China, 0006-01-06
Qi K, Liu W, Yang C, Guan Q, Wu H (2017) Multi-task joint sparse and low-rank representation for the scene classification of high-resolution remote sensing image. Remote Sens Basel 9(1):10. https://doi.org/10.3390/rs9010010
Qi X, Li K, Liu P, Zhou X, Sun M (2020) Deep attention and multi-scale networks for accurate remote sensing image segmentation. IEEE Access 8:146627–146639. https://doi.org/10.1109/ACCESS.2020.3015587
Ren Y, Yu Y, Guan H (2020) DA-CapsUNet: a dual-attention capsule u-net for road extraction from remote sensing imagery. Remote Sens Basel 12(18):2866. https://doi.org/10.3390/rs12182866
Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. arXiv preprint arXiv: 1505.04597
Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Paper presented at the NIPS'17, Red Hook, NY, USA
Saito S, Aoki Y (2015) Building and road detection from large aerial imagery. Paper presented at San Francisco, California, United States. https://doi.org/10.1117/12.2083273
Saito S, Yamashita Y, Aoki Y (2016) Multiple object extraction from aerial imagery with convolutional neural networks. J Imaging Sci Technol 60:104021–104029. https://doi.org/10.2352/J.ImagingSci.Technol.2016.60.1.010402
Sghaier MO, Lepage R (2016) Road extraction from very high resolution remote sensing optical images based on texture analysis and beamlet transform. IEEE J STARS 9(5):1946–1958. https://doi.org/10.1109/JSTARS.2015.2449296
Shamsolmoali P, Zareapoor M, Zhou H, Wang R, Yang J (2020) Road segmentation for remote sensing images using adversarial spatial pyramid networks. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2020.3016086
Shao Z, Zhou Z, Huang X, Zhang Y (2021) MRENet: simultaneous extraction of road surface and road centerline in complex urban scenes from very high-resolution images. Remote Sens Basel 13(2):239. https://doi.org/10.3390/rs13020239
Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal 39(4):640–651. https://doi.org/10.1109/TPAMI.2016.2572683
Shen Z, Liu Z, Li J, Jiang Y, Chen Y, Xue X (2017) DSOD: learning deeply supervised object detectors from scratch. Paper presented at IEEE, Venice, Italy. https://doi.org/10.1109/ICCV.2017.212
Shi Q, Liu X, Li X (2018) Road detection from remote sensing images by generative adversarial networks. IEEE Access 6:25486–25494. https://doi.org/10.1109/ACCESS.2017.2773142
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Comput Sci
Singh P, Dash R (2019) A two-step deep convolution neural network for road extraction from aerial images. Paper presented at Noida, India, 0003-01-03. https://doi.org/10.1109/SPIN.2019.8711639
Singh S, Krishnan S (2020) Filter response normalization layer: eliminating batch dependence in the training of deep neural networks. Paper presented at IEEE, Seattle, WA, USA. https://doi.org/10.1109/CVPR42600.2020.01125
Song J, Li J, Chen H, Wu J (2021a) MapGen-GAN: a fast translator for remote sensing image to map via unsupervised adversarial learning. IEEE J STARS 14:2341–2357. https://doi.org/10.1109/JSTARS.2021.3049905
Song T, Liu T, Zong D, Jiang X, Huang T, Fan H (2021b) Research on road extraction method from remote sensing images based on improved U-net network. Computer Eng Appl 1–12
Sujatha C, Selvathi D (2015) Connected component-based technique for automatic extraction of road centerline in high resolution satellite images. EURASIP J Image Video Process 2015(1):8. https://doi.org/10.1186/s13640-015-0062-9
Sun T, Chen Z, Yang W, Wang Y (2018) Stacked U-Nets with multi-output for road extraction. Paper presented at IEEE, Salt Lake City, UT, USA. https://doi.org/10.1109/CVPRW.2018.00033
Tan X, Xiao Z, Wan Q, Shao W (2021) Scale sensitive neural network for road segmentation in high-resolution remote sensing images. IEEE Geosci Remote Sens Lett 18(3):533–537. https://doi.org/10.1109/LGRS.2020.2976551
Tao Y, Xu M, Zhang F, Du B, Zhang L (2017a) Unsupervised-restricted deconvolutional neural network for very high resolution remote-sensing image classification. IEEE Trans Geosci Remote Sens 55(12):6805–6823. https://doi.org/10.1109/TGRS.2017.2734697
Tao Y, Xu M, Zhong Y, Cheng Y (2017b) GAN-assisted two-stream neural network for high-resolution remote sensing image classification. Remote Sens Basel 9(12):1328. https://doi.org/10.3390/rs9121328
Tao C, Qi J, Li Y, Wang H, Li H (2019) Spatial information inference net: road extraction using road-specific contextual information. ISPRS J Photogramm Remote Sens 158:155–166. https://doi.org/10.1016/j.isprsjprs.2019.10.001
Tian Z, He T, Shen C, Yan Y (2019) Decoders matter for semantic segmentation: data-dependent decoding enables flexible feature aggregation. Paper presented at IEEE, Long Beach, CA, USA. https://doi.org/10.1109/CVPR.2019.00324
Toth C, Jóźków G (2016) Remote sensing platforms and sensors: a survey. ISPRS J Photogramm Remote Sens 115:22–36. https://doi.org/10.1016/j.isprsjprs.2015.10.004
Tunde A, Adeniyi E (2012) Impact of road transport on agricultural development: a Nigerian example. Ethiop J Environ Stud Manag 5(3):232–238. https://doi.org/10.4314/ejesm.v5i3.3
Varia N, Dokania A, Senthilnath J (2018) DeepExt: a convolution neural network for road extraction using RGB images captured by UAV. Paper presented at IEEE, Bangalore, India. https://doi.org/10.1109/SSCI.2018.8628717
Wan J, Xie Z, Xu Y, Chen S, Qiu Q (2021) DA-RoadNet: a dual-attention network for road extraction from high resolution satellite imagery. IEEE J STARS 14:6302–6315. https://doi.org/10.1109/JSTARS.2021.3083055
Wang J, Song J, Chen M, Yang Z (2015) Road network extraction: a neural-dynamic framework based on deep learning and a finite state machine. Int J Remote Sens 36(12):3144–3169. https://doi.org/10.1080/01431161.2015.1054049
Wang W, Yang N, Zhang Y, Wang F, Cao T, Eklund P (2016) A review of road extraction from remote sensing images. J Traffic Transp Eng (English Ed) 3(3):271–282. https://doi.org/10.1016/j.jtte.2016.05.005
Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. Paper presented at IEEE, Lake Tahoe, NV, USA. https://doi.org/10.1109/WACV.2018.00163
Wang B, Qi G, Tang S, Zhang T, Wei Y, Li L, Zhang Y (2019a) Boundary perception guidance: a scribble-supervised semantic segmentation approach. Paper presented at Macao, China, 0008-01-08. https://doi.org/10.24963/ijcai.2019/508
Wang J, Ke S, Jiang B, Deng C, Zhao Y, Liu D, Mu Y, Tan M, Wang X, Liu W, Xiao B (2019b) Deep high-resolution representation learning for visual recognition. arXiv preprint arXiv: 1908.07919
Wang S, Yang H, Wu Q, Zheng Z, Wu Y, Li J (2020) An improved method for road extraction from high-resolution remote-sensing images that enhances boundary information. Sensors Basel 20(7):2064. https://doi.org/10.3390/s20072064
Wang S, Mu X, Yang D, He H, Zhao P (2021) Road extraction from remote sensing images using the inner convolution integrated encoder–decoder network and directional conditional random fields. Remote Sens Basel 13(3):465. https://doi.org/10.3390/rs13030465
Wegner J, Montoya-Zegarra JA, Schindler K (2013) A higher-order CRF model for road network extraction. Paper presented at IEEE, Los Alamitos, CA, USA, 0006-01-06. https://doi.org/10.1109/CVPR.2013.222
Wegner JD, Montoya-Zegarra JA, Schindler K (2015) Road networks as collections of minimum cost paths. ISPRS J Photogramm Remote Sens 108:128–137. https://doi.org/10.1016/j.isprsjprs.2015.07.002
Wei Y, Wang Z, Xu M (2017) Road structure refined CNN for road extraction in aerial image. IEEE Geosci Remote Sens Lett 14(5):709–713. https://doi.org/10.1109/LGRS.2017.2672734
Wei Y, Ji S (2021) Scribble-based weakly supervised deep learning for road surface extraction from remote sensing images. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2021.3061213
Wei Y, Zhang K, Ji S (2020) Simultaneous road surface and centerline extraction from large-scale remote sensing images using CNN-based segmentation and tracing. IEEE Trans Geosci Remote Sens 58(12):8919–8931. https://doi.org/10.1109/TGRS.2020.2991733
Wu Y, He K (2020) Group normalization. Int J Comput vis 128(3):742–755. https://doi.org/10.1007/s11263-019-01198-w
Wu S, Du C, Chen H, Xu Y, Guo N, Jing N (2019) Road extraction from very high resolution images using weakly labeled OpenStreetMap centerline. ISPRS Int J Geo Inf 8(11):478. https://doi.org/10.3390/ijgi8110478
Wu Q, Luo F, Wu P, Wang B, Yang H, Wu Y (2021) Automatic road extraction from high-resolution remote sensing images using a method based on densely connected spatial feature-enhanced pyramid. IEEE J STARS 14:3–17. https://doi.org/10.1109/JSTARS.2020.3042816
Wulamu A, Shi Z, Zhang D, He Z (2019) Multiscale road extraction in remote sensing images. Comput Intell Neurosci 2019:1–9. https://doi.org/10.1155/2019/2373798
Xiao D, Yin L, Fu Y (2021) Open-pit mine road extraction from high-resolution remote sensing images using RATT-UNet. IEEE Geosci Remote Sens Lett. https://doi.org/10.1109/LGRS.2021.3065148
Xie S, Tu Z (2017) Holistically-nested edge detection. Int J Comput vis 125(1):3–18. https://doi.org/10.1007/s11263-017-1004-z
Xie Y, Miao F, Zhou K, Peng J (2019) HsgNet: a road extraction network based on global perception of high-order spatial information. ISPRS Int J Geo Inf 8(12):571. https://doi.org/10.3390/ijgi8120571
Xin J, Zhang X, Zhang Z, Fang W (2019) Road extraction of high-resolution remote sensing images derived from DenseUNet. Remote Sens Basel 11(21):2499. https://doi.org/10.3390/rs11212499
Xu Y, Feng Y, Xie Z, Hu A, Zhang X (2018a) A research on extracting road network from high resolution remote sensing imagery. Paper presented at IEEE, Kunming, China. https://doi.org/10.1109/GEOINFORMATICS.2018.8557042
Xu Y, Xie Z, Feng Y, Chen Z (2018b) Road extraction from high-resolution remote sensing imagery using deep learning. Remote Sens Basel 10(9):1461. https://doi.org/10.3390/rs10091461
Xu Q, Wang D, Luo B (2021a) Faster multiscale capsule network with octave convolution for hyperspectral image classification. IEEE Geosci Remote Sens Lett 18(2):361–365. https://doi.org/10.1109/LGRS.2020.2970079
Xu Z, Shen Z, Li Y, Xia L, Wang H, Li S, Jiao S, Lei Y (2021b) Road extraction in mountainous regions from high-resolution images based on DSDNet and terrain optimization. Remote Sens Basel 13(1):90. https://doi.org/10.3390/rs13010090
Yang C, Wang Z (2020) An ensemble Wasserstein generative adversarial network method for road extraction from high resolution remote sensing images in rural areas. IEEE Access 8:174317–174324. https://doi.org/10.1109/ACCESS.2020.3026084
Yang X, Li X, Ye Y, Lau RYK, Zhang X, Huang X (2019a) Road detection and centerline extraction via deep recurrent convolutional neural network U-Net. IEEE Trans Geosci Remote Sens 57(9):7209–7220. https://doi.org/10.1109/TGRS.2019.2912301
Yang X, Li X, Ye Y, Zhang X, Zhang H, Huang X, Zhang B (2019b) Road detection via deep residual dense U-Net. Paper presented at IEEE, Budapest, Hungary. https://doi.org/10.1109/IJCNN.2019.8851728
Youssef AM, Sefry SA, Pradhan B, Alfadail EA (2016) Analysis on causes of flash flood in Jeddah city (Kingdom of Saudi Arabia) of 2009 and 2011 using multi-sensor remote sensing data and GIS. Geomat Nat Hazards Risk 7(3):1018–1042. https://doi.org/10.1080/19475705.2015.1012750
Yu C, Wang J (2018) BiSeNet: bilateral segmentation network for real-time semantic segmentation. Paper presented at Cham
Yu Y, Gu T, Guan H, Li D, Jin S (2019) Vehicle detection from high-resolution remote sensing imagery using convolutional capsule networks. IEEE Geosci Remote Sens Lett 16(12):1894–1898. https://doi.org/10.1109/LGRS.2019.2912582
Zhang Z, Wang Y (2019) JointNet: a common neural network for road and building extraction. Remote Sens Basel 11(6):696. https://doi.org/10.3390/rs11060696
Zhang J, Chen L, Wang C, Zhuo L, Tian Q, Liang X (2017) Road recognition from remote sensing imagery using incremental learning. IEEE Trans Intell Transp Syst 18(11):2993–3005. https://doi.org/10.1109/TITS.2017.2665658
Zhang X, Zhou X, Lin M, Sun J (2018a) ShuffleNet: an extremely efficient convolutional neural network for mobile devices. Paper presented at IEEE, Salt Lake City, UT, USA. https://doi.org/10.1109/CVPR.2018.00716
Zhang Y, He J, Kan X, Xia G, Zhu L, Ge T (2018b) Summary of road extraction methods for remote sensing images. Comput Eng Appl 54(13):1–10
Zhang Z, Liu Q, Wang Y (2018c) Road extraction by deep residual U-Net. IEEE Geosci Remote Sens Lett 15(5):749–753. https://doi.org/10.1109/LGRS.2018.2802944
Zhang X, Han X, Li C, Tang X, Zhou H, Jiao L (2019a) Aerial image road extraction based on an improved generative adversarial network. Remote Sens Basel 11(8):930. https://doi.org/10.3390/rs11080930
Zhang Y, Xiong Z, Zang Y, Wang C, Li J, Li X (2019b) Topology-aware road network extraction via multi-supervised generative adversarial networks. Remote Sens Basel 11(9):1017. https://doi.org/10.3390/rs11091017
Zhang J, Yu X, Li A, Song P, Liu B, Dai Y (2020a) Weakly-supervised salient object detection via scribble annotations. Paper presented at IEEE, Seattle, WA, USA. https://doi.org/10.1109/CVPR42600.2020.01256
Zhang Y, Zhu Q, Zhong Y, Guan Q, Zhang L, Li D (2020b) A modified D-linknet with transfer learning for road extraction from high-resolution remote sensing. Paper presented at IEEE, Waikoloa, HI, USA. https://doi.org/10.1109/IGARSS39084.2020.9324236
Zhang J, Hu Q, Li J, Ai M (2021) Learning from GPS trajectories of floating car for CNN-based urban road extraction with high-resolution satellite imagery. IEEE Trans Geosci Remote Sens 59(3):1836–1847. https://doi.org/10.1109/TGRS.2020.3003425
Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. Paper presented at IEEE, Honolulu, HI, USA. https://doi.org/10.1109/CVPR.2017.660
Zhe G, Li X, Zhenpo T, Jingyuan B, Delie M (2020) Road network extraction and vectorization of remote sensing images based on deep learning. Paper presented at IEEE, Chongqing, China. https://doi.org/10.1109/ITOEC49072.2020.9141903
Zhong Z, Li J, Cui W, Jiang H (2016) Fully convolutional networks for building and road extraction: preliminary results. Paper presented at IEEE, Beijing, China. https://doi.org/10.1109/IGARSS.2016.7729406
Zhou M, Sui H, Chen S, Wang J, Chen X (2020) BT-RoadNet: a boundary and topologically-aware neural network for road extraction from high-resolution remote sensing imagery. ISPRS J Photogramm Remote Sens 168:288–306. https://doi.org/10.1016/j.isprsjprs.2020.08.019
Zhou K, Xie Y, Gao Z, Miao F, Zhang L (2021) FuNet: a novel road extraction network with fusion of location data and remote sensing imagery. ISPRS Int J Geo Inf 10(1):39. https://doi.org/10.3390/ijgi10010039
Zhou L, Zhang C, Wu M (2018) D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. Paper presented at IEEE:,Salt Lake City, UT, USA. https://doi.org/10.1109/CVPRW.2018.00034
Zhu Y, Yan J, Wang C, Zhou Y (2019) Road detection of remote sensing image based on convolutional neural network. Paper presented at the image and graphics, Cham, 0011-01-11. https://doi.org/10.1007/978-3-030-34110-7_10
Zhu Q, Zhang Y, Wang L, Zhong Y, Guan Q, Lu X, Zhang L, Li D (2021) A global context-aware and batch-independent network for road extraction from VHR satellite imagery. ISPRS J Photogramm Remote Sens 175:353–365. https://doi.org/10.1016/j.isprsjprs.2021.03.016
Funding
This work was sponsored by the National Key R&D Program of China (grant number: 2020YFD1100201) and the Postgraduate Research & Practice Innovation Program of Jiangsu Province (grant number: SJCX21_0040).
Author information
Authors and Affiliations
Contributions
PL conducted the research. QW revised the paper and guided the research, GY, LL and PL were responsible for collecting data and creating the figures. GY, HZ, QW, and PL revised and improved the paper. All authors read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflicts of interest.
Rights and permissions
About this article
Cite this article
Liu, P., Wang, Q., Yang, G. et al. Survey of Road Extraction Methods in Remote Sensing Images Based on Deep Learning. PFG 90, 135–159 (2022). https://doi.org/10.1007/s41064-022-00194-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41064-022-00194-z