Skip to main content
SpringerLink
Log in

A hybrid deep learning–based model for automatic car extraction from high-resolution airborne imagery

  • Original Paper
  • Published:
Applied Geomatics Aims and scope Submit manuscript

Abstract

Automatic car extraction (ACE) from high-resolution airborne imagery (i.e., true-orthophoto) has been a hot research topic in the field of photogrammetry and machine learning. ACE from high-resolution airborne imagery is the most suitable method for control and monitoring practices in large cities such as traffic management. The use of deep learning–based feature extraction methods, such as convolutional neural networks, have been providing state-of-the-art performance in the last few years, particularly, these techniques have been successfully applied to automatic object extraction from images. In this paper, we proposed a novel hybrid method to take advantage of the semantic segmentation of high-resolution airborne imagery to ACE that is realized based on the combination of deep convolutional neural networks and restricted Boltzmann machine (RBM). This hybrid method is called RBMDeepNet. We trained and tested our model on the ISPRS Potsdam and Vaihingen benchmark datasets (non-big data) which is more challenging for ACE. Here, Potsdam data is a true-color dataset, and Vaihingen data is a false-color dataset. The results obtained in the present study showed that the proposed method for ACE from high-resolution airborne imagery achieves a 7% improvement in accuracy with about 10% improvement in processing time compared to similar methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Ahn J, Kwak S (2018) Learning Pixel-Level Semantic Affinity with Image-Level Supervision for Weakly Supervised Semantic Segmentation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Salt Lake City, pp 4981–4990

    Chapter  Google Scholar 

  • Akhtar N, Choubey NS, Ragavendran U (2019) Investigation of Non-natural Information from Remote Sensing Images: A Case Study Approach. In: Anandakumar H, Arulmurugan R, Onn CC (eds) Computational Intelligence and Sustainable Systems: Intelligence and Sustainable Computing. Springer International Publishing, Cham, pp 165–199

    Chapter  Google Scholar 

  • Alani A (2017) Arabic handwritten digit recognition based on restricted Boltzmann machine and convolutional neural networks. Information 8:142. https://doi.org/10.3390/info8040142

    Article  Google Scholar 

  • Arunmozhi A, Park J (2018) Comparison of HOG, LBP and Haar-like features for on-road vehicle detection. In: 2018 IEEE International Conference on Electro/Information Technology (EIT) pp 0362–0367

  • Audebert N, Boulch A, Randrianarivo H et al (2017a) Deep learning for urban remote sensing. In: 2017 Joint Urban Remote Sensing Event (JURSE). IEEE, Dubai, pp 1–4

    Google Scholar 

  • Audebert N, Le Saux B, Lefèvre S (2017b) Segment-before-detect: vehicle detection and classification through semantic segmentation of aerial images. Remote Sens 9:368. https://doi.org/10.3390/rs9040368

    Article  Google Scholar 

  • Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39:2481–2495. https://doi.org/10.1109/tpami.2016.2644615

    Article  Google Scholar 

  • Ball JE, Anderson DT, Chan CS (2017) A comprehensive survey of deep learning in remote sensing: theories, tools and challenges for the community. J Appl Remote Sens 11(1). https://doi.org/10.1117/1.JRS.11.042609

  • Boulila W (2019) A top-down approach for semantic segmentation of big remote sensing images. Earth Sci Inf. https://doi.org/10.1007/s12145-018-00376-7

  • Cha Y-J, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O (2018) Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types: autonomous SHM using deep faster R-CNN. Comput Aided Civ Inf Eng 33:731–747. https://doi.org/10.1111/mice.12334

    Article  Google Scholar 

  • Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 801–818

  • Choi J, Sung K, Yang Y (2007) Multiple Vehicles Detection and Tracking based on Scale-Invariant Feature Transform. In: 2007 IEEE Intelligent Transportation Systems Conference. pp 528–533

  • Chollet F (2018) Deep learning with Python. Manning Publications Co, Shelter Island, New York

    Google Scholar 

  • Erasu D, change L cover, areas U, detection C (2017) Remote sensing-based urban land use/land cover change detection and monitoring. J Remote Sens GIS 06: doi: https://doi.org/10.4172/2469-4134.1000196

  • Fu G, Liu C, Zhou R, Sun T, Zhang Q (2017) Classification for high resolution remote sensing imagery using a fully convolutional network. Remote Sens 9:498. https://doi.org/10.3390/rs9050498

    Article  Google Scholar 

  • Ghaffarian S, Gökasar I (2015) Traffic density measurement by automatic detection of vehicles using gradient vectors from aerial images. Int J Civ Environ Eng 9:5

    Google Scholar 

  • Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, Zhang T, Gao S, Liu J (2019) CE-Net: context encoder network for 2D medical image segmentation. IEEE Trans Med Imaging:1–1. https://doi.org/10.1109/TMI.2019.2903562

  • Hinz S, Baumgartner A (2003) Automatic extraction of urban road networks from multi-view aerial imagery. ISPRS J Photogramm Remote Sens 58:83–98

    Article  Google Scholar 

  • Huang J, Zhang X, Xin Q, Sun Y, Zhang P (2019) Automatic building extraction from high-resolution aerial images and LiDAR data using gated residual refinement network. ISPRS J Photogramm Remote Sens 151:91–105. https://doi.org/10.1016/j.isprsjprs.2019.02.019

    Article  Google Scholar 

  • Kamal S, Jalal A (2015) A hybrid feature extraction approach for human detection, tracking and activity recognition using depth sensors. Arab J Sci Eng 41:1043–1051. https://doi.org/10.1007/s13369-015-1955-8

    Article  Google Scholar 

  • Karimi Nejadasl F, Gorte BGH, Hoogendoorn SP (2006) Optical flow based vehicle tracking strengthened by statistical decisions. ISPRS J Photogramm Remote Sens 61:159–169. https://doi.org/10.1016/j.isprsjprs.2006.09.007

    Article  Google Scholar 

  • Keronen S, Cho K, Raiko T, et al (2013) Gaussian-Bernoulli restricted Boltzmann machines and automatic feature extraction for noise robust missing data mask estimation. 2013 IEEE Int Conf Acoust Speech Signal Process. doi: https://doi.org/10.1109/icassp.2013.6638964

  • Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. 3rd Int Conf Learn Represent ICLR

  • Li Z, Chen Q, Koltun V (2018) Interactive Image Segmentation with Latent Diversity. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Salt Lake City, pp 577–585

    Chapter  Google Scholar 

  • Maggiori E, Tarabalka Y, Charpiat G, Alliez P (2017) Convolutional neural networks for large-scale remote-sensing image classification. IEEE Trans Geosci Remote Sens 55:645–657. https://doi.org/10.1109/TGRS.2016.2612821

    Article  Google Scholar 

  • Masouleh MK, Sadeghian S (2019) Deep learning-based method for reconstructing three-dimensional building cadastre models from aerial images. J Appl Remote Sens 13:024508. https://doi.org/10.1117/1.JRS.13.024508

    Article  Google Scholar 

  • Masouleh MK, Shah-Hosseini R (2018) Fusion of deep learning with adaptive bilateral filter for building outline extraction from remote sensing imagery. J Appl Remote Sens 12(1). https://doi.org/10.1117/1.JRS.12.046018

  • Mou L, Zhu XX (2018) Vehicle instance segmentation from aerial image and video using a multitask learning residual fully convolutional network. IEEE Trans Geosci Remote Sens 56:6699–6711. https://doi.org/10.1109/tgrs.2018.2841808

    Article  Google Scholar 

  • Munoz-Organero M, Ruiz-Blaquez R, Sánchez-Fernández L (2018) Automatic detection of traffic lights, street crossings and urban roundabouts combining outlier detection and deep learning classification techniques based on GPS traces while driving. Comput Environ Urban Syst 68:1–8. https://doi.org/10.1016/j.compenvurbsys.2017.09.005

    Article  Google Scholar 

  • Narayanan P, Borel-Donohue C, Lee H, et al (2018) A real-time object detection framework for aerial imagery using deep neural networks and synthetic training images. Signal Process SensorInformation Fusion Target Recognit XXVII doi: https://doi.org/10.1117/12.2306154

  • Palubinskas G, Kurz F, Reinartz P (2008) Detection of Traffic Congestion in Optical Remote Sensing Imagery. In: IGARSS 2008–2008 IEEE International Geoscience and Remote Sensing Symposium. pp II-426-II–429

  • Pang S, Yang X (2016) Deep convolutional extreme learning machine and its application in handwritten digit classification. Comput Intell Neurosci 2016:1–10. https://doi.org/10.1155/2016/3049632

    Article  Google Scholar 

  • Reinartz P, Lachaise M, Schmeer E, Krauss T, Runge H (2006) Traffic monitoring with serial images from airborne cameras. ISPRS J Photogramm Remote Sens 61:149–158. https://doi.org/10.1016/j.isprsjprs.2006.09.009

    Article  Google Scholar 

  • Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Lect Notes Comput Sci Subser Lect Notes Artif Intell Lect Notes Bioinforma 9351:234–241. https://doi.org/10.1007/978-3-319-24574-4_28

    Article  Google Scholar 

  • Salakhutdinov R, Hinton G (2009) Deep Boltzmann Machines. In: Artificial Intelligence and Statistics. pp 448–455

  • Shah-Hosseini R, Safari A, Homayouni S (2017) Natural hazard damage detection based on object-level support vector data description of optical and SAR Earth observations. Int J Remote Sens 38:3356–3374. https://doi.org/10.1080/01431161.2017.1294777

    Article  Google Scholar 

  • Solimini D (2016) Understanding Earth Observation: The Electromagnetic Foundation of Remote Sensing, 1st edn. 2016 edition. Springer, New York

    Book  Google Scholar 

  • Xu Y, Wu L, Xie Z, Chen Z (2018) Building extraction in very high resolution remote sensing imagery using deep learning and guided filters. Remote Sens 10:144. https://doi.org/10.3390/rs10010144

    Article  Google Scholar 

  • Yan G, Yu M, Yu Y, Fan L (2016) Real-time vehicle detection using histograms of oriented gradients and AdaBoost classification. Optik 127:7941–7951. https://doi.org/10.1016/j.ijleo.2016.05.092

    Article  Google Scholar 

  • Yang M, Yu K, Zhang C et al (2018) DenseASPP for Semantic Segmentation in Street Scenes. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Salt Lake City, pp 3684–3692

    Google Scholar 

  • Zhu XX, Tuia D, Mou L, Xia GS, Zhang L, Xu F, Fraundorfer F (2017) Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geosci Remote Sens Mag 5:8–36. https://doi.org/10.1109/MGRS.2017.2762307

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank all the developers of Keras for providing such open-source and powerful deep learning library. We thank the International Society for Photogrammetry and Remote Sensing for making the Potsdam, and Vaihingen dataset. Finally, we would like to thank the three anonymous referees for their helpful comments.

Author information

Authors and Affiliations

  1. School of Surveying and Geospatial Engineering, College of Engineering, University of Tehran, Tehran, Iran

    Mehdi Khoshboresh Masouleh & Reza Shah-Hosseini

Authors
  1. Mehdi Khoshboresh Masouleh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reza Shah-Hosseini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Reza Shah-Hosseini.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khoshboresh Masouleh, M., Shah-Hosseini, R. A hybrid deep learning–based model for automatic car extraction from high-resolution airborne imagery. Appl Geomat 12, 107–119 (2020). https://doi.org/10.1007/s12518-019-00285-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12518-019-00285-4

Keywords

Search

Navigation