A top-down approach for semantic segmentation of big remote sensing images

Abstract

The increasing amount of remote sensing data has opened the door to new challenging research topics. Nowadays, significant efforts are devoted to pixel and object based classification in case of massive data. This paper addresses the problem of semantic segmentation of big remote sensing images. To do this, we proposed a top-down approach based on two main steps. The first step aims to compute features at the object-level. These features constitute the input of a multi-layer feed-forward network to generate a structure for classifying remote sensing objects. The goal of the second step is to use this structure to label every pixel in new images. Several experiments are conducted based on real datasets and results show good classification accuracy of the proposed approach. In addition, the comparison with existing classification techniques proves the effectiveness of the proposed approach especially for big remote sensing data.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Notes

  1. 1.

    http://kalideos.cnes.fr.

  2. 2.

    Centre National d’Etudes Spatiales – Distribution Airbus DS.

References

  1. [nprtool, 18] (2018) https://www.mathworks.com/help/nnet/ref/nprtool.html, [Accessed: 08- Jan- 2018]

  2. Akcay HG, Aksoy S (2008) Automatic detection of geospatial objects using multiple hierarchical segmentations. IEEE Trans Geosci Remote Sens 46(7):2097–2111

    Google Scholar 

  3. Andrés S, Arvor D, Mougenot I, Libourel T, Durieux L (2017) Ontology-based classification of remote sensing images using spectral rules. Comput Geosci 102:158–166

    Google Scholar 

  4. Ashwini Reddy T, Renuka Devi K., Gangashetty SV (2011) Multilayer Feedforward Neural Network Models for Pattern Recognition Tasks in Earthquake Engineering, International Conference on Advanced Computing, Networking and Security, pp. 154–162

  5. Athanasiadis T, Mylonas P, Avrithis Y, Kollias S (2007) Semantic image segmentation and object labeling. IEEE Trans Circuits Syst Video Technol 17(3):298–312

    Google Scholar 

  6. Boulila W, Farah IR, Saheb Ettabaa K, Solaiman B, Ben Ghézala H (2009) Improving spatiotemporal change detection: a high level fusion approach for discovering uncertain knowledge from satellite image databases. ICDM 58:222–227

    Google Scholar 

  7. Boulila W, Farah IR, Saheb Ettabaa K, Solaiman B, Ben Ghézala H (2010) Spatio-temporal modeling for knowledge discovery in satellite image databases, CORIA COnférence en Recherche d'Information et Applications, pp. 35–49

  8. Boulila W, Farah IR, Saheb Ettabaa K, Solaiman B, Ben Ghézala H (2011) A data mining based approach to predict spatiotemporal changes in satellite images. Int J Appl Earth Obs Geoinf 13(3):386–395

    Google Scholar 

  9. Boulila W, Bouatay A, Farah IR (2014) A probabilistic collocation method for the imperfection propagation: application to land cover change prediction. J Multimedia Process Technol 5(1):12–32

    Google Scholar 

  10. Boulila W, Ayadi Z, Farah IR (2017) Sensitivity analysis approach to model epistemic and aleatory imperfection: application to land cover change prediction model. J Comput Sci 23:58–70

    Google Scholar 

  11. Boulila W, Farah IR, Hussain A (2018) A novel decision support system for the interpretation of remote sensing big data. Earth Sci Inf 11(1):31–45

    Google Scholar 

  12. Bruzzone L, Prieto DF (2001) Unsupervised retraining of a maximum likelihood classifier for the analysis of multitemporal remote sensing images, IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no. 2, pp. 456–460

    Google Scholar 

  13. Chander G, Markham B (2003) Revised Landsat-5 TM radiometric calibration procedures and postcalibration dynamic ranges. IEEE Trans Geosci Remote Sens 41(11):2674–2677

    Google Scholar 

  14. Chander G, Markham BL, Helder DL (2009) Summary of current radiometric calibration coefficients for Landsat MSS, TM, ETM+, and EO-1 ALI sensors. Remote Sens Environ 113(5):893–903

    Google Scholar 

  15. Cheng G, Han J (2016) A survey on object detection in optical remote sensing images. J Photogramm Remote Sens 117:11–28

    Google Scholar 

  16. Congalton R, Green K (2008) Assessing the accuracy of remotely sensed data: principles and practices, 2nd edn. CRC Press, Taylor & Francis, Boca Raton

    Google Scholar 

  17. Conners RW, Trivedi MM, Harlow CA (1984) Segmentation of a high-resolution urban scene using texture operators. Comput Vis Graph Image Process 25:273–310

    Google Scholar 

  18. Diao W, Sun X, Dou F, Yan M, Wang H, Fu K (2015) Object recognition in remote sensing images using sparse deep belief networks. IEEE Geosci Remote Sens Lett 6(10):745–754

    Google Scholar 

  19. Durand N, Derivaux S, Forestier G (2007) Ontology-Based Object Recognition for Remote Sensing Image Interpretation, IEEE International Conference on Tools with Artificial Intelligence, pp. 472–479

  20. Farah IR, Boulila W, Saheb Ettabaa K, Solaiman B, Ben Ahmed M (2008) Interpretation of multi-sensor remote sensing images: Multi-approach fusion of uncertain information. IEEE Trans Geosci Remote Sens 46(12)

    Google Scholar 

  21. Ferchichi A, Boulila W, Farah IR (2017a) Propagating aleatory and epistemic uncertainty in land cover change prediction process. Eco Inform 37:24–37

    Google Scholar 

  22. Ferchichi A, Boulila W, Farah IR (2017b) Towards an uncertainty reduction framework for land-cover change prediction using possibility theory. Vietnam J Comput Sci 4(3):195–209

    Google Scholar 

  23. Ferchichi A, Boulila W, Farah IR (2018) Reducing uncertainties in land cover change models using sensitivity analysis. Knowl Inf Syst 55(3):719–740

    Google Scholar 

  24. Foody GM, Mathur A (2004) A relative evaluation of multiclass image classification by support vector machines. IEEE Trans Geosci Remote Sens 42(6):1335–1343

    Google Scholar 

  25. Haiyang Y, Fuping G (2009) Object recognition of high resolution remote sensing image based on PSWT, International Conference on Image Analysis and Signal Processing, pp. 52–56

  26. Haralick RM, Shapiro LG (1985) Image segmentation techniques. Comput Vis Graph Image Process 29(1):100–132

    Google Scholar 

  27. Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern SMC-3:610–621

    Google Scholar 

  28. Heisel S, Kovačević T, Briesen H, Schembecker G, Wohlgemuth K (2017) Variable selection and training set design for particle classification using a linear and a non-linear classifier. Chem Eng Sci 173:131–144

    Google Scholar 

  29. Huang C, Davis LS, Townshend JRG (2002) An assessment of support vector machines for land cover classification. Int J Remote Sens 23(4):725–749

    Google Scholar 

  30. Lei Z, Fang T, Huo H, Li D (2012) Rotation-invariant object detection of remotely sensed images based on texton forest and Hough voting. IEEE Trans Geosci Remote Sens 50(4):1206–1217

    Google Scholar 

  31. Levenberg K (1944) A method for the solution of certain non-linear problems in least squares. Q Appl Math 2:164–168

    Google Scholar 

  32. Liu J, Li J, Li W, Wu J (2016) Rethinking big data: a review on the data quality and usage issues. ISPRS J Photogramm Remote Sens 115:134–142

    Google Scholar 

  33. Ma X, Shen H, Yang J, Zhang L, Li P (2014) Polarimetric-spatial classification of SAR images based on the fusion of multiple classifiers. IEEE J. Sel Topics Appl Earth Observ 7(3):961–971

    Google Scholar 

  34. MacQueen JB (1967) Some methods for classification and analysis of multivariate observations, Berkeley symposium on mathematical statistics and probability, University of California Press, pp. 281–297

  35. Marmanis D, Schindler K, Wegner JD, Galliani S, Datcu M, Stilla U (2018) Classification with an edge: improving semantic image segmentation with boundary detection. J Photogramm Remote Sens 135:158–172

    Google Scholar 

  36. Ming D, Li J, Wang J, Zhang M (2015) Scale parameter selection by spatial statistics for GeOBIA: using mean-shift based multi-scale segmentation as an example. ISPRS J Photogramm Remote Sens 106:28–41

    Google Scholar 

  37. Mitra P, Uma Shankar B, Pal SK (2004) Segmentation of multispectral remote sensing images using active support vector machines. Pattern Recogn Lett 25:1067–1074

    Google Scholar 

  38. Murthy CS, Raju PV, Badrinath KVS (2003) Classification of wheat crop with multi-temporal images: performance of maximum likelihood and artificial neural networks. Int J Remote Sens 24(23):4871–4890

    Google Scholar 

  39. Rau JY, Jhan JP, Rau RJ (2014) Semiautomatic object-oriented landslide, Recognition Scheme From Multisensor Optical Imagery and DEM. IEEE Trans Geosci Remote Sens 52(2):1336–1349

    Google Scholar 

  40. Ryherd S, Woodcock C (1996) Combining spectral and texture data in the segmentation of remotely sensed images. Photogramm Eng Remote Sens 62(2):181–194

    Google Scholar 

  41. Samuel OW, Asogbon GM, Sangaiah AK, Fang P, Li G (2017) An integrated decision support system based on ANN and Fuzzy_AHP for heart failure risk prediction. Expert Syst Appl 68:163–172

    Google Scholar 

  42. Shotton J, Kohli P (2014) Semantic Image Segmentation, Computer Vision: A Reference Guide, Springer, pp. 713–716

  43. Svozil D, Kvasnicka V, Pospichal J (1997) Introduction to multi-layer feed-forward neural networks. Chemom Intell Lab Syst 39(1):43–62

    Google Scholar 

  44. Trias-Sanz R, Stamon G, Louchet J (2008) Using colour texture and hierarchical segmentation for high-resolution remote sensing. J Photogramm Remote Sens 63(2):156–168

    Google Scholar 

  45. Ulyanov D, Lebedev V, Vedaldi A, Lempitsky V (2016) Texture networks: feed-forward synthesis of textures and stylized images. ICML 48:1349–1357

    Google Scholar 

  46. Vincent L (1993) Grayscale area openings and closings, their efficient implementation and applications,Workshop on Mathematical Morphology and Its Applications to Signal Processing, J. Serra and P. Salembier, Eds., Barcelona, Spain, pp. 22–27

  47. Wang S, Zhang Y, Dong Z, Du S, Ji G, Yan J, Yang J, Wang Q, Feng C, Phillips P (2015a) Feed-forward neural network optimized by hybridization of PSO and ABC for abnormal brain detection. Int J Imaging Syst Technol 25(2):153–164

    Google Scholar 

  48. Wang S, Zhang Y, Ji G, Yang J, Wu J, Wei L (2015b) Fruit classification by wavelet-entropy and feedforward neural network trained by fitness-scaled chaotic ABC and biogeography-based optimization. Entropy 17(8):5711–5728

    Google Scholar 

  49. Yang X, Tridandapani S, Beitler JJ, Yu DS, Yoshida EJ, Curran WJ, Liu T (2012) Ultrasound GLCM texture analysis of radiation-induced parotid-gland injury in head-and-neck cancer radiotherapy: an in vivo study of late toxicity. Med Phys 39(9):5732–5739

    Google Scholar 

  50. Zhang J, Li T, Lu X, Cheng Z (2016a) Semantic classification of high-resolution, Remote-Sensing Images Based on Mid-level Features. IEEE J. Sel Topics Appl Earth Observ 9(6):2343–2353

    Google Scholar 

  51. Zhang L, Zhang L, Du B (2016b) Deep learning for remote sensing data: a technical tutorial on the state of the art. IEEE Geosci Remote Sens Mag 4(2):22–40

    Google Scholar 

  52. Zhang T, Yan W, Li J, Chen J (2016c) Multiclass labeling of very high-resolution remote sensing imagery by enforcing nonlocal shared constraints in multilevel conditional random fields model. IEEE J. Sel Topics Appl Earth Observ 9(7):2854–2867

    Google Scholar 

  53. Zhang AZ, Sun GY, Liu SH, Wang ZJ, Wang P, Ma JS (2017) Multi-scale segmentation of very high resolution remote sensing image based on gravitational field and optimized region merging. Multimed Tools Appl 76(13):15105–15122

    Google Scholar 

  54. Zheng C, Zhang Y, Wang L (2017) Semantic segmentation of remote sensing imagery using an object-based Markov random field model with auxiliary label fields. IEEE Trans Geosci Remote Sens 55(5):3015–3028

    Google Scholar 

Download references

Acknowledgements

The author would like to thank the anonymous reviewers for their valuable comments which were very helpful to improve the manuscript.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Wadii Boulila.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by: H. Babaie

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Boulila, W. A top-down approach for semantic segmentation of big remote sensing images. Earth Sci Inform 12, 295–306 (2019). https://doi.org/10.1007/s12145-018-00376-7

Download citation

Keywords

  • Semantic segmentation
  • Remote sensing images
  • Neural networks
  • Big data