Aerial Scene Understanding Using Deep Wavelet Scattering Network and Conditional Random Field

  • Sandeep Nadella
  • Amarjot SinghEmail author
  • S. N. Omkar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9913)


This paper presents a fast and robust architecture for scene understanding for aerial images recorded from an Unmanned Aerial Vehicle. The architecture uses Deep Wavelet Scattering Network to extract Translation and Rotation Invariant features that are then used by a Conditional Random Field to perform scene segmentation. Experiments are conducted using the proposed framework on two annotated datasets of 1277 images and 300 aerial images, introduced in the paper. An overall pixel accuracy of 81 % and 78 % is achieved for the datasets. A comparison with another similar framework is also presented.


Aerial scene understanding Unmanned aerial vehicle Deep wavelet scattering Conditional random field 


  1. 1.
    Bruna, J., Mallat, S.: Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1872–1886 (2013)CrossRefGoogle Scholar
  2. 2.
    Casella, E., Rovere, A., Pedroncini, A., Mucerino, L., Casella, M., Cusati, L.A., Vacchi, M., Ferrari, M., Firpo, M.: Study of wave runup using numerical models and low-altitude aerial photogrammetry: A tool for coastal management. Estuar. Coast. Shelf Sci. 149, 160–167 (2014)CrossRefGoogle Scholar
  3. 3.
    Christophe, E., Inglada, J.: Robust road extraction for high resolution satellite images. In: 2007 IEEE International Conference on Image Processing, pp. 437–440. IEEE (2007)Google Scholar
  4. 4.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2005)Google Scholar
  5. 5.
    Domke, J.: Learning graphical model parameters with approximate marginal inference. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2454–2467 (2013)CrossRefGoogle Scholar
  6. 6.
    Dubuisson-Jolly, M., Gupta, A.: Color and texture fusion: application to aerial image segmentation and gis updating. Image Vis. Comput. 18, 823–832 (2010)CrossRefGoogle Scholar
  7. 7.
    Ghiasi, M., Amirfattahi, R.: Fast semantic segmentation of aerial images based on color and texture. In: 8th Iranian Conference on Machine Vision and Image Processing (MVIP) (2013)Google Scholar
  8. 8.
    Gould, S., Fulton, R., Koller, D.: Decomposing a scene into geometric and semantically consistent regions. In: International Conference on Computer Vision (ICCV) (2009)Google Scholar
  9. 9.
    Laptev, I., Mayer, H., Lindeberg, T., Eckstein, W., Steger, C., Baumgartner, A.: Automatic extraction of roads from aerial images based on scale space and snakes. Mach. Vis. Appl. 12(1), 23–31 (2000)CrossRefGoogle Scholar
  10. 10.
    Lathuiliere, S., Vu, H., Le, T., Tran, T., Hung, D.: Semantic regions recognition in UAV images sequence. Knowl. Syst. Eng. 326, 313–324 (2015)Google Scholar
  11. 11.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional network for semantic segmentation. In: IEEE Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  12. 12.
    Marmanis, D., Wegner, J.D., Galliani, S., Schindler, K., Datcu, M., Stilla, U.: Semantic segmentation of aerial images with an ensemble of CNNs. ISPRS Ann. Photogrammetry Remote Sens. Spatial Inf. Sci. 3, 473–480 (2016)CrossRefGoogle Scholar
  13. 13.
    Montoya-Zegarra, J., Wegner, J., Ladicky, L., Schindler, K.: Semantic segmentation of aerial images in urban areas with class-specific higher-order cliques. ISPRS Ann. Photogrammetry Remote Sens. Spatial Inf. Sci. 2, 127–133 (2015)CrossRefGoogle Scholar
  14. 14.
    Munoz, D., Bagnell, J.A., Hebert, M.: co-inference for multi-modal scene analysis. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 668–681. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33783-3_48 CrossRefGoogle Scholar
  15. 15.
    Penmetsa, S., Minhuj, F., Singh, A., Omkar, S.: Autonomous UAV for suspicious action detection using pictorial human pose estimation and classification. Electron. Lett. Comput. Vis. Image Anal. 3(1), 18–32 (2014)Google Scholar
  16. 16.
    Rezaeian, M., Amirfattahi, R., Sadri, S.: Semantic segmentation of aerial images using fusion of color and texture features. J. Comput. Secur. 1, 225–238 (2013)Google Scholar
  17. 17.
    Rochery, M., Jermyn, I.H., Zerubia, J.: Higher order active contours. Int. J. Comput. Vis. 69(1), 27–42 (2006)CrossRefGoogle Scholar
  18. 18.
    Şerban, G., Rus, I., Vele, D., Breţcan, P., Alexe, M., Petrea, D.: Flood-prone area delimitation using UAV technology, in the areas hard-to-reach for classic aircrafts: case study in the north-east of apuseni mountains, transylvania. Nat. Hazards, 82, 1–16 (2016)Google Scholar
  19. 19.
    Sifre, L.: Rigid-motion scattering for image classification. Ph.D. thesis (2014)Google Scholar
  20. 20.
    Sifre, L., Mallat, S.: Combined scattering for rotation invariant texture analysis. In: European Symposium on Artificial Neural Networks (ESANN) (2012)Google Scholar
  21. 21.
    Sifre, L., Mallat, S.: Rotation, scaling and deformation invariant scattering for texture discrimination. In: IEEE conference on Computer Vision and Pattern Recognition (CVPR), pp. 1233–1240 (2013)Google Scholar
  22. 22.
    Šmídl, V., Hofman, R.: Tracking of atmospheric release of pollution using unmanned aerial vehicles. Atmos. Environ. 67, 425–436 (2013)CrossRefGoogle Scholar
  23. 23.
    Su, Y., Guo, Q., Fry, D.L., Collins, B.M., Kelly, M., Flanagan, J.P., Battles, J.J.: A vegetation mapping strategy for conifer forests by combining airborne lidar data and aerial imagery. Can. J. Remote Sens. 42(1), 1–15 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of ECENational Institute of TechnologyWarangalIndia
  2. 2.Department of EngineeringUniversity of CambridgeCambridgeUK
  3. 3.Department of Aerospace EngineeringIndian Institute of ScienceBangaloreIndia

Personalised recommendations