Skip to main content
SpringerLink
Log in

Cloud/haze detection in airborne videos using a convolutional neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In airborne videos surveillance, moving object detection and target tracking are the key steps. However, under bad weather conditions, the presence of clouds and haze or even smoke coming from buildings can make the processing of these videos very challenging. Current cloud detection or classification methods only consider a single image. Moreover, the images they use are often captured by satellites or planes at high altitudes with very long ranges to clouds, which can help distinguish cloudy regions from non-cloudy ones. In this paper, a new approach for cloud and haze detection is proposed by exploiting both spatial and temporal information in airborne videos. In this method, several consecutive frames are divided into patches. Then, consecutive patches are collected as patch sets and fed into a deep convolutional neural network. The network is trained to learn the appearance of clouds as well as their motion characteristics. Therefore, instead of relying on single frame patches, the decision on a patch in the current frame is made based on patches from previous and subsequent consecutive frames. This approach, avoids discarding the temporal information about clouds in videos, which may contain important cues for discriminating between cloudy and non-cloudy regions. Experimental results show that using temporal information besides the spatial characteristics of haze and clouds can greatly increase detection accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M (2016) TensorFlow: a system for large-scale machine learning. In: Proceedings of the operating systems design and implementation, vol 16, pp 256–283

  2. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  3. Ackerman SA, Holz RE, Frey R, Eloranta EW, Maddux BC, McGill M (2008) Cloud detection with MODIS. Part II: validation. J Atmos Ocean Technol 25(7):1073–1086

    Article  Google Scholar 

  4. An Z, Shi Z (2015) Scene learning for cloud detection on remote-sensing images. IEEE J Sel Top Appl Earth Obs Remote Sens 8(8):4206–4222

    Article  Google Scholar 

  5. Cheng HY, Weng CC, Chen YY (2012) Vehicle detection in aerial surveillance using dynamic Bayesian networks. IEEE Trans Image Process 21(4):2152–2159

    Article  MathSciNet  Google Scholar 

  6. Fan D-P, Cheng M-M, Liu J-J, Gao S-H, Hou Q, Borji A (2018) Salient objects in clutter: bringing salient object detection to the foreground. In: Proceedings of the European conference on computer vision (ECCV), pp 186–202

  7. Fan D-P, Wang W, Cheng M-M, Shen J (2019) Shifting more attention to video salient object detection. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), pp 8554–8564

  8. Frey RA, Ackerman SA, Liu Y, Strabala KI, Zhang H, Key JR, Wang X (2008) Cloud detection with MODIS. Part I: improvements in the MODIS cloud mask for collection 5. J Atmos Ocean Technol 25(7):1057–1072

    Article  Google Scholar 

  9. Huang X, Zhang Y-J (2017) 300-FPS salient object detection via minimum directional contrast. IEEE Trans Image Process 26(9):4243–4254

    Article  MathSciNet  Google Scholar 

  10. Huang X, Zhang Y-J (2018) Water flow driven salient object detection at 180 fps. Pattern Recognit 76:95–107

    Article  Google Scholar 

  11. Le Hégarat-Mascle S, André C (2009) Use of Markov random fields for automatic cloud/shadow detection on high resolution optical images. ISPRS J Photogramm Remote Sens 64(4):351–366

    Article  Google Scholar 

  12. Ling Z, Gong J, Fan G, Lu X (2018) Optimal transmission estimation via fog density perception for efficient single image defogging. IEEE Trans Multimed 20(7):1699–1711

    Article  Google Scholar 

  13. Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th international conference on machine learning, pp 807–814

  14. Reilly V, Idrees H, Shah M (2010) Detection and tracking of large number of targets in wide area surveillance. In: European conference on computer vision, pp 186–199

  15. Saunders RW, Kriebel KT (1988) An improved method for detecting clear sky and cloudy radiances from AVHRR data. Int J Remote Sens 9(1):123–150

    Article  Google Scholar 

  16. Shi M, Xie F, Zi Y, Yin J (2016) Cloud detection of remote sensing images by deep learning. In: IEEE international geoscience and remote sensing symposium, pp 701–704

  17. Sommer LW, Teutsch M, Schuchert T, Beyerer J (2016) A survey on moving object detection for wide area motion imagery. In: IEEE Winter conference on applications of computer vision (WACV), pp 1–9

  18. Song Y, Li J, Wang X, Chen X (2018) Single image dehazing using ranking convolutional neural network. IEEE Trans Multimed 20(6):1548–1560

    Article  Google Scholar 

  19. Teutsch M, Grinberg M (2016) Robust detection of moving vehicles in wide area motion imagery. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 27–35

  20. Teutsch M, Kruger W (2015) Robust and fast detection of moving vehicles in aerial videos using sliding windows. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 26–34

  21. Wang W, Yuan X, Wu X, Liu Y (2017) Fast image dehazing method based on linear transformation. IEEE Trans Multimed 19(6):1142–1155

    Article  Google Scholar 

  22. Xie F, Shi M, Shi Z, Yin J, Zhao D (2017) Multilevel cloud detection in remote sensing images based on deep learning. IEEE J Sel Top Appl Earth Obs Remote Sens 10(8):3631–3640

    Article  Google Scholar 

  23. Yuan Y, Hu X (2015) Bag-of-words and object-based classification for cloud extraction from satellite imagery. IEEE J Sel Top Appl Earth Obs Remote Sens 8(8):4197–4205

    Article  Google Scholar 

  24. Yuan Y, Li C, Kim J, Cai W, Feng DD (2017) Reversion correction and regularized random walk ranking for saliency detection. IEEE Trans Image Process 27(3):1311–1322

    Article  MathSciNet  Google Scholar 

  25. Zhang Q, Xiao C (2014) Cloud detection of RGB color aerial photographs by progressive refinement scheme. IEEE Trans Geosci Remote Sens 52 (11):7264–7275

    Article  Google Scholar 

  26. Zhao J-X, Liu J-J, Fan D-P, Cao Y, Yang J, Cheng M-M (2019) EGNet: edge guidance network for salient object detection. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 8779–8788

Download references

Author information

Authors and Affiliations

  1. ECE Department, McMaster University, 1280 Main Street West, Hamilton, ON, Canada

    Hamidreza Fazlali, Shahram Shirani, Mike McDonald & Thia Kirubarajan

Authors
  1. Hamidreza Fazlali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shahram Shirani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mike McDonald
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thia Kirubarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamidreza Fazlali.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fazlali, H., Shirani, S., McDonald, M. et al. Cloud/haze detection in airborne videos using a convolutional neural network. Multimed Tools Appl 79, 28587–28601 (2020). https://doi.org/10.1007/s11042-020-09359-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09359-7

Keywords

Search

Navigation