Abstract
Deep learning has become the most powerful tool for action recognition in recent years. The aim of this paper is to investigate the proper convolutional network architecture to classify poleward moving auroras (PMAs). The first challenge is that the auroral images have complex morphological and motion characteristics, so it is difficult to get the discriminative motion information to recognize the PMA events. Second, the imbalanced dataset will cause the serious problems, such as the network tends to identify a given sequence as the majority of events (non-PMAs). To address these issues, we use 3D ResNet-18 to get the motion information of auroral image sequence based on optical flow and also use the spatial attention mechanism to make the network pay more attention to PMA events. The empirical results demonstrate that our method achieves significant performance improvement compared with the previous method.
This work was supported by the National Natural Science Foundation of China under Grant No. 61571353. Student paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Vorobjev, V.G., Gustafsson, G., Starkov, G.V., Feldstein, Y.I., Shevnina, N.F.: Dynamics of day and night aurora during substorms. Planet. Space Sci. 23(2), 269–278 (1975)
Milan, S.E., Yeoman, T.K., Lester, M., Moen, J., Sandholt, P.E.: Post-noon two-minute period pulsating aurora and their relationship to the dayside convection pattern. Ann. Geophys. 17, 877–891 (1999)
Kozlovsky, A., Kangas, J.: Motion and origin of noon high-latitude poleward moving auroral arcs on closed magnetic field lines. J. Geophys. Res.: Space Phys. 107(A2), 1017 (2002)
Wang, B.: Investigation of triggering of poleward moving auroral forms using satellite-imager coordinated observations. J. Geophys. Res.: Space Phys. 121(11), 10–929 (2016)
Xing, Z.Y., et al.: Poleward moving auroral forms (PMAFs) observed at the yellow river station: a statistical study of its dependence on the solar wind conditions. J. Atmos. Solar Terr. Phys. 86, 25–33 (2012)
Fasel, G.J.: Dayside poleward moving auroral forms: a statistical study. J. Geophys. Res.: Space Phys. 100(AZ), 11891–11905 (1995)
Yang, Q., Liang, J., Zejun, H., Xing, Z., Zhao, H.: Automatic recognition of poleward moving auroras from all-sky image sequences based on HMM and SVM. Planet. Space Sci. 69(1), 40–48 (2012)
Zha, S., Luisier, F., Andrews, W., Srivastava, N., Salakhutdinov, R.: Exploiting image-trained CNN architectures for unconstrained video classification. arXiv preprint arXiv:1503.04144 (2015)
Ji, S., Wei, X., Yang, M., Kai, Y.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013)
Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)
Qiu, Z., Yao, T., Mei, T.: Learning spatio-temporal representation with pseudo-3D residual networks. In: proceedings of the IEEE International Conference on Computer Vision, pp. 5533–5541 (2017)
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., Paluri, M.: A closer look at spatiotemporal convolutions for action recognition. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 6450–6459 (2018)
Feichtenhofer, C., Fan, H., Malik, J., He, K.: Slowfast networks for video recognition. arXiv preprint arXiv:1812.03982 (2018)
Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Advances in Neural Information Processing Systems, pp. 568–576 (2014)
Sharma, S., Kiros, R., Salakhutdinov, R.: Action recognition using visual attention. arXiv preprint arXiv:1511.04119 (2015)
Li, Z., Gavrilyuk, K., Gavves, E., Jain, M., Snoek, C.G.M.: VideoLSTM convolves, attends and flows for action recognition. Comput. Vis. Image Underst. 166, 41–50 (2018)
Yang, X., Gao, X., Tao, D., Li, X., Han, B., Li, J.: Shape-constrained sparse and low-rank decomposition for auroral substorm detection. IEEE Trans. Neural Netw. Learn. Syst. 27(1), 32–46 (2016)
Wang, Q., et al.: Spatial texture based automatic classification of dayside aurora in all-sky images. J. Atmos. Solar Terr. Phys. 72(5–6), 498–508 (2010)
Soomro, K., Zamir, A.R., Shah, M.: UCF101: a dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 (2012)
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., Serre, T.: HMDB: a large video database for human motion recognition. In: 2011 International Conference on Computer Vision, pp. 2556–2563. IEEE (2011)
Hara, K., Kataoka, H., Satoh, Y.: Learning spatio-temporal features with 3D residual networks for action recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3154–3160 (2017)
Ferri, C., Hernández-Orallo, J., Modroiu, R.: An experimental comparison of performance measures for classification. Pattern Recogn. Lett. 30(1), 27–38 (2009)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24673-2_3
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Tang, Y., Niu, C., Dong, M., Ren, S., Liang, J. (2019). Poleward Moving Aurora Recognition with Deep Convolutional Networks. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2019. Lecture Notes in Computer Science(), vol 11858. Springer, Cham. https://doi.org/10.1007/978-3-030-31723-2_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-31723-2_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31722-5
Online ISBN: 978-3-030-31723-2
eBook Packages: Computer ScienceComputer Science (R0)