Skip to main content

Advertisement

SpringerLink
Log in

DSSAE-BBOA: deep learning-based weather big data analysis and visualization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The weather forecasting process is used to predict the future atmospheric condition of a specified location. The evolution of the big data era gives the chances to significantly increase the prediction accuracy of weather conditions. In this paper, a deep learning-based stacked sparse autoencoder (DSSAE) has been proposed for predicting the weather condition of a particular area. This model requires a pre-processing approach to obtain essential data from big weather data and increase the prediction model’s speed. For this, the principal component analysis (PCA) is utilized to reduce dimensionality and extraction of features with more significant variance. Also, it integrates the feature selection algorithm based on Binary Butterfly Optimization Algorithm (BBOA) along with a deep stack autoencoder to improve prediction accuracy. The proposed model is validated using the weather data taken from the division of weather underground for short term and long term weather prediction. The simulation consequences illustrate that the proposed model overtakes the existing models in terms of computation time, accuracy and error rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Aksu D, Üstebay S, Aydin MA, Atmaca T (2018) Intrusion detection with comparative analysis of supervised learning techniques and fisher score feature selection algorithm. In: International symposium on computer and information sciences. Springer, Cham, pp 141–149

    Chapter  Google Scholar 

  2. Anjali T, Chandini K, Anoop K, Lajish VL (2019) Temperature prediction using machine learning approaches. In International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT), 1, pp. 1264-1268. IEEE

  3. Arora S, Anand P (2019) Binary butterfly optimization approaches for feature selection. Expert Syst Appl 116:147–160

    Article  Google Scholar 

  4. Bagirov AM, Mahmood A (2018) A comparative assessment of models to predict monthly rainfall in Australia. Water Resour Manag 32(5):1777–1794

    Article  Google Scholar 

  5. Bayatvarkeshi M, Mohammadi K, Kisi O, Fasihi R (2020) A new wavelet conjunction approach for estimation of relative humidity: wavelet principal component analysis combined with ANN. Neural Comput & Applic 32(9):4989–5000

    Article  Google Scholar 

  6. Booz J, Yu W, Xu G, Griffith D, Golmie N (2019) A deep learning-based weather forecast system for data volume and Recency analysis. In International Conference on Computing, Networking, and Communications (ICNC), IEEE, 697–701

  7. Buszta A, Mazurkiewicz J (2015) Climate changes prediction system based on weather big data visualisation. DepCoS-RELCOMEX. Advances in Intelligent Systems and Computing, 365, Springer.

  8. Dalto M, Matusko J, Vasak M (2015) Deep neural networks for ultra-short-term wind forecasting. In Proceedings of the IEEE International Conference on Industrial Technology. 1657–1663.

  9. Dargan S, Kumar M, Ayyagari MR, Kumar G. (2019) A survey of deep learning and its applications: a new paradigm to machine learning. Arch Computational Methods Eng

  10. Deo RC, Şahin M (2015) Application of the artificial neural network model for prediction of monthly standardized precipitation and evapotranspiration index using hydrometeorological parameters and climate indices in eastern Australia. Atmos Res 161–162:65–81

    Article  Google Scholar 

  11. Deo RC, Şahin M (2015) Application of the extreme learning machine algorithm to predict monthly effective drought index in eastern Australia. Atmos Res 153:512–525

    Article  Google Scholar 

  12. Doreswamy GI, Manjunatha BR (2018) Multi-label Classification of Big NCDC Weather Data Using Deep Learning Model.ICSCS 2018. In: Communications in Computer and Information Science,837. Springer, Singapore

    Google Scholar 

  13. Haidar A, Verma B (2018) Monthly rainfall forecasting using one-dimensional deep convolutional neural network. IEEE Access 6:69053–69063

    Article  Google Scholar 

  14. Hardwinarto S, Aipassa M (2015) Rainfall monthly prediction based on artificial neural network: a case study in Tenggarong Station, East Kalimantan-Indonesia. Procedia Comput Sci 59:142–151

    Article  Google Scholar 

  15. Hassani H, Silva ES (2015) Forecasting with big data: a review. Ann Data Sci 2:5–19. https://doi.org/10.1007/s40745-015-0029-9.6

    Article  Google Scholar 

  16. Hegazy AE, Makhlouf MA, El-Tawel GS (2020) Improved salp swarm algorithm for feature selection. J King Saud Univ-Comput Inform Sci 32(3):335–344

    Google Scholar 

  17. Holmstrom M, Liu D, Vo C (2016) Machine learning applied to weather forecasting.

  18. Huda RK, Haider Banka (2019) Efficient feature selection and classification algorithm based on PSO and rough sets. Neural Comput & Applic 31(8):4287–4303

    Article  Google Scholar 

  19. Iraji MS (2019) Deep stacked sparse auto-encoders for prediction of post-operative survival expectancy in thoracic lung cancer surgery. J Appl Biomed 17:75–75. https://doi.org/10.32725/jab.2018.007

    Article  Google Scholar 

  20. Ismail KA, Majid MA, Zain JM, Bakar NAA (2016) Big Data prediction framework for weather Temperature based on MapReduce algorithm. In 2016 IEEE Conference on Open Systems (ICOS), IEEE, pp 13–17

  21. Jiang S, Chin K-S, Wang L, Gang Q, Tsui KL (2017) Modified genetic algorithm-based feature selection combined with pre-trained deep neural network for demand forecasting in outpatient department. Expert Syst Appl 82:216–230

    Article  Google Scholar 

  22. Jiang L, Ge Z, Song Z (2017) Semi-supervised fault classification based on dynamic sparse stacked auto-encoders model. Chemom Intell Lab Syst 168:72–83

    Article  Google Scholar 

  23. Karunakaran V, Iwin Joseph S, Teja R, Suganthi M, Rajasekar V (2019) A wrapper based feature selection approach using bees algorithm for extreme rainfall prediction via weather pattern recognition through svm classifier. Intl J Civil Eng Technol (IJCIET) 10.

  24. Khalili N, Khodashenas SR, Davary K, Baygi MM, Karimaldini F (2016) Prediction of rainfall using artificial neural networks for synoptic station of Mashhad: a case study. Arab J Geosci 9(13):624

    Article  Google Scholar 

  25. Kourou K, Exarchos TP (2015) Machine learning applications in cancer prognosis and prediction. Comput Struct biotechnol J 3:8–17

    Article  Google Scholar 

  26. Larraondo PR et al. (2017) Automating weather forecasts based on convolutional networks.

  27. Lefkovits S, Lefkovits L (2017) Gabor feature selection based on information gain. Procedia Eng 181:892–898

    Article  Google Scholar 

  28. Li J, Liu H (2016) Challenges of feature selection for big data analytics. IEEE Intell Syst 32:9–15. https://doi.org/10.1109/MIS.2017.38

    Article  Google Scholar 

  29. Liu H, Chen C (2019) Multi-objective data-ensemble wind speed forecasting model with stacked sparse autoencoder and adaptive decomposition-based error correction. Appl Energy 254:113686

    Article  Google Scholar 

  30. Liu JNK, Hu Y, You JJ, Chan PW (2014) Deep neural network-based feature representation for weather forecasting. In Proceedings of the International Conference on Artificial Intelligence (ICAI)

  31. Liu JN, Hu Y, He Y, Chan PW, Lai L (2015) Deep neural network modeling for big data weather forecasting. In Information Granularity, Big Data, and Computational Intelligence 389–408.

  32. Mani K, Kalpana P (2016) An efficient feature selection based on bayes theorem, self-information and sequential forward selection. Intl J Inform Eng Electronic Business 8(6):46–54

    Google Scholar 

  33. Mcgovern A, Elmore K, Gagne D, Haupt S, Karstens C, Lagerquist R, Smith T, Williams J (2017) Using artificial intelligence to improve real-time decision-making for high-impact weather. Bull Am Meteorol Soc 98:2073–2090. https://doi.org/10.1175/BAMS-D-16-0123.1

    Article  Google Scholar 

  34. Mehrkanoon S (2019) Deep shared representation, learning for weather elements forecasting. Knowl-Based Syst 179:120–128

    Article  Google Scholar 

  35. Mekanik F, Imteaz MA, Talei A (2016) Seasonal rainfall forecasting by adaptive network-based fuzzy inference system (ANFIS) using large scale climate signals. Climate Dyn 46(9–10):3097–3111

    Article  Google Scholar 

  36. Moradi E, Pepen A (2015) Machine learning framework for early MRI-based Alzheimer's conversion prediction in MCI subjects. Neuroimages 104:398–412

    Article  Google Scholar 

  37. Najafabadi MM, Villanustre F (2015) Deep learning applications and challenges in big data analytics. J Big Data 2:1

    Article  Google Scholar 

  38. Feiping Nie, Yizhen Huang, Xiaoqian Wang, and Heng Huang. (2014) New primal SVM solver with the linear computational cost for big data classifications. In Proceedings of the 31st International Conference on International Conference on Machine Learning, 32.

  39. Olshannikova E, Visualizing AO (2015) Big data with augmented and virtual reality: challenges and research agenda. J Big Data 2(1):22

    Article  Google Scholar 

  40. Palma-Mendoza R-J, Rodriguez D, De-Marcos L (2018) Distributed ReliefF-based feature selection in spark. Knowl Inf Syst 57(1):1–20

    Article  Google Scholar 

  41. Palvanov A, Cho YI. (2019). VisNet: deep convolutional neural networks for forecasting atmospheric visibility. Sensors (Basel)DOI: https://doi.org/10.3390/s19061343, 19.

  42. Patel J, Shah S, Thakkar P (2015) Predicting stock market index using a fusion of machine learning techniques. Expert Syst Appl 42:2162–2172

    Article  Google Scholar 

  43. Poornima S, Pushpalatha M (2019) Prediction of rainfall using intensified lSTM based recurrent neural network with weighted linear units. Atmosphere 10(11):668

    Article  Google Scholar 

  44. Qi G, Zhu Z, Erqinhu K (2018) Fault-diagnosis for reciprocating compressors using big data and machine learning. Simul Model Pract Theory 80:104–127

    Article  Google Scholar 

  45. Roesch I, Günther T (2019) Visualization of neural network predictions for weather forecasting. Comput Graphics Forum 38:209–220

    Article  Google Scholar 

  46. Rolnik D et al (2019) Tackling Climate Change with Machine Learning. arXiv:1906.05433

  47. Rong M, Gong D, Gao XZ (2019) Feature Selection And Its Use In Big Data: Challenges, Methods, and Trends. IEEE Access,1–1. https://doi.org/10.1109/ACCESS.2019.2894366.

  48. Saha M, Santara A, Mitra P, Chakraborty A, Nanjundiah RS (2020) Prediction of the Indian summer monsoon using a stacked autoencoder and ensemble regression model. Int J Forecast

  49. Singh K, Guntuku SC (2014) Big data analytics framework for peer-to-peer botnet detection using random forests. Inf Sci 278:488–497

    Article  Google Scholar 

  50. Sivarajah U, Kamal MM, Irani Z, Weerakkody V (2017) Critical analysis of big data challenges and analytical methods. J Bus Res 70:263–286

    Article  Google Scholar 

  51. Tang J, Liu J. (2016) Visualizing large-scale and high-dimensional data. In Proceedings of the 25th International Conference on World Wide Web, 287–297.

  52. Thaseen IS, Kumar CA (2017) Intrusion detection model using fusion of chi-square feature selection and multi class SVM. J King Saud Univ-Comput Inform Sci 29(4):462–472

    Article  Google Scholar 

  53. Vathsala H, Koolagudi SG (2017) Prediction model for peninsular Indian summer monsoon rainfall using data mining and statistical approaches. Comput Geosci 98:55–63

    Article  Google Scholar 

  54. Voyant C, Notton G, Kalogirou S, Nivet M-L, Paoli C, Motte F, Fouilloy A (2017) Machine learning methods for solar radiation forecasting: a review. Renew Energy 105:569–582

    Article  Google Scholar 

  55. Wan C, Xu Z, Pinson P (2014) Probabilistic forecasting of wind power generation using extreme learning machine. IEEE Trans Power Syst 29(3):1033–1044

    Article  Google Scholar 

  56. Wang X-W, Nie D, Bao-Liang L (2014) Emotional state classification from EEG data using machine learning approach. Neurocomputing 129:94–106

    Article  Google Scholar 

  57. Wu W-C, Hung S-H (2014) DroidDolphin: a dynamic Android malware detection framework using big data and machine learning. In Proceedings of the 2014 Conference on research in adaptive and convergent systems, ACM, 247–252

  58. Xin J, Wang Z, Luxuan Q, Guoren (2015) Wang elastic extreme learning machine for big data classification. Neurocomputing 149:464–471

    Article  Google Scholar 

  59. Yonekura K, Hattori H, Suzuki T (2018) Short-term local weather forecast using dense weather station by the deep neural network. In IEEE International Conference on Big Data (Big Data), IEEE, 1683–1690

  60. Zhai J, Zang L (2018) Ensemble dropout extreme learning machine via fuzzy integral for data classification. Neurocomputing 275:1043–1052

    Article  Google Scholar 

  61. Zhang L, Ma W, Zhang D (2016) Stacked sparse autoencoder in PolSAR data classification using local spatial information. IEEE Geosci Remote Sens Lett 13:1–5. https://doi.org/10.1109/LGRS.2016.2586109

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the Indian Institute of Technology (Indian School of Mines), Dhanbad, Govt. of India. The authors wish to express their gratitude and heartiest thanks to the Department of Computer Science & Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, India, for providing their continuous research support.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, Jharkhand, 826004, India

    Madhukar Rao G & Ramesh Dharavath

Authors
  1. Madhukar Rao G
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ramesh Dharavath
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramesh Dharavath.

Ethics declarations

Conflict of interest

The authors do not have a conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

G, M.R., Dharavath, R. DSSAE-BBOA: deep learning-based weather big data analysis and visualization. Multimed Tools Appl 80, 27471–27493 (2021). https://doi.org/10.1007/s11042-021-11059-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11059-9

Keywords

Search

Navigation