Skip to main content
Book cover

The Proceedings of the Third International Conference on Smart City Applications

SCA 2019: Innovations in Smart Cities Applications Edition 3 pp 1179–1191Cite as

Historical Weather Data Recovery and Estimation

  • 1506 Accesses

Part of the Lecture Notes in Intelligent Transportation and Infrastructure book series (LNITI)

Abstract

In order to make efficient decisions in agriculture, it is imperative to analyse the weather data collected from various sources. These data are generated by automated weather stations. Unfortunately, weather observations may be missing or altered since weather stations may be stopped for maintenance or became out of order. Thus, it would affect significantly the process of data analysis. The purpose of this study is to estimate those missing values by using interpolation methods and others. We study the effectiveness of each method and compare them on different weather attributes. The methods were applied on different patterns of missing values and outliers. The experimental results prove that the two methods based on geographical proximity are performing better.

Keywords

  • Weather data
  • Temperature
  • Rainfall
  • Wind speed
  • Wind direction
  • Missing data
  • Imputation methods
  • RMSE

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-37629-1_85
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   189.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-37629-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   249.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.

References

  1. Menne, M.J., Williams, C.N., Gleason, B.E., Rennie, J.J., Lawrimore, J.H.: The global historical climatology network monthly temperature dataset, version 4. J. Clim. 31, 9835–9854 (2018)

    CrossRef  Google Scholar 

  2. Paltasingh, K.R., Goyari, P., Mishra, R.K.: Measuring weather impact on crop yield using aridity index: evidence from Odisha. Agric. Econ. Res. Rev. 25, 205–216 (2012)

    Google Scholar 

  3. Iizumi, T., Ramankutty, N.: How do weather and climate influence cropping area and intensity? Glob. Food Secur. 4, 46–50 (2015)

    CrossRef  Google Scholar 

  4. Ceglar, A., Turco, M., Toreti, A., Doblas-Reyes, F.J.: Linking crop yield anomalies to large-scale atmospheric circulation in Europe. Agric. For. Meteorol. 240–241, 35–45 (2017)

    CrossRef  Google Scholar 

  5. Ray, D.K., Gerber, J.S., MacDonald, G.K., West, P.C.: Climate variation explains a third of global crop yield variability. Nat. Commun. 6, 1–9 (2015)

    Google Scholar 

  6. Haworth, J., Cheng, T.: Non-parametric regression for space–time forecasting under missing data. Comput. Environ. Urban Syst. 36(6), 538–550 (2012)

    CrossRef  Google Scholar 

  7. Houari, R., Bounceur, A., Kechadi, M.-T., Tari, A.-K., Euler, R.: Dimensionality reduction in data mining: a copula approach. Expert Syst. Appl. 64(12), 247–280 (2016)

    CrossRef  Google Scholar 

  8. Kotsiantis, S., Kostoulas, A., Lykoudis, S., Argiriou, A., Menagias, K.: Filling missing temperature values in weather data banks. In: 2nd IET International Conference on Intelligent Environments (IE 2006), vol. 1, p. 327. IEE, Athens (2006)

    Google Scholar 

  9. Grzymala-Busse, J.W., Hu, M.: A comparison of several approaches to missing attribute values in data mining. In: LNAJ, vol. 2005, pp. 378–385 (2001)

    Google Scholar 

  10. Cong, R.-G., Brady, M.: The interdependence between rainfall and temperature: copula analyses. Sci. World J. 2012, 1–11 (2012)

    CrossRef  Google Scholar 

  11. Medori, M., Michelini, L., Nogues, I., Loreto, F., Calfapietra, C.: The impact of root temperature on photosynthesis and isoprene emission in three different plant species. Sci. World J. 2012, 10 (2012). Article ID 525827

    CrossRef  Google Scholar 

  12. Palomares-Salas, J.C., Agüera-Pérez, A., de la Rosa, J.J.G., Moreno-Muñoz, A.: A novel neural network method for wind speed forecasting using exogenous measurements from agriculture stations. Measurement 55, 295–304 (2014)

    CrossRef  Google Scholar 

  13. Hatfield, J.L., Prueger, J.H.: Temperature extremes: effect on plant growth and development. Weather Clim. Extrem. 10, 4–10 (2015)

    CrossRef  Google Scholar 

  14. Fiala, K., Tůma, I., Holub, P.: Effect of manipulated rainfall on root production and plant belowground dry mass of different grassland ecosystems. Ecosystems 12, 906–914 (2009)

    CrossRef  Google Scholar 

  15. Kant, S., Meshram, S., Sahu, K.C.: Analysis of rainfall data for drought investigation at Agra UP. Recent Res. Sci. Technol. 6(1), 62–64 (2014)

    Google Scholar 

  16. Ben Alaya, M.A., Chebana, F., Ouarda, T.B.M.J.: Probabilistic multisite statistical downscaling for daily precipitation using a bernoulli–generalized pareto multivariate autoregressive model. J. Clim. 28, 2349–2364 (2015)

    CrossRef  Google Scholar 

  17. Huang, B.Q., Rashid, T., Kechadi, M.T.: Multi-context recurrent neural network for time series applications. Int. J. Comput. Intell. 3(1), 45–55 (2006)

    Google Scholar 

  18. Sophia, K., Kanchi, R.R.: Wind speed and direction measurement system using Atmel 89S51 microcontroller. In: International Conference on Inventive Computation Technologies, pp. 1–6 (2016)

    Google Scholar 

  19. De Silva, R.P., Dayawansa, N.D.K., Ratnasiri, M.D.: A comparison of methods used in estimating missing rainfall data. J. Agric. Sci. 3(May), 101–108 (2007)

    Google Scholar 

  20. Jahan, F., Sinha, N.C., Rahman, Md.M., Rahman, Md.M., Mondal, Md.S.H., Islam, M.A.: Comparison of missing value estimation techniques in rainfall data of Bangladesh. Theor. Appl. Climatol. 136, 1115–1131 (2019)

    CrossRef  Google Scholar 

  21. Suhalia, J., Sayang, M.D., Jemain, A.A.: Revised spatial weighting methods for estimation of missing rainfall data. Asia-Pac. J. Atmos. Sci. 44(2), 93–104 (2008)

    Google Scholar 

  22. Sattari, M.-T., Rezazadeh-Joudi, A., Kusiak, A.: Assessment of different methods for estimation of missing data in precipitation studies. Hydrol. Res. 48, 1032–1044 (2017)

    CrossRef  Google Scholar 

  23. Bonan, G.B., Doney, S.C.: Climate, ecosystems, and planetary futures: the challenge to predict life in Earth system models. Science 359, eaam8328 (2018)

    CrossRef  Google Scholar 

  24. Liu, S., Su, H., Tian, J., Zhang, R., Wang, W., Wu, Y.: Evaluating four remote sensing methods for estimating surface air temperature on a regional scale. J. Appl. Meteorol. Climatol. 56, 803–814 (2017)

    CrossRef  Google Scholar 

  25. Stahl, K., Moore, R., Floyer, J., Asplin, M., McKendry, I.: Comparison of approaches for spatial interpolation of daily air temperature in a large region with complex topography and highly variable station density. Agric. For. Meteorol. 139, 224–236 (2006)

    CrossRef  Google Scholar 

  26. Rahman, M.G., Islam, M.Z.: A decision tree based missing value imputation technique for data preprocessing. In: Proceedings of the Ninth Australasian Data Mining Conference, Ballarat, pp. 41–50 (2011)

    Google Scholar 

  27. Yozgatligil, C., Aslan, S., Iyigun, C., Batmaz, I.: Comparison of missing value imputation methods in time series: the case of turkish meteorological data. Theor. Appl. Climatol. 112(1–2), 143–167 (2013)

    CrossRef  Google Scholar 

  28. Zahrah, S.N., Burhanuddin, A., Mohad Deni, S., Mohamed Ramli, N.: Revised normal ratio methods for imputation of missing rainfall data. Sci. Res. J. 13(1), 83 (2016)

    CrossRef  Google Scholar 

  29. Burhanuddin, S.N.Z.A.: Normal ratio in multiple imputation based on bootstrapped sample for rainfall data with missingness. Int. J. GEOMATE 13, 131–137 (2017)

    Google Scholar 

  30. Eskelson, B.N.I., Temesgen, H., Lemay, V., Barrett, T.M., Crookston, N.L., Hudak, A.T.: The roles of nearest neighbor methods in imputing missing data in forest inventory and monitoring databases. Scand. J. For. Res. 24, 235–246 (2009)

    CrossRef  Google Scholar 

  31. Bárdossy, A., Pegram, G.: Infilling missing precipitation records – a comparison of a new copula-based method with other techniques. J. Hydrol. 519(November 2014), 1162–1170 (2014)

    CrossRef  Google Scholar 

  32. Kanda, N., Negi, H.S., Rishi, M.S., Shekhar, M.S.: Performance of various techniques in estimating missing climatological data over snowbound mountainous areas of Karakoram Himalaya: estimation of missing climate data over mountainous areas. Meteorol. Appl. 25, 337–349 (2018)

    CrossRef  Google Scholar 

Download references

Acknowledgements

This research forms part of the CONSUS Programme which is funded under the SFI Strategic Partnerships Programme (16/SPP/3296) and is co-funded by Origin Enterprises Plc.

Author information

Authors and Affiliations

  1. UCD, University College Dublin, Belfield, Dublin 4, Ireland

    Fadoua Rafii & Tahar Kechadi

Authors
  1. Fadoua Rafii
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tahar Kechadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fadoua Rafii .

Editor information

Editors and Affiliations

  1. Faculty of Sciences and Techniques of Tangier, Mediterranean Association of Sciences and Technologies, Tangier, Morocco

    Prof. Dr. Mohamed Ben Ahmed

  2. Faculty of Sciences and Techniques of Tangier, Abdelmalek Essaadi University, Tangier, Morocco

    Dr. Anouar Abdelhakim Boudhir

  3. Polytechnic Institute of Castelo Branco, Castelo Branco, Portugal

    Prof. Dr. Domingos Santos

  4. Hassania School of Public Works, Casablanca, Morocco

    Prof. Dr. Mohamed El Aroussi

  5. Computer Engineering Department, Faculty of Engineering, Karabuk University, Karabük, Turkey

    Prof. Dr. İsmail Rakıp Karas

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Rafii, F., Kechadi, T. (2020). Historical Weather Data Recovery and Estimation. In: Ben Ahmed, M., Boudhir, A., Santos, D., El Aroussi, M., Karas, İ. (eds) Innovations in Smart Cities Applications Edition 3. SCA 2019. Lecture Notes in Intelligent Transportation and Infrastructure. Springer, Cham. https://doi.org/10.1007/978-3-030-37629-1_85

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37629-1_85

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37628-4

  • Online ISBN: 978-3-030-37629-1

  • eBook Packages: EngineeringEngineering (R0)