Abstract
Automatic processing and verification of data obtained in experiments have an essential role in modern science. In the paper, we discuss the assessment of data obtained in meteorological measurements conducted in Biebrza National Park in Poland. The data is essential for understanding the complex environmental processes, such as global warming. The measurements of CO2 flux brings a vast amount of data but suffer from drawbacks like high uncertainty. Part of the data has a high-level of credibility while, others are not reliable. The method of automatic evaluation of data with varied quality is proposed. We use LSTM networks with a weighted square mean error loss function. This approach allows incorporating the information on data reliability in the training process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, OSDI 201, pp. 2265–2836. USENIX Association, USA (2016)
Aubinet, M., Vesala, T., Papale, D. (eds.): Eddy Covariance: A Practical Guide to Measurement and Data Analysis. Springer, Netherlands (2012). https://doi.org/10.1007/978-94-007-2351-1
Baldocchi, D.D.: Assessing the eddy covariance technique for evaluating carbon dioxide exchange rates of ecosystems: past, present and future. Global Change Biol. 9(4), 479–492 (2003). https://doi.org/10.1046/j.1365-2486.2003.00629.x
Che, Z., Purushotham, S., Cho, K., Sontag, D., Liu, Y.: Recurrent neural networks for multivariate time series with missing values. Sci. Rep. 8(1) (2018). https://doi.org/10.1038/s41598-018-24271-9
Christiansen, N.H., Voie, P.E.T., Winther, O., Høgsberg, J.: Comparison of neural network error measures for simulation of slender marine structures. J. Appl. Math. 2014, 1–11 (2014). https://doi.org/10.1155/2014/759834
Dengel, S., Zona, D., Sachs, T., Aurela, M., Jammet, M., Parmentier, F.J.W., Oechel, W., Vesala, T.: Testing the applicability of neural networks as a gap-filling method using CH4 flux data from high latitude wetlands. Biogeosciences 10(12), 8185–8200 (2013). https://doi.org/10.5194/bg-10-8185-2013
Falge, E., et al.: Gap filling strategies for defensible annual sums of net ecosystem exchange. Agric. Forest Meteorol. 107(1), 43–69 (2001). https://doi.org/10.1016/s0168-1923(00)00225-2
Fathalla, A., Salah, A., Li, K., Li, K., Francesco, P.: Deep end-to-end learning for price prediction of second-hand items. Knowl. Inf. Syst. 62(12), 4541–4568 (2020). https://doi.org/10.1007/s10115-020-01495-8
Fortuniak, K., Pawlak, W., Bednorz, L., Grygoruk, M., Siedlecki, M., Zielinski, M.: Methane and carbon dioxide fluxes of a temperate mire in central Europe. Agri. Forest Meteorolog. 232, 306–318 (2017)
Fortuniak, K., Pawlak, W., Siedlecki, M., Chambers, S., Bednorz, L.: Temperate mire fluctuations from carbon sink to carbon source following changes in water table. Sci. Total Environ. 756 (2021). https://doi.org/10.1016/j.scitotenv.2020.144071
Gers, F.A., Eck, D., Schmidhuber, J.: Applying LSTM to time series predictable through time-window approaches. In: Dorffner, G., Bischof, H., Hornik, K. (eds.) ICANN 2001. LNCS, vol. 2130, pp. 669–676. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44668-0_93
Goodfellow, I., Bengio, J., Courville, A., Bach, F.: Deep Learning. MIT Press Ltd (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Ke, J., Zheng, H., Yang, H., Chen, X.M.: Short-term forecasting of passenger demand under on-demand ride services: a spatio-temporal deep learning approach. Transp. Res. Part C Emerging Technol. 85, 591–608 (2017). https://doi.org/10.1016/j.trc.2017.10.016
Kim, Y., et al.: Gap-filling approaches for eddy covariance methane fluxes: a comparison of three machine learning algorithms and a traditional method with principal component analysis. Global Change Biol. 26(3), 1499–1518 (2019). https://doi.org/10.1111/gcb.14845
Moffat, A.M., et al.: Comprehensive comparison of gap-filling techniques for eddy covariance net carbon fluxes. Agric. Forest Meteorol. 147(3–4), 209–232 (2007). https://doi.org/10.1016/j.agrformet.2007.08.011
Papale, D.: Data gap filling. In: Eddy Covariance, pp. 159–172. Springer, Netherlands (2011). https://doi.org/10.1007/978-94-007-2351-1_6
Selvin, S., Vinayakumar, R., Gopalakrishnan, E.A., Menon, V.K., Soman, K.P.: Stock price prediction using LSTM, RNN and CNN-sliding window model. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI). IEEE (2017). https://doi.org/10.1109/icacci.2017.8126078
Spelmen, V.S., Porkodi, R.: A review on handling imbalanced data. In: 2018 International Conference on Current Trends towards Converging Technologies (ICCTCT). IEEE (2018). https://doi.org/10.1109/icctct.2018.8551020
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, pp. 3104–3112. MIT Press, Cambridge (2014)
Acknowledgements
Funding for this research was partly provided (data collection) by the National Science Centre, Poland under the project UMO-2015/17/B/ST10/02187. The site was established in 2012 under project UMO-2011/01/B/ST10/07550 founded by National Science Centre, Poland. The authors thank the authorities of the Biebrza National Park for allowing continuous measurements in the area of the Park.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Podlaski, K., Durka, M., Gwizdałła, T., Miniak-Górecka, A., Fortuniak, K., Pawlak, W. (2021). LSTM Processing of Experimental Time Series with Varied Quality. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2021. ICCS 2021. Lecture Notes in Computer Science(), vol 12747. Springer, Cham. https://doi.org/10.1007/978-3-030-77980-1_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-77980-1_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77979-5
Online ISBN: 978-3-030-77980-1
eBook Packages: Computer ScienceComputer Science (R0)