Advertisement

Feature and Architecture Selection on Deep Feedforward Network for Roll Motion Time Series Prediction

  • Novri SuhermiEmail author
  • Suhartono
  • Santi Puteri Rahayu
  • Fadilla Indrayuni Prastyasari
  • Baharuddin Ali
  • Muhammad Idrus Fachruddin
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 937)

Abstract

The neural architecture and the input features are very substantial in order to build an artificial neural network (ANN) model that is able to perform a good prediction. The architecture is determined by several hyperparameters including the number of hidden layers, the number of nodes in each hidden layer, the series length, and the activation function. In this study, we present a method to perform feature selection and architecture selection of ANN model for time series prediction. Specifically, we explore a deep learning or deep neural network (DNN) model, called deep feedforward network, an ANN model with multiple hidden layers. We use two approaches for selecting the inputs, namely PACF based inputs and ARIMA based inputs. Three activation functions used are logistic sigmoid, tanh, and ReLU. The real dataset used is time series data called roll motion of a Floating Production Unit (FPU). Root mean squared error (RMSE) is used as the model selection criteria. The results show that the ARIMA based 3 hidden layers DNN model with ReLU function outperforms with remarkable prediction accuracy among other models.

Keywords

ARIMA Deep feedforward network PACF Roll motion Time series 

Notes

Acknowledgements

This research was supported by ITS under the scheme of “Penelitian Pemula” No. 1354/PKS/ITS/2018. The authors thank to the Head of LPPTM ITS for funding and to the referees for the useful suggestions.

References

  1. 1.
    Zhang, G., Patuwo, B.E., Hu, M.Y.: Forecasting with artificial neural networks. Int. J. Forecast. 14, 35–62 (1998)CrossRefGoogle Scholar
  2. 2.
    Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Sig. Syst. 2, 303–314 (1989)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Funahashi, K.I.: On the approximate realization of continuous mappings by neural networks. Neural Netw. 2, 183–192 (1989)CrossRefGoogle Scholar
  4. 4.
    Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2, 359–366 (1989)CrossRefGoogle Scholar
  5. 5.
    Chen, Y., He, K., Tso, G.K.F.: Forecasting crude oil prices: a deep learning based model. Proced. Comput. Sci. 122, 300–307 (2017)CrossRefGoogle Scholar
  6. 6.
    Liu, L., Chen, R.C.: A novel passenger flow prediction model using deep learning methods. Transp. Res. Part C: Emerg. Technol. 84, 74–91 (2017)CrossRefGoogle Scholar
  7. 7.
    Qin, M., Li, Z., Du, Z.: Red tide time series forecasting by combining ARIMA and deep belief network. Knowl.-Based Syst. 125, 39–52 (2017)CrossRefGoogle Scholar
  8. 8.
    Qiu, X., Ren, Y., Suganthan, P.N., Amaratunga, G.A.J.: Empirical mode decomposition based ensemble deep learning for load demand time series forecasting. Appl. Soft Comput. 54, 246–255 (2017)CrossRefGoogle Scholar
  9. 9.
    Voyant, C., et al.: Machine learning methods for solar radiation forecasting: a review. Renew. Energy. 105, 569–582 (2017)CrossRefGoogle Scholar
  10. 10.
    Zhao, Y., Li, J., Yu, L.: A deep learning ensemble approach for crude oil price forecasting. Energy Econ. 66, 9–16 (2017)CrossRefGoogle Scholar
  11. 11.
    Hui, L.H., Fong, P.Y.: A numerical study of ship’s rolling motion. In: Proceedings of the 6th IMT-GT Conference on Mathematics, Statistics and its Applications, pp. 843–851 (2010)Google Scholar
  12. 12.
    Nicolau, V., Palade, V., Aiordachioaie, D., Miholca, C.: Neural network prediction of the roll motion of a ship for intelligent course control. In: Apolloni, B., Howlett, Robert J., Jain, L. (eds.) KES 2007. LNCS (LNAI), vol. 4694, pp. 284–291. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-74829-8_35CrossRefGoogle Scholar
  13. 13.
    Zhang, X.L., Ye, J.W.: An experimental study on the prediction of the ship motions using time-series analysis. In: The Nineteenth International Offshore and Polar Engineering Conference (2009)Google Scholar
  14. 14.
    Khan, A., Bil, C., Marion, K., Crozier, M.: Real time prediction of ship motions and attitudes using advanced prediction techniques. In: Congress of the International Council of the Aeronautical Sciences, pp. 1–10 (2004)Google Scholar
  15. 15.
    Wang, Y., Chai, S., Khan, F., Nguyen, H.D.: Unscented Kalman Filter trained neural networks based rudder roll stabilization system for ship in waves. Appl. Ocean Res. 68, 26–38 (2017)CrossRefGoogle Scholar
  16. 16.
    Yin, J.C., Zou, Z.J., Xu, F.: On-line prediction of ship roll motion during maneuvering using sequential learning RBF neural networks. Ocean Eng. 61, 139–147 (2013)CrossRefGoogle Scholar
  17. 17.
    Zhang, G.P.: Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing. 50, 159–175 (2003)CrossRefGoogle Scholar
  18. 18.
    Makridakis, S., Wheelwright, S.C., Hyndman, R.J.: Forecasting: Methods and Applications. Wiley, Hoboken (2008)Google Scholar
  19. 19.
    Wei, W.W.S.: Time Series Analysis: Univariate and Multivariate Methods. Pearson Addison Wesley, Boston (2006)zbMATHGoogle Scholar
  20. 20.
    Tsay, R.S.: Analysis of Financial Time Series. Wiley, Hoboken (2002)CrossRefGoogle Scholar
  21. 21.
    Durbin, J.: The fitting of time-series models. Revue de l’Institut Int. de Statistique/Rev. Int. Stat. Inst. 28, 233 (1960)Google Scholar
  22. 22.
    Levinson, N.: The wiener (root mean square) error criterion in filter design and prediction. J. Math. Phys. 25, 261–278 (1946)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Liang, F.: Bayesian neural networks for nonlinear time series forecasting. Stat. Comput. 15, 13–29 (2005)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Box, G.E.P., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time Series Analysis: Forecasting and Control. Wiley, Hoboken (2015)zbMATHGoogle Scholar
  25. 25.
    Fausett, L.: Fundamentals of Neural Networks: Architectures, Algorithms, and Applications. Prentice-Hall, Inc., Upper Saddle River (1994)zbMATHGoogle Scholar
  26. 26.
    El-Telbany, M.E.: What quantile regression neural networks tell us about prediction of drug activities. In: 2014 10th International Computer Engineering Conference (ICENCO), pp. 76–80. IEEE (2014)Google Scholar
  27. 27.
    Taylor, J.W.: A quantile regression neural network approach to estimating the conditional density of multiperiod returns. J. Forecast. 19, 299–311 (2000)CrossRefGoogle Scholar
  28. 28.
    Han, J., Moraga, C.: The influence of the sigmoid function parameters on the speed of backpropagation learning. In: Mira, J., Sandoval, F. (eds.) IWANN 1995. LNCS, vol. 930, pp. 195–201. Springer, Heidelberg (1995).  https://doi.org/10.1007/3-540-59497-3_175CrossRefGoogle Scholar
  29. 29.
    Karlik, B., Olgac, A.V.: Performance analysis of various activation functions in generalized MLP architectures of neural networks. Int. J. Artif. Intell. Expert Syst. 1, 111–122 (2011)Google Scholar
  30. 30.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning. pp. 807–814. Omnipress, Haifa (2010)Google Scholar
  31. 31.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)zbMATHGoogle Scholar
  32. 32.
    LeCun, Y.A., Bottou, L., Orr, G.B., Müller, K.-R.: Efficient BackProp. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 9–48. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-35289-8_3CrossRefGoogle Scholar
  33. 33.
    De Gooijer, J.G., Hyndman, R.J.: 25 years of time series forecasting. Int. J. Forecast. 22, 443–473 (2006)CrossRefGoogle Scholar
  34. 34.
    Fuller, W.A.: Introduction to Statistical Time Series. Wiley, Hoboken (2009)Google Scholar
  35. 35.
    Phillips, P.C.B., Perron, P.: Testing for a Unit Root in Time Series Regression. Biometrika 75, 335 (1988)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Hobijn, B., Franses, P.H., Ooms, M.: Generalizations of the KPSS-test for stationarity. Stat. Neerl. 58, 483–502 (2004)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Kwiatkowski, D., Phillips, P.C.B., Schmidt, P., Shin, Y.: Testing the null hypothesis of stationarity against the alternative of a unit root. J. Econ. 54, 159–178 (1992)CrossRefGoogle Scholar
  38. 38.
    Hsu, C.W., Chang, C.C., Lin, C.J.: A practical guide to support vector classification. Presented at the (2003)Google Scholar
  39. 39.
    Gensler, A., Henze, J., Sick, B., Raabe, N.: Deep Learning for solar power forecasting — an approach using autoencoder and LSTM neural networks. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 002858–002865. IEEE (2016)Google Scholar
  40. 40.
    Ryu, S., Noh, J., Kim, H.: Deep neural network based demand side short term load forecasting. In: 2016 IEEE International Conference on Smart Grid Communications (SmartGridComm), pp. 308–313. IEEE (2016)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Novri Suhermi
    • 1
    Email author
  • Suhartono
    • 1
  • Santi Puteri Rahayu
    • 1
  • Fadilla Indrayuni Prastyasari
    • 2
  • Baharuddin Ali
    • 3
  • Muhammad Idrus Fachruddin
    • 4
  1. 1.Department of StatisticsInstitut Teknologi Sepuluh Nopember, Kampus ITS SukoliloSurabayaIndonesia
  2. 2.Department of Marine EngineeringInstitut Teknologi Sepuluh Nopember, Kampus ITS SukoliloSurabayaIndonesia
  3. 3.Indonesian Hydrodynamic LaboratoryBadan Pengkajian Dan Penerapan TeknologiSurabayaIndonesia
  4. 4.GDP LaboratoryJakartaIndonesia

Personalised recommendations