Skip to main content
SpringerLink
Log in

Chaotic Time Series Prediction Using Random Fourier Feature Kernel Least Mean Square Algorithm with Adaptive Kernel Size

  • Conference paper
  • First Online:
Modelling, Simulation and Applications of Complex Systems (CoSMoS 2019)

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 359))

Included in the following conference series:

Abstract

The random Fourier feature kernel least mean square (RFF-KLMS) algorithm provides a finite dimensional approximation to the kernel least mean square algorithm with radially symmetric Gaussian kernel. RFF-KLMS was introduced to curb the continuously growing radial basis function (RBF) network which prohibits online application of KLMS. RFF-KLMS assumes a fixed kernel size and the application of the method in nonlinear online regression can be quite tedious because it is not always obvious which kernel size to choose for a particular problem. In this paper, we incorporate a stochastic gradient approach in RFF-KLMS to update the kernel size. The efficacy of the new approach is demonstrated in the online prediction of time series generated from two different chaotic systems. In both examples, the RFF-KLMS algorithm with adaptive kernel size demonstrates very good tracking ability.

The authors would like to acknowledge the financial support from Universiti Sains Malaysia through Research University Grant (RUI) (acc. No. 1001/PMATHS/8011040).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Takens, F.: Detecting strange attractors in turbulence. Lecture Notes in Math. 898, 366–381 (1981)

    Article  MathSciNet  Google Scholar 

  2. Abarbanel, H.D.I.: Analysis of Observed Chaotic Data. Springer-Verlag, New York, Institute for Nonlinear Science (1996)

    Book  Google Scholar 

  3. Huke, J.P., Muldoon, M.R.: Embedding and time series analysis. Math. Today 51(3), 120–123 (2015)

    MathSciNet  Google Scholar 

  4. Zhang, S., Han, M., Xu, M.: Chaotic time series online prediction based on improved kernel adaptive filter. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE, Rio de Janeiro (2018)

    Google Scholar 

  5. Han, M., Zhang, S., Xu, M., Qiu, T., Wang, N.: Multivariate chaotic time series online prediction based on improved kernel recursive least Squares Algorithm. IEEE Trans. Cybern. 49(4), 1160–1172 (2019)

    Article  Google Scholar 

  6. Lu, L., Zhao, H., Chen, B.: Time series prediction using kernel adaptive filter with least mean absolute third loss function. Nonlinear Dyn. 90, 999–1013 (2017)

    Article  MathSciNet  Google Scholar 

  7. Garcia-Vega, S., Zeng, X.-J., Keane, J.: Stock rice prediction using kernel adaptive filtering within a stock market interdependence Approach. Available at SSRN (2018). https://doi.org/10.2139/ssrn.3306250

    Article  Google Scholar 

  8. Georga E.I., Principe, J.C., Polyzos, D., Fotiadis, D. I.: Non-linear dynamic modeling of glucose in type 1 diabetes with kernel adaptive filters. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5897–5900, IEEE, Orlando, FL (2016)

    Google Scholar 

  9. Ouala, S., Nguyen, D., Drumetz, L., Chapron,B., Pascual, A., Collard, F., Gaultier, L., Fablet, R.: Learning latent dynamics for partially-observed chaotic systems. arXiv preprint, arXiv:1907.02452 (2020)

  10. Yin, L., He, Y., Dong, X., Lu, Z.: Adaptive chaotic prediction algorithm of RBF neural network filtering model based on phase Space Reconstruction. J. Comput. 8(6), 1449–1455 (2013)

    Google Scholar 

  11. Feng, T., Yang, S., Han, F.: Chaotic time series prediction using wavelet transform and multi-model hybrid method. J. Vibroeng. 21(7), 1983–1999 (2019)

    Google Scholar 

  12. Kivinen, J., Smola, A., Williamson, R.: Online learning with kernels. In: Advances in Neural Information Processing Systems 14, pp. 785–793. MIT Press (2002)

    Google Scholar 

  13. Comminiello, D., Principe, J.C.: Adaptive Learning Methods for Nonlinear System Modeling. Elsevier (2018)

    Google Scholar 

  14. Chi, M., He, H., Zhang, W.: Nonlinear online classification algorithm with probability. J. Mach. Learn. Res. 20, 33–46 (2011)

    Google Scholar 

  15. Prıncipe, J.C., Liu, W., Haykin, S.: Kernel Adaptive Filtering: A Comprehensive Introduction. 57. John Wiley & Sons (2011)

    Google Scholar 

  16. Paduart, J., Lauwers, L., Swevers, J., Smolders, K., Schoukens, J., Pintelon, R.: Identification of nonlinear systems using Polynomial Nonlinear State Space models. Automatica 46(4), 647–656 (2010)

    Article  MathSciNet  Google Scholar 

  17. Liu, W., Pokharel, P.P., Principe, J.C.: The kernel least-mean-square algorithm. IEEE Trans. Signal Process. 56(2), 543–554 (2008)

    Article  MathSciNet  Google Scholar 

  18. Engel, Y., Mannor, S., Meir, R.: The kernel recursive least-squares algorithm. IEEE Trans. Signal Process. 52(8), 2275–2285 (2004)

    Article  MathSciNet  Google Scholar 

  19. Liu, W., Prıncipe, J.: Kernel affine projection algorithms. EURASIP J. Adv. Signal Process. 2008, 1–12 (2008)

    Google Scholar 

  20. Liu, W., Park, I., Wang, Y., Prıncipe, J.C.: Extended kernel recursive least squares algorithm. IEEE Trans. Signal Process. 57(10), 3801–3814 (2009)

    Article  MathSciNet  Google Scholar 

  21. Platt, J.: A resource-allocating network for function interpolation. Neural Comput. 3(2), 213–225 (1991)

    Article  MathSciNet  Google Scholar 

  22. Liu, W., Park, I., Prıncipe, J.C.: An information theoretic approach of designing sparse kernel adaptive filters. IEEE Trans. Neural Netw. 20(12), 1950–1961 (2009)

    Article  Google Scholar 

  23. Wang, S., Wang, W., Dang, L., Jiang, Y.: Kernel least mean square based on the Nystrom method. Circuits Syst. Signal Process. 38, 3133–3151 (2019)

    Article  Google Scholar 

  24. Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: Proceedings of the 21th Annual Conference on Neural Information Processing Systems (ACNIPS), pp. 1177–1184, Vancouver, BC, Canada (2007)

    Google Scholar 

  25. Singh, A., Ahuja, N., Moulin, P.: Online learning with kernels: overcoming the growing sum problem. In: Proceedings of the 2012 IEEE International Workshop on Machine Learning for Signal Process (MLSP), pp. 1–6, Santander, Spain (2012)

    Google Scholar 

  26. Bouboulis, P., Pougkakiotis, S., Theodoridis, S.: Efficient KLMS and KRLS algorithms: a random fourier feature perspective. In: 2016 IEEE Statistical Signal Processing Workshop (SSP), pp. 1–5, Palma de Mallorca (2016)

    Google Scholar 

  27. Xiong, K., Wang, S.: The online random fourier features conjugate gradient algorithm. IEEE Signal Process. Lett. 26(5), 740–744 (2019)

    Article  Google Scholar 

  28. Racine, J.: An efficient cross-validation algorithm for window width selection for nonparametric kernel regression. Commun. Stat. Simul. Comput. 22(4), 1107–1114 (1993)

    Article  Google Scholar 

  29. Cawley, G.C., Talbot, N.L.: Efficient leave-one-out cross-validation of kernel fischer discriminant classifiers. Pattern Recogn. 36(11), 2585–2592 (2003)

    Article  Google Scholar 

  30. An, S., Liu, W., Venkatesh, S.: Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression. Pattern Recogn. 40(8), 2154–2162 (2007)

    Article  Google Scholar 

  31. Hardle, W.: Applied Nonparametric Regression. Volume 5. Cambridge Univ Press (1990)

    Google Scholar 

  32. Herrmann, E.: Local bandwidth choice in kernel regression estimation. J. Comput. Graph. Stat. 6(1), 35–54 (1997)

    MathSciNet  Google Scholar 

  33. Chen, B., Liang, J., Zheng, N., Príncipe, J.C.: Kernel least mean square with adaptive kernel size. Neurocomput. 191, 95–106 (2016)

    Article  Google Scholar 

  34. Garcia-Vega, S., Zeng, X.-J., Keane, J.: Learning from data streams using kernel least-mean-square with multiple kernel-sizes and adaptive step-size. Neurocomput. 339, 105–115 (2019)

    Article  Google Scholar 

  35. Lorenz, E.N.: Deterministic aperiodic flow. J. Atmos. Sci. 20, 130 (1963)

    Article  Google Scholar 

  36. Zhang, X., Zhu, H., Yao, H.: Analysis of a new three-dimensional chaotic system. Nonlinear Dyn. 67, 335–343 (2012)

    Article  Google Scholar 

  37. Pelikán, E.: Tutorial: forecasting of processes in complex systems for real-world problems. Neural Netw. World 24, 567–589 (2014)

    Article  Google Scholar 

  38. Bolt, E.M.: Regularized forecasting of chaotic dynamical systems. Chaos, Solitons & Fractals. 94, 8–15 (2017)

    Article  MathSciNet  Google Scholar 

  39. Kazem, A., Sharifi, E., Hussain, F.K., Saberi, M., Hussain, O.K.: Support vector regression with chaos-based firefly algorithm for stock market price forecasting. Appl. Soft Comput. 13(2), 947–958 (2013)

    Article  Google Scholar 

  40. Brunton, S.L., Proctor, J.L., Kutz, J.N.: Sparse identification of nonlinear dynamics. Proc. National Acad. Sci. 113(15), 3932–3937 (2016)

    Article  MathSciNet  Google Scholar 

  41. Scholkopf, B., Smola, A.J.: Learning with Kernels Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, USA (2001)

    Google Scholar 

  42. Reed, M., Simon, B.: Methods of Modern Mathematical Physics II: Fourier Analysis. Academic Press, Self-Adjointness (1975)

    MATH  Google Scholar 

  43. Jones, L.K.: A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Ann. Stat. 20(1), 608–613 (1992)

    Article  MathSciNet  Google Scholar 

  44. Liu, Y., Sun, C., Jiang, S.: A kernel least mean square algorithm based on randomized feature networks. Appl. Sci. 8, 458 (2018)

    Article  Google Scholar 

  45. Dong, J., Zheng, Y., Chen, B.: A unified framework of random feature KLMS algorithms and convergence analysis. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8, IEEE, Rio de Janeiro (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Universiti Sains Malaysia, 11800, Penang, Malaysia

    Noor A. Ahmad

  2. Lahore College for Women University, Lahore, Punjab, Pakistan

    Shazia Javed

Authors
  1. Noor A. Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shazia Javed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noor A. Ahmad .

Editor information

Editors and Affiliations

  1. School of Mathematical Sciences, Universiti Sains Malaysia, Penang, Malaysia

    Mohd Hafiz Mohd

  2. School of Mathematical Sciences, Universiti Sains Malaysia, Penang, Malaysia

    Md Yushalify Misro

  3. School of Mathematical Sciences, Universiti Sains Malaysia, Penang, Malaysia

    Syakila Ahmad

  4. Department of Science and Technology, Thuyloi University, Hanoi, Vietnam

    Doanh Nguyen Ngoc

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ahmad, N.A., Javed, S. (2021). Chaotic Time Series Prediction Using Random Fourier Feature Kernel Least Mean Square Algorithm with Adaptive Kernel Size. In: Mohd, M.H., Misro, M.Y., Ahmad, S., Nguyen Ngoc, D. (eds) Modelling, Simulation and Applications of Complex Systems. CoSMoS 2019. Springer Proceedings in Mathematics & Statistics, vol 359. Springer, Singapore. https://doi.org/10.1007/978-981-16-2629-6_17

Download citation

Publish with us

Policies and ethics

Search

Navigation