Skip to main content
SpringerLink
Log in

A comparative evaluation of nonlinear dynamics methods for time series prediction

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

A key problem in time series prediction using autoregressive models is to fix the model order, namely the number of past samples required to model the time series adequately. The estimation of the model order using cross-validation may be a long process. In this paper, we investigate alternative methods to cross-validation, based on nonlinear dynamics methods, namely Grassberger–Procaccia, Kégl, Levina–Bickel and False Nearest Neighbors algorithms. The experiments have been performed in two different ways. In the first case, the model order has been used to carry out the prediction, performed by a SVM for regression on three real data time series showing that nonlinear dynamics methods have performances very close to the cross-validation ones. In the second case, we have tested the accuracy of nonlinear dynamics methods in predicting the known model order of synthetic time series. In this case, most of the methods have yielded a correct estimate and when the estimate was not correct, the value was very close to the real one.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. A manifold (http://en.wikipedia.org/wiki/Manifold) is a mathematical space in which every point has a neighborhood which resembles Euclidean space, but in which the global structure may be more complicated. In a one-dimensional manifold (e.g. a line, a circle) every point has a neighborhood that looks like a segment of a line. In a two-dimensional manifold (e.g. a plane, the surface of a sphere) the neighborhood looks like a disk. \({\mathbb{R}}^n\) is a n-dimensional manifold.

  2. M is diffeomorphic to U iff there is a differentiable map \(m : M\,\mapsto\,U\) whose inverse m −1 exists and is also differentiable.

  3. Takens–Mañé embedding theorem is a consequence of a Whitney Embedding Theorem [32] stating that a generic map from an S-dimensional manifold to a (2S + 1)-dimensional Euclidean space is an embedding, i.e. the image of the S-dimensional manifold is completely unfolded in the larger space. Therefore, two points in the S-dimensional manifold do not map to the same point in the (2S + 1)-dimensional space.

  4. I(λ) is 1 iff condition λ holds, 0 otherwise.

  5. The complexity of effective sorting algorithms (e.g. mergesort and heapsort) is \(\ell \log \ell\), where \(\ell\) is the number of elements that have to be sorted.

  6. In our implementation, we use the linear search having complexity \(O(d \ell^2).\)

  7. Paris-14E Parc Montsouris and DSVC1 time series can be downloaded from http://www.knmi.nl/samenw/eca and http://www.cpdee.ufmg.br/∼MACSIN/services/data/data.htm, respectively.

  8. Windows Vista is a registered trademark of Microsoft Inc.

  9. Mathematica is a registered trademark of Wolfram Inc.

  10. Available on request for further investigations and comparisons.

References

  1. Aguirre LA, Rodrigues GG, Mendes EM (1997) Nonlinear identification and cluster analysis of chaotic attractors from a real implementation of Chua’s circuit. Int J Bifurcat Chaos 6(7):1411–1423

    Google Scholar 

  2. Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716–723

    Article  MATH  MathSciNet  Google Scholar 

  3. Anderson TW (1963) Determination of the order of dependence in normally distributed time series. In: Rosenblatt M (ed) Time series Analysis. Wiley, London, pp 425–446

  4. Arbabanel HDI (1996) Analysis of observed chaotic data. Springer, Berlin

  5. Chua LO, Komuro M, Matsumoto T (1985) The double scroll. IEEE Trans Circuits Syst 32(8):797–818

    Article  MATH  MathSciNet  Google Scholar 

  6. Duda RO, Hart PE, Stork DG (2000) Pattern Classification. Wiley, New York

    Google Scholar 

  7. Eckmann JP, Ruelle D (1985) Ergodic theory of Chaos and strange attractors. Rev Mod Phys 57:617–659

    Article  MathSciNet  Google Scholar 

  8. Grassberger P, Procaccia I (1983) Measuring the strangeness of strange attractors. Physica D9:189–208

    MathSciNet  Google Scholar 

  9. Hirshberg D, Merhav N (1996) Robust methods for model order estimation. IEEE Trans Signal Proces 44:620–628

    Article  Google Scholar 

  10. Hübner U, Weiss CO, Abraham NB, Tang D (1994) Lorenz-Like Chaos in NH3-FIR Lasers. Time Series Prediction. Forecasting the Future and Understanding the Past. Addison Wesley, Reading, pp 73–104

  11. Joachim T (1999) Making large-scale SVM learning practical. Advances in Kernel methods—support vector learning, MIT Press, Cambridge

  12. Kantz H, Schreiber T (1997) Nonlinear time series analysis. Cambridge University Press, London

  13. Kennel MB, Brown R, Arbabanel HDI (1992) Determining embedding dimension for phase-space reconstruction using a geometrical construction. Phys Rev A 45(6):3403–3411

    Article  Google Scholar 

  14. Kégl B (2003) Intrinsic dimension estimation using packing numbers, vol 15. Advances in Neural Information Processing. MIT Press, Cambridge

  15. Mañé R (1981) On the dimension of compact invariant sets of certain nonlinear maps. Dynamical Systems and Turbolence, Warwick 1980. In: Lecture Notes in Mathematics no. 898, Springer, Berlin, pp 230–242

  16. Merhav N (1989) The estimation of the model order in exponential families. IEEE Trans Inf Theory 35(5):1109–1114

    Article  MATH  MathSciNet  Google Scholar 

  17. Müller K-R, Rätsch G, Kohlmorgen J, Smola A, Schölkopf B, Vapnik V (2000) Time series prediction using support vector regression and neural network. In: Higuchi T, Takizawa Y (eds) Proceedings of second international symposium on frontiers of time series modelling: nonparametric approach to knowledge discovery. Institute of mathematical statistic publication

  18. Ott E (1993) Chaos in dynamical systems. Cambridge University Press, London

  19. Packard N, Crutchfield J, Farmer J, Shaw R (1980) Geometry from a time series. Phys Rev Letters 45(1):712–716

    Article  MathSciNet  Google Scholar 

  20. Schölkopf B, Smola A (2002) Learning with Kernels. MIT Press, Cambridge

  21. Rissanen J (1978) Modeling by shortest data description. Automatica 14(5):465–471

    Article  MATH  Google Scholar 

  22. Shawe-Taylor J, Cristianini N (2004) Kernel Methods for Pattern Analysis. Cambridge University Press, London

  23. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464

    Article  MATH  Google Scholar 

  24. Smith RL (1992) Optimal estimation of fractal dimension. Nonlinear modeling and forecasting. In: Casdagli M, Eubank S (eds) SFI studies in the sciences of complexity, vol. XII, Addison-Wesley, New York, pp 115–135

  25. Snyder DL (1975) Random point processes. Wiley, New York

    MATH  Google Scholar 

  26. Stone M (1974) Cross-validatory choice and assessment of statistical prediction. J R Stat Soc 36(1):111–147

    MATH  Google Scholar 

  27. Takens F (1981) Detecting strange attractor in turbolence. Dynamical systems and turbolence, Warwick 1980. In: Lecture Notes in Mathematics no. 898. Springer, Berlin, pp 366–381

  28. Takens F (1985) On the numerical determination of the dimension of an attractor. Dynamical systems and bifurcations. In: Braaksma B, Broer H, Takens F (eds) Proceedings Groningen 1984, Lecture Notes in Mathematics No. 1125, Springer, Berlin, pp 99–106

  29. Tong H (1990) Nonlinear time series. Oxford University Press, NY

  30. Vapnik VN (1998) Statistical Learning Theory. Wiley, New York

    MATH  Google Scholar 

  31. Vaidya PM (1989) An O(n log n) algorithm for the All-Nearest-Neighbors Problem. Discret Comput Geom 4(1):101–115

    Article  MATH  MathSciNet  Google Scholar 

  32. Whitney H (1936) Differentiable manifolds. Ann Math 37:645–680

    Article  MathSciNet  Google Scholar 

  33. Wijngaard JB, Klein Tank AMG, Konnen GP (2003) Homogeneity of 20th century European daily temperature and precipitation series. Int J Clim 23:679–692

    Article  Google Scholar 

Download references

Acknowledgments

The authors wish to thank the anonymous reviewers for their valuable comments.

Author information

Authors and Affiliations

  1. Department of Applied Science, University of Naples Parthenope, Centro Direzionale Isola C4, 80143, Naples, Italy

    Francesco Camastra

  2. Department of Computer Science, University of Sheffield, Regent Court, 211 Portobello Street, Sheffield, S1 4DP, UK

    Maurizio Filippone

Authors
  1. Francesco Camastra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maurizio Filippone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesco Camastra.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Camastra, F., Filippone, M. A comparative evaluation of nonlinear dynamics methods for time series prediction. Neural Comput & Applic 18, 1021–1029 (2009). https://doi.org/10.1007/s00521-009-0266-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-009-0266-y

Keywords

Search

Navigation