Skip to main content
SpringerLink
Log in
Book cover

Time Series Analysis and Applications to Geophysical Systems pp 195–226Cite as

The Innovation Approach to the Identification of Nonlinear Causal Models in Time Series Analysis

  • Conference paper

  • 593 Accesses

Part of the book series: The IMA Volumes in Mathematics and its Applications ((IMA,volume 139))

Abstract

This paper shows how the innovation approach developed by Wiener (1949), Kaiman (1960) and Box and Jenkins (1970) has found wide application in modern nonlinear time series analysis. Nonlinear models, such as the chaos, stochastic or deterministic differential equation models, neural network models and nonlinear AR models developed in the last two decades are reviewed as useful causal models in time series analysis for nonlinear dynamic phenomena in many scientific fields. The merit of the use of the innovation approach in conjunction with these new models is pointed out. Further, the computational efficiency and advantage of RBF-AR models over RBF neural network models is demonstrated in real data analysis of EEG time series of subjects with epilepsy. The advantage of multivariate RBF-ARX models in the modeling of thermal power plants is also shown using numerical results.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akaike H. (1969), “Fitting autoregressive models for prediction,” Ann. Inst. Stat. Math., 21, 243–247.

    Article  MathSciNet  MATH  Google Scholar 

  2. Biscay R., J.C. Jimenez, J.J. Riera, and P.A. Valdes Sosa (1996), “Local linearization method for the numerical solution of stochastic differential equations,” Ann. Inst. Stat. Math., 48, 631–644.

    Article  MathSciNet  MATH  Google Scholar 

  3. Bouchaud J.P. and R. Cont (1998), “A Langevin approach to stock market fluctuations and crashes,” Eur. Phys. J., B6, pp. 543–550.

    Google Scholar 

  4. Box G.E.P. and G.M. Jenkins (1970), Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco.

    MATH  Google Scholar 

  5. Campbell M.J. and A.M. Walker (1977), “A survey of statistical work on the McKenzie river series of annual Canadian lynx trappings for the year 1821–1934, and a new analysis,” J. Roy. Statist. Soc., A 140, 411–431.

    Article  Google Scholar 

  6. Cichocki A. and R. Unbehauen (1993), “Neural Networks for Optimization and Signal Processing,” J. Wiley and Sons, Chichester/New York/Brisbane/Toronto/Singapore.

    Google Scholar 

  7. Doob J.L. (1953), Stochastic Processes, John Wiley and Sons Inc.

    MATH  Google Scholar 

  8. Dyn N. (1989a), “Interpolation and approximation by radial and related functions,” in Approximation theory, VI, Academic Press, London, pp. 211–234.

    Google Scholar 

  9. Dyn N. (1989b), “Interpolation by piecewise-linear radial basis functions,” J. Approximation Theory, 59, 202–223.

    Article  MathSciNet  MATH  Google Scholar 

  10. Frost P.A. and T. Kailath (1971), “An innovation approach to least squares estimation Part III: nonlinear estimation in white noise,” IEEE Trans on Aut. Control, AC-13, pp. 646–655.

    Google Scholar 

  11. Haber R. and L. Keviczky (1999), Nonlinear system identification — input-output modeling approach, Volume 1: Nonlinear system structure identification. Kluwer Academic Publishers, The Netherlands.

    Book  MATH  Google Scholar 

  12. Iino M. and T. Ozaki (2000), “A Nonlinear Model for Financial Dynamics,” in Proceeding of the International Symposium on Frontiers of Time Series Modeling, ISM Tokyo Japan.

    Google Scholar 

  13. Ito K. (1980), “Chaos in the Rikitake two-disc dynamo system,” Earth Planet. Sci. Lett., 51, 451–456.

    Google Scholar 

  14. Jimenez J.C., I. Shoji, and T. Ozaki (1999), “Simulation of stochastic differential equations through the local linearization method: a comparative study,” J. Stat. Phys., 94, 587–602.

    Article  MathSciNet  MATH  Google Scholar 

  15. Jimenez J.C. and T. Ozaki (2000a), “Linear estimation of continuous-discrete linear state space models with multiplicative noise,” Res. Memo., 772, The Institute of Statistical Mathematics, Tokyo.

    Google Scholar 

  16. Jimenez J.C. and T. Ozaki (2000b), “Local Linearization filters for nonlinear continuous-discrete state space models with multiplicative noise,” Res. Memo., 773, The Institute of Statistical Mathematics, Tokyo.

    Google Scholar 

  17. Kailath T. (1968), “An innovation approach to least squares estimation Part I: linear filtering in additive white noise,” IEEE Trans on Aut. Control, AC-13, pp. 646–655.

    Google Scholar 

  18. Kalman R.E. (1960), A New Approach to Linear Filtering and Prediction Problems, Trans. ASME, Series D, J. of Basic Eng., 82, 35–45.

    Article  Google Scholar 

  19. Kloeden P.E. and E. Platen (1995), Numerical Solution of Stochastic Differential Equations, New York, Springer.

    Google Scholar 

  20. Kolmogorov A.N. (1931), “On analytical methods in probability theory,” Math. Ann., 104, 415–458.

    Article  MathSciNet  MATH  Google Scholar 

  21. Kolmogorov A.N. (1941), “Interpolation and extrapolation of stationary random sequences,” Bull. Acad. Sci. USSR, Math. Ser., 5, 3–14.

    MathSciNet  Google Scholar 

  22. Lapedes A. and R. Farber, (1987), “Neural Networks work,” Neural Information Systems, Academic Institute of Physics, pp. 442–456.

    Google Scholar 

  23. Lopes da Silva F.H., A. van Rotterdam, P. Barts, E. van Heusden, and W. Burr (1976), “Models of neuronal populations: the basic mechanisms.” In: M.A. Corner and D.F. Swaab (Eds.), Perspective in Brain Research, Prog. Brain Res., 45, 281–308.

    Google Scholar 

  24. Masani P. and N. Wiener (1959), “Nonlinear prediction,” in Probability and Statistics, John Wiley and Sons, pp. 190–212.

    Google Scholar 

  25. May R. (1976), “Simple mathematical models with very complicated dynamics,” Nature, 261, 459–467.

    Article  Google Scholar 

  26. Mil’stein G.N. (1995), Numerical Integration of Stochastic Differential Equations, Dordrecht: Kluwer Academic Publishers.

    Google Scholar 

  27. Ozaki T. (1980), “Nonlinear time series models for nonlinear random vibrations,” J. of Appl Prob., 17, 84–93.

    Article  MathSciNet  MATH  Google Scholar 

  28. Ozaki T. (1985a), “Nonlinear time series models and dynamical systems,” in Handbook of Statistics, 5, E.J. Hannan et al. (Eds.), North-Holland.

    Google Scholar 

  29. Ozaki T. (1985b), “Statistical identification of storage models with application to stochastic hydrology,” Water Resources Bulletin, 21.

    Google Scholar 

  30. Ozaki T. (1986), “Local Gaussian modelling of stochastic dynamical systems in the analysis of nonlinear random vibrations,” in Essays in Time Series and Allied Processes, Festschrift in honour of Prof. E.J. Hannan, Probability Trust.

    Google Scholar 

  31. Ozaki T. (1992), “Identification of nonlinearities and non-Gaussianities in time series,” in New Directions in Time Series Analysis, Part I, D.R. Brillinger et al. (Eds.), IMA Volumes in Mathematics and Its Application, Springer Verlag, 45, 227–264.

    Google Scholar 

  32. Ozaki T. (1993a), “A local linearization approach to nonlinear filtering,” Int. J. Control, 57, 75–96.

    Article  MathSciNet  MATH  Google Scholar 

  33. Ozaki T. (1993b), “Non-Gaussian characteristics of Exponential Autoregressive processes,” in Developments in Time Series Analysis, Festschrift in honour of Prof. Maurice Priestley, T. Subba Rao (Ed.), Chapman and Hall, pp. 257–273.

    Google Scholar 

  34. Ozaki T. and H. Oda (1978), “Nonlinear time series model identification by Akaike’s information criterion,” Information and Systems, B. Dubuisson (Ed.), Pergamon Press, pp. 83–91.

    Google Scholar 

  35. Ozaki T. and V.H. Ozaki (1989), “Statistical identification of nonlinear dynamics in macroeconomics using nonlinear time series models,” in Statistical Analysis and Forecasting of Economic Structural Change, P. Hackl (Ed.), Springer-Verlag, pp. 345–365.

    Google Scholar 

  36. Ozaki T., P.A. Valdes Sosa, and V. Haggan-Ozaki (1997), “Reconstructing Nonlinear Dynamics from Time Series: with Application to Epilepsy Data Analysis,” J. Signal Processing, 3, 153–162.

    Google Scholar 

  37. Ozaki T., J.C. Jimenez, and V. Haggan-Ozaki (2000), “Role of the likelihood function in the estimation of chaos models,” J. Time Series Analysis, 21, 363–387.

    Article  MathSciNet  MATH  Google Scholar 

  38. Ozaki T., J.C. Jimenez, M. Iino, Z. Shi, and S. Sugawara (2001), “Use of stochastic differential equation models in financial time series analysis: Monitoring and control of currencies in exchange market,” Proceeding of 3rd Japan-US joint seminar on statistical time series analysis, Kyoto, Japan, June 18–22, Sponsored by JSPS-NSF, June 2001, pp. 17–24.

    Google Scholar 

  39. Park J. and I.W. Sandberg (1991), “Universal approximation using radial basis function networks,” Neural Comput., 3, 247–257.

    Google Scholar 

  40. Park J. and I.W. Sandberg (1993), “Approximation and radial basis networks,” Neural Comput., 5, 305–306.

    Article  Google Scholar 

  41. Peng. H., T. Ozaki, and Y. Toyoda (2001), “Nonlinear system identification using radial basis function-based signal-dependent ARX model,” Proc. of 5th IFAC Symposium on Nonlinear Control Systems, Saint-Petersburg, Russia, July 4–6, 2001.

    Google Scholar 

  42. Pogio T. and F. Girosi (1990), “Networks for Approximation and Learning,” Proceeding of the IEEE, 78, 1461–1497.

    Google Scholar 

  43. Roberts S. and L. Tarassenko (1995), “Automated sleep EEG analysis using an RBF network,” in Applications of Neural Networks, A. Murray (Ed.), Kluwer Academic Publishers, pp. 305–322.

    Google Scholar 

  44. Rikitake T (1958), “Oscillations of a system of disc dynamos,” Proc. Camb. Philos. Soc, 54, 89–105.

    Article  MathSciNet  MATH  Google Scholar 

  45. Shi Z., Y. Tamura, and T. Ozaki (1999), “Nonlinear time series modelling with radial basis function-based state-dependent autoregressive model,” Intern. J. of System science, 30, 717–727.

    Article  MATH  Google Scholar 

  46. Shoji I. and T. Ozaki (1997), “Comparative study of estimation methods for continuous time stochastic processes,” Journal of Time Series Analysis, 18, 485–506.

    Article  MathSciNet  MATH  Google Scholar 

  47. Shoji I. and T. Ozaki (1998), “A statistical method of estimation and simulation for systems of stochastic differential equations,” Biometrika, 85, 240–243.

    Article  MathSciNet  MATH  Google Scholar 

  48. Shoji I. (1998), “A comparative study of maximum likelihood estimators for nonlinear dynamical system models,” Int. J. Control, 71, 391–404.

    Article  MathSciNet  MATH  Google Scholar 

  49. Tong H. (1977), “Some comments on the Canadian lynx data,” J. Roy. Soc, A 140, 432–436.

    Article  Google Scholar 

  50. Toyoda Y., K. Oda, and T. Ozaki (1997), “The nonlinear system identification method for advanced control of the fossil power plants”. Proceedings of 11 th IFAC Symposium on System Identification, Fukuoka, Japan, pp. 8–11.

    Google Scholar 

  51. Valdes Sosa P.A., J.C. Jimenez, J. Riera, R. Biscay, and T. Ozaki (1999a), “Nonlinear EEG analysis based on a neural mass model,” Biol. Cybern., 81, 415–424.

    Article  Google Scholar 

  52. Valdes Sosa P., J. Bosch, s, N. Trujillo, R. Biscay, F. Morales, J.L. Hernandez, and T. Ozaki (1999b), “The Statistical Identification of Nonlinear Brain Dynamics: A Progress Report,” in “Nonlinear Dynamics and Brain Functioning,” Nova Science Publishers, edited by N. Pradhan, P.E. Rapp, and R. Sreenivasan.

    Google Scholar 

  53. Vesin J. (1993), “An amplitude-dependent autoregressive signal model based on a radial basis function expansion,” In Proc. Intern. Conf. Acoust. Speech Signal Process., Minnesota, 3, 129–132.

    Google Scholar 

  54. Wold H. (1938), A Study in the Analysis of Stationary Time Series, Uppsala, Sweden: Almqvist and Wicksell.

    Google Scholar 

  55. Wiener N. (1949), Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications, New York: Wiley. Originally issued in MIT Radiation Lab. Report, Cambridge, Mass., February 1942.

    MATH  Google Scholar 

  56. Wiener N. (1961), Cybernetics, 2nd edition, MIT Press, Cambridge, Mass.

    Google Scholar 

  57. Wong E. (1963), “The construction of a class of stationary Markoff process,” Proc. Amer. Math. Soc. Symp. Appl. Math., 16, 264–276.

    Google Scholar 

  58. Zetterberg L.H., L. Kristiansson, and K. Mossberg (1978), “Performance of a model for a local neural populations,” Biol. Cybern., 31, 15–26.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Institute of Statistical Mathematics, 4-6-7 Minami Azabu, Minato-ku, Tokyo, 106-8569, Japan

    T. Ozaki

  2. Mathematics and Physics, Institute of Cybernetics, Cuba

    J. C. Jimenez

  3. Central South University, Changsha, 410083, China

    H. Peng (visiting researcher)

  4. Institute of Statistical Mathematics, USA

    H. Peng (visiting researcher)

  5. Sophia University, Japan

    V. H. Ozaki

Authors
  1. T. Ozaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. C. Jimenez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. V. H. Ozaki
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Statistics, University of California, Berkeley, 367 Evens Hall, Berkeley, CA, 94720-3860, USA

    David R. Brillinger

  2. Department of Earth and Environmental Engineering Henry Krumb School of Mines, Columbia University, 918 Seeley Mudd Building, 500 120th Street, New York, NY, 10027, USA

    Enders Anthony Robinson

  3. Department of Statistics, University of California, Los Angeles, 8142 Math-Science Building, Los Angeles, CA, 90095-1554, USA

    Frederic Paik Schoenberg

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag New York, LLC

About this paper

Cite this paper

Ozaki, T., Jimenez, J.C., Peng, H., Ozaki, V.H. (2004). The Innovation Approach to the Identification of Nonlinear Causal Models in Time Series Analysis. In: Brillinger, D.R., Robinson, E.A., Schoenberg, F.P. (eds) Time Series Analysis and Applications to Geophysical Systems. The IMA Volumes in Mathematics and its Applications, vol 139. Springer, New York, NY. https://doi.org/10.1007/978-1-4684-9386-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-4684-9386-3_11

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4419-1971-7

  • Online ISBN: 978-1-4684-9386-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation