On Gaussian Process NARX Models and Their Higher-Order Frequency Response Functions

  • Keith Worden
  • Graeme Manson
  • Elizabeth J. Cross
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 97)


One of the most versatile and powerful approaches to the identification of nonlinear dynamical systems is the NARMAX (Nonlinear Auto_regressive Moving Average with eXogenous inputs) method. The model represents the current output of a system by a nonlinear regression on past inputs and outputs and can also incorporate a nonlinear noise model in the most general case. Although the NARMAX model is most often given a polynomial form, this is not a restriction of the method and other formulations have been proposed based on nonparametric machine learning paradigms, for example. All of these forms of the NARMAX model allow the computation of Higher-order Frequency Response Functions (HFRFs) which encode the model in the frequency domain and allow a direct interpretation of how frequencies interact in the nonlinear system under study. Recently, a NARX (no noise model) formulation based on Gaussian Process (GP) regression has been developed. One advantage of the GP NARX form is that confidence intervals are a natural part of the prediction process. The objective of the current paper is to discuss the GP formulation and show how to compute the HFRFS corresponding to GP NARX. Examples will be given based on simulated data.


Nonlinear system identification NARMAX models Higher-order Frequency Response Functions (HFRFs) Gaussian processes 



The authors would like to thank Dr James Hensman of the University of Sheffield Centre for Translational Neuroscience for a number of interesting and useful discussions.


  1. 1.
    Leontaritis, I.J., Billings, S.A.: Input-output parametric models for nonlinear systems, part I: deterministic nonlinear systems. Int. J. Control 41, 303–328 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Leontaritis, I.J., Billings, S.A.: Input-output parametric models for nonlinear systems, part II: stochastic nonlinear systems. Int. J. Control 41, 329–344 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Billings, S.A.: Nonlinear System Identification: NARMAX, Methods in the Time, Frequency, and Spatio-Temporal Domains. Wiley, Hoboken (2013)CrossRefGoogle Scholar
  4. 4.
    Billings, S.A., Jamaluddin, H.B., Chen, S.: Properties of neural networks with applications to modelling non-linear dynamical systems. Int. J. Control 55, 193–224 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Chen, S., Billings, S.A., Cowan, C.F.N., Grant, P.M.: Practical identification of NARMAX models using radial basis functions. Int. J. Control 52, 1327–1350 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, New York (1998)Google Scholar
  7. 7.
    Schetzen, M.: The Volterra and Wiener Theories of Nonlinear Systems. John Wiley Interscience Publication, New York (1980).zbMATHGoogle Scholar
  8. 8.
    Worden, K., Tomlinson, G.R.: Nonlinearity in Structural Dynamics: Detection, Modelling and Identification. Institute of Physics Press, Bristol (2001)CrossRefGoogle Scholar
  9. 9.
    Bedrosian, E., Rice, S.O.: The output properties of Volterra systems driven by harmonic and Gaussian inputs. Proc. IEEE 59, 1688–1707 (1971)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Billings, S.A., Tsang, K.M.: Spectral analysis for nonlinear systems, part I: parametric non-linear spectral analysis. Mech. Syst. Signal Process. 3, 319–339 (1989)CrossRefzbMATHGoogle Scholar
  11. 11.
    Chance, J.E., Worden, K., Tomlinson, G.R.: Frequency domain analysis of NARX neural networks. J. Sound Vib. 213, 915–941 (1997)CrossRefGoogle Scholar
  12. 12.
    Murray-Smith, R., Johansen, T. A., Shorten, R.: On transient dynamics, off-equilibrium behaviour and identification in blended multiple model structures. In: European Control Conference, Karlsruhe, BA-14 (1999)Google Scholar
  13. 13.
    Kocijan, J.: Dynamic GP models: an overview and recent developments. In: ASM12, Proc. 6th Int. Conf. Appl. Maths Sim. Mod., pp. 38–43 (2012)Google Scholar
  14. 14.
    Krige, D.G.: A Statistical Approach to Some Mine Valuations and Allied Problems at the Witwatersrand. Master’s Thesis, University of Witwatersrand (1951)Google Scholar
  15. 15.
    Neal, R.M.: Monte Carlo implementation of Gaussian process models for Bayesian regression and classification. Arxiv preprint physics/9701026, (1997)Google Scholar
  16. 16.
    MacKay, D.J.C.: Gaussian processes - a replacement for supervised neural networks. Lecture Notes for Tutorial at Int. Conf. Neural Inf. Proc. Sys. (1997)Google Scholar
  17. 17.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2005)Google Scholar
  18. 18.
    Quinonero-Candelo, Q. Rasmussen, C.E.: A unifying view of sparse approximation Gaussian process regression. J. Mach. Learn. Res. 6, 1939–1959 (2005)MathSciNetGoogle Scholar
  19. 19.
    Snelson, E. Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: AAdvances in Neural Information Processing Systems. MIT Press, Cambridge (2006)Google Scholar
  20. 20.
    Giraud, A.: Approximate Methods for Propagation of Uncertainty with Gaussian Process Models. PhD Thesis, University of Glasgow (2004)Google Scholar
  21. 21.
    Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes: The Art of Scientific Computing, 3rd edn. Cambridge University Press, New York (2007)Google Scholar
  22. 22.
    Worden, K., Hensman, J.J.: Parameter estimation and model selection for a class of hysteretic systems using Bayesian inference. Mech. Syst. Signal Process. 32, 153–169 (2011)CrossRefGoogle Scholar
  23. 23.
    Volterra, V.: Theory of Functionals and of Integral and Integro-Differential Equations. Dover, New York (1959)zbMATHGoogle Scholar
  24. 24.
    Palm, G., Poggio, T.: The Volterra representation and the Wiener expansion: validity and pitfalls. SIAM J. Appl. Math. 33, 195–216 (1997)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Keith Worden
    • 1
  • Graeme Manson
    • 1
  • Elizabeth J. Cross
    • 1
  1. 1.Dynamics Research Group, Department of Mechanical EngineeringUniversity of SheffieldSheffieldUK

Personalised recommendations