Computational Statistics

, Volume 28, Issue 3, pp 1195–1223 | Cite as

Parameter estimation in stochastic differential equations with Markov chain Monte Carlo and non-linear Kalman filtering

  • Isambi S. Mbalawata
  • Simo Särkkä
  • Heikki Haario
Original Paper


This paper is concerned with parameter estimation in linear and non-linear Itô type stochastic differential equations using Markov chain Monte Carlo (MCMC) methods. The MCMC methods studied in this paper are the Metropolis–Hastings and Hamiltonian Monte Carlo (HMC) algorithms. In these kind of models, the computation of the energy function gradient needed by HMC and gradient based optimization methods is non-trivial, and here we show how the gradient can be computed with a linear or non-linear Kalman filter-like recursion. We shall also show how in the linear case the differential equations in the gradient recursion equations can be solved using the matrix fraction decomposition. Numerical results for simulated examples are presented and discussed in detail.


Hamiltonian Monte Carlo Stochastic differential equation  Parameter estimation Markov chain Monte Carlo  Kalman filter  Matrix fraction decomposition 



We would like to acknowledge Aki Vehtari, Antti Solonen, Jouni Hartikainen, and Arno Solin for their contributions. We thank the Department of Mathematics and Physics in Lappeenranta University of Technology, the Department of Biomedical Engineering and Computational Science in Aalto University and the Academy of Finland for their financial support.


  1. Aït-Sahalia Y (2002) Maximum likelihood estimation of discretely sampled diffusions: a closed-form approximation approach. Econometrica 70(1):223–262MathSciNetzbMATHCrossRefGoogle Scholar
  2. Aït-Sahalia Y (2008) Closed-form expansion for multivariate diffusions. Ann Stat 36(2):906–937zbMATHCrossRefGoogle Scholar
  3. Andrieu C, Thoms J (2008) A tutorial on adaptive MCMC. Stat Comput 18(4):343–373MathSciNetCrossRefGoogle Scholar
  4. Andrieu C, Doucet A, Holenstein R (2010) Particle Markov chain Monte Carlo methods. R Stat Soc Ser B Stat Methodol 72(3):269–342MathSciNetCrossRefGoogle Scholar
  5. Berg EVD, Heemink AW, Lin HX, Schoenmakers J (2006) Probability density estimation in stochastic environmental models using reverse representations. Stoch Environ Res Risk Assess 20(1–2):126–139MathSciNetzbMATHCrossRefGoogle Scholar
  6. Berger JO (1985) Statistical decision theory and Bayesian analysis. Springer, BerlinzbMATHCrossRefGoogle Scholar
  7. Beskos A, Papaspiliopoulos O, Roberts G, Fearnhead P (2006) Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion). J R Stat Soc Ser B Stat Methodol 68(3):333–382MathSciNetzbMATHCrossRefGoogle Scholar
  8. Beskos A, Papaspiliopoulos O, Roberts G (2009) Monte Carlo maximum likelihood estimation for discretely observed diffusion processes. Ann Stat 37(1):223–245MathSciNetzbMATHCrossRefGoogle Scholar
  9. Beskos A, Pillai NS, Roberts GO, Sanz-Serna JM, Stuart AM (2010) Optimal tuning of the hybrid Monte Carlo algorithm (submitted)Google Scholar
  10. Bishop CM (1996) Neural networks for pattern recognition. Oxford University, OxfordzbMATHGoogle Scholar
  11. Brandt M, Santa-Clara P (2002) Simulated likelihood estimation of diffusions with an application to exchange rate dynamics in incomplete markets. J Financ Econ 63(2):161–210CrossRefGoogle Scholar
  12. Butcher JC (2003) Numerical methods for ordinary differential equations. Wiley, New YorkzbMATHCrossRefGoogle Scholar
  13. Cappé O, Moulines E, Rydén T (2005) Inference in Hidden Markov models. Springer, BerlinzbMATHGoogle Scholar
  14. Chen L, Qin Z, Liu JS (2001) Exploring hybrid Monte Carlo in Bayesian computation. In: Proceedings of ISBAGoogle Scholar
  15. Doucet A, Godsill SJ, Andrieu C (2000) On sequential Monte Carlo sampling methods for Bayesian filtering. Stat Comput 10(3):197–208CrossRefGoogle Scholar
  16. Duane S, Kennedy AD, Pendleton BJ, Roweth D (1987) Hybrid Monte Carlo. Phys Lett B 195(2):216–222CrossRefGoogle Scholar
  17. Elerian O (1998) A note on the existence of a closed form conditional transition density for the Milstein scheme. Working paper, Nuffield College, Oxford UniversityGoogle Scholar
  18. Elerian O, Chib S, Shephard N (2001) Likelihood inference for discretely observed non-linear diffusions. Econometrica 69(4):959–993MathSciNetzbMATHCrossRefGoogle Scholar
  19. Eraker B (2001) MCMC analysis of diffusion models with application to finance. J Bus Econ Stat 19(2):177–191MathSciNetCrossRefGoogle Scholar
  20. Gelman A, Carlin JB, Stern HS, Rubin DB (2004) Bayesian data analysis. CRC/Chapman and Hall, LondonzbMATHGoogle Scholar
  21. Geyer CJ (1992) Practical Markov chain Monte Carlo. Stat Sci 7(4):473–483MathSciNetCrossRefGoogle Scholar
  22. Gilks WR, Richardson S, Spiegelhalter DJ (1996) Markov chain Monte Carlo in practice. Chapman and Hall/CRC, LondonzbMATHGoogle Scholar
  23. Golightly A, Wilkinson DJ (2006) Bayesian sequential inference for nonlinear multivariate diffusions. Stat Comput 16(4):323–338MathSciNetCrossRefGoogle Scholar
  24. Golightly A, Wilkinson DJ (2008) Bayesian inference for nonlinear multivariate diffusion models observed with error. Comput Stat Data Anal 52(3):1674–1693MathSciNetzbMATHCrossRefGoogle Scholar
  25. Grewal MS, Andrews AP (2001) Kalman filtering, theory and practice using MATLAB. Wiley-IEEE Press, New YorkGoogle Scholar
  26. Holder T, Leimkuhler B, Reich S (2001) Explicit variable step-size and time-reversible integration. Appl Numer Math 39(3–4):367–377MathSciNetzbMATHCrossRefGoogle Scholar
  27. Hurn AS, Lindsay KA (1999) Estimating the parameters of stochastic differential equations. Mathe Comput Simul 48(4–6):373–384MathSciNetzbMATHCrossRefGoogle Scholar
  28. Hurn AS, Lindsay KA, Martin VL (2003) On the efficacy of simulated maximum likelihood for estimating the parameters of stochastic differential equations. J Time Ser Anal 24(1):45–63MathSciNetzbMATHCrossRefGoogle Scholar
  29. Jazwinski AH (1970) Stochastic processes and filtering theory. Academic Press, EdinburghzbMATHGoogle Scholar
  30. Jeisman J (2005) Estimation of the Parameters of stochastic differential equations. Doctoral dissertation, Queensland University of TechnologyGoogle Scholar
  31. Jensen B, Poulsen R (2002) Transition densities of diffusion processes: numerical comparison of approximation techniques. J Deriv 9(4):18–32CrossRefGoogle Scholar
  32. Jones CS (1998) A simple Bayesian method for the analysis of diffusion processes. Working paper, University of PennsylvaniaGoogle Scholar
  33. Julier SJ, Uhlmann JK (2004) Unscented filtering and nonlinear estimation. Proc IEEE 92(3):401–422CrossRefGoogle Scholar
  34. Karatzas I, Shreve SE (1991) Brownian motion and stochastic calculus. Springer, BerlinzbMATHGoogle Scholar
  35. Kloeden PE, Platen E (1999) Numerical solution to stochastic differential equations. Springer, BerlinGoogle Scholar
  36. Liang F, Liu C, Carroll RJ (2010) Advanced Markov chain Monte Carlo methods: learning from past samples. Wiley, ChichesterzbMATHCrossRefGoogle Scholar
  37. Najfeld I, Havel TF (1995) Derivatives of the matrix exponential and their computation. Adv Appl Math 16(3):321–375MathSciNetzbMATHCrossRefGoogle Scholar
  38. Neal RM (2011) MCMC using Hamiltonian dynamics. In: Brooks S, Gelman A, Jones GL, Meng XL (eds) Handbook of Markov chain Monte Carlo, chap 5. Chapman& Hall/CRC, LondonGoogle Scholar
  39. Nielsen JN, Madsen H, Young PC (2000) Parameter estimation in stochastic differential equations: an overview. SIAM J Numer Anal 24:83–94Google Scholar
  40. Ninness B, Henriksen S (2010) Bayesian system identification via Markov chain Monte Carlo techniques. Automatica 46(1):40–51MathSciNetzbMATHCrossRefGoogle Scholar
  41. Øksendal B (2003) Stochastic differential equations: an introduction with applications. Springer, BerlinGoogle Scholar
  42. Ornstein LS, Uhlenbeck GE (1930) On the theory of the Brownian motion. Phys Rev 36(5):823–841zbMATHCrossRefGoogle Scholar
  43. Pedersen AR (1995) A new approach to maximum likelihood estimation for stochastic differential equations based on discrete observations. Scand J Stat 22(1):55–71zbMATHGoogle Scholar
  44. Särkkä S (2006) Recursive Bayesian inference on stochastic differential equations. Doctoral dissertation, Helsinki University of TechnologyGoogle Scholar
  45. Särkkä S (2007) On unscented Kalman filtering for state estimation of continuous-time nonlinear systems. IEEE Trans Autom Control 52(9):1631–1641CrossRefGoogle Scholar
  46. Särkkä S, Vehtari A, Lampinen J (2007) Rao-Blackwellized particle filter for multiple target tracking. Inf Fusion J 8(1):2–15CrossRefGoogle Scholar
  47. Shoji I, Ozaki T (1998) Estimation for nonlinear stochastic differential equations by a local linearization method. Stoch Anal Appl 16(4):733–752MathSciNetzbMATHCrossRefGoogle Scholar
  48. Singer H (2002) Parameter estimation of nonlinear stochastic differential equations: simulated maximum likelihood versus extended Kalman filter and Itô-Taylor expansion. J Comput Graph Stat 11(4):972–995Google Scholar
  49. Singer H (2004) A survey of estimation methods for stochastic differential equastions. In: Proceedings of 6th international conference on social science methodology, AmsterdamGoogle Scholar
  50. Singer H (2011) Continuous-discrete state-space modeling of panel data with nonlinear filter algorithms. AStA Adv Stat Anal 95(4):375–413MathSciNetzbMATHCrossRefGoogle Scholar
  51. Sørensen H (2004) Parametric inference for diffusion processes observed at discrete points in time: a survey. Int Stat Rev 72(3):337–354CrossRefGoogle Scholar
  52. Stramer O, Bognar M (2011) Bayesian inference for irreducible diffusion processes using the pseudo-marginal approach. Bayesian Anal 6(2):231–258MathSciNetCrossRefGoogle Scholar
  53. Stramer O, Bognar M, Schneider P (2010) Bayesian inference for discretely sampled Markov processes with closed-form likelihood expansions. J Financ Econ 8(4):450–480Google Scholar
  54. Young P (1980) Parameter estimation for continuous-time models: a survey. Automatica 17(1):23–39CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Isambi S. Mbalawata
    • 1
  • Simo Särkkä
    • 2
  • Heikki Haario
    • 1
  1. 1.Department of Mathematics and PhysicsLappeenranta University of TechnologyLappeenrantaFinland
  2. 2.Department of Biomedical Engineering and Computational ScienceAalto UniversityAaltoFinland

Personalised recommendations