Smoothing algorithms for state–space models



Two-filter smoothing is a principled approach for performing optimal smoothing in non-linear non-Gaussian state–space models where the smoothing distributions are computed through the combination of ‘forward’ and ‘backward’ time filters. The ‘forward’ filter is the standard Bayesian filter but the ‘backward’ filter, generally referred to as the backward information filter, is not a probability measure on the space of the hidden Markov process. In cases where the backward information filter can be computed in closed form, this technical point is not important. However, for general state–space models where there is no closed form expression, this prohibits the use of flexible numerical techniques such as Sequential Monte Carlo (SMC) to approximate the two-filter smoothing formula. We propose here a generalised two-filter smoothing formula which only requires approximating probability distributions and applies to any state–space model, removing the need to make restrictive assumptions used in previous approaches to this problem. SMC algorithms are developed to implement this generalised recursion and we illustrate their performance on various problems.


Sequential Monte Carlo Two-filter smoothing State–space models Rao-Blackwellisation Non-linear diffusion Parameter estimation 


  1. Andrieu C., Doucet A. (2002). Particle filtering for partially observed Gaussian state–space models. Journal of the Royal Statistical Society B, 64, 4, 827–836.MATHCrossRefMathSciNetGoogle Scholar
  2. Bresler Y. (1986) Two-filter formula for discrete-time non-linear Bayesian smoothing. International Journal of Control 43(2): 629–641MATHCrossRefGoogle Scholar
  3. Briers, M., Doucet, A., Maskell, S. (2004). Smoothing algorithms for state–space models, Technical report, Cambridge University CUED/F-INFENG.TR. 498.Google Scholar
  4. Briers, M., Doucet, A., Singh, S. S. (2005). Sequential auxiliary particle belief propagation. In Eighth international conference of information fusion.Google Scholar
  5. Cappé O., Doucet A., Moulines E., Lavielle M. (1999) Simulation-based methods for blind maximum-likelihood filter identification. Signal Processing 73(1): 3–25MATHCrossRefGoogle Scholar
  6. Chen R., Liu J.S. (2000) Mixture Kalman filters. Journal of the Royal Statistical Society B 62: 493–508MATHCrossRefGoogle Scholar
  7. de Jong P. (1997) The scan sampler. Biometrika 84: 929–937MATHCrossRefMathSciNetGoogle Scholar
  8. Del Moral P. (2004) Feynman-Kac formulae: Genealogical and interacting particle systems with applications, series probability and applications. Springer, New YorkGoogle Scholar
  9. Doucet A., Andrieu C. (2001) Iterative algorithms for state estimation of jump Markov linear systems. IEEE Transactions on Signal Processing, 49(6): 1216–1227CrossRefGoogle Scholar
  10. Doucet A., Godsill S.J., Andrieu C. (2000) On sequential Monte Carlo sampling methods for Bayesian filtering. Statistics and Computing 10: 197–208CrossRefGoogle Scholar
  11. Doucet A., de Freitas J.F.G., Gordon N.J. (2001) Sequential Monte Carlo methods in practice. Springer, New YorkMATHGoogle Scholar
  12. Fearnhead P., Papaspiliopoulos O., Roberts G.O. (2008) Particle filters for partially-observed diffusions. Journal of the Royal Statistical Society B 70(4): 755–777MATHCrossRefMathSciNetGoogle Scholar
  13. Fearnhead, P., Wyncoll, D., Tawn, J. (2008b). A sequential smoothing algorithm with linear computational cost. Technical report, Department of Statistics, Lancaster University.Google Scholar
  14. Godsill S.J., Doucet A., West M. (2004) Monte Carlo smoothing for non-linear time series. Journal of the American Statistical Association 99: 156–168MATHCrossRefMathSciNetGoogle Scholar
  15. Helmick R.E., Blair W.D., Hoffman S.A. (1995) Fixed-interval smoothing for Markovian switching systems. IEEE Transactions on Information Theory 41(6): 1845–1855MATHCrossRefGoogle Scholar
  16. Isard, M., Blake, A. (1998). A smoothing filter for Condensation. In Fifth European conference on computer vision (pp. 767–781).Google Scholar
  17. Kitagawa G. (1987) Non-Gaussian state–space modeling of nonstationary time series. Journal of the American Statistical Association 82: 1032–1063MATHCrossRefMathSciNetGoogle Scholar
  18. Kitagawa G. (1994) The two-filter formula for smoothing and an implementation of the Gaussian-sum smoother. Annals of the Institute of Statistical Mathematics 46(4): 605–623MATHCrossRefMathSciNetGoogle Scholar
  19. Kitagawa G. (1996) Monte Carlo filter and smoother for non-Gaussian non-linear state space models. Journal of Computational and Graphical Statistics 5: 1–25CrossRefMathSciNetGoogle Scholar
  20. Kitagawa, G., Sato, S. (2001). Monte Carlo smoothing and self-organizing state–space model. In A. Doucet, J. F. G. de Freitas, N. J. Gordon (Eds.) Sequential Monte Carlo methods in practice (pp. 177–195). New York: SpringerGoogle Scholar
  21. Klaas, M., Briers, M., De Freitas, N., Doucet, A., Maskell, S., Lang, D. (2006). Fast particle smoothing: If I had a million particles. In International conference on machine learning.Google Scholar
  22. Liu J.S., Chen R. (1998) Sequential Monte Carlo methods for dynamic systems. Journal of the American Statistical Association 93: 1032–1044MATHCrossRefMathSciNetGoogle Scholar
  23. Mayne D.Q. (1966) A solution of the smoothing problem for linear dynamic systems. Automatica 4: 73–92MATHCrossRefGoogle Scholar

Copyright information

© The Institute of Statistical Mathematics, Tokyo 2009

Authors and Affiliations

  1. 1.Information Engineering DivisionCambridge UniversityCambridgeUK
  2. 2.The Institute of Statistical MathematicsTokyoJapan
  3. 3.QinetiQ LtdWorcesterUK

Personalised recommendations