Abstract
We describe a novel deterministic approximate inference technique for conditionally Gaussian state space models, i.e. state space models where the latent state consists of both multinomial and Gaussian distributed variables. The method can be interpreted as a smoothing pass and iteration scheme symmetric to an assumed density filter. It improves upon previously proposed smoothing passes by not making more approximations than implied by the projection onto the chosen parametric form, the assumed density. Experimental results show that the novel scheme outperforms these alternative deterministic smoothing passes. Comparisons with sampling methods suggest that the performance does not degrade with longer sequences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bar-Shalom Y. and Fortmann T. 1988. Tracking and Data Association. Academic Press.
Bar-Shalom Y. and Li. X.-R. 1993. Estimation and Tracking: Principles, Techniques, and Software. Artech House.
Boyen X. and Koller D. 1998. Tractable inference for complex stochastic processes. In Proceedings of the 14th Annual Conference on Uncertanty in Artificial Intelligence.
Carter C. and Kohn R. 1996. Markov chain Monte Carlo in conditionally Gaussian state space models. Biometrika 83(3):589–601.
Chen R. and Liu J. S. 2000. Mixture Kalman filters. Journal of the Royal Statistical Society, Series B 62:493–508.
Cover T. M. and Thomas J. A. 1991. Elements of Information Theory. John Wiley & Sons.pp
Doucet A., de Freitas N., Murphy K., and Russel S. 2001. Rao-Blackwellized particle filtering for dynamic Bayesian networks. In Proceedings of the 17th Annual Conference on Uncertainty in Artificial Intelligence (UAI 2001), San Francisco, CA. Morgan Kaufmann Publishers.
Hamilton J. 1989. A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica 57.
Harrison P. J. and Stevens C. F. 1976. Bayesian forecasting. Journal of the Royal Statistical Society Society B 38:205–247.
Helmick R. E., Blair W. D., and Hoffman S. A. 1995. Fixed-interval smoothing for Markovian switching systems. IEEE Transactions on Information Theory 41:1845–1855.
Heskes T. and Zoeter O. 2002. Expectation propagation for approximate inference in dynamic Bayesian networks. In Proceedings of the 18th Annual Conference on Uncertainty in Artificial Intelligence (UAI), San Francisco, CA. Morgan Kaufmann Publishers.
Kim C. -J. and Nelson Ch. R. 1999. State-Space Models with Regime Switching. MIT Press.
Kitagawa G. 1994. The two-filter formula for smoothing and an implementation of the Gaussian-sum filter. Annals of the Institute of Statistical Mathematics 46(4):605–623.
Kitagawa G. 1996. Monte Carlo filter and smoother for non-gaussian nonlinear state space models. Journal of Computational and Graphical Statistics 5(1):1–25.
Kschischang F. R., Frey B. J. and Loeliger H.-A. 2001. Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory 47(2):498–519.
Lauritzen S. L. 1992. Propagation of probabilities, means, and variances in mixed graphical association models. Journal of the American Statistical Association 87:1098–1108.
Lerner U. and Parr R. 2001. Inference in hybrid networks: Theoretical limits and practical algorithms. In Proceedings of the 17th Annual Conference on Uncertainty in Artificial Intelligence (UAI 2001). Morgan Kaufmann Publishers.
Lerner U., Parr R., Koller D. and Biswas G. 2000. Bayesian fault detection and diagnosis in dynamic systems. In Proceedings of the 17th National Conference on Artificial Intelligence (AAAI), pp. 531–537.
Minka T. 2001 Expectation propagation for approximate Bayesian inference. In Proceedings of the 17th Annual Conference on Uncertainty in Artificial Intelligence (UAI 2001). Morgan Kaufmann Publishers.
Shumway R. H. and Stoffer D. S. 1991. Dynamic linear models with switching. Journal of the American Statistical Association, 86:763–769.
Whittaker J. 1989. Graphical Models in Applied Multivariate Statistics. John Wiley & Sons.
Yedidia J., Freeman W., and Weiss Y. 2001. Generalized belief propagation. In NIPS 13, pp. 689–695.
Zoeter O. and Heskes T. 2005. Change point problems in linear dynamical systems. Journal of Machine Learning Research 6:1999–2026.
Zoeter O. and Heskes T. 2003. Hierarchical visualization of time-series data using switching linear dynamical systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(10):1202–1214.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zoeter, O., Heskes, T. Deterministic approximate inference techniques for conditionally Gaussian state space models. Stat Comput 16, 279–292 (2006). https://doi.org/10.1007/s11222-006-8770-8
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11222-006-8770-8