State Inference in Variational Bayesian Nonlinear State-Space Models

  • Tapani Raiko
  • Matti Tornio
  • Antti Honkela
  • Juha Karhunen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3889)


Nonlinear source separation can be performed by inferring the state of a nonlinear state-space model. We study and improve the inference algorithm in the variational Bayesian blind source separation model introduced by Valpola and Karhunen in 2002. As comparison methods we use extensions of the Kalman filter that are widely used inference methods in tracking and control theory. The results in stability, speed, and accuracy favour our method especially in difficult inference problems.


Independent Component Analysis Source Separation Blind Source Separation Inference Algorithm Speech Spectrum 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Valpola, H., Karhunen, J.: An unsupervised ensemble learning method for nonlinear dynamic state-space models. Neural Computation 14(11), 2647–2692 (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    Cichocki, A., Zhang, L., Choi, S., Amari, S.-I.: Nonlinear dynamic independent component analysis using state-space and neural network models. In: Proc. of the 1st Int. Workshop on Independent Component Analysis and Signal Separation (ICA 1999), Aussois, France, January 11-15, pp. 99–104 (1999)Google Scholar
  3. 3.
    Anderson, B., Moore, J.: Optimal Filtering. Prentice-Hall, Englewood Cliffs (1979)zbMATHGoogle Scholar
  4. 4.
    Koivunen, V., Enescu, M., Oja, E.: Adaptive algorithm for blind separation from noisy time-varying mixtures. Neural Computation 13, 2339–2357 (2001)CrossRefzbMATHGoogle Scholar
  5. 5.
    Julier, S., Uhlmann, J.: A new extension of the Kalman filter to nonlinear systems. In: Int. Symp. Aerospace/Defense Sensing, Simul. and Controls (1997)Google Scholar
  6. 6.
    Wan, E.A., van der Merwe, R.: The unscented Kalman filter. In: Haykin, S. (ed.) Kalman Filtering and Neural Networks, pp. 221–280. Wiley, New York (2001)CrossRefGoogle Scholar
  7. 7.
    Honkela, A., Valpola, H.: Unsupervised variational Bayesian learning of nonlinear models. In: Saul, L., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 17, pp. 593–600. MIT Press, Cambridge (2005)Google Scholar
  8. 8.
    Psiaki, M.: Backward-smoothing extended Kalman filter. Journal of Guidance, Control, and Dynamics 28 (September–October 2005)Google Scholar
  9. 9.
    Doucet, A., de Freitas, N., Gordon, N.J.: Sequential Monte Carlo Methods in Practice. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  10. 10.
    Everson, R., Roberts, S.: Particle filters for non-stationary ICA. In: Girolami, M. (ed.) Advances in Independent Component Analysis, pp. 23–41. Springer, Heidelberg (2000)Google Scholar
  11. 11.
    Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. The Computer Journal 7, 149–154 (1964)CrossRefMathSciNetzbMATHGoogle Scholar
  12. 12.
    Raiko, T., Tornio, M.: Learning nonlinear state-space models for control. In: Proc. Int. Joint Conf. on Neural Networks (IJCNN 2005), Montreal, Canada, pp. 815–820 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Tapani Raiko
    • 1
  • Matti Tornio
    • 1
  • Antti Honkela
    • 1
  • Juha Karhunen
    • 1
  1. 1.Neural Networks Research CentreHelsinki University of Technology, HUTEspooFinland

Personalised recommendations