A Quasi-stochastic Gradient Algorithm for Variance-Dependent Component Analysis

  • Aapo Hyvärinen
  • Shohei Shimizu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4132)


We discuss the blind source separation problem where the sources are not independent but are dependent only through their variances. Some estimation methods have been proposed on this line. However, most of them require some additional assumptions: a parametric model for their dependencies or a temporal structure of the sources, for example. In previous work, we have proposed a generalized least squares approach using fourth-order moments to the blind source separation problem in the general case where those additional assumptions do not hold. In this article, we develop a simple optimization algorithm for the least squares approach, or a quasi-stochastic gradient algorithm. The new algorithm is able to estimate variance-dependent components even when the number of variables is large and the number of moments is computationally prohibitive.


Independent Component Analysis Independent Component Analysis Blind Source Separation Generalize Little Square Independent Component Analysis Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jutten, C., Hérault, J.: Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture. Signal Processing 24, 1–10 (1991)MATHCrossRefGoogle Scholar
  2. 2.
    Comon, P.: Independent component analysis. a new concept? Signal Processing 36, 62–83 (1994)CrossRefGoogle Scholar
  3. 3.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent component analysis. Wiley, New York (2001)CrossRefGoogle Scholar
  4. 4.
    Bach, F.R., Jordan, M.I.: Tree-dependent component analysis. In: Proc. the 18th Conference on Uncertainty in Artificial Intelligence, UAI-2002 (2002)Google Scholar
  5. 5.
    Hyvärinen, A.: A unifying model for blind separation of independent sources. Signal Processing 85, 1419–1427 (2005)MATHCrossRefGoogle Scholar
  6. 6.
    Hyvärinen, A., Hoyer, P.O., Inki, M.: Topographic independent component analysis. Neural Computation 13, 1525–1558 (2001)Google Scholar
  7. 7.
    Hyvärinen, A., Hurri, J.: Blind separation of sources that have spatiotemporal dependencies. Signal Processing 84, 247–254 (2004)MATHCrossRefGoogle Scholar
  8. 8.
    Kawanabe, M., Müller, K.R.: Estimating functions for blind separation when sources have variance-dependencies. In: Proc. 5th International Conference on ICA and Blind Source Separation, Granada, Spain, pp. 136–143 (2004)Google Scholar
  9. 9.
    Shimizu, S., Hyvärinen, A., Kano, Y.: A generalized least squares approach to blind separation of sources which have variance dependency. In: Proc. IEEE Workshop on Statistical Signal Processing, SSP 2005 (2005)Google Scholar
  10. 10.
    Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. on Neural Networks 10, 626–634 (1999)CrossRefGoogle Scholar
  11. 11.
    Ferguson, T.S.: A method of generating best asymptotically normal estimates with application to estimation of bacterial densities. Annals of Mathematical Statistics 29, 1046–1062 (1958)MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications 20, 303–353 (1998)MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Hyvärinen, A.: The fixed-point algorithm and maximum likelihood estimation for independent component analysis. Neural Processing Letters 10, 1–5 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Aapo Hyvärinen
    • 1
  • Shohei Shimizu
    • 1
    • 2
  1. 1.Helsinki Institute for Information TechnologyUniversity of HelsinkiFinland
  2. 2.The Institute of Statistical MathematicsJapan

Personalised recommendations