Approximate Joint Diagonalization Using a Natural Gradient Approach

  • Arie Yeredor
  • Andreas Ziehe
  • Klaus-Robert Müller
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3195)


We present a new algorithm for non-unitary approximate joint diagonalization (AJD), based on a “natural gradient”-type multi-plicative update of the diagonalizing matrix, complemented by step-size optimization at each iteration. The advantages of the new algorithm over existing non-unitary AJD algorithms are in the ability to accommodate non-positive-definite matrices (compared to Pham’s algorithm), in the low computational load per iteration (compared to Yeredor’s AC-DC algorithm), and in the theoretically guaranteed convergence to a true (possibly local) minimum (compared to Ziehe et al.’s FFDiag algorithm).


Independent Component Analysis Independent Component Analysis Blind Source Separation Natural Gradient Target Matrice 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cardoso, J.F., Souloumiac, A.: Blind beamforming for non Gaussian signals. IEE - Proceedings -F 140, 362–370 (1993)Google Scholar
  2. 2.
    Belouchrani, A., Abed-Meraim, K., Cardoso, J.F., Moulines, E.: A blind source separation technique using second-order statistics. IEEE Trans. Signal Processing 45, 434–444 (1997)CrossRefGoogle Scholar
  3. 3.
    Ziehe, A., Nolte, G., Curio, G., Müller, K.R.: OFI: Optimal filtering algorithms for source separation. In: Proc. ICA 2000, Helsinki, Finland, pp. 127–132 (2000)Google Scholar
  4. 4.
    Belouchrani, A., Amin, M.G.: Blind source separation based on time-frequency signal representations. IEEE Trans. Signal Processing 46, 2888–2897 (1998)CrossRefGoogle Scholar
  5. 5.
    Yeredor, A.: Blind source separation via the second characteristic function. Signal Processing 80, 897–902 (2000)zbMATHCrossRefGoogle Scholar
  6. 6.
    Murata, N., Ikeda, S., Ziehe, A.: An approach to blind source separation based on temporal structure of speech signals. Neurocomputing 41, 1–24 (2001)zbMATHCrossRefGoogle Scholar
  7. 7.
    Rahbar, K., Reilly, J.P., Manton, J.H.: Blind identification of MIMO FIR systems driven by quasistationary sources using second-order statistics: A frequency domain approach. IEEE Trans. Signal Processing 52, 406–417 (2004)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Cardoso, J.F., Souloumiac, A.: Jacobi angles for simultaneous diagonalization. SIAM Journal on Matrix Analysis and Applications 17, 161–164 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Cardoso, J.F.: On the performance of orthogonal source separation algorithms. In: Proceedings of EUSIPCO 1994, pp. 776–779 (1994)Google Scholar
  10. 10.
    Yeredor, A.: Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation. IEEE Trans. Signal Processing 50, 1545–1553 (2002)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Ziehe, A., Laskov, P., Müller, K.R., Nolte, G.: A linear least-squares algorithm for joint diagonalization. In: Proceedings ICA 2003, pp. 469–474 (2003)Google Scholar
  12. 12.
    Pham, D.T.: Joint approximate diagonalization of positive definite matrices. SIAM J. on Matrix Anal. and Appl. 22, 1136–1152 (2001)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Amari, S.I., Douglas, S.: Why natural gradient. In: ICASSP 1998, vol. 2, pp. 1213–1216 (1998)Google Scholar
  14. 14.
    Joho, M., Mathis, H.: Joint diagonalization of correlation matrices by using gradient methods with application to blind signal separation. In: Proc. of IEEE Sensor Array and Multichannel Signal Processing Workshop SAM, pp. 273–277 (2002)Google Scholar
  15. 15.
    Amari, S., Cichocki, A., Yang, H.H.: A new learning algorithm for blind source separation. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Advances in Neural Information Processing Systems, vol. 8, pp. 757–763. MIT Press, Cambridge (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Arie Yeredor
    • 1
  • Andreas Ziehe
    • 2
  • Klaus-Robert Müller
    • 2
  1. 1.School of Electrical EngineeringTel-Aviv UniversityIsrael
  2. 2.Fraunhofer FIRSTGermany

Personalised recommendations