The Independence Assumption: Dependent Component Analysis

  • Allan Kardec Barros
Part of the Perspectives in Neural Computing book series (PERSPECT.NEURAL)


Redundancy reduction as a form of neural coding has been a topic of great research interest since the early sixties. A number of strategies have been proposed, but the one which is attracting most attention recently assumes that this coding is carried out so that the output signals are as independent as possible. In this work, we go one step further and propose an algorithm to separate non-orthogonal signals (i.e., dependent signals) based on the minimization of the output mutual spectral overlap. Indeed, separating independent sources turns to be a special case of this strategy. Moreover, we show that this principle can also be used to separate spectrally overlapping signals by exploiting their higher-order cyclostationary properties. We also suggest a numerically-efficient algorithm which searches for the learning step size in a way that avoids divergence.


Independent Component Analysis Independent Component Analysis Blind Source Separation Blind Signal Independent Component Analysis Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    S. Amari, A. Cichocki, H. H. Yang. A new learning algorithm for blind signal separation. In D. S. Touretzky, M. C. Mozer and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems, 8, MIT Press, Cambridge, MA, 1996.Google Scholar
  2. 2.
    S. Amari and A. Cichocki. Adaptive blind signal processing: neural network approaches, Proceedings IEEE (invited paper), 86: 2026–2048, 1998.CrossRefGoogle Scholar
  3. 3.
    Y. Baram and Z. Roth. Density shaping by neural networks with application to classification, estimation and forecasting. CIS report no. 9420. Center for Intelligent Systems, Technion, Israel, 1994.Google Scholar
  4. 4.
    H. B. Barlow, Possible principles underlying the transformations of sensory messages. In W. Rosenblith editor Sensory Communication, pp. 217–234. MIT Press, Cambridge, MA, 1961.Google Scholar
  5. 5.
    H. B. Barlow, Unsupervised learning. Neural Computation, 1: 295–311, 1989.CrossRefGoogle Scholar
  6. 6.
    A. K. Barros, A. Mansour and N. Ohnishi, Removing artifacts from ECG signals using independent components analysis”. Neurocomputing, 22: 173–186, 1998.MATHCrossRefGoogle Scholar
  7. 7.
    A. J. Bell and T. J. Sejnowski, An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7: 1129–1159, 1995.CrossRefGoogle Scholar
  8. 8.
    J-F. Cardoso and B. Hvam Laheld. Equivariant adaptive source separation. IEEE Trans, on Signal Process, 3017–3030, 1996.Google Scholar
  9. 9.
    A. Cichocki and L. Moszczynski. A new learning algorithm for blind separation of sources. Electronics Letters, 28(21): 1986–1987, 1992.CrossRefGoogle Scholar
  10. 10.
    P. Comon. Independent component analysis, a new concept? Signal Processing, 24: 287–314, 1994.CrossRefGoogle Scholar
  11. 11.
    G. Deco and W. Brauer. Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures. Neural Networks, 8: 525–535, 1995.CrossRefGoogle Scholar
  12. 12.
    W. Gardner (editor). Cyclostationarity in communications and signal processign. IEEE Press, 1994.Google Scholar
  13. 13.
    A. Hyvärinen and E. Oja. A fast fixed-point algorithm for independent component analysis. Neural Computation 9: 1483–1492, 1997.CrossRefGoogle Scholar
  14. 14.
    C. Jutten and J. Hárault. Independent component analysis versus PCA. Proc. EUSIPCO, pp. 643–646, 1988.Google Scholar
  15. 15.
    S. Makeig, T-P. Jung, D. Ghahremani, A. J. Bell, T. J. Sejnowski. Blind separation of event-related brain responses into independent components. Proc. Natl. Acad. Sci. USA, 94: 10979–10984, 1997.CrossRefGoogle Scholar
  16. 16.
    J-P. Nadal and N. Parga. Non-linear neurons in the low noise limit: a factorial code maximises information transfer. Network, 5: 565–581, 1994.MATHCrossRefGoogle Scholar
  17. 17.
    D. Nuzillard and J-M. Nuzillard. Blind source separation applied to non-orthogonal signals. Proc. ICA ‘99, 25–30, 1999.Google Scholar
  18. 18.
    J.A. Simpson, and E.S.C Weiner. The Oxford English Dictionary, 2nd edn. Clarendon Press, Oxford, 1989.Google Scholar
  19. 19.
    A. Papoulis. Probability, random variables, and stochastic processes. McGraw-Hill, 1991.Google Scholar

Copyright information

© Springer-Verlag London 2000

Authors and Affiliations

  • Allan Kardec Barros

There are no affiliations available

Personalised recommendations