Skip to main content

Caveats with Stochastic Gradient and Maximum Likelihood Based ICA for EEG

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 10169)


Stochastic gradient (SG) is the most commonly used optimization technique for maximum likelihood based approaches to independent component analysis (ICA). It is in particular the default solver in public implementations of Infomax and variants. Motivated by experimental findings on electroencephalography (EEG) data, we report some caveats which can impact the results and interpretation of neuroscience findings. We investigate issues raised by controlling the step size in gradient updates combined with early stopping conditions, as well as initialization choices which can artificially generate biologically plausible brain sources, so called dipolar sources. We provide experimental evidence that pushing the convergence of Infomax using non stochastic solvers can reduce the number of highly dipolar components and provide a mathematical explanation of this fact. Results are presented on public EEG data.


  • Independent component analysis (ICA)
  • Maximum likelihood
  • Stochastic gradient method
  • Infomax
  • Electroencephalography (EEG)
  • Neuroscience

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-53547-0_27
  • Chapter length: 11 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-53547-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.


  1. Amari, S.-I.: Natural gradient works efficiently in learning. Neural Comput. 10(2), 251–276 (1998)

    CrossRef  Google Scholar 

  2. Bell, A.J., Sejnowski, T.J.: An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7(6), 1129–1159 (1995)

    CrossRef  Google Scholar 

  3. Bertsekas, D.: Nonlinear Programming. Athena Scientific, Cambridge (1999)

    MATH  Google Scholar 

  4. Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Lechevallier, Y., Saporta, G. (eds.) Proceedings of COMPSTAT’2010, pp. 177–186. Springer, Heidelberg (2010)

    Google Scholar 

  5. Cardoso, J.F.: Infomax and maximum likelihood for blind source separation. IEEE Signal Process. Lett. 4, 112–114 (1997)

    CrossRef  Google Scholar 

  6. Cardoso, J.F.: Blind signal separation: statistical principles. Proc. IEEE 86(10), 2009–2025 (1998)

    CrossRef  Google Scholar 

  7. Cardoso, J.F., Laheld, B.H.: Equivariant adaptive source separation. IEEE Trans. Sig. Proc. 44(12), 3017–3030 (1996)

    CrossRef  Google Scholar 

  8. Delorme, A., Makeig, S.: EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134(1), 9–21 (2004)

    CrossRef  Google Scholar 

  9. Delorme, A., Palmer, J., Onton, J., Oostenveld, R., Makeig, S., et al.: Independent EEG sources are dipolar. PloS One 7(2), e30135 (2012)

    CrossRef  Google Scholar 

  10. Eldar, Y.C., Oppenheim, A.V.: MMSE whitening and subspace whitening. IEEE Trans. Inf. Theor. 49(7), 1846–1851 (2003)

    MathSciNet  CrossRef  MATH  Google Scholar 

  11. Gramfort, A., et al.: MNE software for MEG and EEG data. Neuroimage 86, 446–460 (2014)

    CrossRef  Google Scholar 

  12. Jutten, C., Herault, J.: Blind separation of sources, Part I: an adaptive algorithm based on neuromimetic architecture. Signal Process. 24(1), 1–10 (1991)

    CrossRef  MATH  Google Scholar 

  13. Makeig, S., Bell, A.J., Jung, T.P., Sejnowski, T.J.: Independent component analysis of electroencephalographic data. In: NIPS, pp. 145–151 (1996)

    Google Scholar 

  14. Nunez, S.: Electric Fields of the Brain: The Neurophysics of EEG. Oxford University Press, New York (2006)

    CrossRef  Google Scholar 

  15. Onton, J., Delorme, A., Makeig, S.: Frontal midline EEG dynamics during working memory. Neuroimage 27(2), 341–356 (2005)

    CrossRef  Google Scholar 

  16. Pham, D.T., Garat, P.: Blind separation of mixture of independent sources through a quasi-maximum likelihood approach. IEEE Trans. Signal Process. 45(7), 1712–1725 (1997)

    CrossRef  MATH  Google Scholar 

  17. Schaul, T., Zhang, S., LeCun, Y.: No more pesky learning rates. In: Proceedings of ICML Conference (2013)

    Google Scholar 

  18. Scherg, M., Berg, P.: Use of prior knowledge in brain electromagnetic source analysis. Brain Topogr. 4(2), 143–150 (1991)

    CrossRef  Google Scholar 

Download references


This work was funded by the Paris-Saclay Center for Data Science with partial support from the European Research Council (ERC-YStG-676943).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Jair Montoya-Martínez .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Montoya-Martínez, J., Cardoso, JF., Gramfort, A. (2017). Caveats with Stochastic Gradient and Maximum Likelihood Based ICA for EEG. In: Tichavský, P., Babaie-Zadeh, M., Michel, O., Thirion-Moreau, N. (eds) Latent Variable Analysis and Signal Separation. LVA/ICA 2017. Lecture Notes in Computer Science(), vol 10169. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-53546-3

  • Online ISBN: 978-3-319-53547-0

  • eBook Packages: Computer ScienceComputer Science (R0)