Effective Blind Source Separation Based on the Adam Algorithm

  • Michele Scarpiniti
  • Simone Scardapane
  • Danilo Comminiello
  • Raffaele Parisi
  • Aurelio Uncini
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 69)


In this paper, we derive a modified InfoMax algorithm for the solution of Blind Signal Separation (BSS) problems by using advanced stochastic methods. The proposed approach is based on a novel stochastic optimization approach known as the Adaptive Moment Estimation (Adam) algorithm. The proposed BSS solution can benefit from the excellent properties of the Adam approach. In order to derive the new learning rule, the Adam algorithm is introduced in the derivation of the cost function maximization in the standard InfoMax algorithm. The natural gradient adaptation is also considered. Finally, some experimental results show the effectiveness of the proposed approach.


Blind source separation Stochastic optimization Adam algorithm Infomax algorithm Natural gradient 


  1. 1.
    Amari, S.: Natural gradient works efficiently in learning. Neural Comput. 10(3), 251–276 (1998)CrossRefGoogle Scholar
  2. 2.
    Araki, S., Mukai, R., Makino, S., Nishikawa, T., Saruwatari, H.: The fundamental limitation of frequency domain blind source separation for convolutive mixtures of speech. IEEE Trans. Speech Audio Process. 11(2), 109–116 (2003)CrossRefMATHGoogle Scholar
  3. 3.
    Bell, A.J., Sejnowski, T.J.: An information-maximisation approach to blind separation and blind deconvolution. Neural Comput. 7(6), 1129–1159 (1995)CrossRefGoogle Scholar
  4. 4.
    Boulmezaoud, T.Z., El Rhabi, M., Fenniri, H., Moreau, E.: On convolutive blind source separation in a noisy context and a total variation regularization. In: Proceedings of IEEE Eleventh International Workshop on Signal Processing Advances in Wireless Communications (SPAWC2010), pp. 1–5. Marrakech (20–23 June 2010)Google Scholar
  5. 5.
    Choi, S., Cichocki, A., Park, H.M., Lee, S.Y.: Blind source separation and independent component analysis: a review. Neural Inf. Process. Lett. Rev. 6(1), 1–57 (2005)Google Scholar
  6. 6.
    Cichocki, A., Amari, S.: Adaptive Blind Signal and Image Processing. Wiley (2002)Google Scholar
  7. 7.
    Comon, P., Jutten, C. (eds.): Handbook of Blind Source Separation. Springer (2010)Google Scholar
  8. 8.
    Douglas, S.C., Gupta, M.: Scaled natural gradient algorithm for instantaneous and convolutive blind source separation. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2007), vol. 2, pp. 637–640 (2007)Google Scholar
  9. 9.
    Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12(7), 2121–2159 (2011)MathSciNetMATHGoogle Scholar
  10. 10.
    Haykin, S. (ed.): Unsupervised Adaptive Filtering, vol. 2: Blind Source Separation. Wiley (2000)Google Scholar
  11. 11.
    Inuso, G., La Foresta, F., Mammone, N., Morabito, F.C.: Wavelet-ICA methodology for efficient artifact removal from electroencephalographic recordings. In: Proceedings of International Joint Conference on Neural Networks (IJCNN2007)Google Scholar
  12. 12.
    Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (ICLR2015), pp. 1–13 (2015). arXiv:1412.6980
  13. 13.
    Liu, J.Q., Feng, D.Z., Zhang, W.W.: Adaptive improved natural gradient algorithm for blind source separation. Neural Comput. 21(3), 872–889 (2009)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Papoulis, A.: Probability, Random Variables and Stochastic Processes. McGraw-Hill (1991)Google Scholar
  15. 15.
    Pascanu, R., Bengio, Y.: Revisiting natural gradient for deep networks. In: International Conference on Learning Representations (April 2014)Google Scholar
  16. 16.
    Scarpiniti, M., Vigliano, D., Parisi, R., Uncini, A.: Generalized splitting functions for blind separation of complex signals. Neurocomputing 71(10–12), 2245–2270 (2008)CrossRefGoogle Scholar
  17. 17.
    Smaragdis, P.: Blind separation of convolved mixtures in the frequency domain. Neurocomputing 22(21–34) (1998)Google Scholar
  18. 18.
    Thomas, P., Allen, G., August, N.: Step-size control in blind source separation. In: International Workshop on Independent Component Analysis and Blind Source Separation, pp. 509–514 (2000)Google Scholar
  19. 19.
    Tieleman, T., Hinton, G.: Lecture 6.5—RMSProp. Technical report, COURSERA: Neural Networks for Machine Learning (2012)Google Scholar
  20. 20.
    Vigliano, D., Scarpiniti, M., Parisi, R., Uncini, A.: Flexible nonlinear blind signal separation in the complex domain. Int. J. Neural Syst. 18(2), 105–122 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Michele Scarpiniti
    • 1
  • Simone Scardapane
    • 1
  • Danilo Comminiello
    • 1
  • Raffaele Parisi
    • 1
  • Aurelio Uncini
    • 1
  1. 1.Department of Information Engineering, Electronics and Telecommunications (DIET)“Sapienza” University of RomeRomeItaly

Personalised recommendations