Auditory Feedback for Brain Computer Interface Management – An EEG Data Sonification Approach

  • Tomasz M. Rutkowski
  • Francois Vialatte
  • Andrzej Cichocki
  • Danilo P. Mandic
  • Allan Kardec Barros
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4253)

Abstract

An auditory feedback for Brain Computer Interface (BCI) applications is proposed. This is achieved based on the so-called sonification of the mental states of humans, captured by Electro-Encephalogram (EEG) recordings. Two time-frequency signal decomposition techniques, the Bump Modelling and Empirical Mode Decomposition (EMD), are used to map the EEG recordings onto musical scores. This auditory feedback proves to have extremely high potential in the development of on-line BCI interfaces. Examples based on the responses from visual stimuli support the analysis.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain–computer interfaces for communication and control. Clinical Neurophysiology 113, 767–791 (2002)CrossRefGoogle Scholar
  2. 2.
    Wolpaw, J.R., McFarland, D.J.: Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proceedings of National Academy of Sciences of the United States America 101(51), 17849–17854 (2004)CrossRefGoogle Scholar
  3. 3.
    Niedermeyer, E., Da Silva, F.L. (eds.): Electroencephalography: Basic Principles, Clinical Applications, and Related Fields, 5th edn. Lippincott Williams & Wilkins (2004)Google Scholar
  4. 4.
    Kelly, S.P., Lalor, E.C., Finucane, C., McDarby, G., Reilly, R.B.: Visual spatial attention control in an independent brain-computer interface. IEEE Transactions on Biomedical Engineering 52(9), 1588–1596 (2005)CrossRefGoogle Scholar
  5. 5.
    Miranda, E., Brouse, A.: Interfacing the brain directly with musical systems: On developing systems for making music with brain signals. LEONARDO 38(4), 331–336 (2005)CrossRefGoogle Scholar
  6. 6.
    Jovanov, E., Starcevic, D., Samardzic, A., Marsh, A., Obrenovic, Z.: EEG analysis in a telemedical virtual world. Future Generation Computer Systems 15, 255–263 (1999)CrossRefGoogle Scholar
  7. 7.
    Vialatte, F.: Modelisation en bosses pour l’analyse des motifs oscillatoires reproductibles dans l’activite de populations neuronales: applications a l’apprentissage olfactif chez l’animal et a la detection precoce de la maladie d’Alzheimer. PhD thesis, Paris VI University, Paris (2005), http://www.neurones.espci.fr/Theses_PS/VIALATTE_F.pdf
  8. 8.
    Vialatte, F., Cichocki, A., Dreyfus, G., Musha, T., Rutkowski, T., Gervais, R.: Blind source separation and sparse bump modelling of time frequency representation of EEG signals: New tools for early detection of Alzheimer’s disease. In: Proceedings of the 2005 IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing, Mystic CT, USA, pp. 27–32. IEEE, Los Alamitos (2005)CrossRefGoogle Scholar
  9. 9.
    Rilling, G., Flandrin, P., Goncalves, P.: On empirical mode decomposition and its algorithms. In: Proceedings of IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing, NSIP 2003. IEEE, Los Alamitos (2003)Google Scholar
  10. 10.
    Huang, N., Shen, Z., Long, S., Wu, M., Shih, H., Zheng, Q., Yen, N.C., Tung, C., Liu, H.: The empirical mode decomposition and the hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 454(1971), 903–995 (1998)MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Eerola, T., Toiviainen, P.: MIR in Matlab: The MIDI toolbox. In: Proceedings of The 5th International Conference on Music Information Retrieval, ISMIR 2004, Barcelona, Spain, Audiovisual Institute, Universitat Pompeu Fabra, pp. 22–27 (2004)Google Scholar
  12. 12.
    Przybyszewski, A., Rutkowski, T.: Processing of the incomplete representation of the visual world. In: Proceedings of the First Warsaw International Seminar on Intelligent Systems, WISIS 2004, Warsaw, Poland (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Tomasz M. Rutkowski
    • 1
  • Francois Vialatte
    • 1
  • Andrzej Cichocki
    • 1
  • Danilo P. Mandic
    • 2
  • Allan Kardec Barros
    • 3
  1. 1.Laboratory for Advanced Brain Signal ProcessingBrain Science Institute RIKENJapan
  2. 2.Department of Electrical and Electronic EngineeringImperial College LondonUnited Kingdom
  3. 3.Laboratory for Biological Information ProcessingUniversidade Federal do MaranhāoBrazil

Personalised recommendations