Skip to main content
SpringerLink
Log in
Book cover

Brain-Computer Interface Research pp 61–72Cite as

An Affective BCI Using Multiple ERP Components Associated to Facial Emotion Processing

  • Chapter
  • First Online:

Part of the book series: SpringerBriefs in Electrical and Computer Engineering ((BRIEFSELECTRIC))

Abstract

P300-based brain computer interfaces (BCIs) have successfully demonstrated that attention to an oddball stimulus can enhance the P300 component of the event-related potential (ERP) phase-locked to the event. However, it was unclear whether the more sophisticated face-evoked potentials can also be modulated by related mental tasks under the oddball paradigm. This study investigated ERP responses to image stimuli of objects, neutral faces, and emotional faces when subjects perform attention, face recognition and discrimination of emotional facial expressions respectively under the oddball paradigm. The results revealed the significant difference between target and non-target ERPs for each mental task. In addition, significant differences among the three mental tasks were observed for vertex-positive potential (VPP) over the fronto-central sites, late positive potential (LPP)/P3b over the centro-parietal sites and N250 over the occipito-temporal sites. These findings indicate that a novel affective BCI paradigm can be developed based on detection of multiple ERP components reflecting human face encoding and emotion processing. The high classification performance for single-trial emotional face-related ERPs demonstrated the effectiveness of the affective BCI.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The video of experimental stimuli is available at http://www.bsp.brain.riken.jp/bci/emotiaonfacefull.avi.

  2. 2.

    The videos are available at http://www.bsp.brain.riken.jp/bci/.

  3. 3.

    see http://www.bsp.brain.riken.jp/bci/inverted_face_paradigm.wmv.

  4. 4.

    see http://www.bsp.brain.riken.jp/bci/RotatingFace.wmv.

  5. 5.

    see http://www.bsp.brain.riken.jp/bci/RSVP_Face.wmv.

References

  • L. Acqualagnav, M. Treder, M. Schreuder, B. Blankertz, A novel brain-computer interface based on the rapid serial visual presentation paradigm, in Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE (IEEE, 2010), pp. 2686–2689

    Google Scholar 

  • T. Dennis, C. Chen, Emotional face processing and attention performance in three domains: neurophysiological mechanisms and moderating effects of trait anxiety. Int. J. Psychophysiol. 65(1), 10–19 (2007)

    Article  Google Scholar 

  • M. Eimer, Effects of face inversion on the structural encoding and recognition of faces: evidence from event-related brain potentials. Cogn. Brain Res. 10(1–2), 145–158 (2000)

    Article  Google Scholar 

  • L. Farwell, E. Donchin, Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70(6), 510–523 (1988)

    Article  Google Scholar 

  • A. Furdea, S. Halder, D. Krusienski, D. Bross, F. Nijboer, N. Birbaumer, A. Kübler, An auditory oddball (P300) spelling system for brain-computer interfaces. Psychophysiology 46(3), 617–625 (2009)

    Article  Google Scholar 

  • F. Guo, B. Hong, X. Gao, S. Gao, A brain-computer interface using motion-onset visual evoked potential. J. Neural Eng. 5(4), 477–485 (2008)

    Article  Google Scholar 

  • R. Itier, M. Taylor, Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: a repetition study using ERPs. NeuroImage 15(2), 353–372 (2002)

    Article  Google Scholar 

  • R. Itier, M. Taylor, N170 or N1? spatiotemporal differences between object and face processing using ERPs. Cereb. Cortex 14(2), 132–142 (2004)

    Article  Google Scholar 

  • J. Jin, B. Allison, X. Wang, C. Neuper, A combined brain-computer interface based on P300 potentials and motion-onset visual evoked potentials. J. Neurosci. Methods 205(2), 265–276 (2012)

    Article  Google Scholar 

  • T. Kaufmann, S. Schulz, C. Grünzinger, A. Kübler, Flashing characters with famous faces improves ERP-based brain-computer interface performance. J. Neural Eng. 8(5), 056016 (2011)

    Article  Google Scholar 

  • K. Lee, T. Lee, S. Yoon, Y. Cho, J. Choi, H. Kim, Neural correlates of top-down processing in emotion perception: an ERP study of emotional faces in white noise versus noise-alone stimuli. Brain Res. 1337, 56–63 (2010)

    Article  Google Scholar 

  • A. Lenhardt, M. Kaper, H. Ritter, An adaptive P300-based online brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 16(2), 121–130 (2008)

    Article  Google Scholar 

  • W. Luo, W. Feng, W. He, N. Wang, Y. Luo, Three stages of facial expression processing: ERP study with rapid serial visual presentation. NeuroImage 49(2), 1857–1867 (2010)

    Article  Google Scholar 

  • S. Makeig, G. Leslie, T. Mullen, D. Sarma, N. Bigdely-Shamlo, C. Kothe, First demonstration of a musical emotion BCI, in Affective Computing and Intelligent Interaction, 2011, pp. 487–496

    Google Scholar 

  • S. Martens, N. Hill, J. Farquhar, B. Schölkopf, Overlap and refractory effects in a brain-computer interface speller based on the visual P300 event-related potential. J. Neural Eng. 6, 026003 (2009)

    Article  Google Scholar 

  • T. Marzi, M. Viggiano, Interplay between familiarity and orientation in face processing: an ERP study. Int. J. Psychophysiol. 65(3), 182–192 (2007)

    Article  Google Scholar 

  • M. Moscovitch, G. Winocur, M. Behrmann, What is special about face recognition? nineteen experiments on a person with visual object agnosia and dyslexia but normal face recognition. J. Cogn. Neurosci. 9(5), 555–604 (1997)

    Article  Google Scholar 

  • S. Nasr, H. Esteky, A study of N250 event-related brain potential during face and non-face detection tasks. J. Vis. 9(5), 5 (2009)

    Article  Google Scholar 

  • M.F. Neumann, T.N. Mohamed, S.R. Schweinberger, Face and object encoding under perceptual load: ERP evidence. NeuroImage 54(4), 3021–3027 (2011)

    Article  Google Scholar 

  • A. Nijholt, B. Allison, S. Dunne, D. Heylen et al., Affective brain-computer interfaces (abci 2011), in Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction-Volume Part II (Springer-Verlag (2011), pp. 435–435

    Google Scholar 

  • A. Onishi, Y. Zhang, Q. Zhao, A. Cichocki, Fast and reliable P300-based BCI with facial images, in Proceedings of the 5th International Brain-Computer Interface Conference, 2011, pp. 192–195

    Google Scholar 

  • B. Rossion, J. Delvenne, D. Debatisse, V. Goffaux, R. Bruyer, M. Crommelinck, J. Guérit, Spatio-temporal localization of the face inversion effect: an event-related potentials study. Biol. psychol. 50(3), 173–189 (1999)

    Article  Google Scholar 

  • B. Sadeh, A. Zhdanov, I. Podlipsky, T. Hendler, G. Yovel, The validity of the face-selective ERP N170 component during simultaneous recording with functional MRI. Neuroimage 42(2), 778–786 (2008)

    Article  Google Scholar 

  • S. Schweinberger, V. Huddy, A. Burton, N250r: a face-selective brain response to stimulus repetitions. NeuroReport 15(9), 1501 (2004)

    Article  Google Scholar 

  • M. Thulasidas, C. Guan, J. Wu, Robust classification of EEG signal for brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 14(1), 24–29 (2006)

    Article  Google Scholar 

  • G. Townsend, B. LaPallo, C. Boulay, D. Krusienski, G. Frye, C. Hauser, N. Schwartz, T. Vaughan, J. Wolpaw, E. Sellers, A novel P300-based brain-computer interface stimulus presentation paradigm: moving beyond rows and columns. Clin. Neurophysiol. 121(7), 1109–1120 (2010)

    Article  Google Scholar 

  • M. Treder, B. Blankertz, (C) overt attention and visual speller design in an ERP-based brain-computer interface. Behav. Brain Funct. 6(1), 28 (2010)

    Article  Google Scholar 

  • M. Willis, R. Palermo, D. Burke, C. Atkinson, G. McArthur, Switching associations between facial identity and emotional expression: a behavioural and ERP study. Neuroimage 50(1), 329–339 (2010)

    Article  Google Scholar 

  • J. Wolpaw, N. Birbaumer, D. McFarland, G. Pfurtscheller, T. Vaughan, Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)

    Google Scholar 

  • P. Xu, P. Yang, X. Lei, D. Yao, An enhanced probabilistic LDA for multi-class brain computer interface. PloS One 6(1), e14634 (2011)

    Article  Google Scholar 

  • Y. Zhang, Q. Zhao, J. Jin, X. Wang, A. Cichocki, A novel BCI based on ERP components sensitive to configural processing of human faces. J. Neural Eng. 9(2), 026018 (2012)

    Article  Google Scholar 

  • Q. Zhao, A. Onishi, Y. Zhang, J. Cao, L. Zhang, A. Cichocki, A novel oddball paradigm for affective BCIs using emotional faces as stimuli. Lecture Notes in Computer Science, vol. 7062, 2011, pp. 279–286

    Google Scholar 

Download references

Acknowledgments

The work was supported by JSPS Grants-in-Aid for Scientific Research (grant number 24700154) and the National Natural Science Foundation of China (grant number 61202155).

Author information

Authors and Affiliations

  1. Laboratory for Advanced Brain Signal Processing, Brain Science Institute, RIKEN, Saitama, Japan

    Qibin Zhao, Yu Zhang, Akinari Onishi & Andrzej Cichocki

Authors
  1. Qibin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Akinari Onishi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrzej Cichocki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qibin Zhao .

Editor information

Editors and Affiliations

  1. Guger Technologies OG, Sierningstrasse 14, Schiedlberg, 4521, Austria

    Christoph Guger

  2. , Cognitive Science Department, University of California, Baltimore Drive 5700Apt 167, San Diego, 91942, California, USA

    Brendan Z. Allison

  3. g.tec Guger Technologies OG, Herbersteinstrasse 60, Graz, 8020, Austria

    Günter Edlinger

Rights and permissions

Reprints and permissions

Copyright information

© 2013 The Author(s)

About this chapter

Cite this chapter

Zhao, Q., Zhang, Y., Onishi, A., Cichocki, A. (2013). An Affective BCI Using Multiple ERP Components Associated to Facial Emotion Processing. In: Guger, C., Allison, B., Edlinger, G. (eds) Brain-Computer Interface Research. SpringerBriefs in Electrical and Computer Engineering. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36083-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-36083-1_7

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-36082-4

  • Online ISBN: 978-3-642-36083-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation