Abstract
Human visual processing is of such complexity that, despite decades of focused research, many basic questions remain unanswered. Although we know that the inferotemporal cortex is a key region in object recognition, we don’t fully understand its physiologic role in brain function, nor do we have the full set of tools to explore this question. Here we show that electrical potentials from the surface of the human brain contain enough information to decode a subject’s perceptual state accurately, and with fine temporal precision. Electrocorticographic (ECoG) arrays were placed over the inferotemporal cortical areas of seven subjects. Pictures of faces and houses were quickly presented while each subject performed a simple visual task. Results showed that two well-known types of brain signals—event-averaged broadband power and event-averaged raw potential—can independently or together be used to classify the presented image. When applied to continuously recorded brain activity, our decoding technique could accurately predict whether each stimulus was a face, house, or neither, with ~20 ms timing error. These results provide a roadmap for improved brain-computer interfacing tools to help neurosurgeons, research scientists, engineers, and, ultimately, patients.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
G.K. Aguirre, E. Zarahn, M. D’Esposito, An area within human ventral cortex sensitive to “building” stimuli: evidence and implications. Neuron 21, 373–383 (1998)
C.M. Bishop, Neural Networks for Pattern Recognition (Oxford University Press, Oxford, 1995)
R.M. Cichy, D. Pantazis, A. Oliva, Resolving human object recognition in space and time. Nat. Neurosci. 17, 455–462 (2014)
S.M. Crouzet, H. Kirchner, S.J. Thorpe, Fast saccades toward faces: face detection in just 100 ms. J. Vis. 10(16), 11–17 (2010)
A.D. Engell, G. McCarthy, The relationship of gamma oscillations and face-specific ERPs recorded subdurally from occipitotemporal cortex. Cereb. Cortex 21, 1213–1221 (2011)
A.D. Engell, G. McCarthy, Repetition suppression of face-selective evoked and induced EEG recorded from human cortex. Human Brain Map (2014). doi:10.1002/hbm.22467
A.D. Engell, S. Huettel, G. McCarthy, The fMRI BOLD signal tracks electrophysiological spectral perturbations, not event-related potentials. NeuroImage 59, 2600–2606 (2012)
R. Epstein, N. Kanwisher, A cortical representation of the local visual environment. Nature 392, 598–601 (1998)
C. Guger, B.Z. Allison, G.R. Müller-Putz, in Brain-Computer Interface Research: A State-of-the-Art Summary 4 (in press)
D. Hermes, K.J. Miller, H.J. Noordmans, M.J. Vansteensel, N.F. Ramsey, Automated electrocorticographic electrode localization on individually rendered brain surfaces. J. Neurosci. Methods 185, 293–298 (2010)
C.P. Hung, G. Kreiman, T. Poggio, J.J. DiCarlo, Fast readout of object identity from macaque inferior temporal cortex. Science 310, 863–866 (2005)
C. Jacques, N. Witthoft, K.S. Weiner, B.L. Foster, V. Rangarajan, D. Hermes, K.J. Miller, J. Parvizi, K. Grill-Spector, Corresponding ECoG and fMRI category-selective signals in Human ventral temporal cortex. Neuropsychologia (2015)
N. Kanwisher, J. McDermott, M.M. Chun, The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 17, 4302–4311 (1997)
R. Kiani, H. Esteky, K. Mirpour, K. Tanaka, Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. J. Neurophysiol. 97, 4296–4309 (2007)
H. Kirchner, S.J. Thorpe, Ultra-rapid object detection with saccadic eye movements: visual processing speed revisited. Vis. Res. 46, 1762–1776 (2006)
H. Liu, Y. Agam, J.R. Madsen, G. Kreiman, Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex. Neuron 62, 281–290 (2009)
K.J. Miller, L.B. Sorensen, J.G. Ojemann, M. den Nijs, Power-law scaling in the brain surface electric potential. PLoS Comput. Biol. 5, e1000609 (2009)
K.J. Miller, C.J. Honey, D. Hermes, R.P. Rao, M. denNijs, J.G. Ojemann, Broadband changes in the cortical surface potential track activation of functionally diverse neuronal populations. NeuroImage 85(Pt 2), 711–720 (2014)
K.J. Miller, D. Hermes, N. Witthoft, R.P. Rao, J.G. Ojemann, The physiology of perception in human temporal lobe is specialized for contextual novelty. J. Neurophysiol. 114, 256–263 (2015a)
K.J. Miller, G. Schalk, D. Hermes, J.G. Ojemann, R.P.N. Rao, Spontaneous Decoding of the Timing and Content of Human Object Perception from Cortical Surface Recordings Reveals Complementary Information in the Event-Related Potential and Broadband Spectral Change (2015b)
B. Porat, A Course in Digital Signal Processing (Wiley, New York, 1997)
A. Puce, T. Allison, J.C. Gore, G. McCarthy, Face-sensitive regions in human extrastriate cortex studied by functional MRI. J. Neurophysiol. 74, 1192–1199 (1995)
G. Schalk, J. Mellinger, A Practical Guide to Brain-Computer Interfacing with BCI2000 (Springer, London, 2010)
G. Schalk, D.J. McFarland, T. Hinterberger, N. Birbaumer, J.R. Wolpaw, BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51(6), 1034–1043 (2004)
I. Simanova, M. van Gerven, R. Oostenveld, P. Hagoort, Identifying object categories from event-related EEG: toward decoding of conceptual representations. PLoS ONE 5, e14465 (2010)
M.E. van de Nieuwenhuijzen, A.R. Backus, A. Bahramisharif, C.F. Doeller, O. Jensen, M.A. van Gerven, MEG-based decoding of the spatiotemporal dynamics of visual category perception. NeuroImage 83, 1063–1073 (2013)
Acknowledgments
We are grateful to the patients and staff at Harborview Hospital in Seattle. We are grateful for helpful discussions with Kalanit Grill-Spector and Bharathi Jagadeesh. This work was supported by National Aeronautics and Space Administration Graduate Student Research Program (KJM), the NIH (R01-NS065186 (KJM, JGO, RPNR), T32-EY20485 (DH), R01-EB006356 (GS), R01-EB00856 (GS) and P41-EB018783 (GS)), the NSF (EEC-1028725 (RPNR)), the US Army Research Office (W911NF-07-1-0415 (GS), W911NF-08-1-0216 (GS) and W911NF-14-1-0440 (GS)), and Fondazione Neurone (GS).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 The Author(s)
About this chapter
Cite this chapter
Miller, K.J., Schalk, G., Hermes, D., Ojemann, J.G., Rao, R.P.N. (2015). Near-Instantaneous Classification of Perceptual States from Cortical Surface Recordings. In: Guger, C., Müller-Putz, G., Allison, B. (eds) Brain-Computer Interface Research. SpringerBriefs in Electrical and Computer Engineering. Springer, Cham. https://doi.org/10.1007/978-3-319-25190-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-25190-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25188-2
Online ISBN: 978-3-319-25190-5
eBook Packages: Computer ScienceComputer Science (R0)