Abstract
Event Related Potentials (ERPs) are widely used to study category-selective EEG responses to visual stimuli, such as the face-selective N170 component. Typically, this is done by flashing stimuli at the point of static gaze fixation. While allowing for good experimental control, these paradigms ignore the dynamic role of eye-movements in natural vision. Fixation-related potentials (FRPs), obtained using simultaneous EEG and eye-tracking, overcome this limitation. Various studies have used FRPs to study processes such as lexical processing, target detection and attention allocation. The goal of this study was to carefully compare face-sensitive activity time-locked to an abrupt stimulus onset at fixation, with that time-locked to a self-generated fixation on a stimulus. Twelve participants participated in three experimental conditions: Free-viewing (FRPs), Cued-viewing (FRPs) and Control (ERPs). We used a multiple regression approach to disentangle overlapping activity components. Our results show that the N170 face-effect is evident for the first fixation on a stimulus, whether it follows a self-generated saccade or stimulus appearance at fixation point. The N170 face-effect has similar topography across viewing conditions, but there were major differences within each stimulus category. We ascribe these differences to an overlap of the fixation-related lambda response and the N170. We tested the plausibility of this account using dipole simulations. Finally, the N170 exhibits category-specific adaptation in free viewing. This study establishes the comparability of the free-viewing N170 face-effect with the classic event-related effect, while highlighting the importance of accounting for eye-movement related effects.
This is a preview of subscription content, log in via an institution to check access.
dipoles used in the model. b Position (loc) and orientation (ori) of the modeled dipoles in Talairach coordinates. c The dynamics of each dipole. P1\lambda dipoles peak at 90 ms, N1 dipoles peak at 170 ms
Similar content being viewed by others
Notes
In our design, categorical information of the stimulus was hindered in parafoveal view (see "Materials and Methods" section).
Saccade detection was qualitatively verified by comparison to the Engbert et al. paradigm (Engbert & Kliegl, 2003). Note that the fixations analyzed in this study follow macro- rather than micro-saccades.
Note we used a very large time window to preclude possible long term effects of fixation onset but this might not be necessary and adds a large number of predictors to the model.
Memory problems occurred when we used the default MATLAB solver for the regression analysis since our design matrix was extremely large. An alternative to solving the regression in Python is to use the LSMR solver for MATLAB as used in the Unfold toolbox (Ehinger & Dimigen, 2019).
References
Amihai I, Deouell LY, Bentin S (2011) Neural adaptation is related to face repetition irrespective of identity: a reappraisal of the N170 effect. Exp Brain Res 209(2):193–204. https://doi.org/10.1007/s00221-011-2546-x
Andersson R, Larsson L, Holmqvist K, Stridh M, Nyström M (2017) One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms. Behav Res Methods 49(2):616–637. https://doi.org/10.3758/s13428-016-0738-9
Baccino T, Manunta Y (2005) Eye-fixation-related potentials: insight into parafoveal processing. J Psychophysiol 19(3):204–215. https://doi.org/10.1027/0269-8803.19.3.204
Bentin S, Allison T, Puce A, Perez E, McCarthy G (1996) Electrophysiological studies of face perception in humans. J Cogn Neurosci 8(6):551–565. https://doi.org/10.1162/jocn.1996.8.6.551
Bentin S, Deouell LY, Soroker N (1999) Selective visual streaming in face recognition: evidence from developmental prosopagnosia. NeuroReport 10(4):823–827. https://doi.org/10.1097/00001756-199903170-00029
Brouwer A-M, Reuderink B, Vincent J, van Gerven MAJ, van Erp JBF (2013) Distinguishing between target and nontarget fixations in a visual search task using fixation-related potentials. J Vis 13(3):17–17. https://doi.org/10.1167/13.3.17
Buonocore A, Dimigen O, Melcher D (2020) Post-saccadic face processing is modulated by pre-saccadic preview: evidence from fixation-related potentials. J Neurosci. https://doi.org/10.1523/jneurosci.0861-19.2020
Busse L, Katzner S, Tillmann C, Treue S (2008) Effects of attention on perceptual direction tuning curves in the human visual system. J Vis 8(9):2–2. https://doi.org/10.1167/8.9.2
Cao X, Ma X, Qi C (2015) N170 adaptation effect for repeated faces and words. Neuroscience 294:21–28. https://doi.org/10.1016/j.neuroscience.2015.03.009
Carrasco M (2011) Visual attention: the past 25 years. Vis Res 51(13):1484–1525. https://doi.org/10.1016/j.visres.2011.04.012
Clark VP, Fan S, Hillyard SA (1995) Identification of early visual evoked potential generators by retinotopic and topographic analyses. Hum Brain Mapp 2(3):170–187. https://doi.org/10.1002/hbm.460020306
Dandekar S, Privitera C, Carney T, Klein SA (2012) Neural saccadic response estimation during natural viewing. J Neurophysiol 107(6):1776–1790. https://doi.org/10.1152/jn.00237.2011
de Lissa P, McArthur G, Hawelka S, Palermo R, Mahajan Y, Degno F, Hutzler F (2019) Peripheral preview abolishes N170 face-sensitivity at fixation: using fixation-related potentials to investigate dynamic face processing. Vis Cognit 27(9–10):740–759. https://doi.org/10.1080/13506285.2019.1676855
Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134(1):9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009
Devillez H, Guyader N, Guérin-Dugué A (2015) An eye fixation–related potentials analysis of the P300 potential for fixations onto a target object when exploring natural scenes. J Vis 15(13):20. https://doi.org/10.1167/15.13.20
Di Russo F, Martínez A, Sereno MI, Pitzalis S, Hillyard SA (2002) Cortical sources of the early components of the visual evoked potential. Hum Brain Mapp 15(2):95–111. https://doi.org/10.1002/hbm.10010
Dimigen O, Ehinger BV (2019) Analyzing combined eye-tracking/EEG experiments with (non) linear deconvolution models. BioRxiv. https://doi.org/10.1101/735530
Dimigen O, Kliegl R, Sommer W (2012) Trans-saccadic parafoveal preview benefits in fluent reading: a study with fixation-related brain potentials. NeuroImage 62(1):381–393. https://doi.org/10.1016/j.neuroimage.2012.04.006
Dimigen O, Sommer W, Hohlfeld A, Jacobs AM, Kliegl R (2011) Coregistration of eye movements and EEG in natural reading: analyses and review. J Exp Psychol Gen 140(4):552–572. https://doi.org/10.1037/a0023885
Dimigen O, Valsecchi M, Sommer W, Kliegl R (2009) Human microsaccade-related visual brain responses. J Neurosci 29(39):12321–12331. https://doi.org/10.1523/JNEUROSCI.0911-09.2009
Duncan-Johnson CC, Donchin E (1977) On quantifying surprise: the variation of event-related potentials with subjective probability. Psychophysiology 14:456–467
Egeth HE, Yantis S (1997) Visual attention: control, representation, and time course. Annu Rev Psychol 48(1):269–297. https://doi.org/10.1146/annurev.psych.48.1.269
Ehinger BV, Dimigen O (2019) Unfold: an integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis. PeerJ 2019(10):1–33. https://doi.org/10.7717/peerj.7838
Eimer M, Gosling A, Nicholas S, Kiss M (2011) The N170 component and its links to configural face processing: a rapid neural adaptation study. Brain Res 1376:76–87. https://doi.org/10.1016/j.brainres.2010.12.046
Engbert R, Kliegl R (2003) Microsaccades uncover the orientation of covert attention. Vis Res 43(9):1035–1045. https://doi.org/10.1016/S0042-6989(03)00084-1
Feuerriegel D, Churches OF, Keage HAD (2015) Is neural adaptation of the N170 category-specific? Effects of adaptor stimulus duration and interstimulus interval. Int J Psychophysiol 96(1):8–15. https://doi.org/10.1016/j.ijpsycho.2015.02.030
Fischer T, Graupner ST, Velichkovsky BM, Pannasch S (2013) Attentional dynamics during free picture viewing: evidence from oculomotor behavior and electrocortical activity. Front Syst Neurosci 7(17):1–9. https://doi.org/10.3389/fnsys.2013.00017
Frey A, Ionescu G, Lemaire B, López-orozco F, Baccino T, Guérin-dugué A (2013) Decision-making in information seeking on texts: an eye-fixation-related potentials investigation. Front Syst Neurosci 7:39. https://doi.org/10.3389/fnsys.2013.00039
Friston K, Adams RA, Perrinet L, Breakspear M (2012) Perceptions as hypotheses: saccades as experiments. Front Psychol 3:151. https://doi.org/10.3389/fpsyg.2012.00151
Gallant JL, Connor CE, Van Essen DC (1998) Neural activity in areas V1, V2 and V4 during free viewing of natural scenes compared to controlled viewing. NeuroReport 9(7):1673–1678. https://doi.org/10.1097/00001756-199805110-00075
Henderson JM, Anes MD (1994) Roles of object-file review and type priming in visual identification within and across eye fixations. J Exp Psychol 20(4):826–839. https://doi.org/10.1037/0096-1523.20.4.826
Henderson JM, Williams CC, Falk RJ (2005) Eye movements are functional during face learning. Mem Cognit 33(1):98–106. https://doi.org/10.3758/BF03195300
Hillyard SA, Anllo-Vento L (1998) Event-related brain potentials in the study of visual selective attention. Proc Natl Acad Sci USA 95(3):781–787. https://doi.org/10.1073/pnas.95.3.781
Hooge I, Nyström M, Cornelissen T, Holmqvist K (2015) The art of braking: post saccadic oscillations in the eye tracker signal decrease with increasing saccade size. Vis Res 112:55–67. https://doi.org/10.1016/j.visres.2015.03.015
Hsiao JHW, Cottrell G (2008) Two fixations suffice in face recognition. Psychol Sci 19(10):998–1006. https://doi.org/10.1111/j.1467-9280.2008.02191.x
Huber-Huber C, Buonocore A, Dimigen O, Hickey C, Melcher D (2019) The peripheral preview effect with faces: combined EEG and eye-tracking suggests multiple stages of trans-saccadic predictive and non-predictive processing. NeuroImage 200:344–362. https://doi.org/10.1016/j.neuroimage.2019.06.059
Hutzler F, Braun M, Võ MLH, Engl V, Hofmann M, Dambacher M, Jacobs AM (2007) Welcome to the real world: validating fixation-related brain potentials for ecologically valid settings. Brain Res 1172(1):124–129. https://doi.org/10.1016/j.brainres.2007.07.025
Itier RJ, Taylor MJ (2004) Source analysis of the N170 to faces and objects. NeuroReport 15(8):1261–1265. https://doi.org/10.1097/01.wnr.0000127827.73576.d8
Jeffreys DA, Tukmachi ESA (1992) The vertex-positive scalp potential evoked by faces and by objects. Exp Brain Res 91(2):340–350. https://doi.org/10.1007/BF00231668
Joyce C, Rossion B (2005) The face-sensitive N170 and VPP components manifest the same brain processes: the effect of reference electrode site. Clin Neurophysiol 116(11):2613–2631. https://doi.org/10.1016/j.clinph.2005.07.005
Kamienkowski JE, Ison MJ, Quiroga RQ, Sigman M (2012) Fixation-related potentials in visual search: a combined EEG and eye tracking study. J Vis 12(7):4–4. https://doi.org/10.1167/12.7.4
Kapoula Z, Robinson DA (1986) Saccadic undershoot is not inevitable: saccades can be accurate. Vis Res 26(5):735–743. https://doi.org/10.1016/0042-6989(86)90087-8
Kaunitz LN, Kamienkowski JE, Varatharajah A, Sigman M, Quiroga RQ, Ison MJ (2014) Looking for a face in the crowd: fixation-related potentials in an eye-movement visual search task. NeuroImage 89:297–305. https://doi.org/10.1016/j.neuroimage.2013.12.006
Kazai K, Yagi A (2003) Comparison between the lambda response of eye-fixation-related potentials and the P100 component of pattern-reversal visual evoked potentials. Cogn Affect Behav Neurosci 3(1):46–56. https://doi.org/10.3758/CABN.3.1.46
Kazai K, Yagi A (2005) Contrast dependence of lambda response. Int Congr Ser 1278:61–64. https://doi.org/10.1016/j.ics.2004.11.053
Keren AS, Yuval-greenberg S, Deouell LY (2010) Saccadic spike potentials in gamma-band EEG: characterization, detection and suppression. NeuroImage 49(3):2248–2263. https://doi.org/10.1016/j.neuroimage.2009.10.057
Kloth N, Schweinberger SR, Kovács G (2010) Neural correlates of generic versus gender-specific face adaptation. J Cogn Neurosci 22(10):2345–2356. https://doi.org/10.1162/jocn.2009.21329
Kornrumpf B, Niefind F, Sommer W, Dimigen O (2016) Neural correlates of word recognition: a systematic comparison of natural reading and rapid serial visual presentation. J Cogn Neurosci 28:1374–1391. https://doi.org/10.1162/jocn_a_00977
Kovács G, Zimmer M, Bankó É, Harza I, Antal A, Vidnyánszky Z (2006) Electrophysiological correlates of visual adaptation to faces and body parts in humans. Cereb Cortex 16(5):742–753. https://doi.org/10.1093/cercor/bhj020
Kovács G, Zimmer M, Volberg G, Lavric I, Rossion B (2013) Electrophysiological correlates of visual adaptation and sensory competition. Neuropsychologia 51(8):1488–1496. https://doi.org/10.1016/j.neuropsychologia.2013.04.016
Kusunoki M, Gottlieb J, Goldberg ME (2000) The lateral intraparietal area as a salience map: the representation of abrupt onset, stimulus motion, and task relevance. Vis Res 40(10–12):1459–1468. https://doi.org/10.1016/S0042-6989(99)00212-6
MacEvoy SP, Hanks TD, Paradiso MA (2008) Macaque V1 activity during natural vision: effects of natural scenes and saccades. J Neurophysiol 99(2):460–472. https://doi.org/10.1152/jn.00612.2007
Mangun GR (1995) Neural mechanisms of visual selective attention. Psychophysiology 32(1):4–18. https://doi.org/10.1111/j.1469-8986.1995.tb03400.x
Maris E, Oostenveld R (2007) Nonparametric statistical testing of EEG- and MEG-data. J Neurosci Methods 164(1):177–190. https://doi.org/10.1016/j.jneumeth.2007.03.024
Martínez A, Anllo-Vento L, Sereno MI, Frank LR, Buxton RB, Dubowitz DJ et al (1999) Involvement of striate and extrastriate visual cortical areas in spatial attention. Nat Neurosci 2(4):364–369. https://doi.org/10.1038/7274
Martinez-Conde S, Macknik SL, Hubel DH (2004) The role of fixational eye movements in visual perception. Nat Rev Neurosci 5(3):229–240. https://doi.org/10.1038/nrn1348
Maurer U, Rossion B, McCandliss BD (2008) Category specificity in early perception: face and word N170 responses differ in both lateralization and habituation properties. Front Hum Neurosci 2:1–7. https://doi.org/10.3389/neuro.09.018.2008
Melcher D, Colby CL (2008) Trans-saccadic perception. Trends Cogn Sci 12(12):466–473. https://doi.org/10.1016/j.tics.2008.09.003
Meyberg S, Sommer W, Dimigen O (2017) How microsaccades relate to lateralized ERP components of spatial attention: a co-registration study. Neuropsychologia 99:64–80. https://doi.org/10.1016/j.neuropsychologia.2017.02.023
Min X, Zhai G, Gu K (2015) Visual attention on human face: 2015 visual communications and image processing (VCIP). IEEE. https://doi.org/10.1109/VCIP.2015.7457922
Nemrodov D, Itier RJ (2011) The role of eyes in early face processing: a rapid adaptation study of the inversion effect. Br J Psychol 102(4):783–798. https://doi.org/10.1111/j.2044-8295.2011.02033.x
Nikolaev AR, Jurica P, Nakatani C, Plomp G, van Leeuwen C (2013) Visual encoding and fixation target selection in free viewing: presaccadic brain potentials. Front Syst Neurosci 7:26. https://doi.org/10.3389/fnsys.2013.00026
Nikolaev AR, Meghanathan RN, van Leeuwen C (2016) Combining EEG and eye movement recording in free viewing: pitfalls and possibilities. Brain Cogn 107:55–83. https://doi.org/10.1016/j.bandc.2016.06.004
Oostenveld R, Fries P, Maris E, Schoffelen J-M (2011) FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci 2011:156869. https://doi.org/10.1155/2011/156869
Polich J (2007) Updating P300: an integrative theory of P3a and P3b. Clin Neurophysiol 118(10):2128–2148. https://doi.org/10.1016/j.clinph.2007.04.019
Prablanc C, Jeannerod M (1975) Corrective saccades: dependence on retinal reafferent signals. Vis Res 15(4):465–469. https://doi.org/10.1016/0042-6989(75)90022-X
Purpura KP, Kalik SF, Schiff ND (2003) Analysis of perisaccadic field potentials in the occipitotemporal pathway during active vision. J Neurophysiol 90(5):3455–3478. https://doi.org/10.1152/jn.00011.2003
Rajkai C, Lakatos P, Chen CM, Pincze Z, Karmos G, Schroeder CE (2008) Transient cortical excitation at the onset of visual fixation. Cereb Cortex 18(1):200–209. https://doi.org/10.1093/cercor/bhm046
Ries AJ, Slayback D, Touryan J (2018) The fixation-related lambda response: effects of saccade magnitude, spatial frequency, and ocular artifact removal. Int J Psychophysiol 134:1–8. https://doi.org/10.1016/j.ijpsycho.2018.09.004
Rosenholtz R, Raj A, Balas BJ, Ilie L (2018) A summary statistic representation in peripheral vision explains visual search Ruth. J Vis 12(2012):1–17. https://doi.org/10.1167/12.4.14.Introduction
Rossion B (2014) Understanding face perception by means of human electrophysiology. Trends Cogn Sci 18(6):310–318. https://doi.org/10.1016/j.tics.2014.02.013
Rossion B, Gauthier I, Tarr MJ, Despland P, Bruyer R, Linotte S, Crommelinck M (2000) The N170 occipito-temporal component is delayed and enhanced to inverted faces but not to inverted objects. NeuroReport 11(1):69–72. https://doi.org/10.1097/00001756-200001170-00014
Rossion B, Jacques C (2012) The N170: understanding the time course of face perception in the human brain. In: Luck SJ, Kappenman ES (eds) Oxford library of psychology. The Oxford handbook of event-related potential components. Oxford University Press, Oxford, pp 115–141
Rousselet GA, Mace MJM, Fabre-Thorpe M (2004) Animal and human faces in natural scenes: how specific to human faces is the N170 ERP component? J Vis 4(1):2–2. https://doi.org/10.1167/4.1.2
Sagiv N, Bentin S (2001) Structural encoding of human and schematic faces: holistic and part-based processes. J Cogn Neurosci 13(7):937–951. https://doi.org/10.1162/089892901753165854
Schendan HE, Ganis G (2013) Face-specificity is robust across diverse stimuli and individual people, even when interstimulus variance is zero. Psychophysiology 50(3):287–291. https://doi.org/10.1111/psyp.12013
Schotter ER, Angele B, Rayner K (2012) Parafoveal processing in reading. Atten Percep Psychophys 74(1):5–35. https://doi.org/10.3758/s13414-011-0219-2
Schroeder CE, Wilson DA, Radman T, Scharfman H, Lakatos P (2010) Dynamics of active sensing and perceptual selection. Curr Opin Neurobiol 20(2):172–176. https://doi.org/10.1016/j.conb.2010.02.010
Smith NJ, Kutas M (2015a) Regression-based estimation of ERP waveforms: I The rERP framework. Psychophysiology 52(2):157–168. https://doi.org/10.1111/psyp.12317
Smith NJ, Kutas M (2015b) Regression-based estimation of ERP waveforms: II. Nonlinear effects, overlap correction, and practical considerations. Psychophysiology 52(2):169–181. https://doi.org/10.1111/psyp.12320
Soto V, Tyson-Carr J, Kokmotou K, Roberts H, Cook S, Fallon N, Stancak A (2018) Brain responses to emotional faces in natural settings: a wireless mobile EEG recording study. Front Psychol 9:1–15. https://doi.org/10.3389/fpsyg.2018.02003
Tabernero J, Artal P (2014) Lens oscillations in the human eye: implications for post-saccadic suppression of vision. PLoS ONE 9(4):e95764. https://doi.org/10.1371/journal.pone.0095764
Thickbroom GW, Knezevič W, Carroll WM, Mastaglia FL (1991) Saccade onset and offset lambda waves: relation to pattern movement visually evoked potentials. Brain Res 551(1–2):150–156. https://doi.org/10.1016/0006-8993(91)90927-N
Towler J, Gosling A, Duchaine B, Eimer M (2012) The face-sensitive N170 component in developmental prosopagnosia. Neuropsychologia 50(14):3588–3599. https://doi.org/10.1016/j.neuropsychologia.2012.10.017
Wynn JK, Lee J, Horan WP, Green MF (2008) Using event related potentials to explore stages of facial affect recognition deficits in schizophrenia. Schizophr Bull 34(4):679–687. https://doi.org/10.1093/schbul/sbn047
Yagi A (1979) Lambda waves associated with offset of saccades: a subject with large lambda waves. Biol Psychol 8(3):235–238. https://doi.org/10.1016/0301-0511(79)90051-6
Yagi A (1981) Visual signal detection and lambda responses. Electroencephalogr Clin Neurophysiol 52(6):604–610. https://doi.org/10.1016/0013-4694(81)91434-6
Yantis S, Jonides J (1984) Abrupt visual onsets and selective attention: evidence from visual search. J Exp Psychol Hum Percept Perform 10(5):601–621. https://doi.org/10.1037//0096-1523.10.5.601
Yarbus AL (1967) Eye movements and vision. Neuropsychologia 6(4):389–390. https://doi.org/10.1016/0028-3932(68)90012-2
Yovel G (2016) Neural and cognitive face-selective markers: an integrative review. Neuropsychologia 83:5–13. https://doi.org/10.1016/j.neuropsychologia.2015.09.026
Acknowledgements
This work is supported by grant 1902/14 from the Israel Science Foundation to L.Y.D. L.Y.D is supported by the Jack H. Skirball research fund. We thank Dr. Edden Gerber for his MATLAB implementation of the pre-processing procedure. We thank Chen Gueta, Namoi Revel and Chen Berkman with their help in the data collection, Oded Wertheimer for programming, Gal Chen for help with the statistical analysis and all the lab members for their valuable comments.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interests.
Ethical Approval
All procedures complied with the ethical regulations of the Hebrew University of Jerusalem for experiments involving human subjects and was approved by the institutional review board and subjects signed an informed consent prior to their participation. The study was funded by a grant from the Israel Science foundation to LYD.
Additional information
Handling Editor: Micah M. Murray.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Auerbach-Asch, C.R., Bein, O. & Deouell, L.Y. Face Selective Neural Activity: Comparisons Between Fixed and Free Viewing. Brain Topogr 33, 336–354 (2020). https://doi.org/10.1007/s10548-020-00764-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10548-020-00764-7