Advertisement

Cognitive Neurodynamics

, Volume 13, Issue 6, pp 503–512 | Cite as

Measurement of cognitive dynamics during video watching through event-related potentials (ERPs) and oscillations (EROs)

  • Emel Erdogdu
  • Elif Kurt
  • Adil Deniz Duru
  • Atilla Uslu
  • Canan Başar-Eroğlu
  • Tamer DemiralpEmail author
Research Article
  • 126 Downloads

Abstract

Event-related potentials (ERPs) and oscillations (EROs) are reliable measures of cognition, but they require time-locked electroencephalographic (EEG) data to repetitive triggers that are not available in continuous sensory input streams. However, such real-life-like stimulation by videos or virtual-reality environments may serve as powerful means of creating specific cognitive or affective states and help to investigate dysfunctions in psychiatric and neurological disorders more efficiently. This study aims to develop a method to generate ERPs and EROs during watching videos. Repeated luminance changes were introduced on short video segments, while EEGs of 10 subjects were recorded. The ERP/EROs time-locked to these distortions were analyzed in time and time–frequency domains and tested for their cognitive significance through a long term memory test that included frames from the watched videos. For each subject, ERPs and EROs corresponding to video segments of recalled images with 25% shortest and 25% longest reaction times were compared. ERPs produced by transient luminance changes displayed statistically significant fluctuations both in time and time–frequency domains. Statistical analyses showed that a positivity around 450 ms, a negativity around 500 ms and delta and theta EROs correlated with memory performance. Few studies mixed video streams with simultaneous ERP/ERO experiments with discrete task-relevant or passively presented auditory or somatosensory stimuli, while the present study, by obtaining ERPs and EROs to task-irrelevant events in the same sensory modality as that of the continuous sensory input, produces minimal interference with the main focus of attention on the video stream.

Keywords

Event-related potentials Event-related oscillations Cognitive dynamics Wavelet transform Continous sensory input 

Notes

Acknowledgements

The present work was supported by Scientific Research Projects Coordination Unit of Istanbul University Projects 1400-42514/2014 and 23083-58951/2015. The author EE received the DAAD stipend (57140539) during the period of this project.

Compliance with ethical standards

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Ethical approval

This study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethical Committee of the Istanbul Faculty of Medicine of Istanbul University.

References

  1. Ademoglu A, Demiralp T, Yordanova J, Kolev V, Devrim M (1998) Decomposition of event-related brain potentials into multicomponents using wavelet transform. Appl Sig Proc 5:142–151CrossRefGoogle Scholar
  2. Allison BZ, Polich J (2008) Workload assessment of computer gaming using a single-stimulus event-related potential paradigm. Biol Psychol 77(3):277–283.  https://doi.org/10.1016/j.biopsycho.2007.10.014 CrossRefPubMedGoogle Scholar
  3. Backus AR, Schoffelen JM, Szebényi S, Hanslmayr S, Doeller CF (2016) Hippocampal-prefrontal theta oscillations support memory integration. Curr Biol 26(4):450–457.  https://doi.org/10.1016/j.cub.2015.12.048 CrossRefPubMedGoogle Scholar
  4. Basar E, Bullock TH (eds) (1992) Induced rhythms in the brain. Birkhäuser, BostonGoogle Scholar
  5. Basar E, Demiralp T, Schürmann M, Basar-Eroglu C, Ademoglu A (1999) Oscillatory brain dynamics, wavelet analysis, and cognition. Brain Lang 66(1):146–183.  https://doi.org/10.1006/brln.1998.2029 CrossRefPubMedGoogle Scholar
  6. Basar E, Schürmann M, Demiralp T, Basar-Eroglu C, Ademoglu A (2001) Event-related oscillations are ‘real brain responses’-wavelet analysis and new strategies. Int J Psychophysiol 39(2–3):91–127.  https://doi.org/10.1016/S0167-8760(00)00135-5 CrossRefPubMedGoogle Scholar
  7. Basar E, Basar-Eroglu C, Güntekin B, Yener GG (2013) Brain’s alpha, beta, gamma, delta, and theta oscillations in neuropsychiatric diseases: proposal for biomarker strategies. Suppl Clin Neurophysiol 62:19–54CrossRefGoogle Scholar
  8. Basar-Eroglu C, Basar E, Demiralp T, Schürmann M (1992) P300-response: possible psychophysiological correlates in delta and theta frequency channels. A review. Int J Psychophysiol 13(2):161–179.  https://doi.org/10.1016/0167-8760(92)90055-G CrossRefPubMedGoogle Scholar
  9. Basar-Eroglu C, Schmiedt-Fehr C, Mathes B (2013) Auditory evoked alpha oscillations imply reduced anterior and increased posterior amplitudes in schizophrenia. Suppl Clin Neurophysiol 62:121–129CrossRefGoogle Scholar
  10. Bayram A, Bayraktaroglu Z, Karahan E, Erdogan B, Bilgic B, Ozker M, Kasikci I, Duru AD, Ademoglu A, Oztürk C, Arikan K, Tarhan N, Demiralp T (2011) Simultaneous EEG/fMRI analysis of the resonance phenomena in steady-state visual evoked responses. Clin EEG Neurosci 42(2):98–106CrossRefGoogle Scholar
  11. Bayram A, Karahan E, Bilgiç B, Ademoglu A, Demiralp T (2016) Achromatic temporal-frequency responses of human lateral geniculate nucleus and primary visual cortex. Vis Res 127:177–185CrossRefGoogle Scholar
  12. Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433–436.  https://doi.org/10.1163/156856897X00357 CrossRefGoogle Scholar
  13. Carvalho S, Leite J, Galdo-Álvarez S, Gonçalves ÓF (2011) Psychophysiological correlates of sexually and non-sexually motivated attention to film clips in a workload task. PLoS ONE 6(12):e29530.  https://doi.org/10.1371/journal.pone.0029530 CrossRefPubMedPubMedCentralGoogle Scholar
  14. Demiralp T, Ademoglu A (2001) Decomposition of event-related brain potentials into multiple functional components using wavelet transform. Clin Electroencephalogr 32(3):122–138.  https://doi.org/10.1177/155005940103200307 CrossRefPubMedGoogle Scholar
  15. Demiralp T, Ademoglu A, Comerchero M, Polich J (2001) Wavelet analysis of P3a and P3b. Brain Topogr 13(4):251–267CrossRefGoogle Scholar
  16. Demiralp T, Bayraktaroglu Z, Lenz D, Junge S, Busch NA, Maess B, Ergen M, Herrmann CS (2007) Gamma amplitudes are coupled to theta phase in human EEG during visual perception. Int J Psychophysiol 64(1):24–30CrossRefGoogle Scholar
  17. Ergen M, Marbach S, Brand A, Basar-Eroglu C, Demiralp T (2008) P3 and delta band responses in visual oddball paradigm in schizophrenia. Neurosci Lett 440(3):304–308CrossRefGoogle Scholar
  18. Ergen M, Yildirim E, Uslu A, Gürvit H, Demiralp T (2012) P3 response during short-term memory retrieval revisited by a spatio-temporal analysis. Int J Psychophysiol 84(2):205–210CrossRefGoogle Scholar
  19. Ergen M, Saban S, Kirmizi-Alsan E, Uslu A, Keskin-Ergen Y, Demiralp T (2014) Time-frequency analysis of the event-related potentials associated with the Stroop test. Int J Psychophysiol 94(3):463–472.  https://doi.org/10.1016/j.ijpsycho.2014.08.177 CrossRefPubMedGoogle Scholar
  20. Escera C, Alho K, Schröger E, Winkler IW (2000) Involuntary attention and distractibility as evaluated with event-related brain potentials. Audiol Neurotol 5(3–4):151–166.  https://doi.org/10.1159/000013877 CrossRefGoogle Scholar
  21. Grossman A, Morlet J (1984) Decomposition of Hardy functions into square integrable wavelets of constant shape. SIAM J Math Anal 15:723–736CrossRefGoogle Scholar
  22. Güntekin B, Saatçi E, Yener G (2008) Decrease of evoked delta, theta and alpha coherences in Alzheimer patients during a visual oddball paradigm. Brain Res 1235:109–116CrossRefGoogle Scholar
  23. Herrmann CS, Grigutsch M, Busch NA (2005) EEG oscillations and wavelet analysis. In: Handy TC (ed) Event-related potentials: a methods handbook. MIT Press, Cambridge, pp 229–259Google Scholar
  24. Kirmizi-Alsan E, Bayraktaroglu Z, Gurvit H, Keskin YH, Emre M, Demiralp T (2006) Comparative analysis of event-related potentials during Go/NoGo and CPT: decomposition of electrophysiological markers of response inhibition and sustained attention. Brain Res 1104(1):114–128CrossRefGoogle Scholar
  25. Klimesch W (1999) EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res Brain Res Rev 29(2–3):169–195.  https://doi.org/10.1016/S0165-0173(98)00056-3 CrossRefPubMedGoogle Scholar
  26. Kramer AF, Trejo LJ, Humphrey D (1995) Assessment of mental workload with task-irrelevant auditory probes. Biol Psychol 40(1–2):83–100.  https://doi.org/10.1016/0301-0511(95)05108-2 CrossRefPubMedGoogle Scholar
  27. Lang PJ, Bradley MM, Cuthbert BN (1997) International affective picture system (IAPS): technical manual and affective ratings. NIMH Cent Study Emot Atten 1:39–58Google Scholar
  28. Liu B, Wang Z, Jin Z, Song G (2010) The influence on cognitive processing from the switches of shooting angles in videos of real-world events: an ERP study. Neurosci Lett 469(3):370–374.  https://doi.org/10.1016/j.neulet.2009.12.030 CrossRefPubMedGoogle Scholar
  29. McArthur GM, Bishop DVM, Proudfoot M (2003) Do video sounds interfere with auditory event-related potentials? Behav Res Methods Instrum Comput 35(2):329–333.  https://doi.org/10.3758/BF03202561 CrossRefPubMedGoogle Scholar
  30. Mertens R, Polich J (1997) P300 from a single-stimulus paradigm: passive versus active tasks and stimulus modality. Electroencephalogr Clin Neurophysiol 104(6):488–497.  https://doi.org/10.1016/S0168-5597(97)00041-5 CrossRefPubMedGoogle Scholar
  31. Nessler D, Johnson R, Bersick M, Friedman D (2006) On why the elderly have normal semantic retrieval but deficient episodic encoding: a study of left inferior frontal ERP activity. Neuroimage 30(1):299–312CrossRefGoogle Scholar
  32. Polich J (2007) Updating P300: an integrative theory of P3a and P3b. Clin Neurophysiol 118(10):2128–2148.  https://doi.org/10.1016/j.clinph.2007.04.019 CrossRefPubMedPubMedCentralGoogle Scholar
  33. Roy RN, Bonnet S, Charbonnier S, Campagne A (2016) Efficient workload classification based on ignored auditory probes: a proof of concept. Front Hum Neurosci 10:519.  https://doi.org/10.3389/fnhum.2016.00519 CrossRefPubMedPubMedCentralGoogle Scholar
  34. Sauseng P, Klimesch W, Doppelmayr M, Hanslmayr S, Schabus M, Gruber WR (2004) Theta coupling in the human electroencephalogram during a working memory task. Neurosci Lett 354(2):123–126.  https://doi.org/10.1016/j.neulet.2003.10.002 CrossRefPubMedGoogle Scholar
  35. Scholz S, Schneider SL, Rose M (2017) Differential effects of ongoing EEG beta and theta power on memory formation. PLoS ONE 12(2):e0171913.  https://doi.org/10.1371/journal.pone.0171913 CrossRefPubMedPubMedCentralGoogle Scholar
  36. Selimbeyoglu A, Keskin-Ergen Y, Demiralp T (2012) What if you are not sure? Electroencephalographic correlates of subjective confidence level about a decision. Clin Neurophysiol 123(6):1158–1167.  https://doi.org/10.1016/j.clinph.2011.10.037 CrossRefPubMedGoogle Scholar
  37. Shigemitsu Y, Nittono H (2008) Assessing interest level during movie watching with brain potentials. In: Poster session presented at the second international workshop on Kansei, Fukuoka, Japan, Proceedings, pp 39–42Google Scholar
  38. Shigemitsu Y, Nittono H, Hori T (2007) Assessing the allocation of attention to audiovisual experience with the P300 elicited by vibratory probe stimuli. Jpn J Physiol Psychol Psychophysiol 25(3):277–285.  https://doi.org/10.5674/jjppp1983.25.277 CrossRefGoogle Scholar
  39. Sitnikova T, Kuperberg G, Holcomb PJ (2003) Semantic integration in videos of real-world events: an electrophysiological investigation. Psychophysiology 40(1):160–164.  https://doi.org/10.1111/1469-8986.00016 CrossRefPubMedGoogle Scholar
  40. Sitnikova T, Holcomb PJ, Kiyonaga KA, Kuperberg GR (2008) Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. J Cogn Neurosci 20(11):2037–2057.  https://doi.org/10.1162/jocn.2008.20143 CrossRefPubMedPubMedCentralGoogle Scholar
  41. Sugimoto F, Katayama JI (2013) Somatosensory P2 reflects resource allocation in a game task: assessment with an irrelevant probe technique using electrical probe stimuli to shoulders. Int J Psychophysiol 87(2):200–204.  https://doi.org/10.1016/j.ijpsycho.2013.01.007 CrossRefPubMedGoogle Scholar
  42. Suzuki J, Nittono H, Hori T (2005) Level of interest in video clips modulates event-related potentials to auditory probes. Int J Psychophysiol 55(1):35–43.  https://doi.org/10.1016/j.ijpsycho.2004.06.001 CrossRefPubMedGoogle Scholar
  43. Takeda Y, Kimura M (2014) The auditory N1 amplitude for task-irrelevant probes reflects visual interest. Int J Psychophysiol 94(1):35–41.  https://doi.org/10.1016/j.ijpsycho.2014.07.007 CrossRefPubMedGoogle Scholar
  44. Tarnanas I, Tsolaki M, Nef T, Müri RM, Mosimann UP (2014) Can a novel computerized cognitive screening test provide additional information for early detection of Alzheimer’s disease? Alzheimers Dement 10(6):790–798.  https://doi.org/10.1016/j.jalz.2014.01.002 CrossRefPubMedGoogle Scholar
  45. Tulving E, Kapur S, Craik FI, Moscovitch M, Houle S (1994) Hemispheric encoding/retrieval asymmetry in episodic memory: positron emission tomography findings. PNAS 91(6):2016–2020CrossRefGoogle Scholar
  46. Ullsperger P, Freude G, Erdmann U (2001) Auditory probe sensitivity to mental workload changes—an event-related potential study. Int J Psychophysiol 40(3):201–209.  https://doi.org/10.1016/S0167-8760(00)00188-4 CrossRefPubMedGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Institute of Psychology and Cognition ResearchUniversity of BremenBremenGermany
  2. 2.Hulusi Behçet Life Sciences Research LaboratoryIstanbul UniversityÇapa, IstanbulTurkey
  3. 3.Aziz Sancar Institute of Experimental Medicine, Department of NeuroscienceIstanbul UniversityÇapa, IstanbulTurkey
  4. 4.Department of Physical Education and Sports Teaching, Faculty of Sport SciencesMarmara UniversityBeykoz, IstanbulTurkey
  5. 5.Department of Physiology, Istanbul Faculty of MedicineIstanbul UniversityÇapa, IstanbulTurkey
  6. 6.Department of Psychology, Faculty of Arts and SciencesIzmir University of EconomicsBalçova, IzmirTurkey

Personalised recommendations