Resampling approach to statistical inference: Bootstrapping from event-related potentials data

  • Francesco Di NoceraEmail author
  • Fabio Ferlazzo


We propose the use of the bootstrap resampling technique as a tool to assess the within-subject reliability of experimental modulation effects on event-related potentials (ERPs). The assessment of the within-subject reliability is relevant in all those cases when the subject score is obtained by some estimation procedure, such as averaging. In these cases, possible deviations from the assumptions on which the estimation procedure relies may lead to severely biased results and, consequently, to incorrect functional inferences. In this study, we applied bootstrap analysis to data from an experiment aimed at investigating the relationship between ERPs and memory processes. ERPs were recorded from two groups of subjects engaged in a recognition memory task. During the study phase, subjects in Group A were required to make an orthographic judgment on 160 visually presented words, whereas subjects in Group B were only required to pay attention to the words. During the test phase all subjects were presented with the 160 previously studied words along with 160 new words and were required to decide whether the current word was “old” or “new.” To assess the effect of word imagery value, half of the words had a high imagery value and half a low imagery value. Analyses of variance performed on ERPs showed that an imagery-induced modulation of the old/new effect was evident only for subjects who were not engaged in the orthographic task during the study phase. This result supports the hypothesis that this modulation is due to some aspect of the recognition memory process and not to the stimulus encoding operations that occur during the recognition memory task. However, bootstrap analysis on the same data showed that the old/new effect on ERPs was not reliable for all the subjects. This result suggests that only a cautious inference can be made from these data.


Recognition Memory Study Phase Mean Amplitude Recognition Memory Task Current Word 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Bartolini, U., Tavaglini, C., &Zampolli, A. (1971).Lessico di frequenza della lingua italiana contemporanea [Usage norms for contemporary Italian]. Milan: IBM Italia.Google Scholar
  2. Beisteiner, R., Huter, D., Edward, V., &Koch, G. (1997). Brain potentials with old/new distinction of non-words and geometric figures.Electroencephalography & Clinical Neurophysiology,99, 517–526.CrossRefGoogle Scholar
  3. Bentin, S. (1987). Event-related potentials, semantic process, and expectancy factors in word recognition.Brain & Language,31, 308–327.CrossRefGoogle Scholar
  4. Blair, R. C., &Karniski, W. (1993). An alternative method for significance testing of waveform difference potentials.Psychophysiology,30, 518–524.CrossRefPubMedGoogle Scholar
  5. De Weerd, J. P. C. (1981). A posteriori time-varying filtering of averaged evoked potentials.Biological Cybernetics,41, 211–222.CrossRefPubMedGoogle Scholar
  6. De Weerd, J. P. C., &Martens, W. L. J. (1978). Theory and practice of a posteriori “Wiener” filtering of averaged evoked potentials.Biological Cybernetics,30, 81–94.CrossRefGoogle Scholar
  7. Di Nocera, F., &Ferlazzo, F. (1999). ERPs and cognition: Reliability and control [Abstract].Journal of Psychophysiology,13, 207.Google Scholar
  8. Di Nocera, F., Ferlazzo, F., &Gentilomo, A. (1996). Stabilité interindividuelle de l’effet de modulation de la composante P300 des potentiels évoqués cognitifs dans une condition de double tâche.Psychologie Française,41, 365–374.Google Scholar
  9. Donaldson, D. I., &Rugg, M. D. (1998). Recognition memory for new associations: Electrophysiological evidence for the role of recollection.Neuropsychologia,36, 377–395.CrossRefPubMedGoogle Scholar
  10. Efron, B. (1979). Bootstrap methods: Another look at the jackknife.Annals of Statistics,7, 1–26.CrossRefGoogle Scholar
  11. Efron, B., &Tibshirani, R. J. (1993).An introduction to bootstrap. New York: Chapman & Hall.Google Scholar
  12. Farwell, L. A., &Donchin, E. (1991). The truth will out: Interrogative polygraphy (“lie detection”) with event-related brain potentials.Psychophysiology,28, 531–547.CrossRefPubMedGoogle Scholar
  13. Ferlazzo, F., Conte, S., &Gentilomo, A. (1993a). Event-related potentials and recognition memory: The effect of word imagery value.International Journal of Psychophysiology,15, 115–122.CrossRefPubMedGoogle Scholar
  14. Ferlazzo, F., Conte, S., &Gentilomo, A. (1993b). Event-related potentials and recognition memory within the levels of processing framework.NeuroReport,4, 667–670.CrossRefPubMedGoogle Scholar
  15. Ferlazzo, F., &Di Nocera, F. (1998). The serial position effect on ERPs recorded in a cued-recall task [Abstract].International Journal of Psychophysiology,30, 226.CrossRefGoogle Scholar
  16. Ferlazzo, F., Di Nocera, F., &Di Segni, S. (1998). Inside cognitive processes: Using ERPs to investigate the time course of the serial position effect in a cued recall paradigm.General Psychology,1, 155–168.Google Scholar
  17. Friedman, D. (1990). Cognitive event-related potentials components during continuous recognition memory for pictures.Psychophysiology,27, 136–148.CrossRefPubMedGoogle Scholar
  18. Friedman, D., &Sutton, S. (1987). Event-related potentials during continuous recognition memory. In R. Johnson, Jr., J.W. Rohrbaugh, & R. Parasuraman (Eds.),Current trends in event related potentials research (Electroencephalography & Clinical Neurophysiology, Suppl. 40, pp. 316–321). Amsterdam: Elsevier.Google Scholar
  19. Humphrey, D. G., &Kramer, A. F. (1994). Toward a psychophysiological assessment of dynamic changes in mental workload.Human Factors,36, 3–26.PubMedGoogle Scholar
  20. Jasper, H. (1958). The 10–20 electrode system of the International Federation.Electroencephalography & Clinical Neurophysiology,10, 371–375.Google Scholar
  21. Johnson, R., Kreiter, K., Russo, B., &Zhu, J. (1998). A spatiotemporal analysis of recognition-related event-related brain potentials.International Journal of Psychophysiology,29, 83–104.CrossRefPubMedGoogle Scholar
  22. Karis, D., Fabiani, M., &Donchin, E. (1984). “P300” and memory: Individual differences in the von Restorff effect.Cognitive Psychology,16, 177–216.CrossRefGoogle Scholar
  23. Karniski, W., Blair, R. C., &Snider, A. D. (1994). An exact statistical method for comparing topographic maps, with any number of subjects and electrodes.Brain Topography,6, 203–210.CrossRefPubMedGoogle Scholar
  24. Neville, H., Kutas, M., &Schmidt, A. L. (1982). Event-related potential studies of cerebral specialization during reading.Brain & Language,16, 300–315.CrossRefGoogle Scholar
  25. Noldy-Cullum, N. E., &Stelmack, R. M. (1987). Recognition memory for pictures and words: The effect of incidental and intentional learning on N400. In R. Johnson, Jr., J. W. Rohrbaugh, & R. Parasuraman (Eds.),Current trends in event-related potentials research (Electroencephalography & Clinical Neurophysiology, Suppl. 40, pp. 350–354). Amsterdam: Elsevier.Google Scholar
  26. Paivio, A. (1965). Abstractness, imagery, and meaningfulness in paired-associate learning.Journal of Verbal Learning & Verbal Behavior,4, 32–38.CrossRefGoogle Scholar
  27. Pratt, H., Erez, A., &Geva, A. B. (1994). Lexicality and modality effects on evoked potentials in a memory scanning task.Brain & Language,46, 353–367.CrossRefGoogle Scholar
  28. Rugg, M. D. (1985). The effect of semantic priming and word repetition on event-related potentials.Psychophysiology,22, 642–647.CrossRefPubMedGoogle Scholar
  29. Rugg, M. D. (1995). ERP studies of memory. In M. D. Rugg & M. G. H. Coles (Eds.),Electrophysiology of mind (pp. 132–170). Oxford: Oxford University Press.Google Scholar
  30. Rugg, M. D., Furda, J., &Lorist, M. (1988). The effects of task on the modulation of event-related potentials by word repetition.Psychophysiology,25, 55–63.CrossRefPubMedGoogle Scholar
  31. Rugg, M. D., Mark, R. E., Walla, P., Schloerscheidt, A. M., Birch, C. S., &Allan, K. (1998). Dissociation of the neural correlates of implicit and explicit memory.Nature,392, 595–598.CrossRefPubMedGoogle Scholar
  32. Rugg, M. D., &Nagy, M. E. (1987). Lexical contribution to nonword-repetition effects: Evidence from event-related potentials.Memory & Cognition,15, 473–481.CrossRefGoogle Scholar
  33. Rugg, M. D., Schloerscheidt, A. M., &Mark, R. E. (1998). An electrophysiological comparison of two indices of recollection.Journal of Memory & Language,39, 47–69.CrossRefGoogle Scholar
  34. Sanquist, T. F., Rohrbaugh, J. W., Syndulko, K., &Lindsley, D. B. (1980). Electrocortical signs of level of processing: Perceptual analysis and recognition memory.Psychophysiology,17, 568–576.CrossRefPubMedGoogle Scholar
  35. Schloerscheidt, A. M., &Rugg, M. D. (1997). Recognition memory for words and pictures: An event-related potential study.NeuroReport,8, 3281–3285.CrossRefPubMedGoogle Scholar
  36. Smith, M. E., &Guster, K. (1993). Decomposition of recognition memory event-related potentials yields target, repetition and retrieval effects.Electroencephalography & Clinical Neurophysiology,86, 335–343.CrossRefGoogle Scholar
  37. Wasserman, S., &Bockenholt, U. (1989). Bootstrapping: An application to psychophysiology.Psychophysiology,26, 208–221.CrossRefPubMedGoogle Scholar
  38. Woody, C. D. (1967). Characterization of an adaptive filter for the analysis of variable latency neuroelectric signals.Medical & Biological Engineering,5, 539–553.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc 2000

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of Rome “La Sapienza,”RomeItaly

Personalised recommendations