Functional Near-Infrared Spectroscopy and Electroencephalography: A Multimodal Imaging Approach

  • Anna C. Merzagora
  • Meltem Izzetoglu
  • Robi Polikar
  • Valerie Weisser
  • Banu Onaral
  • Maria T. Schultheis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5638)


Although neuroimaging has greatly expanded our knowledge about the brain-behavior relation, combining multiple neuroimaging modalities with complementing strengths can overcome some limitations encountered when using a single modality. Valuable candidates for a multimodal approach are functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG). fNIRS is an imaging technology that localizes hemodynamic changes within the cortex. However, hemodynamic activation is an intrinsically slow process. On the other hand, EEG has excellent time resolution by directly measuring the manifestation of the brain electrical activity at the scalp. Based on their complementary strengths, the integration of fNIRS and EEG may provide higher spatiotemporal resolution than either method alone. In this effort, we integrate fNIRS and EEG to evaluate the behavioral performance of six healthy adults in a working memory task. To this end, features extracted from fNIRS and EEG were used separately, as well as in combination, and their performances were compared against each other.


multimodal neuroimaging functional near-infrared spectroscopy EEG pattern classification working memory n-back P300 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sergent, J.: Brain-imaging studies of cognitive functions. Trends Neurosci. 17, 221–227 (1994)CrossRefPubMedGoogle Scholar
  2. 2.
    Stufflebeam, S.M., Rosen, B.R.: Mapping cognitive function. Neuroimaging Clin N Am. 17, 469–484, viii-ix (2007)Google Scholar
  3. 3.
    Malhi, G.S., Lagopoulos, J.: Making sense of neuroimaging in psychiatry. Acta Psychiatr Scand 117, 100–117 (2008)PubMedGoogle Scholar
  4. 4.
    Eliassen, J.C., Boespflug, E.L., Lamy, M., Allendorfer, J., Chu, W.J., Szaflarski, J.P.: Brain-mapping techniques for evaluating poststroke recovery and rehabilitation: a review. Top Stroke Rehabil 15, 427–450 (2008)CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Calvert, G.A.: Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11, 1110–1123 (2001)CrossRefPubMedGoogle Scholar
  6. 6.
    Lalanne, C., Lorenceau, J.: Crossmodal integration for perception and action. J. Physiol. Paris 98, 265–279 (2004)CrossRefPubMedGoogle Scholar
  7. 7.
    Lewine, J.D., Orrison, W.W.J.: Clinical electroencephalography and event-related potentials. In: Orrison, W.W.J., Lewine, J.D., Sanders, J.A., Hartshorne, M.F. (eds.) Functional brain imaging, pp. 327–368. Mosby-Year Book, Boston (1995)CrossRefGoogle Scholar
  8. 8.
    Lewine, J.D., Orrison, W.W.J.: Magnetoenecephalography and magnetic source imaging. In: Orrison, W.W.J., Lewine, J.D., Sanders, J.A., Hartshorne, M.F. (eds.) Functional brain imaging, pp. 369–418. Mosby-Year Book, Boston (1995)CrossRefGoogle Scholar
  9. 9.
    Raichle, M.E., Mintun, M.A.: Brain work and brain imaging. Ann. Rev. Neurosci. 29, 449–476 (2006)CrossRefPubMedGoogle Scholar
  10. 10.
    Logothetis, N.K., Wandell, B.A.: Interpreting the BOLD Signal. Annu. Rev. Physiol. 66, 735–769 (2004)CrossRefPubMedGoogle Scholar
  11. 11.
    George, J.S., Aine, C.J., Mosher, J.C., Schmidt, D.M., Ranken, D.M., Schlitt, H.A., Wood, C.C., Lewine, J.D., Sanders, J.A., Belliveau, J.W.: Mapping function in the human brain with magnetoencephalography, anatomical magnetic resonance imaging, and functional magnetic resonance imaging. J. Clin. Neurophysiol. 12, 406–431 (1995)CrossRefPubMedGoogle Scholar
  12. 12.
    He, B., Lian, J.: High-resolution spatio-temporal functional neuroimaging of brain activity. Crit. Rev. Biomed. Eng. 30, 283–306 (2002)CrossRefPubMedGoogle Scholar
  13. 13.
    Ritter, P., Villringer, A.: Simultaneous EEG-fMRI. Neurosci. Biobehav. Rev. 30, 823–838 (2006)CrossRefPubMedGoogle Scholar
  14. 14.
    Schiff, N.D.: Multimodal neuroimaging approaches to disorders of consciousness. J. Head Trauma Rehabil. 21, 388–397 (2006)CrossRefPubMedGoogle Scholar
  15. 15.
    Villringer, A.: Functional neuroimaging. Optical approaches Adv. Exp. Med. Biol. 413, 1–18 (1997)CrossRefPubMedGoogle Scholar
  16. 16.
    Izzetoglu, M., Izzetoglu, K., Bunce, S., Ayaz, H., Devaraj, A., Onaral, B., Pourrezaei, K.: Functional near-infrared neuroimaging. IEEE Trans Neural Syst. Rehabil. Eng. 13, 153–159 (2005)CrossRefPubMedGoogle Scholar
  17. 17.
    Jobsis, F.F.: Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters. Science 198, 1264–1267 (1977)CrossRefPubMedGoogle Scholar
  18. 18.
    Izzetoglu, M., Bunce, S.C., Izzetoglu, K., Onaral, B., Pourrezaei, K.: Functional brain imaging using near-infrared technology. IEEE Eng. Med. Biol. Mag. 26, 38–46 (2007)CrossRefPubMedGoogle Scholar
  19. 19.
    Kok, A.: Event-related-potential (ERP) reflections of mental resources: a review and synthesis. Biol. Psychol. 45, 19–56 (1997)CrossRefPubMedGoogle Scholar
  20. 20.
    Johnson Jr., R.: On the neural generators of the P300 component of the event-related potential. Psychophysiology 30, 90–97 (1993)CrossRefPubMedGoogle Scholar
  21. 21.
    Kok, A.: On the utility of P3 amplitude as a measure of processing capacity. Psychophysiology 38, 557–577 (2001)CrossRefPubMedGoogle Scholar
  22. 22.
    Chance, B., Anday, E., Nioka, S., Zhou, S., Hong, L., Worden, K., Li, C., Murray, T., Ovetsky, Y., Pidikiti, D., Thomas, R.: A novel method for fast imaging of brain function, non-invasively, with light. Opt. Express 2, 411–423 (1998)CrossRefPubMedGoogle Scholar
  23. 23.
    Owen, A.M., McMillan, K.M., Laird, A.R., Bullmore, E.: N-back working memory paradigm: a meta-analysis of normative functional neuroimaging studies. Hum. Brain Mapp. 25, 46–59 (2005)CrossRefPubMedGoogle Scholar
  24. 24.
    Delpy, D.T., Cope, M., van der Zee, P., Arridge, S., Wray, S., Wyatt, J.: Estimation of optical pathlength through tissue from direct time of flight measurement. Phys. Med. Biol. 33, 1433–1442 (1988)CrossRefPubMedGoogle Scholar
  25. 25.
    Jung, T.P., Makeig, S., Humphries, C., Lee, T.W., McKeown, M.J., Iragui, V., Sejnowski, T.J.: Removing electroencephalographic artifacts by blind source separation. Psychophysiology 37, 163–178 (2000)CrossRefPubMedGoogle Scholar
  26. 26.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern classification. Wiley Interscience, New York (2001)Google Scholar
  27. 27.
    Kuncheva, L.I.: Combining pattern classifiers: Methods and algorithms. Wiley Interscience, New York (2004)CrossRefGoogle Scholar
  28. 28.
    Parikh, D., Polikar, R.: An Ensemble-Based Incremental Learning Approach to Data Fusion. IEEE Trans Sys. Man Cyber 37, 437–450 (2007)CrossRefGoogle Scholar
  29. 29.
    Kennan, R.P., Horovitz, S.G., Maki, A., Yamashita, Y., Koizumi, H., Gore, J.C.: Simultaneous recording of event-related auditory oddball response using transcranial near infrared optical topography and surface EEG. Neuroimage 16, 587–592 (2002)CrossRefPubMedGoogle Scholar
  30. 30.
    Butti, M., Pastori, A., Merzagora, A., Zucca, C., Bianchi, A., Reni, G., Cerutti, S.: Multimodal analysis of a sustained attention protocol: Continuous Performance Test assessed with Near Infrared Spectroscopy and EEG. In: 28th Annual International Conference of the IEEE EMBS, pp. 1040–1043 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Anna C. Merzagora
    • 1
  • Meltem Izzetoglu
    • 1
  • Robi Polikar
    • 1
    • 2
  • Valerie Weisser
    • 3
  • Banu Onaral
    • 1
  • Maria T. Schultheis
    • 1
    • 3
  1. 1.School of Biomedical Engineering, Science and Health SystemsDrexel UniversityPhiladelphiaU.S.A.
  2. 2.Department of Electrical and Computer EngineeringRowan UniversityGlassboroU.S.A.
  3. 3.Department of PsychologyDrexel UniversityPhiladelphiaU.S.A.

Personalised recommendations