Using fNIRS to Measure Mental Workload in the Real World

  • Evan M. Peck
  • Daniel Afergan
  • Beste F. Yuksel
  • Francine Lalooses
  • Robert J. K. Jacob
Chapter

Abstract

In the past decade, functional near-infrared spectroscopy (fNIRS) has seen increasing use as a non-invasive brain sensing technology. Using optical signals to approximate blood-oxygenation levels in localized regions of the brain, the appeal of the fNIRS signal is that it is relatively robust to movement artifacts and comparable to fMRI measures. We provide an overview of research that builds towards the use of fNIRS to monitor user workload in real world environments, and eventually to act as input to biocybernetic systems. While there are still challenges for the use of fNIRS in real world environments, its unique characteristics make it an appealing alternative for monitoring the cognitive processes of a user.

References

  1. Ayaz H, Shewokis PA, Bunce S, Izzetoglu K, Willems B, Onaral B (2012) Optical brain monitoring for operator training and mental workload assessment. NeuroImage 59(1):36–47Google Scholar
  2. Baddeley AD (1992) Working memory. Science 255(5044):556–559CrossRefGoogle Scholar
  3. Bor D, Cumming N, Scott CEL, Owen AM (2004) Prefrontal cortical involvement in verbal encoding strategies. Eur J Neurosci 19:3365–3370CrossRefGoogle Scholar
  4. Bor D, Duncan J, Wiseman RJ, Owen AM (2003) Encoding strategies dissociate prefrontal activity from working memory demand. Neuron 37:361–367CrossRefGoogle Scholar
  5. Braver TS, Cohen JD, Nystrom LE, Jonides J, Smith EE, Noll DC (1997) A parametric study of prefrontal cortex involvement in human working memory. Neuroimage 5:49–62CrossRefGoogle Scholar
  6. Bunce SC, Izzetoglu K, Ayaz H, Shewokis P, Izzetoglu M, Pourrezaei K, Onaral B (2011) Implementation of fNIRS for monitoring levels of expertise and mental workload. In: Foundations of Augmented Cognition. Directing the Future of Adaptive Systems. Springer, Berlin, pp 13–22 Google Scholar
  7. Burgess PW, Quayle A, Frith CD (2001) Brain regions involved in prospective memory as determined by positron emission tomography. Neuropsychologia 39:545–555CrossRefGoogle Scholar
  8. Cahill L, Uncapher M, Kilpatrick L, Alkire MT, Turner J (2004) Sex-related hemispheric lateralization of amygdala function in emotionally influenced memory: an fMRI investigation. Learn Mem 11(3):261–266CrossRefGoogle Scholar
  9. Chance B, Anday E, Nioka S, Zhou S, Hong L, Worden K, Li C et al (1998) A novel method for fast imaging of brain function, non-invasively, with light. Opt Express 2(10):41123CrossRefGoogle Scholar
  10. Christoff K, Gabrieli JDE (2000) The frontopolar cortex and human cognition: evidence for a rostrocaudal hierarchical organisation within the human prefrontal cortex. Psychobiology 28:168–186Google Scholar
  11. Christoff K, Prabhakaran V, Dorfman J, Zhao Z, Kroger JK, Holyoak KJ, Gabrieli JD (2001) Rostrolateral prefrontal cortex involvement in relational integration during reasoning. Neuroimage 14(5):1136–1149Google Scholar
  12. Cleveland WS, McGill R (1984) Graphical Perception: Theory, experimentation, and the application to the development of graphical methods. J Am Stat Assoc 387:531–554CrossRefMathSciNetGoogle Scholar
  13. Cohen JD, Perlstein WM, Braver TS, Nystrom LE, Noll DC, Jonides J, Smith EE (1997) Temporal dynamics of brain activation during a working memory task. Nature 386:604–608CrossRefGoogle Scholar
  14. Cui X, Bray S, Reiss A (2010) Speeded near infrared spectroscopy (NIRS) response detection. PLoS One 5(11):e15474CrossRefGoogle Scholar
  15. Davis MH, Meunier F, Marslen-Wilson WD (2004) Neural responses to morphological, syntactic, and semantic properties of single words: an fMRI study. Brain Lang 89(3):439–449CrossRefGoogle Scholar
  16. D’Esposito M, Zarahn E, Aguirre G (1999) Event-related functional MRI: implications for cognitive psychology. Psychol Bull 125(1):155–164CrossRefGoogle Scholar
  17. Dove A, Rowe JB, Brett M, Owen AM (2001) Neural correlates of passive and active encoding and retrieval: a 3T fMRI study. Neuroimage 13(Suppl):660CrossRefGoogle Scholar
  18. Franceschini MA, Joseph DK, Huppert TJ, Diamond SG, Boas DA (2006) Diffuse optical imaging of the whole head. J Biomed Opt 11(5):054007CrossRefGoogle Scholar
  19. Gevins AS, Cutillo BC (1993) Neuroelectric evidence for distributed processing in human working memory. Electroencephalogr Clin Neurophysiol 87:128–143CrossRefGoogle Scholar
  20. Girouard A, Solovey E, Hirshfield L, Chauncey K, Sassaroli A, Fantini S, Jacob RJK (2009) Distinguishing difficulty levels with non-invasive brain activity measurements. Interact 2009:440–452Google Scholar
  21. Gore JC (2003) Principles and practice of functional MRI of the human brain. J Clin Investig 112(1):4–9CrossRefGoogle Scholar
  22. Grabenhorst F, Rolls ET (2011) Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci 15(2):5667CrossRefGoogle Scholar
  23. Herff C, Heger D, Putze F, Guan C, Schultz T (2012) Cross-subject classification of speaking modes using fNIRS. ICONIP 2012:417–424Google Scholar
  24. Hirshfield LM, Solovey ET, Girouard A, Kebinger J, Jacob RJK, Sassaroli A, Fantini S (2009) Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy. In: CHI 2009Google Scholar
  25. Hirshfield LM, Gulotta R, Hirshfield S, Hincks S, Russell M, Ward R, Williams T, Jacob RJK (2011) This is your brain on interfaces: enhancing usability testing with functional near-infrared spectroscopy. In: CHI 2011Google Scholar
  26. Hockey GRJ (1997) Compensatory control in the regulation of human performance under stress and high workload: a cognitive-energetical framework. Biol Psychol 45:73–93CrossRefGoogle Scholar
  27. Izzetoglu K, Ayaz H, Menda J (2011) Applications of functional near infrared imaging: case study on UAV ground controller. In: Schmorrow DD, Fidopiastis CM (eds) Foundations of augmented cognition. Springer, New York, pp 608–617 Google Scholar
  28. Jonides J, Smith EE, Koeppe RA, Awh E, Minoshima S, Mintun MA (1993) Spatial working memory in humans as revealed by PET. Nature 363:623–625CrossRefGoogle Scholar
  29. Koechlin E, Corrado G, Pietrini P, Grafman J (2000) Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning. Proc Nat Acad Sci. 97(13):7651–7656Google Scholar
  30. Kroger JK, Sabb FW, Fales CL, Bookheimer SY, Cohen MS, Holyoak KJ (2002) Recruitment of anterior dorsolateral prefrontal cortex in human reasoning: a parametric study of relational complexity. Cereb Cortex 12:477–485Google Scholar
  31. Liu T, Saito H, Oi M (2012) Distinctive activation patterns under intrinsically versus extrinsically driven cognitive loads in prefrontal cortex: a near-infrared spectroscopy study using a driving video game. Neuroscience letters, 506(2):220–224Google Scholar
  32. Luu S, Chau T (2008) Decoding subjective preference from single-trial near-infrared spectroscopy signals. J Neural Eng 6:058001 Google Scholar
  33. Miller G (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 63(2):8197Google Scholar
  34. Minati L, Grisoli M, Franceschetti S, Epifani F, Granvillano A, Medford N, Harrison N et al (2012) Neural signatures of economic parameters during decision-making: a functional MRI (FMRI), electroencephalography (EEG) and autonomic monitoring study. Brain Topogr 25(1):73–96Google Scholar
  35. Moghimi S, Kushki A, Power S, Guerguerian AM, Chau T (2012) Automatic detection of a prefrontal cortical response to emotionally rated music using multi-channel near-infrared spectroscopy. J Neural Eng 9(2):026022Google Scholar
  36. Owen AM, McMillan KM, Laird AR, Bullmore E (2005) N-back working memory paradigm: a meta-analysis of normative functional neuroimaging studies. Hum Brain Mapp 25(1):46–59CrossRefGoogle Scholar
  37. Peck EM, Afergan D, Jacob RJK (2013a) Investigation of fNIRS brain sensing as input to information filtering systems. In: Augmented human 2013Google Scholar
  38. Peck EM, Yuksel BF, Ottley A, Jacob RJK, Chang R (2013b) Using fNIRS brain sensing to evaluate information visualization interfaces. In: CHI 2013Google Scholar
  39. Ramnani N, Owen AM (2004) Anterior prefrontal cortex: insights into function from anatomy and neuroimaging. Nat Rev Neurosci 5:184–194CrossRefGoogle Scholar
  40. Repovš G, Baddeley A (2006) The multi-component model of working memory: explorations in experimental cognitive psychology. Neuroscience 139:5–21CrossRefGoogle Scholar
  41. Repovš G, Bresjanac M (2006) Cognitive neuroscience of working memory: a prologue. Neuroscience 139:1–3CrossRefGoogle Scholar
  42. Rugg MD, Fletcher PC, Allan K, Frith CD, Frackowiak RS, Dolan RJ (1998) Neural correlates of memory retrieval during recognition memory and cued recall. Neuroimage 8:262–273CrossRefGoogle Scholar
  43. Sase I, Takatsuki A, Seki J, Yanagida T, Seiyama A (2012) Noncontact backscatter-mode near-infrared time-resolved imaging system: preliminary study for functional brain mapping. J Biomed Opt 11(5):054006Google Scholar
  44. Solovey ET, Girouard A, Chauncey K, Hirshfield LM, Sassaroli A, Zheng F, Fantini S, Jacob RJK (2009) Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines. In: UIST 2009Google Scholar
  45. Solovey ET, Lalooses F, Chauncey K, Weaver D, Scheutz M, Sassaroli A, Fantini S, Jacob RJK (2011) Sensing cognitive multitasking for a brain-based adaptive user interface. In: CHI 2011Google Scholar
  46. Solovey ET, Schermerhorn P, Scheutz M, Sassaroli A, Fantini S, Jacob RJK (2012) Brainput: enhancing interactive systems with streaming fNIRS brain input. In: CHI 2012Google Scholar
  47. Strangman G, Culver JP, Thompson JH, Boas DA (2002) A quantitative comparison of simultaneous BOLD fMRI and NIRS recordings during functional brain activation. NeuroImage 17(2):719731CrossRefGoogle Scholar
  48. Tsunashima H, Yanagisawa K (2009) Measurement of brain function of car driver using functional near-infrared spectroscopy (fNIRS). Comput Intell Neurosci 2009:164958Google Scholar
  49. Tulving E (1983) Elements of episodic memory. Clarendon, OxfordGoogle Scholar
  50. Wickens CD (2002) Multiple resources and performance prediction. Theor Issues Ergon Sci 3:159–177Google Scholar
  51. Wildey C, MacFarlane D, Khan B, Tian F, Liu H, Alexandrakis G (2010) Improved fNIRS using a novel brush optrode. In: Laser scienceGoogle Scholar
  52. Vidaurre C, Sannelli C, Muller K-R, Blankertz B (2010) Machine-learning-based coadaptive calibration for brain-computer interfaces. Neural Comput 816:791816Google Scholar
  53. Villringer A, Chance B (1997) Non-invasive optical spectroscopy and imaging of human brain function. Trends Neurosci 20(10):43542CrossRefGoogle Scholar
  54. Yurtsever G, Ayaz H, Kepics F, Onaral B (2003) Wireless, continuous wave near infrared spectroscopy system for monitoring brain activity. In: Bioengineering conference, pp 53–53Google Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • Evan M. Peck
    • 1
  • Daniel Afergan
    • 1
  • Beste F. Yuksel
    • 1
  • Francine Lalooses
    • 1
  • Robert J. K. Jacob
    • 1
  1. 1.Tufts UniversityMedfordUSA

Personalised recommendations