Advertisement

Combining Electroencephalograph and Functional Near Infrared Spectroscopy to Explore Users’ Mental Workload

  • Leanne M. Hirshfield
  • Krysta Chauncey
  • Rebecca Gulotta
  • Audrey Girouard
  • Erin T. Solovey
  • Robert J. K. Jacob
  • Angelo Sassaroli
  • Sergio Fantini
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5638)

Abstract

We discuss the physiological metrics that can be measured with electroencephalography (EEG) and functional near infrared spectroscopy (fNIRs). We address the functional and practical limitations of each device, and technical issues to be mindful of when combining the devices. We also present machine learning methods that can be used on concurrent recordings of EEG and fNIRs data. We discuss an experiment that combines fNIRs and EEG to measure a range of user states that are of interest in HCI. While our fNIRS machine learning results showed promise for the measurement of workload states in HCI, our EEG results indicate that more research must be done in order to combine these two devices in practice.

Keywords

fNIRs EEG near infrared spectroscopy workload 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Izzetoglu, K., et al.: Functional Optical Brain Imaging Using Near-Infrared During Cogni-tive Tasks. International Journal of Human-Computer Interaction 17(2), 211–231 (2004)CrossRefGoogle Scholar
  2. 2.
    John, M.S., et al.: Overview of the DARPA Augmented Cognition Technical Integration Experiment. International Journal of Human-Computer Interaction 17(2), 131–149 (2004)CrossRefGoogle Scholar
  3. 3.
    Tong, Y., et al.: Concurrent recordings of electrical evoked potentials and near-infrared responses to brain activation. Tufts University (2007)Google Scholar
  4. 4.
    Savran, A., et al.: Emotion Detection in the Loop from Brain Signals and Facial Images. In: eNTERFACE 2006, Dubrovnik, Croatia (2006)Google Scholar
  5. 5.
    Salvatori, G., et al.: Combining Near-Infrared Spectroscopy and Electroencephalography to Monitor Brain Function. In: Instrumentation and Measurement Technology Conference. IEEE, Sorrento (2006)Google Scholar
  6. 6.
    Anderson, C., Sijercic, Z.: Classification of EEG Signals from Four Subjects During Five Mental Tasks. In: Solving Engineering Problems with Neural Networks: Proceedings of the Conference on Engineering Applications in Neural Networks, pp. 407–414 (1996)Google Scholar
  7. 7.
    Berka, C., et al.: Real-Time Analysis of EEG Indexes of Alertness, Cognition, and Memory Acquired With a Wireless EEG Headset. International Journal of Human Computer Interaction 17(2), 151–170 (2004)CrossRefGoogle Scholar
  8. 8.
    Lee, J.C., Tan, D.S.: Using a Low-Cost Electroencephalograph for Task Classification in HCI Research. In: ACM Symposium on User Interface Software and Technology (2006)Google Scholar
  9. 9.
    Grimes, D., et al.: Feasibility and Pragmatics of Classifying Working Memory Load with an Electroencephalograph. In: CHI 2008 Conference on Human Factors in Computing Systems, Florence, Italy (2008)Google Scholar
  10. 10.
    Gevins, A., et al.: Flight helmet EEG system, in Final Tech Report AL/CF-SR-1993-0007. Sam Technology, San Fracisco, CA (1993)Google Scholar
  11. 11.
    Mathan, S., et al.: Neurophysiological Estimation of Interruptibility: Demonstrating Feasibility in a Field Context. In: Proceedings of the 4th International Conference of the Augmented Cognition Society, Baltimore, MD (2007)Google Scholar
  12. 12.
    Chance, B., et al.: A novel method for fast imaging of brain function, non-invasively, with light. Optics Express 10(2), 411–423 (1988)Google Scholar
  13. 13.
    Parasuraman, R., Caggiano, D.: Neural and Genetic Assays of Human Mental Workload. In: Quantifying Human Information Processing, Lexington Books (2005)Google Scholar
  14. 14.
    Hirshfield, L.M., et al.: Brain Measurement for Usability Testing and Adaptive Interfaces: An Example of Uncovering Syntactic Workload in the Brain Using Functional Near Infrared Spectroscopy. In: Conference on Human Factors in Computing Systems: Proceeding of the twenty-seventh annual SIGCHI conference on Human factors in computing systems (2009)Google Scholar
  15. 15.
    Sassaroli, A., et al.: Discrimination of mental workload levels in human subjects with functional near-infrared spectroscopy. The Journal of Innovative Optical Health Sciences (accepted, 2009)Google Scholar
  16. 16.
    Son, I.-Y., et al.: Human performance assessment using fNIR. In: Proceedings of SPIE The International Society for Optical Engineering, vol. 5797, pp. 158–169 (2005)Google Scholar
  17. 17.
    Sitaram, R., et al.: Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. NeuroImage 34(1416) (2007)Google Scholar
  18. 18.
    Lin, J., et al.: A Symbolic Representation of Time Series, with Implications for Streaming Algorithms. In: Proceedings of the 8th ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery, San Diego, CA (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Leanne M. Hirshfield
    • 1
  • Krysta Chauncey
    • 1
  • Rebecca Gulotta
    • 1
  • Audrey Girouard
    • 1
  • Erin T. Solovey
    • 1
  • Robert J. K. Jacob
    • 1
  • Angelo Sassaroli
    • 2
  • Sergio Fantini
    • 2
  1. 1.Human Computer Interaction LabUSA
  2. 2.Biomedical Engineering DepartmentTufts UniversityMedfordUSA

Personalised recommendations