Abstract
Functional near-infrared spectroscopy (fNIRS) is an emerging non-invasive, lightweight imaging tool which can measure blood oxygenation levels in the brain. In this chapter, we describe the fNIRS device and its potential within the realm of human-computer interaction (HCI). We discuss research that explores the kinds of states that can be measured with fNIRS, and we describe initial research and prototypes that can use this objective, real time information about users’ states as input to adaptive user interfaces.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bunce SC, Izzetoglu M, Izzetoglu K, Onaral B, Pourrezaei K (2006) Functional near infrared spectroscopy: An emerging neuroimaging modality. IEEE Eng Med Biol Mag 25(4):54–62. Special issue on Clinical Neuroengineering
Chenier F, Sawan M (2007) A new brain imaging device based on fNIRS. Paper presented at the BIOCAS 2007, pp 1–4
Coyle S, Ward T, Markham C (2004) Physiological noise in near-infrared spectroscopy: Implications for optical brain computer interfacing. Paper presented at the Proc EMBS, pp 4540–4543
Cutrell E, Tan DS (2007) BCI for passive input in HCI. Paper presented at the ACM CHI’08 Conference on Human Factors in Computing Systems Workshop on Brain-Computer Interfaces for HCI and Games
Devaraj A, Izzetoglu M, Izzetoglu K, Onaral B (2004) Motion artifact removal for fNIR spectroscopy for real world application areas. Proc SPIE 5588:224–229
Dietterich TG (2002) Machine learning for sequential data: A review. Struct Syntactic Stat Pattern Recognit 2396:15–30
Ehlis A-C, Bähne CG, Jacob CP, Herrmann MJ, Fallgatter AJ (2008) Reduced lateral prefrontal activation in adult patients with attention-deficit/hyperactivity disorder (ADHD) during a working memory task: A functional near-infrared spectroscopy (fNIRS) study. J Psychiatr Res 42(13):1060–1067
Fairclough SH (2009) Fundamentals of physiological computing. Interact Comput 21(1–2):133–145
Fuller D, Sullivan J, Essif E, Personius K, Fregosi R (1995) Measurement of the EMG-force relationship in a human upper airway muscle. J Appl Physiol 79(1):270–278
Gevins A, Smith M (2003) Neurophysiological measures of cognitive workload during human-computer interaction. Theor Issues Ergon Sci 4:113–131
Girouard A, Solovey E, Hirshfield L, Chauncey K, Sassaroli A, Fantini S, et al (2009) Distinguishing difficulty levels with non-invasive brain activity measurements. In: Human-Computer Interaction—INTERACT 2009, pp 440–452
Grimes D, Tan DS, Hudson SE, Shenoy P, Rao RPN (2008) Feasibility and pragmatics of classifying working memory load with an electroencephalograph. Paper presented at the Proc CHI’08, pp 835–844
Guhe M, Liao W, Zhu Z, Ji Q, Gray WD, Schoelles MJ (2005) Non-intrusive measurement of workload in real-time. Paper presented at the 49th Annual Conference of the Human Factors and Ergonomics Society, pp 1157–1161
Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): Results of empirical and theorical research. In: Hancock P, Meshkati N (eds) Human Mental Workload, Amsterdam, pp 139–183
Herrmann MJ, Huter T, Plichta MM, Ehlis A-C, Alpers GW, Mühlberger A, et al. (2008) Enhancement of activity of the primary visual cortex during processing of emotional stimuli as measured with event-related functional near-infrared spectroscopy and event-related potentials. Hum Brain Mapp 29(1):28–35
Hirshfield LM, Girouard A, Solovey ET, Jacob RJK, Sassaroli A, Tong Y, et al (2007) Human-computer interaction and brain measurement using functional near-infrared spectroscopy. Paper presented at the Proceedings of the ACM UIST’07 Symposium on User Interface Software and Technology
Hirshfield LM, Chauncey K, Gulotta R, Girouard A, Solovey ET, Jacob RJ, et al (2009a) Combining electroencephalograph and functional near infrared spectroscopy to explore users’ mental workload. Paper presented at the Proceedings of the 5th International Conference on Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience: Held as Part of HCI International 2009, San Diego, CA, pp 239–247
Hirshfield LM, Solovey ET, Girouard A, Kebinger J, Jacob RJK, Sassaroli A, et al (2009b) Brain measurement for usability testing and adaptive interfaces: An example of uncovering syntactic workload with functional near infrared spectroscopy. Paper presented at the Proceedings of the 27th International Conference on Human Factors in Computing Systems, Boston, MA, USA, pp 2185–2194
Hoshi Y (2009) Near-infrared spectroscopy for studying higher cognition neural correlates of thinking, pp 83–93
Iqbal ST, Zheng XS, Bailey BP (2004) Task-evoked pupillary response to mental workload in human-computer interaction. Paper presented at the CHI’04 extended abstracts on Human Factors in Computing Systems, Vienna, Austria, pp 1477–1480
Izzetoglu K, Bunce S, Onaral B, Pourrezaei K, Chance B (2004) Functional optical brain imaging using near-infrared during cognitive tasks. IJHCI 17(2):211–231
Jacob RJK (1993) Eye movement-based human-computer interaction techniques: Toward non-command interfaces. In: Hix HRHaD (ed) Advances in Human-Computer Interaction, vol. 4. Ablex Publishing Co, Norwood, NJ, pp 151–190
John MS, Kobus D, Morrison J, Schmorrow D (2004) Overview of the DARPA augmented cognition technical integration experiment. Int J Hum Comput Interact 17(2):131–149
Kok A (1997) Event-related-potential (ERP) reflections of mental resources: A review and synthesis. Biol Psychol 45:19–56
Lee JC, Tan DS (2006) Using a low-cost electroencephalograph for task classification in HCI research. Paper presented at the Proc ACM Symposium on User Interface Software and Technology 2006, pp 81–90
Mandryk R, Atkins M, Inkpen K (2006) A continuous and objective evaluation of emotional experience with interactive play environments. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems, Montreal, Quebec, Canada, pp 1027–1036
Mappus RL, Venkatesh GR, Shastry C, Israeli A Jackson MM (2009) An fNIR Based BMI for Letter Construction Using Continuous Control. Paper presented at the Proc CHI 2009 Extended Abstracts, pp 3571–3576
Marshall S, Pleydell-Pearce C, Dickson B (2003) Integrating psychophysiological measures of cognitive workload and eye movements to detect strategy shifts. Paper presented at the IEEE Proceedings of the 36th Annual Hawaii International Conference on System Sciences, 130b
Matsuda G, Hiraki K (2006) Sustained decrease in oxygenated hemoglobin during video games in the dorsal prefrontal cortex: A NIRS study of children. NeuroImage 29(3):706–711
Matthews F, Pearlmutter BA, Ward TE, Soraghan C, Markham C (2008) Hemodynamics for brain-computer interfaces. Signal Process Mag IEEE 25(1):87–94
Millán JdR, Renkens F, Mouriño J, Gerstner W (2004) Brain-actuated interaction. Artif Intell 159(1-2):241–259
Morioka S, Yamada M, Komori T (2008) Frontal lobe activity during the performance of spatial tasks: fNIRS study. J Phys Ther Sci 20:135–139
Nagamitsu S, Nagano M, Tamashita Y, Takashima S, Matsuishi T (2006) Prefrontal cerebral blood volume patterns while playing video games—A near-infrared spectroscopy study. Brain Develop 28:315–321
Nijholt A, Tan D, Allison B, Millán JdR, Graimann B (2008) Brain-computer interfaces for HCI and games. Paper presented at the CHI’08 extended abstracts on Human Factors in Computing Systems, Florence, Italy, pp 3925–3928
Ramnani N, Owen AM (2004) Anterior prefrontal cortex: insights into function from anatomy and neuroimaging. Nat Rev Neurosci 5(3):184–194
Shneiderman B, Plaisant C (2005) Designing the User Interface: Strategies for Effective Human-Computer Interaction, 4th edn. Addison-Wesley, Reading, Mass
Sitaram R, Zhang H, Guan C, Thulasidas M, Hoshi Y, Ishikawa A, et al. (2007) Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. NeuroImage 34(4):1416–1427
Solovey ET, Girouard A, Chauncey K, Hirshfield LM, Sassaroli A, Zheng F, et al (2009) Using fNIRS brain sensing in realistic HCI settings: Experiments and guidelines. Paper presented at the ACM UIST’09 Symposium on User Interface Software and Technology, pp 157–166
Villringer A, Chance B (1997) Non-invasive optical spectroscopy and imaging of human brain function. Trends Neurosci 20:435–442
Acknowledgements
The authors would like to thank the HCI research group at Tufts University; Michel Beaudoin-Lafon, Wendy Mackay, and the In |Situ| research group; and Desney Tan and Dan Morris at Microsoft Research. We thank the NSF (Grant Nos. IIS-0713506 and IIS-0414389), the Canadian National Science and Engineering Research Council, the US Air Force Research Laboratory, and the US Army NSRDEC for support of this research. Any opinions, findings, and conclusions or recommendations expressed in this chapter are those of the authors and do not necessarily reflect the views of these organizations.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag London Limited
About this chapter
Cite this chapter
Girouard, A. et al. (2010). From Brain Signals to Adaptive Interfaces: Using fNIRS in HCI. In: Tan, D., Nijholt, A. (eds) Brain-Computer Interfaces. Human-Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-84996-272-8_13
Download citation
DOI: https://doi.org/10.1007/978-1-84996-272-8_13
Publisher Name: Springer, London
Print ISBN: 978-1-84996-271-1
Online ISBN: 978-1-84996-272-8
eBook Packages: Computer ScienceComputer Science (R0)