Truthiness: Challenges Associated with Employing Machine Learning on Neurophysiological Sensor Data

  • Mark CostaEmail author
  • Sarah Bratt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9743)


The use of neurophysiological sensors in HCI research is increasing in use and sophistication, largely because such sensors offer the potential benefit of providing “ground truth” in studies, and also because they are expected to underpin future adaptive systems. Sensors have shown significant promise in the efforts to develop measurements to help determine users’ mental and emotional states in real-time, allowing the system to use that information to adjust user experience.

Most of the sensors used generate a substantial amount of data, a high dimensionality and volume of data that requires analysis using powerful machine learning algorithms. However, in the process of developing machine learning algorithms to make sense of the data and subject’s mental or emotional state under experimental conditions, researchers often rely on existing and imperfect measures to provide the “ground truth” needed to train the algorithms.

In this paper, we highlight the different ways in which researchers try to establish ground truth and the strengths and limitations of those approaches. The paper concludes with several suggestions and specific areas that require more discussion.


Machine learning Cognitive data Method validity fNIRS Neuro-physiological sensors  


  1. Ang, K.K., Yu, J., Guan, C.: Extracting and selecting discriminative features from high density NIRS-Based BCI for numerical cognition. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2012). doi: 10.1109/IJCNN.2012.6252604
  2. An, J., Lee, J., Ahn, C.: An efficient GP approach to recognizing cognitive tasks from fNIRS neural signals. Sci. China Inf. Sci. 56(10), 1–7 (2013). doi: 10.1007/s11432-013-5001-8 MathSciNetCrossRefGoogle Scholar
  3. Balconi, M., Grippa, E., Vanutelli, M.E.: What hemodynamic (fNIRS), Electrophysiological (EEG) and autonomic integrated measures can tell us about emotional processing. Brain Cogn. 95, 67–76 (2015). doi: 10.1016/j.bandc.2015.02.001 CrossRefGoogle Scholar
  4. Bandara, D., Song, S., Hirshfield, L., Velipasalar, S.: A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data. In: HCI International 2016 Conference Proceedings. Springer, Toronto (In press)Google Scholar
  5. Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)CrossRefGoogle Scholar
  6. Cuzzocreo, J.L., Yassa, M.A., Verduzco, G., Honeycutt, N.A., Scott, D.J., Bassett, S.S.: Effect of handedness on fMRI activation in the medial temporal lobe during an auditory verbal memory task. Hum. Brain Mapp. 30(4), 1271–1278 (2009). doi: 10.1002/hbm.20596 CrossRefGoogle Scholar
  7. Fairclough, S., Gilleade, K.: Advances in Physiological Computing. Springer Science & Business Media, New York (2014)CrossRefGoogle Scholar
  8. Gateau, T., Durantin, G., Lancelot, F., Scannella, S., Dehais, F.: Real-time state estimation in a flight simulator using fNIRS. PLoS ONE 10(3), e0121279 (2015). doi: 10.1371/journal.pone.0121279 CrossRefGoogle Scholar
  9. Girouard, A., Solovey, E.T., Hirshfield, L.M., Chauncey, K., Sassaroli, A., Fantini, S., Jacob, R.J.: Distinguishing difficulty levels with non-invasive brain activity measurements. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5726, pp. 440–452. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. Girouard, A.: Towards adaptive user interfaces using real time fNIRS. Tufts University, Medford, MA, USA (2010)Google Scholar
  11. Hoskin, R.: The dangers of self-report. Sci. Brainwaves, 3 March 2012.
  12. James, D.R., et al.: Cognitive burden estimation for visuomotor learning with fNIRS. In: Jiang, T., Navab, N., Pluim, J.P., Viergever, M.A. (eds.) MICCAI 2010, Part III. LNCS, vol. 6363, pp. 319–326. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. Lawlor-Savage, L., Goghari, V.M.: Working memory training in schizophrenia and healthy populations. Behav. Sci. 4(3), 301–319 (2014). doi: 10.3390/bs4030301 CrossRefGoogle Scholar
  14. Liu, N., Cui, X., Bryant, D.M., Glover, G.H., Reiss, A.L.: Inferring deep-brain activity from cortical activity using functional near-infrared spectroscopy. Biomed. Opt. Express. 6(3), 1074–1089 (2015). doi: 10.1364/BOE.6.001074 CrossRefGoogle Scholar
  15. Lottridge, D.: Evaluating human computer interaction through self-rated emotion. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5727, pp. 860–863. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  16. Marx, A.-M., Ehlis, A.-C., Furdea, A., Holtmann, M., Banaschewski, T., Brandeis, D., Rothenberger, A., et al.: Near-Infrared Spectroscopy (NIRS) neurofeedback as a treatment for children with Attention Deficit Hyperactivity Disorder (ADHD)—a pilot study. Front. Hum. Neurosci. 8, 1038 (2015). doi: 10.3389/fnhum.2014.01038 CrossRefGoogle Scholar
  17. Morrison, A.B., Conway, A.R., Chein, J.M.: Primacy and recency effects as indices of the focus of attention. Front. Hum. Neurosci. 8 (2014). doi: 10.3389/fnhum.2014.00006
  18. Naseer, N., Hong, K.-S.: fNirs-based brain-computer interfaces: a review. Front. Hum. Neurosci. 9 (2015). doi: 10.3389/fnhum.2015.00003
  19. Noah, J.A., Ono, Y., Nomoto, Y., Shimada, S., Tachibana, A., Zhang, X., Bronner, S., Hirsch, J.: fMRI validation of fNIRS measurements during a naturalistic task. J.Visualized Exp. 100 (2015). doi: 10.3791/52116
  20. Olson, J.S., Kellogg, W.A.: Ways of Knowing in HCI. Springer Science & Business, New York (2014)CrossRefGoogle Scholar
  21. Paulhus, D.L., Vazire, S.: Thse self-report method. In: Robins, R.W., Chris Fraley, R., Krueger, R.F. (eds.) Handbook of Research Methods in Personality Psychology, pp. 224–239. Guilford Press, New York (2009)Google Scholar
  22. Rek, M., Romero, N., van Boeijen, A.: Motivation to self-report: capturing user experiences in field studies. In: Collazos, C., Liborio, A., Rusu, C. (eds.) CLIHC 2013. LNCS, vol. 8278, pp. 111–114. Springer, Heidelberg (2013). doi: CrossRefGoogle Scholar
  23. Rusnock, C., Borghetti, B., McQuaid, I.: Objective-analytical measures of workload – the third pillar of workload triangulation? In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2015. LNCS, vol. 9183, pp. 124–135. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  24. Sackett, P.R, Larson Jr., J.R.: Research strategies and tactics in industrial and organizational psychology. Dunnette, M.D., Hough, L.M. (eds.) Handbook of Industrial and Organizational Psychology, vol. 1, 2nd edn. Consulting Psychologists Press, Palo Alto (1990)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.School of Information StudiesSyracuse UniversitySyracuseUSA
  2. 2.M.I.N.D. Lab S.I. Newhouse School of Public CommunicationsSyracuse UniversitySyracuseUSA

Personalised recommendations