Truthiness: Challenges Associated with Employing Machine Learning on Neurophysiological Sensor Data
The use of neurophysiological sensors in HCI research is increasing in use and sophistication, largely because such sensors offer the potential benefit of providing “ground truth” in studies, and also because they are expected to underpin future adaptive systems. Sensors have shown significant promise in the efforts to develop measurements to help determine users’ mental and emotional states in real-time, allowing the system to use that information to adjust user experience.
Most of the sensors used generate a substantial amount of data, a high dimensionality and volume of data that requires analysis using powerful machine learning algorithms. However, in the process of developing machine learning algorithms to make sense of the data and subject’s mental or emotional state under experimental conditions, researchers often rely on existing and imperfect measures to provide the “ground truth” needed to train the algorithms.
In this paper, we highlight the different ways in which researchers try to establish ground truth and the strengths and limitations of those approaches. The paper concludes with several suggestions and specific areas that require more discussion.
KeywordsMachine learning Cognitive data Method validity fNIRS Neuro-physiological sensors
- Ang, K.K., Yu, J., Guan, C.: Extracting and selecting discriminative features from high density NIRS-Based BCI for numerical cognition. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2012). doi: 10.1109/IJCNN.2012.6252604
- Bandara, D., Song, S., Hirshfield, L., Velipasalar, S.: A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data. In: HCI International 2016 Conference Proceedings. Springer, Toronto (In press)Google Scholar
- Girouard, A., Solovey, E.T., Hirshfield, L.M., Chauncey, K., Sassaroli, A., Fantini, S., Jacob, R.J.: Distinguishing difficulty levels with non-invasive brain activity measurements. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5726, pp. 440–452. Springer, Heidelberg (2009)CrossRefGoogle Scholar
- Girouard, A.: Towards adaptive user interfaces using real time fNIRS. Tufts University, Medford, MA, USA (2010)Google Scholar
- Hoskin, R.: The dangers of self-report. Sci. Brainwaves, 3 March 2012. http://www.sciencebrainwaves.com/the-dangers-of-self-report/
- Marx, A.-M., Ehlis, A.-C., Furdea, A., Holtmann, M., Banaschewski, T., Brandeis, D., Rothenberger, A., et al.: Near-Infrared Spectroscopy (NIRS) neurofeedback as a treatment for children with Attention Deficit Hyperactivity Disorder (ADHD)—a pilot study. Front. Hum. Neurosci. 8, 1038 (2015). doi: 10.3389/fnhum.2014.01038 CrossRefGoogle Scholar
- Morrison, A.B., Conway, A.R., Chein, J.M.: Primacy and recency effects as indices of the focus of attention. Front. Hum. Neurosci. 8 (2014). doi: 10.3389/fnhum.2014.00006
- Naseer, N., Hong, K.-S.: fNirs-based brain-computer interfaces: a review. Front. Hum. Neurosci. 9 (2015). doi: 10.3389/fnhum.2015.00003
- Noah, J.A., Ono, Y., Nomoto, Y., Shimada, S., Tachibana, A., Zhang, X., Bronner, S., Hirsch, J.: fMRI validation of fNIRS measurements during a naturalistic task. J.Visualized Exp. 100 (2015). doi: 10.3791/52116
- Paulhus, D.L., Vazire, S.: Thse self-report method. In: Robins, R.W., Chris Fraley, R., Krueger, R.F. (eds.) Handbook of Research Methods in Personality Psychology, pp. 224–239. Guilford Press, New York (2009)Google Scholar
- Rek, M., Romero, N., van Boeijen, A.: Motivation to self-report: capturing user experiences in field studies. In: Collazos, C., Liborio, A., Rusu, C. (eds.) CLIHC 2013. LNCS, vol. 8278, pp. 111–114. Springer, Heidelberg (2013). doi:http://link.springer.com/chapter/10.1007/978-3-319-03068-5_19 CrossRefGoogle Scholar
- Sackett, P.R, Larson Jr., J.R.: Research strategies and tactics in industrial and organizational psychology. Dunnette, M.D., Hough, L.M. (eds.) Handbook of Industrial and Organizational Psychology, vol. 1, 2nd edn. Consulting Psychologists Press, Palo Alto (1990)Google Scholar