Abstract
Research on adaptive systems based on human psychophysiological assessments has been growing rapidly over the last decade. One fundamental component of such a system is human state assessment, on which the adaptation depends, at least partly. This is critical as the confidence of operators in such system will be determined to some extent by the accuracy of the models responsible for the assessment. This chapter presents work carried out in order to better understand the relationship between bio-behavioral data, psychological state, and operational performance of the operators. Modeling physiological parameters and performance was performed through a manipulation of three factors. The first factor was the size of smoothing window for performance. The second factor was the performance decrement threshold for labelling functional and sub-functional states. Finally, the third factor was the mode of classification being either prospective or descriptive. We used two types of classifiers, a linear and a non-linear classifier, and compared performance. Insights emerging from this work support that the use of multiple sources of bio-behavioral data, combined in a non-linear fashion, increases the psychometric qualities of state classifiers. This suggests that if such systems are to be used in safety-critical systems, they should be implemented using a wide variety of sensors to increase classification accuracies of performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Carter, R., Cheuvront, S.N., Sawka, M.N.: Operator Functional State Assessment (l’évaluation de l’aptitude opérationnelle de l’opérateur humain). Army research institute (2004)
Bracken, B.K., Palmon, N., Romero, V., Pfautz, J., Cooke, N.J.: A prototype toolkit for sensing and modeling individual and team state. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 58, pp. 949–953 (2014). https://doi.org/10.1177/1541931214581199
Durkee, K.T., Pappada, S.M., Ortiz, A.E., Feeney, J.J., Galster, S.M.: System decision framework for augmenting human performance using real-time workload classifiers. Presented at the 2015 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), Orlando, FL (2015)
Yin, Z., Zhang, J.: Operator functional state classification using least-square support vector machine based recursive feature elimination technique. Comput. Methods Programs Biomed. 113, 101–115 (2014). https://doi.org/10.1016/j.cmpb.2013.09.007
Durantin, G., Gagnon, J.-F., Tremblay, S., Dehais, F.: Using near infrared spectroscopy and heart rate variability to detect mental overload. Behav. Brain Res. 259, 16–23 (2014). https://doi.org/10.1016/j.bbr.2013.10.042
Hogervorst, M.A., Brouwer, A.-M., van Erp, J.B.F.: Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload. Front. Neurosci. 8 (2014). https://doi.org/10.3389/fnins.2014.00322
Tobon, D.V., Falk, T., Maier, M.: MS-QI: a modulation spectrum-based ECG quality index for telehealth applications. IEEE Trans. Biomed. Eng. 99, 1 (2014)
Overbeek, T.J., van Boxtel, A., Westerink, J.H.: Respiratory sinus arrhythmia responses to cognitive tasks: effects of task factors and RSA indices. Biol. Psychol. 99, 1–14 (2014)
Wijsman, J., Grundlehner, B., Liu, H., Penders, J., Hermens, H.: Wearable physiological sensors reflect mental stress state in office-like situations. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), pp. 600–605 (2013). https://doi.org/10.1109/acii.2013.105
Williamon, A., Aufegger, L., Wasley, D., Looney, D., Mandic, D.P.: Complexity of physiological responses decreases in high-stress musical performance. J. R. Soc. Interface 10 (2013). https://doi.org/10.1098/rsif.2013.0719
Régis, N., et al.: Formal detection of attentional tunneling in human operator-automation interactions. IEEE Trans. Hum. Mach. Syst. 44(3), 326–336 (2014)
Matthews, G., Reinerman-Jones, L.E., Barber, D.J., Abich, J.: The psychometrics of mental workload multiple measures are sensitive but divergent. Hum. Factors J. Hum. Factors Ergon. Soc. 57, 125–143 (2015). https://doi.org/10.1177/0018720814539505
Gaillard, A.W.: Fatigue assessment and performance protection. NATO Sci. Ser. Sub Ser. I Life Behav. Sci. 355, 24–35 (2003)
Eggemeier, F.T., Wilson, G.F., Kramer, A.F., Damos, D.L.: Workload Assessment in Multi-Task Environments. Multiple-Task Performance, pp. 207–216 (1991)
Wilson, G.F., Russell, C.A.: Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Hum. Factors 45, 635–643 (2003)
Nourbakhsh, N., Wang, Y., Chen, F.: GSR and blink features for cognitive load classification. In: Kotzé, P., Marsden, G., Lindgaard, G., Wesson, J., Winckler, M. (eds.) Human-Computer Interaction–INTERACT. LNCS, vol. 8117, pp. 159–166. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40483-2_11
Brouwer, A.-M., Zander, T.O., van Erp, J.B.F., Korteling, J.E., Bronkhorst, A.W.: Using neurophysiological signals that reflect cognitive or affective state: six recommendations to avoid common pitfalls. Front. Neurosci. 9 (2015). https://doi.org/10.3389/fnins.2015.00136
Gagnon, J.-F., Gagnon, O., Lafond, D., Parent, M., Tremblay, S.: A systematic assessment of operational metrics for modeling operator functional state: In: Proceedings of the 3rd International Conference on Physiological Computing Systems, pp. 15–23 SCITEPRESS - Science and Technology Publications, Lisbon, Portugal (2016). https://doi.org/10.5220/0005921600150023
Poole, A., Ball, L.J.: Eye tracking in human-computer interaction and usability research: current status and future prospects. In: Encyclopedia of Human-Computer Interaction, pp. 211–219 (2005). https://doi.org/10.4018/978-1-59140-562-7
Salvucci, D.D., Goldberg, J.H.: Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 Symposium on Eye Tracking Research & Applications, pp. 71–78. ACM (2000)
Boonnithi, S., Phongsuphap, S.: Comparison of heart rate variability measures for mental stress detection. Comput. Cardiol. 2011, 85–88 (2011)
Rodríguez-Liñares, L., Vila, X., Mendez, A., Lado, M., Olivieri, D.: RHRV: an R-based software package for heart rate variability analysis of ECG recordings. In: 3rd Iberian Conference in Systems and Information Technologies (CISTI 2008), Vigo, Spain (2008)
Gagnon, O., Lafond, D., Gagnon, J-F., Parizeau, M.: Comparing methods for assessing operator functional state. In: Proceedings of the 2016 IEEE International Inter-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), San Diego, CA, USA, 21–25 March 2016
Oh, H., et al.: A composite cognitive workload assessment system in pilots under various task demands using ensemble learning. In: Schmorrow, Dylan D., Fidopiastis, Cali M. (eds.) AC 2015. LNCS (LNAI), vol. 9183, pp. 91–100. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20816-9_10
Kuhn, M.: caret: classification and regression training. Astrophys. Source Code Libr. 1, 05003 (2015)
Torgo, L.: Data Mining with R, Learning with Case Studies. Chapman and Hall/CRC (2010). http://www.dcc.fc.up.pt/~ltorgo/DataMiningWithR
Wang, Z., Hope, R.M., Wang, Z., Ji, Q., Gray, W.D.: Cross-subject workload classification with a hierarchical Bayes model. NeuroImage Neuroergon. Hum. Brain Action Work 59, 64–69 (2012). https://doi.org/10.1016/j.neuroimage.2011.07.094
Acknowledgments
This research was supported by a Mitacs internship awarded to Mark Parent, funded by NSERC and Thales Canada. The authors would also like to thank Margot Beugniot for her participation in data collection.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Gagnon, JF., Gagnon, O., Lafond, D., Parent, M., Tremblay, S. (2019). Bio-behavioral Modeling of Workload and Performance. In: Holzinger, A., Pope, A., Plácido da Silva, H. (eds) Physiological Computing Systems. PhyCS PhyCS PhyCS 2016 2017 2018. Lecture Notes in Computer Science(), vol 10057. Springer, Cham. https://doi.org/10.1007/978-3-030-27950-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-27950-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27949-3
Online ISBN: 978-3-030-27950-9
eBook Packages: Computer ScienceComputer Science (R0)