Cognitive Neurodynamics

, Volume 7, Issue 5, pp 395–407 | Cite as

Predictive modeling of human operator cognitive state via sparse and robust support vector machines

  • Jian-Hua ZhangEmail author
  • Pan-Pan Qin
  • Jörg Raisch
  • Ru-Bin Wang
Research Article


The accurate prediction of the temporal variations in human operator cognitive state (HCS) is of great practical importance in many real-world safety-critical situations. However, since the relationship between the HCS and electrophysiological responses of the operator is basically unknown, complicated and uncertain, only data-based modeling method can be employed. This paper is aimed at constructing a data-driven computationally intelligent model, based on multiple psychophysiological and performance measures, to accurately estimate the HCS in the context of a safety-critical human–machine system. The advanced least squares support vector machines (LS-SVM), whose parameters are optimized by grid search and cross-validation techniques, are adopted for the purpose of predictive modeling of the HCS. The sparse and weighted LS-SVM (WLS-SVM) were proposed by Suykens et al. to overcome the deficiency of the standard LS-SVM in lacking sparseness and robustness. This paper adopted those two improved LS-SVM algorithms to model the HCS based solely on a set of physiological and operator performance data. The results showed that the sparse LS-SVM can obtain HCS models with sparseness with almost no loss of modeling accuracy, while the WLS-SVM leads to models which are robust in case of noisy training data. Both intelligent system modeling approaches are shown to be capable of capturing the temporal fluctuation trends of the HCS because of their superior generalization performance.


Human operator cognitive state Least squares support vector machine Sparseness Robustness Regressive model 



The authors would also like to thank Prof. D Manzey, Technical University Berlin, Germany, for providing the AUTO-CAMS software used in our OFS data acquisition experiments. The work supported by the National Natural Science Foundation of China (under Grant No. 61075070 and Key Grant No. 11232005) and a Senior Research Fellowship from the Max Planck Institute for Dynamics of Complex Technical Systems, Magdeburg, Germany.


  1. Bradshaw JM et al (2002) Adjustable autonomy and human-agent teamwork in practice: an interim report on space applications, Chapter 0, IEEE computer society foundation for intelligent physical agents (FIPA)Google Scholar
  2. Backs RW, Boucsein W (eds) (2000) Engineering psychophysiology: issues and applications. Lawrence Erlbaum Associates, MahwahGoogle Scholar
  3. Bainbridge L (1983) Ironies of automation. Automatica 19(6):775–779CrossRefGoogle Scholar
  4. Chen Z, Cao J, Cao Y et al (2008) An empirical EEG analysis in brain death diagnosis for aults. Cogn Neurodyn 2:257–271PubMedCrossRefGoogle Scholar
  5. Colucci F (1995) Rotorcraft Pilot’s Associate update: the army’s largest science and technology program. Vertiflite, March/April 1995, 16–20Google Scholar
  6. Fitts PM (1951) Some basic questions in designing an air-navigation and air-traffic control system. In: Moray N (ed) Ergonomics major writings, vol 4. Taylor & Francis, London, pp 367–383Google Scholar
  7. Gaillard AWK, Kramer AF (2000) Theoretical and methodological issues in psychophysiological research. In: Backs RW, Boucsein W (eds) Engineering psychophysiology: issues and applications. Lawrence Erlbaum Associates, Mahwah, pp 31–58Google Scholar
  8. Gao J, Hu J, Tung W–W (2011) Complexity measures of brain wave dynamics. Cogn Neurodyn 5:171–182PubMedCrossRefGoogle Scholar
  9. Gevins A, Smith ME (1999) Detecting transient cognitive impairment with EEG pattern recognition methods. Aviat Space Environ Med 70:1018–1024PubMedGoogle Scholar
  10. Gevins A, Smith ME, Leong H (1998) Monitoring working memory load during computer-based tasks with EEG pattern recognition methods. Hum Factors 40:79–91PubMedCrossRefGoogle Scholar
  11. Greene KA, Bauer KW, Wilson GF et al (2000) Selection of psychophysiological features for classifying air traffic controller workload in neural networks. Smart Eng Syst Des 2:315–330Google Scholar
  12. Hammer JM, Small RL (1995) An intelligent interface in an associate system. In: Rouse WB (ed) Human/technology interaction in complex systems, vol 7. JAI Press, Greenwich, pp 1–44Google Scholar
  13. Hampel FR, Rousseeuw PJ, Stahel WA (1986) Robust statistics—the approach based on influence functions. Wiley, New YorkGoogle Scholar
  14. Hancock PA, Desmond PA (2001) Stress, workload and fatigue. Lawrence Erlbaum Associates, MahwahGoogle Scholar
  15. Hockey GRJ (1997) Compensatory control in the regulation of human performance under stress and high workload: a cognitive-energetical framework. Biol Psychol 45:73–93PubMedCrossRefGoogle Scholar
  16. Hockey GRJ (2003) Operator functional state: the assessment and prediction of human performance degradation in complex tasks. IOS Press, AmsterdamGoogle Scholar
  17. Hockey GRJ, Wastell DG, Sauer J (1998) Effects of sleep deprivation and user-interface on complex performance: a multilevel analysis of compensatory control. Hum Factors 40:233–253PubMedCrossRefGoogle Scholar
  18. Hockey GRJ, Nickel P, Roberts AC, Roberts MH (2009) Sensitivity of candidate markers of psychophysiological strain to cyclical changes in manual control load during simulated process control. Appl Ergon 40:1011–1018PubMedCrossRefGoogle Scholar
  19. Hoyt R (ed) (2010) Real-time physiological and psycho-physiological status monitoring. NATO RTO publication RTO-TR-HFM-132, NATO Research and Technology Organization, Neuilly sur Seine, July 2010Google Scholar
  20. Jorna PGAM (1993) Heart rate and workload variations in actual and simulated flight. Ergonomics 36(9):1043–1054PubMedCrossRefGoogle Scholar
  21. Kuriyagawa Y, Kageyama I (1999) A modeling of heart rate variability to estimate mental work load. In: Proceedings of IEEE international conference on systems, man, and cybernetics (SMC’99), vol 2, pp 294–299Google Scholar
  22. Lee S-Y, Song H-A, Amari S-I (2012) A new discriminant NMF algorithm and its application to the extraction of subtle emotional differences in speech. Cogn Neurodyn 6:525–535CrossRefGoogle Scholar
  23. Nickel P, Roberts AC, Hockey GRJ (2005) Assessment of high risk operator functional state markers in dynamical systems–preliminary results and implications. In: Proceedings of human factors and ergonomics society Europe chapter annual meeting, Turin, 26–28 Oct 2005Google Scholar
  24. Parasuraman R (2000) Designing automation for human use: empirical studies and quantitative models. Ergonomics 43:931–951PubMedCrossRefGoogle Scholar
  25. Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Trans SMC Part A 30(3):286–297Google Scholar
  26. Pockett S, Whalen S, McPhail AVH, Freeman WJ (2007) Topography, independent component analysis and dipole source analysis of movement related potentials. Cogn Neurodyn 1:327–340PubMedCrossRefGoogle Scholar
  27. Prinzel LJ, Freeman FG, Scerbo MW, Mikulka PJ, Pope AT (2000) A closed-loop system for examining psychophysiological measures for adaptive task allocation. Int J Aviat Psychol 10:393–410PubMedCrossRefGoogle Scholar
  28. Qin P–P, Zhang J-H (2012) LSSVM regressive model based analysis of operator functional state in a human-machine system (in Chinese). Space Med Med Eng 25(1):35–41Google Scholar
  29. Rouse WB (1976) Adaptive allocation of decision making responsibility between supervisor and computer. In: Sheridan TB, Johannsen G (eds) Monitoring behavior and supervisory control. Plenum Press, New York, pp 295–306CrossRefGoogle Scholar
  30. Rouse WB (1977) Human-computer interaction in multi-task situations. IEEE Trans SMC 7:384–392Google Scholar
  31. Rousseeuw PJ, Leroy A (1987) Robust regression and outlier detection. Wiley, New YorkCrossRefGoogle Scholar
  32. Russell CA, Wilson GF (1998) Air traffic controller functional state classification using neural networks. In: Proceedings of the artificial neural networks in engineering conference, vol 8, pp 649–654Google Scholar
  33. Scerbo MW, Freeman FG, Mikulka PJ (2000) A biocybernetic system for adaptive automation. In: Backs RW, Boucsein W (eds) Engineering psychophysiology: issues and applications. Lawrence Erlbaum Associates, Mahwah, pp 241–254Google Scholar
  34. Scerbo MW, Freeman FG, Mikulka PJ, Parasurmann R, Di Nocero F, Prinzel LJ III (2001) The efficacy of psychophysiological measures for implementing adaptive technology, NASA/TP-2001-211018, JuneGoogle Scholar
  35. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300CrossRefGoogle Scholar
  36. Suykens JAK, Lukas L, Vandewalle J (2000) Sparse least squares support vector machines classifiers. In: Proceedings of the European symposium on artificial neural networks (ESANN’2000), vol 4, Bruges, pp 37–42Google Scholar
  37. Suykens JAK, De Brabanter J, Lukas L, Vandewalle J (2002) Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48(1–4):85–105CrossRefGoogle Scholar
  38. Trejo LJ, Wheeler KR, Jorgensen CC et al (2003) Multi-modal neuroelectric interface development. IEEE Trans Neural Syst Rehabil Eng 11(2):199–204PubMedCrossRefGoogle Scholar
  39. Vapnik VN (1995) The nature of statistical learning theory. Springer, New YorkCrossRefGoogle Scholar
  40. Wang R, Zhang J, Zhang Y, Wang X (2012) Assessment of human operator functional state using a novel differential evolution optimization based adaptive fuzzy model. Biomed Signal Process Control 7:490–498CrossRefGoogle Scholar
  41. Werner G (2012) From brain states to mental phenomena via phase space transitions and renormalization group transformation: proposal of a theory. Cogn Neurodyn 6:199–202PubMedCrossRefGoogle Scholar
  42. Wilson GF (2001) In-flight psychophysiological monitoring. In: Fahrenberg F, Myrtek M (eds) Progress in ambulatory monitoring. Hogrefe and Huber Publishers, Seattle, pp 435–454Google Scholar
  43. Wilson GF (2002a) Psychophysiological test methods and procedures. In: Charlton SG, O’Brien TG (eds) Handbook of human factors testing and evaluation. Lawrence Erlbaum Associates, Inc, Mahwah, pp 157–180Google Scholar
  44. Wilson GF (2002b) An analysis of mental workload in pilots during flight using multiple psychophysiological measures. Int J Aviat Psychol 12:3–18CrossRefGoogle Scholar
  45. Wilson GF (2002c) Adaptive aiding implemented by psychophysiologically determined operator functional state. In: Proceedings of the NATO RTO-HFM symposium on the roles of humans in intelligent and automated systems, Warsaw, 7–9 Oct 2002; Also published in NATO RTO-MP-088, pp 18-1–18-8, Oct 2003Google Scholar
  46. Wilson GF, Eggemerier FT (1991) Physiological measures of workload in multi-task environments. In: Damos D (ed) Multiple-task performance, pp 329–360Google Scholar
  47. Wilson GF, Fisher F (1991) The use of cardiac and eye blink measures to determine flight segment in F4 crews. Aviat Space Environ Med 62:959–961PubMedGoogle Scholar
  48. Wilson GF, Fisher F (1995) Cognitive task classification based upon topographic EEG data. Biol Psychol 40:239–250PubMedCrossRefGoogle Scholar
  49. Wilson GF, Schlegel RE (eds) (2004) Operator functional state assessment, NATO RTO Publication RTO-TR-HFM-104, NATO Research and Technology Organization, Neuilly sur Seine, Feb 2004Google Scholar
  50. Wilson GF, Lambert JD, Russell CA (2000) Performance enhancement with real-time physiologically controlled adaptive aiding. In: Proceedings of the IEA 2000/HFES 2000 congress, vol 3, pp 61–64Google Scholar
  51. Zhang Q, Lee M (2012) Analyzing the dynamics of emotional scene sequence using recurrent neuro-fuzzy network. Cogn Neurodyn, published online: 17 Aug 2012. doi:  10.1007/s11571-012-9216-y
  52. Zhang J-H, Nassef A, Mahfouf M et al (2006) Modeling and analysis of HRV under physical and mental workloads. In: Proceedings of the 6th IFAC symposium on modeling and control in biomedical systems, Reims, pp 189–194, 20–22 Sept 2006Google Scholar
  53. Zhang J-H, Wang X-Y, Mahfouf M et al (2008) Use of heart rate variability analysis for quantitatively assessing operator’s mental workload. In: Proceedings of the international conference on biomedical engineering and informatics (BMEI), vol 1, Sanya, pp 668–672Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Jian-Hua Zhang
    • 1
    • 2
    Email author
  • Pan-Pan Qin
    • 1
  • Jörg Raisch
    • 3
    • 4
  • Ru-Bin Wang
    • 2
  1. 1.Department of AutomationEast China University of Science and TechnologyShanghaiPeople’s Republic of China
  2. 2.Institute of Cognitive NeurodynamicsEast China University of Science and TechnologyShanghaiPeople’s Republic of China
  3. 3.Control Systems GroupTechnische Universität BerlinBerlinGermany
  4. 4.Systems and Control Theory GroupMax Planck Institute for Dynamics of Complex Technical SystemsMagdeburgGermany

Personalised recommendations