Advertisement

A semi-supervised inattention detection method using biological signal

  • 350 Accesses

  • 2 Citations

Abstract

Recently, operations research methods have been utilized for biological data analysis as a huge amount of biological data becomes available. One of popular applications of the data analysis is inattention detection of operators in human–machine interaction systems using electroencephalography (EEG) signal. Most of the previous studies on the inattention detection employed supervised learning approaches, but their results have potential bias since they rely on imperfect assumptions for the acquisition of mental state labels, attention and inattention, due to the absence of the standardized measure for the mental states. Instead, we consider unsupervised learning approach, where no labeled data is required. In order to address the low performance of unsupervised learning approaches, attention duration for which an operator sustains his/her attention from the beginning of performing a task and relevance levels between four attributes of EEG signal and mental states are exploited. In this regard, we propose a semi-supervised inattention detection method (SID), in which attention duration and attributes-weights of EEG signal are respectively utilized as a small portion of labeled data for semi-supervised learning and weights for similarity calculation. Specifically, cumulative sum algorithm is used for the determination of the attention duration, and constrained attributes-weighting clustering algorithm is used for the estimation of attributes-weights. From experiments using real-world dataset, SID outperformed the compared methods, and it is expected that the adoption of SID will contribute to the enhancement of the operators’ safety.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Akin, M., Kurt, M. B., Sezgin, N., & Bayram, M. (2008). Estimating vigilance level by using EEG and EMG signals. Neural Computing and Applications, 17(3), 227–236.

  2. Alfaro, C., Cano-Montero, J., Gmez, J., Moguerza, J. M., & Ortega, F. (2013). A multi-stage method for content classification and opinion mining on weblog comments. Annals of Operations Research, 236(1), 1–17.

  3. Balasubramanian, V., Adalarasu, K., & Gupta, A. (2011). EEG based analysis of cognitive fatigue during simulated driving. International Journal of Industrial and Systems Engineering, 7(2), 135–149.

  4. Basu, S., Banerjee, A., & Mooney, R. J. (2001). Semi-supervised clustering by seeding. In Proceedings of the international conference on machine learning (pp. 27–34).

  5. Belyavin, A., & Wright, N. A. (1987). Changes in electrical activity of the brain with vigilance. Electroencephalography and Clinical Neurophysiology, 66(2), 137–144.

  6. Berka, C., Levendowski, D. J., Lumicao, M. N., Yau, A., Davis, G., Zivkovic, V. T., et al. (2007). EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviation, Space, and Environmental Medicine, 78(5), B231–B244.

  7. Caldwell, J., Hall, K., & Erickson, B. (2002). EEG data collected from helicopter pilots in flight are sufficiently sensitive to detect increased fatigue from sleep deprivation. The International Journal of Aviation Psychology, 12(1), 19–32.

  8. Chan, E. Y., Ching, W. K., Ng, M. K., & Huang, J. Z. (2004). An optimization algorithm for clustering using weighted dissimilarity measures. Pattern Recognition, 37(5), 943–952.

  9. Chaovalitwongse, W. A., Prokopyev, O. A., & Pardalos, P. M. (2006). Electroencephalogram (EEG) time series classification: Applications in epilepsy. Annals of Operations Research, 148(1), 227–250.

  10. Choi, Y., Kwon, N., Lee, S., Shin, Y., Ryo, C. Y., & Park, J., et al. (2014). Hypovigilance detection for ucav operators based on a hidden Markov model. Computational and Mathematical Methods in Medicine, 2014, 1–13.

  11. Choi, Y., Kwon, N., Yoon, J., Jeon, S., Paeng, B. H., & Park, J., et al. (2013a). A semi-supervised attributes-weighting clustering method for inattention prediction in human–machine interaction systems. In Proceedings of the Asia pacific industrial engineering and management systems conference.

  12. Choi, Y., Kwon, N., Yoon, J., Park, J., Lee, H., Kim, Y., et al. (2013b). A BCI based ground control framework for attention maintenance of UAV operators. Entrue Journal of Information Technology, 12(1), 101–115.

  13. Craig, A., Tran, Y., Wijesuriya, N., & Nguyen, H. (2012). Regional brain wave activity changes associated with fatigue. Psychophysiology, 49(4), 574–582.

  14. Dempster, A. P,. Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society Series B (Methodological), 39(1), 1–38.

  15. Doroshenkov, L. G., Konyshev, V. A., & Selishchev, S. V. (2007). Classification of human sleep stages based on EEG processing using hidden Markov models. Biomedical Engineering, 41(1), 25–28.

  16. Egner, T., & Gruzelier, J. H. (2004). EEG biofeedback of low beta band components: Frequency-specific effects on variables of attention and event-related brain potentials. Clinical Neurophysiology, 115(1), 131–139.

  17. Elkan, C., & Noto, K. (2008). Learning classifiers from only positive and unlabeled data. In Proceedings of the ACM international conference on knowledge discovery and data mining (pp. 213–220).

  18. Endsley, M. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 32–64.

  19. Friedrichs, F., & Yang, B. (2010). Camera-based drowsiness reference for driver state classification under real driving conditions. In Proceedings of the IEEE intelligent vehicles symposium (pp. 101–106).

  20. Frigui, H., & Nasraoui, O. (2004). Unsupervised learning of prototypes and attribute weights. Pattern Recognition, 37(3), 567–581.

  21. Harmony, T., Fernndez, T., Silva, J., Bernal, J., Daz-Comas, L., Reyes, A., et al. (1996). EEG delta activity: An indicator of attention to internal processing during performance of mental tasks. International Journal of Psychophysiology, 24(12), 161–171.

  22. Haufe, S., Treder, M. S., Gugler, M. F., Sagebaum, M., Curio, G., & Blankertz, B. (2011). EEG potentials predict upcoming emergency brakings during simulated driving. Journal of Neural Engineering, 8(5). doi:10.1088/1741-2560/8/5/056001.

  23. Huang, R. S., Kuo, C. J., Tsai, L. L., & Chen, O. T. C. (1996). EEG pattern recognition-arousal states detection and classification. In Proceedings of the IEEE international conference on neural networks (pp. 641–646).

  24. Jap, B. T., Lal, S., & Fischer, P. (2011). Comparing combinations of EEG activity in train drivers during monotonous driving. Expert Systems with Applications, 38(1), 996–1003.

  25. Ji, Q., & Yang, X. (2001). Real time visual cues extraction for monitoring driver vigilance. Lecture Notes in Computer Science, 2095, 107–124.

  26. Kaida, K., Takahashi, M., Akerstedt, T., Nakata, A., Otsuka, Y., Haratani, T., et al. (2006). Validation of the karolinska sleepiness scale against performance and EEG variables. Clinical Neurophysiology, 117(7), 1574–1581.

  27. Kim, K., Choi, Y., & Park, J. (2014a). A conditional feature utilization approach to itemset retrieval in online shopping services. Journal of Electronic Commerce Research, 15(4), 317.

  28. Kim, K., Chung, B. S., Choi, Y., Lee, S., Jung, J. Y., & Park, J. (2014b). Language independent semantic kernels for short-text classification. Expert Systems with Applications, 41(2), 735–743.

  29. Klimesch, W., Doppelmayr, M., Russegger, H., Pachinger, T., & Schwaiger, J. (1998). Induced alpha band power changes in the human EEG and attention. Neuroscience Letters, 244(2), 73–76.

  30. Lal, S. K. L., & Craig, A. (2002). Driver fatigue: Electroencephalography and psychological assessment. Psychophysiology, 39(3), 313–321.

  31. Larue, G., Rakotonirainy, A., & Pettitt, A. (2010). Real-time performance modelling of a sustained attention to response task. Ergonomics, 53(10), 1205–1216.

  32. Lee, S., Ha, J., Na, O., & Na, S. (2003). The cusum test for parameter change in time series models. Scandinavian Journal of Statistics, 30(4), 781–796.

  33. Liu, J., Zhang, C., & Zheng, C. (2010). EEG-based estimation of mental fatigue by using KPCAHMM and complexity parameters. Biomedical Signal Processing and Control, 5(2), 124–130.

  34. Lukasiak, P., Baewicz, J., & Miostan, M. (2009). Some operations research methods for analyzing protein sequences and structures. Annals of Operations Research, 175(1), 9–35.

  35. Mackworth, N. H. (1948). The breakdown of vigilance durning prolonged visual search. Quarterly Journal of Experimental Psychology, 1(1), 6–21.

  36. Mishchenko, Y., & Kaya, M. (2015). Detecting the attention state of an operator in continuous attention task using EEG-based brain-computer interface. In Proceedings of the signal processing and communications applications conference (pp. 232–235).

  37. Nilsson, T., Nelson, T., & Carlson, D. (1997). Development of fatigue symptoms during simulated driving. Accident Analysis and Prevention, 29(4), 479–488.

  38. Nuechterlein, K., Parasuraman, R., & Jiang, Q. (1983). Visual sustained attention: Image degradation produces rapid sensitivity decrement over time. Science, 220(4594), 327–329.

  39. Pattyn, N., Neyt, X., Henderickx, D., & Soetens, E. (2008). Psychophysiological investigation of vigilance decrement: Boredom or cognitive fatigue? Physiology and Behavior, 93(1), 369–378.

  40. Peiris, M., Jones, R., Davidson, P., Carroll, G., & Bones, P. (2006). Frequent lapses of responsiveness during an extended visuomotor tracking task in nonsleepdeprived subjects. Journal of Sleep Research, 15(3), 291–300.

  41. Plotnikov, A., Stakheika, N., De Gloria, A., Schatten, C., Bellotti, F., & Berta, R., et al. (2012). Exploiting real-time eeg analysis for assessing flow in games. In Proceedings of the IEEE international conference on advanced learning technologies (pp. 688–689).

  42. Punsawad, Y., Aempedchr, S., Wongsawat, Y., & Panichkun, M. (2011). Weighted-frequency index for EEG-based mental fatigue alarm system. International Journal of Applied Biomedical Engineering, 4(1), 36–41.

  43. Putman, P., Verkuil, B., Arias-Garcia, E., Pantazi, I., & van Schie, C. (2014). EEG theta/beta ratio as a potential biomarker for attentional control and resilience against deleterious effects of stress on attention. Cognitive, Affective, and Behavioral Neuroscience, 14(2), 782–791.

  44. Sahin, H., & Feng, Q. (2009). A mathematical framework for sequential passenger and baggage screening to enhance aviation security. Computers and Industrial Engineering, 57(1), 148–155.

  45. Sauter, C., Danker-Hopfe, H., Loretz, E., Zeitlhofer, J., Geisler, P., & Popp, R. (2013). The assessment of vigilance: Normative data on the siesta sustained attention test. Sleep Medicine, 14(6), 542–548.

  46. Shi, L. C., & Lu, B. L. (2008). Dynamic clustering for vigilance analysis based on EEG. In Proceedings of the international conference of the IEEE engineering in medicine and biology society (pp. 54–57).

  47. Shi, L. C., Yu, H., & Lu, B. L. (2007). Semi-supervised clustering for vigilance analysis based on EEG. In Proceedings of the international joint conference on neural networks (pp. 1518–1523).

  48. Sun, H., Bi, L., Lu, X., Fan, B., & Guo, Y. (2015a). Vigilance analysis based on EEG band power using support vector machine. In Proceedings of the international congress on image and signal processing (pp. 1090–1094).

  49. Sun, H., Yang, Y., Sourina, O., Huang, G. B., Klanner, F., & Denk, C., et al. (2015b). Estimating vigilance from EEG using manifold clustering guided by instantaneous lapse rate. In Proceedings of the international conference on information, communications and signal processing (pp. 1–5).

  50. Thiffault, P., & Bergeron, J. (2003). Monotony of road environment and driver fatigue: A simulator study. Accident Analysis and Prevention, 35(3), 381–391.

  51. Ting, P. H., Hwang, J. R., Doong, J. L., & Jeng, M. C. (2008). Driver fatigue and highway driving: A simulator study. Physiology and Behavior, 94(3), 448–453.

  52. Upadhyay, R., Padhy, P. K., & Kankar, P. K. (2016). Application of s-transform for automated detection of vigilance level using EEG signals. Journal of Biological Systems, 24(1), 1–27.

  53. Uusberg, A., Uibo, H., Kreegipuu, K., & Allik, J. (2013). EEG alpha and cortical inhibition in affective attention. International Journal of Psychophysiology, 89(1), 26–36.

  54. Vijay, J., & Subhashini, J. (2013). An efficient brain tumor detection methodology using K-means clustering algoriftnn. In Proceedings of the international conference on communications and signal processing (pp. 653–657).

  55. Wang, L., & Pan, C. (2014). Robust level set image segmentation via a local correntropy-based K-means clustering. Pattern Recognition, 47(5), 1917–1925.

  56. West, J. V., & Lane-Cummings, K. (2007). Microsoft flight simulator X for pilots: Real world training. New York: Wiley.

  57. Yeo, M. V. M., Li, X., Shen, K., & Wilder-Smith, E. P. V. (2009). Can svm be used for automatic eeg detection of drowsiness during car driving? Safety Science, 47(1), 115–124.

  58. Yusoff, I. A., Isa, N. A. M., & Hasikin, K. (2013). Automated two-dimensional K-means clustering algorithm for unsupervised image segmentation. Computers and Electrical Engineering, 39(3), 907–917.

  59. Zhang, B., Wang, J., & Fuhlbrigge, T. (2010). A review of the commercial brain-computer interface technology from perspective of industrial robotics. In Proceedings of the IEEE international conference on automation and logistics (pp. 379–384).

  60. Zhang, Z., Luo, D., Rasim, Y., Li, Y., Meng, G., Xu, J., et al. (2016). A vehicle active safety model: Vehicle speed control based on driver vigilance detection using wearable EEG and sparse representation. Sensors, 16(2), 242.

  61. Zhao, C., Zhao, M., Liu, J., & Zheng, C. (2012). Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. Accident Analysis and Prevention, 45, 83–90.

Download references

Acknowledgements

This work was supported by the BK21 Plus Program (Center for Sustainable and Innovative Industrial Systems, Department of Industrial Engineering, Seoul National University) funded by the Ministry of Education, Korea (No. 21A20130012638) and by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT and future Planning (No. 2013R1A2A2A03013947).

Author information

Correspondence to Dongmin Shin.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Choi, Y., Park, J. & Shin, D. A semi-supervised inattention detection method using biological signal. Ann Oper Res 258, 59–78 (2017). https://doi.org/10.1007/s10479-017-2406-6

Download citation

Keywords

  • Inattention detection
  • Biological signal
  • Semi-supervised method
  • Cumulative sum algorithm
  • Constrained attributes-weighting clustering algorithm