Skip to main content
SpringerLink
Log in

Extracting and supplementing method for EEG signal in manufacturing workshop based on deep learning of time–frequency correlation

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

The quality of EEG signals is extremely important for brain–computer interface systems, especially in complex environments such as manufacturing floors. At present, there are few studies on the recovery methods of damaged EEG signals. Therefore, a TFCMI–CNN–LSTM hybrid model of time–frequency correlation analysis combined with deep learning is proposed to predict the missing data of EEG signals. We designed a brain fatigue experiment in a noisy environment to simulate the actual work situation of workers in the workshop, and verified the performance of the prediction model combined with the public EEG dataset BCI Competition IV 2a. The prediction results of the data set BCI Competition IV 2a show that the average RMSE between the prediction results of the TFCMI–CNN–LSTM model and the true value is 3.512, the MAE is 2.787, the Spearman Rank is 0.828, which can effectively restore abnormal EEG signals. This paper studies the importance of the CNN module in the model. The results show that the CNN module can greatly reduce the time spent in model training, and is more suitable for real-time EEG signal acquisition systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Chiwewe, T. M., Mbuya, C. F., & Hancke, G. P. (2015). Using cognitive radio for interference-resistant industrial wireless sensor networks: An overview. IEEE Transactions on Industrial Informatics, 11(6), 1466–1481. https://doi.org/10.1109/TII.2015.2491267

    Article  Google Scholar 

  • Cho, K., Merriecboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078

  • Clevert, D. A., Unterthiner, T., & Hochreiter, S. (2015). Fast and accurate deep network learning by exponential linear units (elus). arXiv:1511.07289

  • Colditz, P. B., Burke, C. J., & Celka, P. (2001). Digital processing of EEG signals. IEEE Engineering in Medicine and Biology Magazine, 20(5), 21–22. https://doi.org/10.1109/51.956815

    Article  Google Scholar 

  • Gao, Y., Zhao, P., Li, G., & Li, H. (2021). Seismic noise attenuation by signal reconstruction: An unsupervised machine learning approach. Geophysical Prospecting, 69(5), 984–1002. https://doi.org/10.1111/1365-2478.13070

    Article  Google Scholar 

  • Gholami, A., Bonakdari, H., Zaji, A. H., Fenjan, S. A., & Akhtari, A. A. (2018). New radial basis function network method based on decision trees to predict flow variables in a curved channel. Neural Computing and Applications, 30(9), 2771–2785. https://doi.org/10.1007/s00521-017-2875-1

    Article  Google Scholar 

  • Gianotti, L. R., Lobmaier, J. S., Calluso, C., Dahinden, F. M., & Knoch, D. (2018). Theta resting EEG in TPJ/pSTS is associated with individual differences in the feeling of being looked at. Social Cognitive and Affective Neuroscience, 13(2), 216–223. https://doi.org/10.1093/scan/nsx143

    Article  Google Scholar 

  • Graves, A. (2013). Generating sequences with recurrent neural networks. arXiv:1308.0850

  • Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., & Schmidhuber, J. (2009). A novel connectionist system for unconstrained handwriting recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5), 855–868. https://doi.org/10.1109/TPAMI.2008.137

    Article  Google Scholar 

  • Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R., & Schmidhuber, J. (2016). LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2222–2232. https://doi.org/10.1109/TNNLS.2016.2582924

    Article  Google Scholar 

  • Hartmann, K. G., Schirrmeister, R. T., & Ball, T. (2018). EEG-GAN: Generative adversarial networks for electroencephalograhic (EEG) brain signals. arXiv:1806.01875

  • Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction. Springer.

    Book  Google Scholar 

  • Hersche, M., Rellstab, T., Schiavone, P. D., Cavigelli, L., Benini, L., & Rahimi, A. (2018). Fast and accurate multiclass inference for MI-BCIs using large multiscale temporal and spectral features. In 2018 26th European Signal Processing Conference (EUSIPCO). 3–7 Sept. 2018. pp. 1690–1694

  • Huang, S., Cai, N., Pacheco, P. P., Narrandes, S., Wang, Y., & Xu, W. (2018). Applications of support vector machine (SVM) learning in cancer genomics. Cancer Genomics-Proteomics, 15(1), 41–51. https://doi.org/10.21873/cgp.20063

    Article  Google Scholar 

  • Jas, M., Engemann, D. A., Bekhti, Y., Raimondo, F., & Gramfort, A. (2017). Autoreject: Automated artifact rejection for MEG and EEG data. Neuroimage, 159, 417–429. https://doi.org/10.1016/j.neuroimage.2017.06.030

    Article  Google Scholar 

  • Jozefowicz, R., Zaremba, W., & Sutskever, I. (2015). An empirical exploration of recurrent network architectures. In International conference on machine learning. PMLR, pp. 2342–2350.

  • Kouziokas, G. N. (2020). A new W-SVM kernel combining PSO-neural network transformed vector and Bayesian optimized SVM in GDP forecasting. Engineering Applications of Artificial Intelligence, 92, 103650. https://doi.org/10.1016/j.engappai.2020.103650

    Article  Google Scholar 

  • Liu, X., Wei, X., Guo, L., Liu, Y., Song, Q., & Jamalipour, A. (2019). Turning the signal interference into benefits: Towards indoor self-powered visible light communication for IoT devices in industrial radio-hostile environments. IEEE Access: Practical Innovations, Open Solutions, 7, 24978–24989. https://doi.org/10.1109/ACCESS.2019.2900696

    Article  Google Scholar 

  • Lotte, F., & Guan, C. (2011). Regularizing common spatial patterns to improve BCI designs: Unified theory and new algorithms. IEEE Transactions on Biomedical Engineering, 58(2), 355–362. https://doi.org/10.1109/TBME.2010.2082539

    Article  Google Scholar 

  • Lu, C. F., Teng, S., Hung, C. I., Tseng, P. J., Lin, L. T., Lee, P. L., et al. (2011). Reorganization of functional connectivity during the motor task using EEG time-frequency cross mutual information analysis. Clinical Neurophysiology, 122(8), 1569–1579. https://doi.org/10.1016/j.clinph.2011.01.050

    Article  Google Scholar 

  • Marques, A. G., Segarra, S., Leus, G., & Ribeiro, A. (2015). Sampling of graph signals with successive local aggregations. IEEE Transactions on Signal Processing, 64(7), 1832–1843. https://doi.org/10.1109/TSP.2015.2507546

    Article  Google Scholar 

  • Mognon, A., Jovicich, J., Bruzzone, L., & Buiatti, M. (2011). ADJUST: An automatic EEG artifact detector based on the joint use of spatial and temporal features. Psychophysiology, 48(2), 229–240. https://doi.org/10.1111/j.1469-8986.2010.01061.x

    Article  Google Scholar 

  • Nolan, H., Whelan, R., & Reilly, R. B. (2010). FASTER: Fully automated statistical thresholding for EEG artifact rejection. Journal of Neuroscience Methods, 192(1), 152–162. https://doi.org/10.1016/j.jneumeth.2010.07.015

    Article  Google Scholar 

  • Pfurtscheller, G., & Silva, F. H. L. (1999). Event related EEG /MEG synchronization and desynchronization: Basic principles. Clinical Neurophysiology, 110(11), 1842–1857. https://doi.org/10.1016/S1388-2457(99)00141-8

  • Sakai, T., Shoji, T., Yoshida, N., Fukumori, K., Tanaka, Y., & Tanaka, T. (2020). SCALPNET: Detection of spatiotemporal abnormal intervals in epileptic EEG using convolutional neural networks. In ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 1244–1248.

  • Sayed, A., & Ibrahim, A. (2018). Recent developments in systematic sampling: A review. Journal of Statistical Theory and Practice, 12(2), 290–310. https://doi.org/10.1080/15598608.2017.1353456

    Article  Google Scholar 

  • Silvestre-Blanes, J., Almeida, L., Marau, R., & Pedreiras, P. (2010). Online QoS management for multimedia real-time transmission in industrial networks. IEEE Transactions on Industrial Electronics, 58(3), 1061–1071. https://doi.org/10.1109/TIE.2010.2049711

    Article  Google Scholar 

  • Subha, D. P., Joseph, P. K., Acharya, R., & Lim, C. M. (2010). EEG signal analysis: A survey. Journal of medical systems, 34(2), 195–212. https://doi.org/10.1007/s10916-008-9231-z

    Article  Google Scholar 

  • Sun, Y., Wang, X., & Tang, X. (2014). Deep learning face representation from predicting 10,000 classes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1891–1898.

  • Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Advances in neural information processing systems, p. 27.

  • Tangermann, M., Müller, K. R., Aertsen, A., Birbaumer, N., Braun, C., Brunner, C., et al. (2012). Review of the BCI competition IV. Frontiers in Neuroscience, 6, 55. https://doi.org/10.3389/fnins.2012.00055

    Article  Google Scholar 

  • Wan, Z., Yang, R., Huang, M., Zeng, N., & Liu, X. (2021). A review on transfer learning in EEG signal analysis. Neurocomputing, 421, 1–14. https://doi.org/10.1016/j.neucom.2020.09.017

    Article  Google Scholar 

  • Wang, Z., Horng, G., Hsu, T., Aripriharta, A., & Jong, G. (2020). Heart sound signal recovery based on time series signal prediction using a recurrent neural network in the long short-term memory model. The Journal of Supercomputing, 76(11), 8373–8390. https://doi.org/10.1007/s11227-019-03096-x

    Article  Google Scholar 

  • Xue, H., Huynh, D. Q., & Reynolds, M. (2018). SS-LSTM: A hierarchical LSTM model for pedestrian trajectory prediction. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp. 1186–1194.

  • Yan, R., Liao, J., Yang, J., Sun, W., Nong, M., & Li, F. (2021). Multi-hour and multi-site air quality index forecasting in Beijing using CNN, LSTM, CNN-LSTM, and spatiotemporal clustering. Expert Systems with Applications, 169, 114513. https://doi.org/10.1016/j.eswa.2020.114513

    Article  Google Scholar 

  • Yasoda, K., Ponmagal, R., Bhuvaneshwari, K., & Venkatachalam, K. (2020). Automatic detection and classification of EEG artifacts using fuzzy kernel SVM and wavelet ICA (WICA). Soft Computing, 24(21), 16011–16019. https://doi.org/10.1007/s00500-020-04920-w

    Article  Google Scholar 

  • Zeinolabedini, M., & Najafzadeh, M. (2019). Comparative study of different wavelet-based neural network models to predict sewage sludge quantity in wastewater treatment plant. Environmental Monitoring and Assessment, 191(3), 163. https://doi.org/10.1007/s10661-019-7196-7

    Article  Google Scholar 

  • Zhang, K., & Luo, Y. (2020). Effects of worker fatigue on assembly line balancing. In 2020 IEEE 11th international conference on software engineering and service science (ICSESS). IEEE, pp. 254–257.

  • Zhang, W., Wei, Z., Wang, B., & Han, X. (2016). Measuring mixing patterns in complex networks by Spearman rank correlation coefficient. Physica A: Statistical Mechanics and its Applications, 451, 440–450. https://doi.org/10.1016/j.physa.2016.01.056

    Article  Google Scholar 

  • Zhang, Z. (2018). Improved adam optimizer for deep neural networks. In 2018 IEEE/ACM 26th international symposium on quality of service (IWQoS). IEEE, pp. 1–2.

Download references

Acknowledgements

This research was funded by the National Natural Science Foundation (NSFC) of China under Grant No. 51775325, the Young Eastern Scholars Program of Shanghai under Grant No. QD2016033, and the Hong Kong Scholars Program of China under Grant No. XJ2013015. The authors would like to thank the anonymous reviewers for their constructive comments that will help us to improve the quality of this manuscript.

Author information

Authors and Affiliations

  1. School of Mechatronic Engineering and Automation, Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, Shanghai University, Shanghai, 200444, China

    Bin Ren & Yunjie Pan

Authors
  1. Bin Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunjie Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Ren.

Ethics declarations

Conflict of interest

The authors declare that there are no conflict of interests, we do not have any possible conflicts of interest. We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Tables 8, 9, and 10.

Table 8 The mathematical symbols with TFCMI and their meanings
Full size table
Table 9 The mathematical symbols with CNN–LSTM and their meanings
Full size table
Table 10 The mathematical symbols with performance evaluation indicators and their meanings
Full size table

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, B., Pan, Y. Extracting and supplementing method for EEG signal in manufacturing workshop based on deep learning of time–frequency correlation. J Intell Manuf 34, 3179–3196 (2023). https://doi.org/10.1007/s10845-022-01997-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-01997-y

Keywords

Search

Navigation