Abstract
The negative effects of stress on well-being demonstrate the need for real-time detection. The increasing prevalence of wearables with AI capabilities to continually monitor vital signs like heart rate and blood pressure highlights their growing value in promptly identifying stress. This paper presents a new approach for real-time stress detection employing an LSTM-based deep learning model, a technique with promising outcomes for time-series data analysis. Our model is trained on raw PPG signals from the WESAD dataset covering diverse stress scenarios. By learning patterns and fluctuations over time, it can effectively distinguish stressed and non-stressed states. The training uses only the raw PPG signals, segments them into windows, and creates labeled data for supervised learning. To enable real-time detection, we explore deploying our trained model on STM32H7xx microcontrollers equipped with a Cortex-M7 core offering low-power and hardware acceleration. We implement the LSTM model leveraging their capabilities for efficient inference. This implementation process involves optimizing the model and converting it into a format compatible with the microcontrollers. Within this study, we employ key TensorFlow toolkit optimization methods, including quantization-aware training (QAT), pruning, prune-preserving quantization-aware training (PQAT), and post-training quantization (PTQ), along with the TensorFlow Lite (TFL) toolkit, to evaluate and compare the outcomes obtained from applying these methods to the baseline model. Our goal is to select the most effective approach for the processor, enabling real-time detection. Through the utilization of these techniques, our objective is to reduce the size of the model and the necessary processing resources, such as RAM size, while ensuring that a high level of accuracy is maintained. Our results show the capability of the optimized LSTM model to accurately detect stress from raw PPG data on resource-constrained, low-power STM32H7xx MCUs. The final optimized model requires only 170 Kbytes of RAM, a nearly 12 times reduction in size, while still achieving a high accuracy of 87.76% when performing inference on the microcontroller.
Similar content being viewed by others
References
Cohen, S.; Janicki-Deverts, D.; Miller, G.E.: Psychological stress and disease. JAMA 298(14), 1685 (2007). https://doi.org/10.1001/jama.298.14.1685
McEwen, B.S.: Physiology and neurobiology of stress and adaptation: central role of the brain. Physiol. Rev. 87(3), 873–904 (2007). https://doi.org/10.1152/physrev.00041.2006
Vildjiounaite, E.; Huotari; V., Kallio; J., Kyllönen; V., Mäkelä; SM., Gimel’farb G.: “Detection of prolonged stress in smart office,” 1253–1261 (2019). https://doi.org/10.1007/978-3-030-01177-2_90.
Derevenco, P.; Popescu, G.; Deliu, N.: Stress assessment by means of questionnaires. Rom J Physiol Physiol Sci 37(1–4), 39–49 (2000)
Aristizabal, S., et al.: The feasibility of wearable and self-report stress detection measures in a semi-controlled lab environment. IEEE Access 9, 102053–102068 (2021). https://doi.org/10.1109/ACCESS.2021.3097038
Attar, E.T.; Balasubramanian, V.; Subasi, E.; Kaya, M.: Stress analysis based on simultaneous heart rate variability and EEG monitoring. IEEE J. Transl. Eng. Heal. Med. 9, 1–7 (2021). https://doi.org/10.1109/JTEHM.2021.3106803
Praveena, G.; Mathana, J.M.: “Review on stress detection and management techniques using nano EEG sensors,” in 2022 IEEE International Conference on Nanoelectronics, Nanophotonics, Nanomaterials, Nanobioscience & Nanotechnology (5NANO), IEEE, Apr 1–7 (2022). doi: https://doi.org/10.1109/5NANO53044.2022.9828889.
Massaro, S.; Pecchia, L.: Heart rate variability (HRV) analysis: a methodology for organizational neuroscience. Organ. Res. Methods 22(1), 354–393 (2019). https://doi.org/10.1177/1094428116681072
Ishaque, S.; Khan, N.; Krishnan, S.: Physiological signal analysis and stress classification from VR simulations using decision tree methods. Bioengineering 10(7), 766 (2023). https://doi.org/10.3390/bioengineering10070766
Pavel et al., A.M.: “Heart rate variability analysis for the prediction of EEG grade in infants with hypoxic ischaemic encephalopathy within the first 12 h of birth,” Front. Pediatr. (2023). https://doi.org/10.3389/fped.2022.1016211.
Alugubelli, N.; Abuissa, H.; Roka, A.: Wearable devices for remote monitoring of heart rate and heart rate variability—what we know and what is coming. Sensors 22(22), 8903 (2022). https://doi.org/10.3390/s22228903
Hasanpoor, Y.; Motaman, K.; Tarvirdizadeh, B.; Alipour, K.; Ghamari, M.: “Stress detection using ppg signal and combined deep CNN-MLP network,” in 2022 29th National and 7th International Iranian Conference on Biomedical Engineering, ICBME 2022, Institute of Electrical and Electronics Engineers Inc., 223–228 (2022). https://doi.org/10.1109/ICBME57741.2022.10052957.
Hasanpoor, Y.; Tarvirdizadeh, B.; Alipour, K.; Ghamari, M.: “Stress assessment with convolutional neural network using PPG signals,” in 2022 10th RSI International Conference on Robotics and Mechatronics (ICRoM), IEEE, Nov 472–477 (2022). https://doi.org/10.1109/ICRoM57054.2022.10025071.
Gedam, S.; Paul, S.: A review on mental stress detection using wearable sensors and machine learning techniques. IEEE Access 9, 84045–84066 (2021). https://doi.org/10.1109/ACCESS.2021.3085502
E. Smets et al.,: “Comparison of machine learning techniques for psychophysiological stress detection,” 13–22 (2016). https://doi.org/10.1007/978-3-319-32270-4_2.
Awasthi, K.; Nanda, P.; Suma, K.V.: “Performance analysis of machine learning techniques for classification of stress levels using PPG signals,” in 2020 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT), IEEE, Jul. 1–6 (2020). https://doi.org/10.1109/CONECCT50063.2020.9198481.
Golgouneh, A.; Tarvirdizadeh, B.: Fabrication of a portable device for stress monitoring using wearable sensors and soft computing algorithms. Neural Comput. Appl. 32(11), 7515–7537 (2020). https://doi.org/10.1007/s00521-019-04278-7
Keshan, N.; Parimi, P.V.; Bichindaritz, I.: “Machine learning for stress detection from ECG signals in automobile drivers,” in 2015 IEEE International Conference on Big Data (Big Data), IEEE, Oct 2661–2669 (2015). https://doi.org/10.1109/BigData.2015.7364066.
Saeed, S.M.U.; Anwar, S.M.; Khalid, H.; Majid, M.; Bagci, U.: EEG based classification of long-term stress using psychological labeling. Sensors 20(7), 1886 (2020). https://doi.org/10.3390/s20071886
Castaldo, R.; Xu, W.; Melillo, P.; Pecchia, L.; Santamaria, L.; James, C.: “Detection of mental stress due to oral academic examination via ultra-short-term HRV analysis,” in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, Aug, 3805–3808 (2016). https://doi.org/10.1109/EMBC.2016.7591557.
Rodríguez-Arce, J.; Lara-Flores, L.; Portillo-Rodríguez, O.; Martínez-Méndez, R.: Towards an anxiety and stress recognition system for academic environments based on physiological features. Comput. Methods Programs Biomed. 190, 105408 (2020). https://doi.org/10.1016/j.cmpb.2020.105408
Setiawan, R.; Budiman, F.; Basori, W.I.: “Stress diagnostic system and digital medical record based on internet of things,” in 2019 International Seminar on Intelligent Technology and Its Applications (ISITIA), IEEE, Aug. 348–353 (2019). https://doi.org/10.1109/ISITIA.2019.8937273.
Bobade P.; Vani, M.: “Stress detection with machine learning and deep learning using multimodal physiological data,” in 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA), IEEE, Jul. 51–57 (2020). https://doi.org/10.1109/ICIRCA48905.2020.9183244.
Zagajewski, B.; Kluczek, M.; Raczko, E.; Njegovec, A.; Dabija, A.; Kycko, M.: Comparison of random forest, support vector machines, and neural networks for post-disaster forest species mapping of the Krkonoše/Karkonosze transboundary biosphere reserve. Remote Sens. 13(13), 2581 (2021). https://doi.org/10.3390/rs13132581
Ursutiu, D.; Nascov, V.; Samoila, C.; Moga, M.: “Microcontroller technologies in low power applications,” in 2012 15th International Conference on Interactive Collaborative Learning (ICL), IEEE, Sep 1–5 (2012). doi: https://doi.org/10.1109/ICL.2012.6402096.
Mou, L., et al.: Driver stress detection via multimodal fusion using attention-based CNN-LSTM. Expert Syst. Appl. 173, 114693 (2021). https://doi.org/10.1016/j.eswa.2021.114693
Han, H.; Byun, K; Kang, H.G.: “A deep learning-based stress detection algorithm with speech signal,” in Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, New York, NY, USA: ACM, Oct, 11–15 (2018). https://doi.org/10.1145/3264869.3264875.
Motaman, K.; Alipour, K.; Tarvirdizadeh, B.; Ghamari, M.: “A stress detection model based on LSTM network using solely raw PPG signals,” in 2022 10th RSI International Conference on Robotics and Mechatronics (ICRoM), IEEE, Nov 485–490 (2022). https://doi.org/10.1109/ICRoM57054.2022.10025256.
Schmidt, P.; Reiss, A.; Duerichen, R.; Marberger, C.; Van Laerhoven, K.: “Introducing WESAD, a multimodal dataset for wearable stress and affect detection,” in Proceedings of the 20th ACM International Conference on Multimodal Interaction, New York, NY, USA: ACM, Oct. 400–408 (2018). https://doi.org/10.1145/3242969.3242985.
“WESAD: Multimodal dataset for wearable stress and affect Detection|Ubiquitous Computing.” https://www.eti.uni-siegen.de/ubicomp/home/datasets/icmi18/index.html.en?lang=en (accessed 03 Mar 2024).
Maqsood, S.; Xu, S.; Springer, M.; Mohawesh, R.: A benchmark study of machine learning for analysis of signal feature extraction techniques for blood pressure estimation using photoplethysmography (PPG). IEEE Access 9, 138817–138833 (2021). https://doi.org/10.1109/ACCESS.2021.3117969
Jacob et al., B.: “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference,” Dec. (2017). [Online]. Available: http://arxiv.org/abs/1712.05877
Han, T.; Li, D.; Liu, J.; Tian, L.; Shan, Y.: “Improving low-precision network quantization via bin regularization,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, Oct. 5241–5250 (2021). https://doi.org/10.1109/ICCV48922.2021.00521.
Li, L.; Zhu, J.; Sun, M-T.: “Deep learning based method for pruning deep neural networks,” in 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), IEEE, Jul 312–317 (2019) https://doi.org/10.1109/ICMEW.2019.00-68.
Zhu M. Gupta, S.: “To prune, or not to prune: exploring the efficacy of pruning for model compression,” Oct. (2017). [Online]. Available: http://arxiv.org/abs/1710.01878
Yu, F.; Cui, L.; Wang, P.; Han, C.; Huang, R.; Huang, X.: EasiEdge: a novel global deep neural networks pruning method for efficient edge computing. IEEE Internet Things J. 8(3), 1259–1271 (2021). https://doi.org/10.1109/JIOT.2020.3034925
Predić, B.; Vukić, U.; Saračević, M.; Karabašević, D.; Stanujkić, D.: The possibility of combining and implementing deep neural network compression methods. Axioms 11(5), 229 (2022). https://doi.org/10.3390/axioms11050229
Bai, H.; Hou, L.; Shang, L.; Jiang, X.; King, I.; Lyu M.R.: “Towards efficient post-training quantization of pre-trained language models,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., Curran Associates, Inc.,1405–1418. (2022). [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/096347b4efc264ae7f07742fea34af1f-Paper-Conference.pdf
Kamruzzaman, J.; Begg, R.; Sarker, R.: “Overview of Artificial Neural Networks and their Applications in Healthcare,” in Neural Networks in Healthcare, IGI Global, 1–19. (2006). https://doi.org/10.4018/978-1-59140-848-2.ch001.
Immonen, R.; Hämäläinen, T.: Tiny machine learning for resource-constrained microcontrollers. J. Sensors 2022, 1–11 (2022). https://doi.org/10.1155/2022/7437023
Ingolfsson, T.M.; Wang, X.; Hersche, M; Burrello, A.; Cavigelli, L.; Benini, L.: “ECG-TCN: wearable cardiac arrhythmia detection with a temporal convolutional network,” in 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), IEEE, Jun 1–4 (2021). https://doi.org/10.1109/AICAS51828.2021.9458520.
Faraone A.; Delgado-Gonzalo, R.: “Convolutional-recurrent neural networks on low-power wearable platforms for cardiac arrhythmia detection,” (2020). https://doi.org/10.1109/AICAS48895.2020.9073950.
Zhu, T.; Kuang, L.; Li, K.; Zeng, J.; Herrero, P.; Georgiou, P.: “Blood glucose prediction in type 1 diabetes using deep learning on the edge,” in 2021 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, May 1–5 (2021). https://doi.org/10.1109/ISCAS51556.2021.9401083.
Zanetti, R.; Aminifar, A.; Atienza, D.: “Robust epileptic seizure detection on wearable systems with reduced false-alarm rate,” in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), IEEE, Jul 4248–4251 (2020). https://doi.org/10.1109/EMBC44109.2020.9175339.
Risso et al., M.: “Robust and Energy-Efficient PPG-Based Heart-Rate Monitoring,” in 2021 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, May 1–5 (2021). doi: https://doi.org/10.1109/ISCAS51556.2021.9401282.
Gokul, H.; Suresh, P.; Hari Vignesh, B.; Pravin Kumaar, R.; Vijayaraghavan, V.: “Gait recovery system for parkinson’s disease using machine learning on embedded platforms,” in 2020 IEEE International Systems Conference (SysCon), IEEE, Aug. 1–8 (2020). https://doi.org/10.1109/SysCon47679.2020.9275930.
Oden L.; Witt, T.: “Fall-detection on a wearable micro controller using machine learning algorithms,” in 2020 IEEE International Conference on Smart Computing (SMARTCOMP), IEEE, Sep 296–301 (2020). https://doi.org/10.1109/SMARTCOMP50058.2020.00067.
Sucerquia, A.; López, J.; Vargas-Bonilla, J.: SisFall: a fall and movement dataset. Sensors 17(12), 198 (2017). https://doi.org/10.3390/s17010198
Merenda, M.; Astrologo, M.; Laurendi, D.; Romeo, V.; Della Corte, F.G.: “A novel fitness tracker using edge machine learning,” in 2020 IEEE 20th Mediterranean Electrotechnical Conference ( MELECON), IEEE, Jun 212–215 (2020). https://doi.org/10.1109/MELECON48756.2020.9140602.
Casadei, B.C.; Gumiero, A.; Tantillo, G.; Della Torre, L.; Olmo, G.: “Systolic blood pressure estimation from PPG signal using ANN,” Electronics, 11, 18, 2909, (2022). https://doi.org/10.3390/electronics11182909.
Kwiatkowski, K.K.; Pau, D.P.; Leung, T.; Di Marco, O.: “Phonocardiogram segmentation with tiny computing,” in 2023 IEEE International Conference on Consumer Electronics (ICCE), IEEE, Jan 1–4 (2023). https://doi.org/10.1109/ICCE56470.2023.10043562.
Whang, S.E.; Lee, J.-G.: Data collection and quality challenges for deep learning. Proc. VLDB Endow. 13(12), 3429–3432 (2020). https://doi.org/10.14778/3415478.3415562
Jmal, A.; Barioul, R.; Meddeb Makhlouf, A.; Fakhfakh, A.; Kanoun, O.: “An embedded ANN raspberry PI for inertial sensor based human activity recognition,” in Springer, 375–385 (2020). https://doi.org/10.1007/978-3-030-51517-1_34.
Gao, H.; Wu, X.; Shi, C.; Gao, Q.; Geng, J.: “A LSTM-based realtime signal quality assessment for photoplethysmogram and remote photoplethysmogram,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, Jun 3826–3835 (2021). https://doi.org/10.1109/CVPRW53098.2021.00424.
Smagulova K.; James, A.P.: “Overview of long short-term memory neural networks,” in Modeling and Optimization in Science and Technologies, Springer Verlag, 139–153 (2020). https://doi.org/10.1007/978-3-030-14524-8_11.
Li, S.; Walls, R.J; Guo, T.: “Characterizing and modeling distributed training with transient cloud GPU servers,” in 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS), IEEE, Nov 943–953 (2020). https://doi.org/10.1109/ICDCS47774.2020.00097.
Wang, Y.E.; Wei, G.-Y.; Brooks, D.: “Benchmarking TPU, GPU, and CPU Platforms for Deep Learning,” Jul. (2019). [Online]. Available: http://arxiv.org/abs/1907.10701
Varghese, B.; Wang, N.; Barbhuiya, S.; Kilpatrick, P.; Nikolopoulos, D.S.: “Challenges and Opportunities in Edge Computing,” 2016 IEEE Int. Conf. Smart Cloud, 20–26 (2016). [Online]. Available: https://api.semanticscholar.org/CorpusID:17046469
Ren, J.; Pan, Y.; Goscinski, A.M.; Beyah, R.A.: “Edge Computing for the Internet of Things,” IEEE Netw, vol. 32, pp. 6–7, (2018). [Online]. Available: https://api.semanticscholar.org/CorpusID:38206525
Zhang, Y.; Wijerathne, D.; Li, Z.; Mitra, T.: “Power-performance characterization of TinyML systems,” in 2022 IEEE 40th International Conference on Computer Design (ICCD), IEEE, Oct 644–651 (2022). https://doi.org/10.1109/ICCD56317.2022.00099.
Mukherjee, N.; Mukhopadhyay, S.; Gupta, R.: Real-time mental stress detection technique using neural networks towards a wearable health monitor. Meas. Sci. Technol. 33(4), 044003 (2022). https://doi.org/10.1088/1361-6501/ac3aae
Urosevic V.; Dimitrijevic, S.: “Optimum input sequence size for a sliding window-based LSTM neural network used in short-term electrical load forecasting,” in 2021 29th Telecommunications Forum (TELFOR), IEEE, Nov. 1–4 (2021). https://doi.org/10.1109/TELFOR52709.2021.9653206.
S. A and S. K.: “Drug Analyser Using Neural Networks with the Use of Transfer Learning Techniques,” Int. Res. J. Adv. Sci. Hub, vol. 3, no. Special Issue ICIES-2021 4S, 22–25, Apr (2021). https://doi.org/10.47392/irjash.2021.105.
M.S.M.P. Mrs. A. Pavithra,: “TensorFlow in Deep learning,” (2019). [Online]. Available: https://api.semanticscholar.org/CorpusID:150864082
van den Dool, W.; Blankevoort, T.; Welling, M.; Asano, Y.: “Efficient Neural PDE-Solvers Using Quantization Aware Training,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, Oct. 1423–1432 (2023).
Narang, S.; Elsen, E.; Diamos, G.; Sengupta, S.: “Exploring sparsity in recurrent neural networks,” Apr (2017). [Online]. Available: http://arxiv.org/abs/1704.05119
Hawks, B.; Duarte, J.; Fraser, N.J.; Pappalardo, A.; Tran, N.; Umuroglu, Y.: Ps and Qs: quantization-aware pruning for efficient low latency neural network inference. Front Artif. Intell. (2021). https://doi.org/10.3389/frai.2021.676564
Krishnamoorthi, R.: “Quantizing deep convolutional networks for efficient inference: A whitepaper,” ArXiv, vol. abs/1806.0, (2018). [Online]. Available: https://api.semanticscholar.org/CorpusID:49356451
Orășan, I.L.; Seiculescu, C.; Căleanu, C.D.: “Benchmarking TensorFlow Lite Quantization Algorithms for Deep Neural Networks,” 2022 IEEE 16th Int. Symp. Appl. Comput. Intell. Informatics, pp. 221–226, (2022). [Online]. Available: https://api.semanticscholar.org/CorpusID:253123929
Li J.; Álvarez, R.: “On the quantization of recurrent neural networks,” ArXiv, vol. abs/2101.0, (2021). [Online]. Available: https://api.semanticscholar.org/CorpusID:231602957
“STM32H743IIT6-eStore-STMicroelectronics.” https://estore.st.com/en/stm32h743iit6-cpn.html (accessed 26 Feb. 2024).
“Buy STM32H750VBT6-ST Online Store.” https://estore.st.com/en/stm32h750vbt6-cpn.html (accessed 26 Feb. 2024).
“Winbond W25Q16JV Serial NOR Flash.” https://www.techdesign.com/market/categories/flash/code-storage-flash-memory/serial-nor-flash?search=W25Q16JV&utm_source=winbond&utm_medium=product&utm_campaign=winbond_product (accessed 26 Feb 2024).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rostami, A., Tarvirdizadeh, B., Alipour, K. et al. Real-Time Stress Detection from Raw Noisy PPG Signals Using LSTM Model Leveraging TinyML. Arab J Sci Eng (2024). https://doi.org/10.1007/s13369-024-09095-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s13369-024-09095-2