Skip to main content
SpringerLink
Log in

A novel deep learning architecture and MINIROCKET feature extraction method for human activity recognition using ECG, PPG and inertial sensor dataset

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The research in human activity recognition has gained prominence in various applications, including healthcare, medical, and surveillance. The earlier popular techniques which relied on images or video sequences to perform classification are susceptible to noise, line of sight, and light conditions. The wireless wearable sensors provide a robust alternative to these techniques for data collection and classification. Towards this, we propose the application of Minimally Random Convolutional Kernel Transform (MINIROCKET) for feature extraction on sensor data. The extracted features are then used by classifiers for activity recognition. To this end, we employed two publicly available datasets containing heart rate sensors and motion sensor data on various activities. Further, we showed that the application of MINIROCKET requires significantly less computational time compared to other existing models.

Additionally, we propose a novel deep learning based Double stacked Convolutional and LSTM (DCLS) architecture to provide the baseline and showed that classification through MINIROCKET’s features yields superior results compared to best deep learning models at a less computational expense. The results of our experiments are compared with other baseline models for varied sampling time window sizes and have shown greater accuracies. In addition, we report the best combination of the sampling time window size and the appropriate model to achieve the best accuracy, minimum false positives, or minimum false negatives depending on the requirement. This helps in developing a multi-criteria decision making system for human activity recognition using wearable sensor devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Algorithm 2
Fig. 4
Algorithm 3
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Cicirelli F, Fortino G, Giordano A, Guerrieri A, Spezzano G, Vinci A (2016) On the design of smart homes: a framework for activity recognition in home environment. J Med Syst 40(9):1–17

    Article  Google Scholar 

  2. Rashidi P, Cook DJ (2009) Keeping the resident in the loop: adapting the smart home to the user. IEEE Trans Syst Man Cybern-Part Syst Humans 39(5):949–959

    Article  Google Scholar 

  3. Patel S, Park H, Bonato P, Chan L, Rodgers M (2012) A review of wearable sensors and systems with application in rehabilitation. J Neuroeng Rehab 9(1):1–17

    Article  Google Scholar 

  4. Mazilu S, Blanke U, Hardegger M, Tröster G, Gazit E, Hausdorff JM (2014) Gaitassist: a daily-life support and training system for parkinson’s disease patients with freezing of gait. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 2531–2540

  5. Kranz M, Möller A, Hammerla N, Diewald S, Plötz T, Olivier P, Roalter L (2013) The mobile fitness coach: towards individualized skill assessment using personalized mobile devices. Pervasive Mobile Comput 9(2):203–215

    Article  Google Scholar 

  6. Stiefmeier T, Roggen D, Ogris G, Lukowicz P, Tröster G (2008) Wearable activity tracking in car manufacturing. IEEE Pervasive Comput 7(2):42–50

    Article  Google Scholar 

  7. Attal F, Mohammed S, Dedabrishvili M, Chamroukhi F, Oukhellou L, Amirat Y (2015) Physical human activity recognition using wearable sensors. Sensors 15(12):31314–31338

    Article  Google Scholar 

  8. Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2017) Towards unsupervised physical activity recognition using smartphone accelerometers. Multimed Tools Appl 76(8):10701–10719

    Article  Google Scholar 

  9. Keihani D, Kargarfard M, Mokhtari M (2014) Cardiac effects of exercise rehabilitation on quality of life, depression and anxiety in patients with heart failure patients. J Fund Mental Health 17(1):13–19

    Google Scholar 

  10. Jarchi D, Casson AJ (2017) Description of a database containing wrist ppg signals recorded during physical exercise with both accelerometer and gyroscope measures of motion. Data 2(1):1

    Article  Google Scholar 

  11. Boukhechba M, Cai L, Wu C, Barnes LE (2019) Actippg: using deep neural networks for activity recognition from wrist-worn photoplethysmography (ppg) sensors. Smart Health 14:100082

    Article  Google Scholar 

  12. Zhang Y, Liu B, Zhang Z (2015) Combining ensemble empirical mode decomposition with spectrum subtraction technique for heart rate monitoring using wrist-type photoplethysmography. Biomed Signal Process Contr 21:119–125

    Article  Google Scholar 

  13. Chen L, Liu X, Peng L, Wu M (2021) Deep learning based multimodal complex human activity recognition using wearable devices. Appl Intell 51(6):4029–4042

    Article  Google Scholar 

  14. Moghadam ZB, NOGHONDAR MS, Goshvarpour A (2021) Novel delayed poincare’s plot indices of photoplethysmogram for classification of physical activities. Appl Med Inform 43(1):43–55

    Google Scholar 

  15. Mehrang S, Pietilä J, Korhonen I (2018) An activity recognition framework deploying the random forest classifier and a single optical heart rate monitoring and triaxial accelerometer wrist-band. Sensors 18 (2):613

    Article  Google Scholar 

  16. Tang Y, Zhang L, Teng Q, Min F, Song A (2022) Triple cross-domain attention on human activity recognition using wearable sensors. IEEE Trans Emerging Topics Computat Intell

  17. Ciucurel C, Georgescu L, Iconaru EI (2018) Ecg response to submaximal exercise from the perspective of golden ratio harmonic rhythm. Biomed Signal Process Contr 40:156–162

    Article  Google Scholar 

  18. Tang Y, Zhang L , Min F, He J (2022) Multi-scale deep feature learning for human activity recognition using wearable sensors. IEEE Trans Industr Electr

  19. Gholamiangonabadi D, Grolinger K (2022) Personalized models for human activity recognition with wearable sensors: deep neural networks and signal processing. Appl Intell:1–21

  20. Reiss A, Indlekofer I, Schmidt P, Van Laerhoven K (2019) Deep ppg: large-scale heart rate estimation with convolutional neural networks. Sensors 19(14):3079

    Article  Google Scholar 

  21. Alessandrini M, Biagetti G, Crippa P, Falaschetti L, Turchetti C (2021) Recurrent neural network for human activity recognition in embedded systems using ppg and accelerometer data. Electronics 10 (14):1715

    Article  Google Scholar 

  22. Brophy E, Muehlhausen W, Smeaton AF, Ward TE (2020) Cnns for heart rate estimation and human activity recognition in wrist worn sensing applications. In: 2020 IEEE international conference on pervasive computing and communications workshops (PerCom Workshops). IEEE, pp 1–6

  23. Elshafei M, Costa DE, Shihab E (2021) On the impact of biceps muscle fatigue in human activity recognition. Sensors 21(4):1070

    Article  Google Scholar 

  24. Ma C, Li W, Cao J, Du J, Li Q, Gravina R (2020) Adaptive sliding window based activity recognition for assisted livings. Inform Fusion 53:55–65

    Article  Google Scholar 

  25. Bondugula RK, Udgata SK, Bommi NS (2021) A novel weighted consensus machine learning model for covid-19 infection classification using ct scan images. Arab J Sci Eng:1–12

  26. Gholamiangonabadi D, Kiselov N, Grolinger K (2020) Deep neural networks for human activity recognition with wearable sensors: leave-one-subject-out cross-validation for model selection. IEEE Access 8:133982–133994

    Article  Google Scholar 

  27. Aydemir T, Şahin M, Aydemir O (2020) A new method for activity monitoring using photoplethysmography signals recorded by wireless sensor. J Med Bio Eng 40(6):934–942

    Article  Google Scholar 

  28. Biagetti G, Crippa P, Falaschetti L, Saraceni L, Tiranti A, Turchetti C (2020) Dataset from ppg wireless sensor for activity monitoring. Data Brief 29:105044

    Article  Google Scholar 

  29. van Gent P, Farah H, van Nes N, van Arem B (2019) Heartpy: a novel heart rate algorithm for the analysis of noisy signals. Transportat Res Part Traffic Psychol Behav 66:368–378

    Article  Google Scholar 

  30. Peng C, Zhang X, Yu G, Luo G, Sun J (2017) Large kernel matters–improve semantic segmentation by global convolutional network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4353–4361

  31. Dempster A, Petitjean F, Webb GI (2020) Rocket: exceptionally fast and accurate time series classification using random convolutional kernels. Data Min Knowl Disc 34(5):1454–1495

    Article  MathSciNet  MATH  Google Scholar 

  32. Dempster A, Schmidt DF, Webb GI (2021) Minirocket: a very fast (almost) deterministic transform for time series classification. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp 248–257

  33. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  34. Bondugula RK, Sivangi KB, Udgata SK (2022) Identification of schizophrenic individuals using activity records through visualization of recurrent networks. In: Udgata SK, Sethi S, Gao X-Z (eds) Intelligent Systems. Springer Nature Singapore, pp 653–664

  35. Nweke HF, Teh YW, Al-Garadi MA, Alo UR (2018) Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges. Expert Syst Appl 105:233–261

    Article  Google Scholar 

  36. Zhao B, Lu H, Chen S, Liu J, Wu D (2017) Convolutional neural networks for time series classification. J Syst Eng Electron 28(1):162–169

    Article  Google Scholar 

  37. Brophy E, Veiga JJD, Wang Z, Ward TE (2018) A machine vision approach to human activity recognition using photoplethysmograph sensor data. In: 2018 29th Irish signals and systems conference (ISSC). IEEE, pp 1–6

  38. Mahmud T, Akash SS, Fattah SA, Zhu W-P, Ahmad MO (2020) Human activity recognition from multi-modal wearable sensor data using deep multi-stage lstm architecture based on temporal feature aggregation. In: 2020 IEEE 63rd international midwest symposium on circuits and systems (MWSCAS). IEEE, pp 249–252

  39. Almanifi ORA, Khairuddin IM, Razman MAM, Musa RM, Majeed APA (2022) Human activity recognition based on wrist ppg via the ensemble method. ICT Express

  40. Biagetti G, crippa P, Falaschetti L, Orcioni S, Turchetti C (2017) Human activity recognition using accelerometer and photoplethysmographic signals. In: International conference on intelligent decision technologies. Springer, pp 53–62

Download references

Author information

Authors and Affiliations

  1. AI Lab, School of Computer and Information Sciences, University of Hyderabad, Hyderabad, India

    Rohit Kumar Bondugula, Siba K Udgata & Kaushik Bhargav Sivangi

Authors
  1. Rohit Kumar Bondugula
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siba K Udgata
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kaushik Bhargav Sivangi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Siba K Udgata.

Ethics declarations

Ethics approval

We have used the secondary data available in the public domain and have not conducted any experiments involving human beings in this study.

Conflict of Interests

The authors declare that there is no conflict of interest in this work.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bondugula, R.K., Udgata, S.K. & Sivangi, K.B. A novel deep learning architecture and MINIROCKET feature extraction method for human activity recognition using ECG, PPG and inertial sensor dataset. Appl Intell 53, 14400–14425 (2023). https://doi.org/10.1007/s10489-022-04250-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04250-4

Keywords

Search

Navigation