Skip to main content
SpringerLink
Log in

Time Analysis in Human Activity Recognition

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Continuous human activity recognition from inertial signals is performed by splitting these temporal signals into time windows and identifying the activity in each window. Defining the appropriate window duration has been the target of several previous works. In most of these analyses, the recognition performance increases with the windows duration until an optimal value and decreases or saturates for longer windows. This paper evaluates several strategies to combine sub-window information inside a window, obtaining important improvements for long windows. This evaluation was performed using a state-of-the-art human activity recognition system based on Convolutional Neural Networks (CNNs). This deep neural network includes convolutional layers to learn features from signal spectra and additional fully connected layers to classify the activity at each window. All the analyses were carried out using two public datasets (PAMAP2 and USC-HAD) and a Leave-One-Subject-Out (LOSO) cross-validation. For 10-s windows, the accuracy increased from 90.1 (± 0.66) to 94.27 (± 0.46) in PAMAP2 and from 80.54 (± 0.73) to 84.46 (± 0.67) in USC-HAD. For 20-s windows, the improvements were from 92.66 (± 0.58) to 96.35 (± 0.38) (PAMAP2) and from 78.39 (± 0.76) to 86.36 (± 0.57) (USC-HAD).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Zhuang Z, Xue Y (2019) Sport-related human activity detection and recognition using a smartwatch. Sensors 19(22):5001. https://doi.org/10.3390/s19225001

    Article  Google Scholar 

  2. Fang H, Hu C (2014) Recognizing Human Activity in Smart Home Using Deep Learning algorithm. In 33rd Chinese Control Conference (CCC), Nanjing, Peoples of Pepublic China, 2014. Jul 28–30 2014, in Chinese Control Conference, pp 4716–4720. [Online]. Available: <Go to ISI>://WOS:000366482804147

  3. Khan AT, Li S, Cao X (2021) Control framework for cooperative robots in smart home using bio-inspired neural network. Measurement 167:108253. https://doi.org/10.1016/j.measurement.2020.108253

    Article  Google Scholar 

  4. Tamoor KA, Shuai L (2021) Human guided cooperative robotic agents in smart home using beetle antennae search. Sci China Inf Sci. https://doi.org/10.1007/s11432-020-3073-5

    Article  Google Scholar 

  5. Gil-Martín M, Montero JM, San-Segundo R (2019) Parkinson’s disease detection from drawing movements using convolutional neural networks. Electronics 8(8):907. https://doi.org/10.3390/electronics8080907 (In English)

    Article  Google Scholar 

  6. Neto et al. (2016) Recognition of Human Activities using the User's Context and the Activity Theory for Risk Prediction. (in English), Proceedings of the 18th International Conference on Enterprise Information Systems, Vol 2 (Iceis), Proceedings Paper pp 282–289 https://doi.org/10.5220/0005832202820289.

  7. Li Z, Zuo W, Li S (2020) Zeroing dynamics method for motion control of industrial upper-limb exoskeleton system with minimal potential energy modulation. Measurement 163:107964. https://doi.org/10.1016/j.measurement.2020.107964

    Article  Google Scholar 

  8. Li Z, Li C, Li S, Cao X (2019) A Fault-tolerant method for motion planning of industrial redundant manipulator. IEEE Trans Ind Informatics 16(12):7469–7478. https://doi.org/10.1109/TII.2019.2957186

    Article  Google Scholar 

  9. Gil-Martín M, Antúnez-Durango J, San-Segundo R (2020) Adaptation and selection techniques based on deep learning for human activity recognition using inertial sensors. Eng Proceed 2(1):22

    Google Scholar 

  10. Gil-Martin M, San-Segundo R, de Cordoba R, Manuel Pardo J (2020) Robust biometrics from motion wearable sensors using a D-vector approach. Neural Process Lett. https://doi.org/10.1007/s11063-020-10339-z

    Article  Google Scholar 

  11. San-Segundo R, Navarro-Hellín H, Torres-Sánchez R, Hodgins J, De la Torre F (2019) Increasing robustness in the detection of freezing of gait in Parkinson’s disease. Electronics 8(2):119. https://doi.org/10.3390/electronics8020119

    Article  Google Scholar 

  12. Lu H, Jin L, Luo X, Liao B, Guo D, Xiao L (2019) RNN for solving perturbed time-varying underdetermined linear system with double bound limits on residual errors and state variables. IEEE Trans Industr Inf 15(11):5931–5942. https://doi.org/10.1109/tii.2019.2909142

    Article  Google Scholar 

  13. Lin CY, Chen BY, Wu WC, Lin EY, Tsai CL (2015) Human Action Recognition Based on Non-negative Matrix Factorization. In Asia-Pacific-Signal-and-Information-Processing-Association Annual Summit and Conference (APSIPA ASC), Hong Kong, Peoples Rebublic of China. In Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, 2015, pp 1091-1093. [Online]. Available: <Go to ISI>://WOS:000382954100206

  14. Luo X, Zhou M, Shang M, Li S, Xia Y (2016) A novel approach to extracting non-negative latent factors from non-negative big sparse matrices. IEEE Access 4:2649–2655. https://doi.org/10.1109/access.2016.2556680

    Article  Google Scholar 

  15. Luo X, Zhou M, Li S, Wu D, Liu Z, Shang M (2021) Algorithms of unconstrained non-negative latent factor analysis for recommender systems. IEEE Trans Big Data 7(1):227–240. https://doi.org/10.1109/tbdata.2019.2916868

    Article  Google Scholar 

  16. Gil-Martín M, San-Segundo R, Fernández-Martínez F, de Córdoba R (2020) Human activity recognition adapted to the type of movement. Comput Electr Eng 88:106822. https://doi.org/10.1016/j.compeleceng.2020.106822

    Article  Google Scholar 

  17. Reiss A, and Stricker D (2012) "Creating and benchmarking a new dataset for physical activity monitoring. Presented at the Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments, Heraklion, Crete, Greece, [Online]. Available: https://doi.org/10.1145/2413097.2413148.

  18. Kasnesis P, Patrikakis CZ, Venieris IS (2019) PerceptionNet: a deep convolutional neural network for late sensor fusion. Intell Syst Appl 868:101–119. https://doi.org/10.1007/978-3-030-01054-6_7

    Article  Google Scholar 

  19. Ignatov A (2018) Real-time human activity recognition from accelerometer data using convolutional neural networks. Appl Soft Comput 62:915–922. https://doi.org/10.1016/j.asoc.2017.09.027

    Article  Google Scholar 

  20. San-Segundo R, Echeverry-Correa JD, Salamea C, Pardo JM (2016) Human activity monitoring based on hidden markov models using a smartphone. IEEE Instrum Measure Magazine 19(6):27–31. https://doi.org/10.1109/mim.2016.7777649

    Article  Google Scholar 

  21. Khaire P, Kumar P, Imran J (2018) Combining CNN streams of RGB-D and skeletal data for human activity recognition. Pattern Recogn Lett 115:107–116. https://doi.org/10.1016/j.patrec.2018.04.035

    Article  Google Scholar 

  22. Abdel-Salam R, Mostafa R, and Hadhood M (2021) Human activity recognition using wearable sensors: review, challenges, evaluation benchmark. p. arXiv:2101.01665

  23. Tanjid Hasan Tonmoy M, Mahmud S, Mahbubur Rahman AKM, Ashraful Amin M, and Ali AA, Hierarchical self attention based autoencoder for open-set human activity recognition. p. arXiv:2103.04279

  24. Mahmud S et al. (2020) Human activity recognition from wearable sensor data using self-attention. p. arXiv:2003.09018

  25. Khan AT, Cao X, Li S, Hu B, Katsikis VN (2021) Quantum beetle antennae search: a novel technique for the constrained portfolio optimization problem. Sci China-Inf Sci 64(5):152204. https://doi.org/10.1007/s11432-020-2894-9

    Article  MathSciNet  Google Scholar 

  26. Banos O, Galvez J-M, Damas M, Pomares H, Rojas I (2014) Window size impact in human activity recognition. Sensors 14(4):6474–6499. https://doi.org/10.3390/s140406474

    Article  Google Scholar 

  27. Pirttikangas S, Fujinami K, Nakajima T (2006) Feature selection and activity recognition from wearable sensors. Ubiquitous Comput Syst, Proceed 4239:516–527

    Article  Google Scholar 

  28. Stikic M, Huynh T, Van Laerhoven K, Schiele B (2008) ADL Recognition Based on the Combination of RFID and Accelerometer Sensing. 2008 2nd International Conference on Pervasive Computing Technologies for Healthcare. pp 245

  29. Sun L, Zhang D, Li B, Guo B, and Li S (2010) Activity recognition on an accelerometer embedded mobile phone with varying positions and orientations. Ubiquitous Intelligence and Computing, vol. 6406, p 548

  30. Mannini A, Intille SS, Rosenberger M, Sabatini AM, Haskell W (2013) Activity recognition using a single accelerometer placed at the wrist or ankle. Med Sci Sports Exerc 45(11):2193–2203. https://doi.org/10.1249/MSS.0b013e31829736d6

    Article  Google Scholar 

  31. Mehrang S, Pietila J, Korhonen I (2018) An activity recognition framework deploying the random forest classifier and a single optical heart rate monitoring and Triaxial accelerometer wrist-band. Sensors 18(2):613. https://doi.org/10.3390/s18020613

    Article  Google Scholar 

  32. Zhang M, Sawchuk AA, and Assoc Comp M (2012) USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. Ubicomp'12: Proceedings of the 2012 Acm International Conference on Ubiquitous Computing pp 1036–1043

  33. Bulling A, Blanke U, Schiele B (2014) A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput Surv (CSUR) 46(3):1–33. https://doi.org/10.1145/2499621

    Article  Google Scholar 

  34. Weiss NA (2017) Introductory statistics. Pearson, London

    MATH  Google Scholar 

  35. Roggen D, Calatroni A, Rossi M, Holleczek T, and Forster K (2010) Collecting complex activity datasets in highly rich networked sensor environments. 2010 Seventh International Conference on Networked Sensing Systems (INSS 2010), pp 233–40 https://doi.org/10.1109/inss.2010.5573462.

Download references

Funding

The work leading to these results has been supported by AMIC (MINECO, TIN2017-85854-C4-4-R), and CAVIAR (MINECO, TEC2017-84593-C2-1-R) projects partially funded by the European Union. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for this research.

Author information

Authors and Affiliations

  1. Speech Technology Group, Information Processing and Telecomunications Center, E.T.S.I. Telecomunicación, Universidad Politécnica de Madrid, 28040, Madrid, Spain

    Manuel Gil-Martín, Rubén San-Segundo, Fernando Fernández-Martínez & Javier Ferreiros-López

Authors
  1. Manuel Gil-Martín
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rubén San-Segundo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fernando Fernández-Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Javier Ferreiros-López
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M. Gil-Martín: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Visualization, Roles/Writing—original draft. R. San-Segundo: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing—review & editing. F. Fernández-Martínez: Conceptualization, Funding acquisition, Project administration, Supervision, Validation, Writing—review & editing. J. Ferreiros-López: Conceptualization, Funding acquisition, Project administration, Supervision, Validation, Writing—review & editing.

Corresponding author

Correspondence to Manuel Gil-Martín.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gil-Martín, M., San-Segundo, R., Fernández-Martínez, F. et al. Time Analysis in Human Activity Recognition. Neural Process Lett 53, 4507–4525 (2021). https://doi.org/10.1007/s11063-021-10611-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10611-w

Keywords

Search

Navigation