Skip to main content

Advertisement

SpringerLink
Log in

A human activity recognition framework for grossly labeled smartphone sensing data through combining genetic algorithm with multiple instance multiple label learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human Activity Recognition through smartphones plays a crucial role in several medical state-of-affairs like patient monitoring, eldercare, and post-surgery recovery. Most of these works require precisely labeled accelerometer data for training supervised learning classifiers. Precise labeling of smartphone sensing data is difficult in real life due to the non-uniform gait of individual users with unpredictable activity transitions in between. Selecting an optimal number of features for the classification of such grossly labeled smartphone accelerometer instances is hardly investigated in the literature. Hence, a semi-supervised learning approach, that combines Multi-Instance Multi-label (MIML) learning with a Genetic Algorithm (GA) is proposed here. Rather than labeling a single instance, activity bags are designed for classification. GA selects the optimal sets of features for such grossly labeled bags of featured instances. MIML-kNN is chosen to be the classifier that is integrated with GA to predict the single, double or triple activity combinations that are often performed in daily life. Interestingly, the proposed framework can also predict component activities from an unknown activity combination even when it is not trained with such combinations. The framework is implemented for a real dataset collected from 8 users and it is found to be working adequately with less than half the set of features giving an average precision of 94% for even triple and double activity combinations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. (https://play.google.com/store/apps/details?id=com.peterhohsy.gsensor_debug&hl=en)

References

  1. Baldominos A, Isasi P, Saez Y (2017) Feature selection for physical activity recognition using genetic algorithms. In: 2017 IEEE Congress on Evolutionary Computation (CEC). IEEE, pp 2185–2192

  2. Batool M, Jalal A, Kim K (2019) Sensors technologies for human activity analysis based on svm optimized by pso algorithm. In: 2019 International Conference on Applied and Engineering Mathematics (ICAEM). IEEE, pp 145–150

  3. Bouaguel W (2016) A new approach for wrapper feature selection using genetic algorithm for big data. In: Intelligent and Evolutionary Systems. Springer, pp 75–83

  4. Chen J, Sun Y, Sun S (2021) Improving human activity recognition performance by data fusion and feature engineering. Sensors 21(3):692

    Article  Google Scholar 

  5. Chen L, Hoey J, Nugent CD, Cook DJ, Yu Z (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C (Appl Rev) 42 (6):790–808

    Article  Google Scholar 

  6. Chowdhary CL, Mouli PVSSRC (2013) Image registration with new system for ensemble of images of multi-sensor registration. World Appl Sci J 26 (1):45–50

    Google Scholar 

  7. Dernbach S, Das B, Krishnan NC, Thomas BL, Cook DJ (2012) Simple and complex activity recognition through smart phones. In: 2012 Eighth International Conference on Intelligent Environments. IEEE, pp 214–221

  8. El-Maaty AMA, Wassal AG (2018) Hybrid ga-pca feature selection approach for inertial human activity recognition. In: 2018 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, pp 1027–1032

  9. Fang H, He L, Si H, Liu P, Xie X (2014) Human activity recognition based on feature selection in smart home using back-propagation algorithm. ISA Trans 53(5):1629–1638

    Article  Google Scholar 

  10. Gashi S, Di Lascio E, Santini S (2021) Multi-class multi-label classification for cooking activity recognition. In: Human Activity Recognition Challenge. Springer, pp 75–89

  11. Guan X, Raich R, Wong W-K (2016) Efficient multi-instance learning for activity recognition from time series data using an auto-regressive hidden markov model. In: International Conference on Machine Learning. PMLR, pp 2330–2339

  12. Gupta P, Dallas T (2014) Feature selection and activity recognition system using a single triaxial accelerometer. IEEE Trans Biomed Eng 61(6):1780–1786

    Article  Google Scholar 

  13. Hu Y, Wang B, Sun Y, An J, Wang Z (2020) Genetic algorithm–optimized support vector machine for real-time activity recognition in health smart home. Int J Distrib Sens Netw 16(11):1–13

    Article  Google Scholar 

  14. Huh J-H, Seo Y-S (2019) Understanding edge computing: Engineering evolution with artificial intelligence. IEEE Access 7:164229–164245. https://doi.org/10.1109/ACCESS.2019.2945338

    Article  Google Scholar 

  15. Ijaz MF, Alfian G, Syafrudin M, Rhee J (2018) Hybrid prediction model for type 2 diabetes and hypertension using dbscan-based outlier detection, synthetic minority over sampling technique (smote), and random forest. Appl Sci 8(8):1325

    Article  Google Scholar 

  16. Ijaz MF, Attique M, Son Y (2020) Data-driven cervical cancer prediction model with outlier detection and over-sampling methods. Sensors 20(10):2809

    Article  Google Scholar 

  17. Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In: Proceedings of the 23rd ACM International Conference on Multimedia. ACM, pp 1307–1310

  18. Li J, Tian L, Chen L, Wang H, Cao T, Yu L (2019) Optimal feature selection for activity recognition based on ant colony algorithm. In: 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA). IEEE, pp 2356–2362

  19. Lu Z, Chung YY, Yeung HWF, Zandavi SM, Zhi W, Yeh W-C (2017) Using hidden markov model to predict human actions with swarm intelligence. In: International Conference on Neural Information Processing. Springer, pp 21–30

  20. Mohamed R, Zainudin MNS, Sulaiman MN, Perumal T, Mustapha N (2018) Multi-label classification for physical activity recognition from various accelerometer sensor positions. J Inf Commun Technol 17(2):209–231

    Google Scholar 

  21. Nguyen Truc DT, Huynh T-T, Pham H-A (2018) An improved human activity recognition by using genetic algorithm to optimize feature vector. In: 2018 10th International Conference on Knowledge and Systems Engineering (KSE). IEEE, pp 123–128

  22. Panigrahi R, Borah S, Bhoi AK, Ijaz MF, Pramanik M, Jhaveri RH, Chowdhary CL (2021) Performance assessment of supervised classifiers for designing intrusion detection systems: A comprehensive review and recommendations for future research. Mathematics 9(6):690

    Article  Google Scholar 

  23. Panigrahi R, Borah S, Bhoi AK, Ijaz MF, Pramanik M, Kumar Y, Jhaveri RH (2021) A consolidated decision tree-based intrusion detection system for binary and multiclass imbalanced datasets. Mathematics 9(7):751

    Article  Google Scholar 

  24. Peng L, Chen L, Wu X, Guo H, Chen G (2016) Hierarchical complex activity representation and recognition using topic model and classifier level fusion. IEEE Trans Biomed Eng 64(6):1369–1379

    Article  Google Scholar 

  25. Read J, Martino L, Hollmén J (2017) Multi-label methods for prediction with sequential data. Pattern Recogn 63:45–55

    Article  Google Scholar 

  26. Saha J, Chowdhury C, Ghosh D, Bandyopadhyay S (2020) A detailed human activity transition recognition framework for grossly labeled data from smartphone accelerometer. Multimed Tools Appl:1–22

  27. Saha J, Chowdhury C, Roy Chowdhury I, Biswas S, Aslam N (2018) An ensemble of condition based classifiers for device independent detailed human activity recognition using smartphones. Information 9(4):94

    Article  Google Scholar 

  28. Saha J, Ghosh D, Chowdhury C, Bandyopadhyay S (2021) Smart handheld based human activity recognition using multiple instance multiple label learning. Wirel Pers Commun 117(2):923–943

    Article  Google Scholar 

  29. Saputri TRD, Khan AM, Lee S-W (2014) User-independent activity recognition via three-stage ga-based feature selection. Int J Distrib Sens Netw 10 (3):706287

    Article  Google Scholar 

  30. Savvaki S, Tsagkatakis G, Panousopoulou A, Tsakalides P (2017) Matrix and tensor completion on a human activity recognition framework. IEEE J Biomed Health Inf 21(6):1554–1561

    Article  Google Scholar 

  31. Shoaib M, Bosch S, Incel OD, Scholten H, Havinga PJM (2016) Complex human activity recognition using smartphone and wrist-worn motion sensors. Sensors 16(4):426

    Article  Google Scholar 

  32. Stikic M, Larlus D, Schiele B (2009) Multi-graph based semi-supervised learning for activity recognition. In: 2009 international symposium on wearable computers. IEEE, pp 85–92

  33. Stikic M, Schiele B (2009) Activity recognition from sparsely labeled data using multi-instance learning. In: International Symposium on Location-and Context-Awareness. Springer, pp 156–173

  34. Toda T, Inoue S, Tanaka S, Ueda N (2014) Training human activity recognition for labels with inaccurate time stamps. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, pp 863–872

  35. Vaizman Y, Weibel N, Lanckriet G (2018) Context recognition in-the-wild: Unified model for multi-modal sensors and multi-label classification. Proc ACM Interact Mob Wearable Ubiquit Technol 1(4):1–22

    Article  Google Scholar 

  36. Vecchio A, Mulas F, Cola G (2017) Posture recognition using the interdistances between wearable devices. IEEE Sens Lett 1(4):1–4

    Article  Google Scholar 

  37. Wang H, Ke R, Li J, An Y, Wang K, Yu L (2018) A correlation-based binary particle swarm optimization method for feature selection in human activity recognition. Int J Distrib Sens Netw 14(4):1550147718772785

    Google Scholar 

  38. Wang Z, Wu D, Chen J, Ghoneim A, Hossain MA (2016) A triaxial accelerometer-based human activity recognition via eemd-based features and game-theory-based feature selection. IEEE Sens J 16(9):3198–3207

    Article  Google Scholar 

  39. Wannenburg J, Malekian R (2016) Physical activity recognition from smartphone accelerometer data for user context awareness sensing. IEEE Trans Syst Man Cybern Syst 47(12):3142–3149

    Article  Google Scholar 

  40. Woznowski PR, King R, Harwin W, Craddock I (2016) Activity recognition framework for healthcare applications: ontology, labelling strategies, and best practice. In: Proceedings of the International Conference on Internet of Things and Big Data 2016. SciTePress, pp 369–377

  41. Zainudin MN Shah, Mohamed R, Sulaiman MN, Perumal T, Mustapha N, Nazri ASA (2017) Multi-label classification using label combination to recognize human activity based on various sensor positions. In: Proceedings of the 6th International Conference on Computing and Informatics, ICOCI’2017

  42. Zhang M (2010) A k-nearest neighbor based multi-instance multi-label learning algorithm. In: 2010 22nd IEEE International Conference on Tools with Artificial Intelligence, vol 2, pp 207–212

  43. Zhang M, Sawchuk AA (2013) Human daily activity recognition with sparse representation using wearable sensors. IEEE J Biomed Health Inf 17(3):553–560

    Article  Google Scholar 

  44. Zolfaghari S, Keyvanpour MR (2016) Sarf: Smart activity recognition framework in ambient assisted living. In: 2016 Federated Conference on Computer Science and Information Systems (FedCSIS). IEEE, pp 1435–1443

Download references

Author information

Authors and Affiliations

  1. Department of CSE, Jadavpur University, Kolkata, India

    Sajan Rajak, Debasish Bose & Chandreyee Chowdhury

  2. Techno Main SaltLake, Kolkata, India

    Anindita Saha

Authors
  1. Sajan Rajak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debasish Bose
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anindita Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chandreyee Chowdhury
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chandreyee Chowdhury.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rajak, S., Bose, D., Saha, A. et al. A human activity recognition framework for grossly labeled smartphone sensing data through combining genetic algorithm with multiple instance multiple label learning. Multimed Tools Appl 81, 24887–24911 (2022). https://doi.org/10.1007/s11042-022-12261-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12261-z

Keywords

Search

Navigation