Abstract
Human Activity Recognition (HAR) has been studied for decades, from data collection, learning models, to post-processing and result interpretations. However, the inherent hierarchy in the activities remains relatively under-explored, despite its significant impact on model performance and interpretation. In this paper, we propose H-HAR, by rethinking the HAR tasks from a fresh perspective by delving into their intricate global label relationships. Rather than building multiple classifiers separately for multi-layered activities, we explore the efficacy of a flat model enhanced with graph-based label relationship modeling. Being hierarchy-aware, the graph-based label modeling enhances the fundamental HAR model, by incorporating intricate label relationships into the model. We validate the proposal with a multi-label classifier on complex human activity data. The results highlight the advantages of the proposal, which can be vertically integrated into advanced HAR models to further enhance their performances.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Banerjee, S., Akkaya, C., Perez-Sorrosal, F., Tsioutsiouliklis, K.: Hierarchical transfer learning for multi-label text classification. In: ACL, pp. 6295–6300 (2019)
Banos, O., Garcia, R., Saez, A.: MHEALTH Dataset. UCI Machine Learning Repository (2014). https://doi.org/10.24432/C5TW22
Chen, H., Ma, Q., Lin, Z., Yan, J.: Hierarchy-aware label semantics matching network for hierarchical text classification. In: ACL, pp. 4370–4379 (2021)
Debache, I., Jeantet, L., Chevallier, D., Bergouignan, A., Sueur, C.: A lean and performant hierarchical model for human activity recognition using body-mounted sensors. Sensors 20(11), 3090 (2020)
Dumais, S., Chen, H.: Hierarchical classification of web content. In: SIGIR, pp. 256–263 (2000)
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., Krishnan, D.: Supervised contrastive learning. NeurIPS 33, 18661–18673 (2020)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (2017)
Leutheuser, H., Schuldhaus, D., Eskofier, B.M.: Hierarchical, multi-sensor based classification of daily life activities: comparison with state-of-the-art algorithms using a benchmark dataset. PLoS ONE 8(10), e75196 (2013)
Reyes-Ortiz, J.L., Oneto, L., Samà, A., Parra, X., Anguita, D.: Transition-aware human activity recognition using smartphones. Neurocomputing 171, 754–767 (2016)
Shimura, K., Li, J., Fukumoto, F.: HFT-CNN: learning hierarchical category structure for multi-label short text categorization. In: ACL, pp. 811–816 (2018)
Thu, N.T.H., Han, D.S.: HiHAR: a hierarchical hybrid deep learning architecture for wearable sensor-based human activity recognition. IEEE Access 9, 145271–145281 (2021)
Tonioni, A., Di Stefano, L.: Domain invariant hierarchical embedding for grocery products recognition. CVIU 182, 81–92 (2019)
Wang, Z., Wang, P., Huang, L., Sun, X., Wang, H.: Incorporating hierarchy into text encoder: a contrastive learning approach for hierarchical text classification. In: ACL, pp. 7109–7119 (2022)
Wu, Z., Pan, S., Long, G., Jiang, J., Zhang, C.: Graph wavenet for deep spatial-temporal graph modeling. In: IJCAI, pp. 1907–1913 (2019)
Zhang, S., McCullagh, P., Nugent, C., Zheng, H.: Activity monitoring using a smart phone’s accelerometer with hierarchical classification. In: 2010 Sixth International Conference on Intelligent Environments, pp. 158–163. IEEE (2010)
Zhang, X., Zhou, F., Lin, Y., Zhang, S.: Embedding label structures for fine-grained feature representation. In: CVPR, pp. 1114–1123 (2016)
Zheng, Y.: Human activity recognition based on the hierarchical feature selection and classification framework. J. Elect. Comput. Eng. 2015, 34–34 (2015)
Zhou, J., et al.: Hierarchy-aware global model for hierarchical text classification. In: ACL, pp. 1106–1117 (2020)
Zuo, J., Arvanitakis, G., Hacid, H.: On handling catastrophic forgetting for incremental learning of human physical activity on the edge. In: EDBT (2023)
Zuo, J., Arvanitakis, G., Ndhlovu, M., Hacid, H.: Magneto: edge AI for human activity recognition - privacy and personalization. In: EDBT (2024)
Zuo, J., Zeitouni, K., Taher, Y.: Exploring interpretable features for large time series with se4tec. In: EDBT (2019)
Zuo, J., Zeitouni, K., Taher, Y.: SMATE: semi-supervised spatio-temporal representation learning on multivariate time series. In: ICDM, pp. 1565–1570 (2021)
Zuo, J., Zeitouni, K., Taher, Y., Garcia-Rodriguez, S.: Graph convolutional networks for traffic forecasting with missing values. DMKD 37(2), 913–947 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zuo, J., Hacid, H. (2024). Re-thinking Human Activity Recognition with Hierarchy-Aware Label Relationship Modeling. In: Yang, DN., Xie, X., Tseng, V.S., Pei, J., Huang, JW., Lin, J.CW. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2024. Lecture Notes in Computer Science(), vol 14649. Springer, Singapore. https://doi.org/10.1007/978-981-97-2262-4_1
Download citation
DOI: https://doi.org/10.1007/978-981-97-2262-4_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2264-8
Online ISBN: 978-981-97-2262-4
eBook Packages: Computer ScienceComputer Science (R0)