Abstract
While Human Action Recognition (HAR) using motion capture can perform well with high accuracy, it requires a high computational cost for recording and post-processing. To avoid this, we build a HAR system using 3D pose estimation from the single-camera video instead of motion capture. One drawback in this approach is that the performance is considerably dependent on the camera position. This paper investigates how we can use the pose estimate constantly without the effect of camera position even when the camera position in the test data is changed. We augment the data by rotating around the 3D pose estimate to improve the accuracy when using different camera positions in the test data and in the training data. The strategy of augmenting training data shows improvements up to 55.7% in accuracy, compared with the case of 2D pose with no augmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bulling, A., Blanke, U., Schiele, B.: A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv. 46(3) (2014). https://doi.org/10.1145/2499621. https://doi.org/10.1145/2499621
Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2d pose estimation using part affinity fields. In: CVPR (2017)
Ionescu, C., Papava, D., Olaru, V., Sminchisescu, C.: Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence 36(7), 1325–1339 (2014)
Kohei, A., Tsuyoshi, O., Sozo, I.: Human action recognition using 3d pose estimation. Proceedings of the 21st SOFT Kyushu Chapter Annual Conference pp. 40–43 (2019)
Liaw, A., Wiener, M.: Classification and regression by randomforest. Forest 23, (2001)
Martinez, J., Hossain, R., Romero, J., Little, J.J.: A simple yet effective baseline for 3d human pose estimation. In: ICCV (2017)
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., Bajcsy, R.: Berkeley mhad: A comprehensive multimodal human action database. In: WACV, pp. 53–60. IEEE Computer Society (2013). http://dblp.uni-trier.de/db/conf/wacv/wacv2013.html#OfliCKVB13
Ohashi, H., Al-Naser, M.O.A., Ahmed, S., Akiyama, T., Sato, T., Nguyen, P., Nakamura, K., Dengel, A.: Augmenting wearable sensor data with physical constraint for dnn-based human-action recognition. In: Time Series Workshop. Time Series Workshop @ ICML, located at ICML 2017, August 11-11, Sydney, Australia (2017)
Okita, T., Inoue, S.: Activity recognition: Translation across sensor modalities using deep learning. In: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, UbiComp ’18, pp. 1462–1471. Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3267305.3267512. https://doi.org/10.1145/3267305.3267512
Pavllo, D., Feichtenhofer, C., Grangier, D., Auli, M.: 3d human pose estimation in video with temporal convolutions and semi-supervised training. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Takasaki, C., Takefusa, A., Nakada, H., Oguchi, M.: A study of action recognition using pose data toward distributed processing over edge and cloud. In: 2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), pp. 111–118 (2019). https://doi.org/10.1109/CloudCom.2019.00027
Takeda, S., Lago, P., Okita, T., Inoue, S.: Reduction of marker-body matching work in activity recognition using motion capture. In: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC ’19 Adjunct, pp. 835–842. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3341162.3345591. https://doi.org/10.1145/3341162.3345591
Um, T.T., Pfister, F.M.J., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., Kuliundefined, D.: Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks. In: Proceedings of the 19th ACM International Conference on Multimodal Interaction, ICMI ’17, pp. 216–220. Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3136755.3136817. https://doi.org/10.1145/3136755.3136817
Varol, G., Laptev, I., Schmid, C., Zisserman, A.: Synthetic humans for action recognition from unseen viewpoints (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Adachi, K., Lago, P., Okita, T., Inoue, S. (2021). Improvement of Human Action Recognition Using 3D Pose Estimation. In: Ahad, M.A.R., Inoue, S., Roggen, D., Fujinami, K. (eds) Activity and Behavior Computing. Smart Innovation, Systems and Technologies, vol 204. Springer, Singapore. https://doi.org/10.1007/978-981-15-8944-7_2
Download citation
DOI: https://doi.org/10.1007/978-981-15-8944-7_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-8943-0
Online ISBN: 978-981-15-8944-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)