Abstract
Human–robot interaction requires a robust estimate of human motion in real time. This work presents a fusion algorithm for joint center positions tracking from multiple depth cameras to improve human motion analysis accuracy. The main contribution is the proposed algorithm based on body tracking measurement fusion with an extended Kalman filter and anthropomorphic constraints, independent of sensors. As an illustration of the use of this algorithm, this paper presents the direct comparison of joint center positions estimated with a reference stereophotogrammetric system and the ones estimated with the new Kinect 3 (Azure Kinect) sensor and its older version the Kinect 2 (Kinect for Windows). The experiment was made in two parts, one for each model of Kinect, by comparing raw and merging body tracking data of a two-sided Kinect with the proposed algorithm. The proposed approach improves body tracker data for Kinect 3 which does not have the same characteristics as Kinect 2. This study shows also the importance of defining good heuristics to merge data depending on how the body tracking works. Thus, with proper heuristics, the joint center position estimates are improved by at least 14.6 %. Finally, we propose an additional comparison between Kinect 2 and Kinect 3 exhibiting the pros and cons of the two sensors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Microsoft Kinect—Windows app development. [Online]. Available: https://developer.microsoft.com/en-us/windows/kinect
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., Sheikh, Y.: OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields (2018). arXiv:1812.08008 [cs]
Moon, S., Park, Y., Ko, D.W., Suh, I.H.: Multiple kinect sensor fusion for human skeleton tracking using kalman filtering. Int. J. Adv. Robot. Syst. 13(2), 65 (2016)
Feng, S., Murray-Smith, R.: Fusing Kinect sensor and inertial sensors with multi-rate Kalman filter. In: IET Conference on Data Fusion Target Tracking 2014: Algorithms and Applications (DF TT 2014), pp. 1–8 (2014)
Colombel, J., Bonnet, V., Daney, D., Dumas, R., Seilles, A., Charpillet, F.: Physically consistent whole-body kinematics assessment based on an rgb-d sensor. Application to simple rehabilitation exercises. Sensors 20(10), 2848 (2020). Number: 10 Publisher: Multidisciplinary Digital Publishing Institute. [Online]. Available: https://www.mdpi.com/1424-8220/20/10/2848
Wang, Q., Kurillo, G., Ofli, F., Bajcsy, R.: Evaluation of pose tracking accuracy in the first and second generations of microsoft kinect. In: 2015 International Conference on Healthcare Informatics, pp. 380–389 (2015)
Pagliari, D., Pinto, L.: Calibration of kinect for xbox one and comparison between the two generations of microsoft sensors. Sensors 15(11), 27569–27589 (2015)
Davis, R.B., Õunpuu, S., Tyburski, D., Gage, J.R.: A gait analysis data collection and reduction technique. Hum. Mov. Sci. 10(5), 575–587 (1991)
Bell, A.L., Brand, R.A., Pedersen, D.R.: Prediction of hip joint centre location from external landmarks. Hum. Mov. Sci. 8(1), 3–16 (1989)
Rab, G., Petuskey, K., Bagley, A.: A method for determination of upper extremity kinematics. Gait Posture 15(2), 113–119 (2002)
Wu, G., Siegler, S., Allard, P., Kirtley, C., Leardini, A., Rosenbaum, D., Whittle, M., D’Lima, D.D., Cristofolini, L., Witte, H., Schmid, O., Stokes, I.: Standardization and Terminology Committee of the International Society of Biomechanics, “ISB recommendation on definitions of joint coordinate system of various joints for the reporting of human joint motion–part I: ankle, hip, and spine. International society of biomechanics. J. Biomech. 35(4), 543–548 (2002)
Wu, G., van der Helm, F.C.T., (DirkJan) Veeger, H.E.J., Makhsous, M., Van Roy, P., Anglin, C., Nagels, J., Karduna, A.R., McQuade, K., Wang, X., Werner, F.W., Buchholz, B.: ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion–Part II: shoulder, elbow, wrist and hand. J. Biomech. 38(5), 981–992 (2005)
Khalil, W., Creusot, D.: SYMORO+: a system for the symbolic modelling of robots. Robotica 15(2), 153–161 (1997)
Roecker, J.A., McGillem, C.D.: Comparison of two-sensor tracking methods based on state vector fusion and measurement fusion. IEEE Trans. Aerosp. Electron. Syst. 24(4), 447–449 (1988)
Gan, Q., Harris, C.J.: Comparison of two measurement fusion methods for Kalman-filter-based multisensor data fusion. IEEE Trans. Aerosp. Electron. Syst. 37(1), 273–279 (2001)
Yeung, K.-Y., Kwok, T.H., Wang, C.: Improved skeleton tracking by duplex kinects: a practical approach for real-time applications. J. Comput. Inf. Sci. Eng. 13, 041007 (2013)
Otto, M.M., Agethen, P., Geiselhart, F., Rietzler, M., Gaisbauer, F., Rukzio, E.: Presenting a holistic framework for scalable, marker-less motion capturing: skeletal tracking performance analysis, sensor fusion algorithms and usage in automotive industry. J. Virtual Reality Broadcast. 13(3) (2017)
Haralick, R.M., Joo, H., Lee, C., Zhuang, X., Vaidya, V.G., Kim, M.B.: Pose estimation from corresponding point data. IEEE Trans. Syst. Man Cybern. 19(6), 1426–1446 (1989)
Tripathy, S.R., Chakravarty, K., Sinha, A.: Constrained particle filter for improving kinect based measurements. In: 2018 26th European Signal Processing Conference (EUSIPCO), pp. 306–310 (2018)
Skals, S., Rasmussen, K.P., Bendtsen, K.M., Yang, J., Andersen, M.S.: A musculoskeletal model driven by dual microsoft kinect sensor data. Multibody Syst. Dyn. 41(4), 297–316 (2017)
Naeemabadi, M., Dinesen, B., Andersen, O.K., Hansen, J.: Investigating the impact of a motion capture system on Microsoft Kinect v2 recordings: a caution for using the technologies together. PLOS ONE 13(9), e0204052 (2018)
Banos, O., Calatroni, A., Damas, M., Pomares, H., Rojas, I., Sagha, H., del, J., Millán, R., Troster, G., Chavarriaga, R., Roggen, D.: Kinect=IMU? Learning MIMO signal mappings to automatically translate activity recognition systems across sensor modalities. In: 2012 16th International Symposium on Wearable Computers, pp. 92–99 (2012). iSSN: 2376-8541
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Colombel, J., Daney, D., Bonnet, V., Charpillet, F. (2021). Markerless 3D Human Pose Tracking in the Wild with Fusion of Multiple Depth Cameras: Comparative Experimental Study with Kinect 2 and 3. In: Ahad, M.A.R., Inoue, S., Roggen, D., Fujinami, K. (eds) Activity and Behavior Computing. Smart Innovation, Systems and Technologies, vol 204. Springer, Singapore. https://doi.org/10.1007/978-981-15-8944-7_8
Download citation
DOI: https://doi.org/10.1007/978-981-15-8944-7_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-8943-0
Online ISBN: 978-981-15-8944-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)