Abstract
In this paper, we propose a novel human body pose refinement method that relies on an existing single-frame pose detector and uses an optical flow algorithm in order to increase quality of output trajectories. First, a pose estimation algorithm such as OpenPose is applied and the error of keypoint position measurement is calculated. Then, the velocity of each keypoint in frame coordinate space is estimated by an optical flow algorithm, and results are merged through a Kalman filter. The resulting trajectories for a set of experimental videos were calculated and evaluated by metrics, which showed a positive impact of optical flow velocity estimations. Our algorithm may be used as a preliminary step to further joint trajectory processing, such as action recognition.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Andriluka, M., Roth, S., Schiele, B.: People-tracking-by-detection and people-detection-by-tracking. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)
Antonucci, A., Magnago, V., Palopoli, L., Fontanelli, D.: Performance assessment of a people tracker for social robots. In: 2019 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), pp. 1–6. IEEE (2019)
Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields arXiv:1812.08008 (2018)
Jalles, J.T.: Structural time series models and the Kalman filter: aconcise review (2009)
Kalman, R.E., et al.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)
Kocabas, M., Karagoz, S., Akbas, E.: MultiPoseNet: fast multi-person pose estimation using pose residual network. In: Proceedings of the 15th European Conference, Munich, Germany, 8–14 September 2018, Part XI, pp. 437–453 (2018). https://doi.org/10.1007/978-3-030-01252-626
Kroeger, T., Timofte, R., Dai, D., Van Gool, L.: Fast optical flow using dense inverse search. In: European Conference on Computer Vision, pp. 471–488. Springer (2016)
Li, X.R., Jilkov, V.P.: Survey of maneuvering target tracking. Part I. Dynamic models. IEEE Trans. Aerosp. Electron. Syst. 39(4), 1333–1364 (2003)
Lin, L., Lu, Y., Pan, Y., Chen, X.: Integrating graph partitioning and matching for trajectory analysis in video surveillance. IEEE Trans. Image Process. 21(12), 4844–4857 (2012)
Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer (2014)
Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Zhao, X., Kim, T.K.: Multiple object tracking: a literature review. arXiv preprint arXiv:14097618 (2014)
Ning, G., Zhang, Z., Huang, C., Ren, X., Wang, H., Cai, C., He, Z.: Spatially supervised recurrent convolutional neural networks for visual object tracking. In: 2017 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–4. IEEE (2017)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Xiu, Y., Li, J., Wang, H., Fang, Y., Lu, C.: Pose flow: efficient online pose tracking. In: British Machine Vision Conference (BMVC) arXiv:1802.00977 (2018)
Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: DeepFlow: large displacement optical flow with deep matching. In: Proceedings of the IEEE International Conference on Computer Vision 2013, pp. 1385–1392 (2013)
Andriluka, M., Iqbal, U., Insafutdinov, E., Pishchulin, L., Milan, A., Gall, J., Schiele, B.: Posetrack: a benchmark for human pose estimation and tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5167–5176 (2018)
Xiao, B., Wu, H., Wei, Y.: Simple baselines for human pose estimation and tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 466–481 (2018)
Acknowledgment
Authors thank XIMEA corp. CEO Max Larin for XIMEA cameras during the summer of 2019 for data collection and experiments with the adaptive video surveillance system.
Also, we would like to thank the MRTech directors Igor Dvoretskiy and Aleksandr Kiselev for the numerous and extensive discussions about video system architecture and video processing solutions and Fyodor Serzhenko from FastVideo corp. for comments about GPU usage for the on-board video processing.
Boris Karapetyan provided us with the skeleton visualization software components.
The reported study was funded by the RFBR, project number 19-29-09090.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Khelvas, A., Gilya-Zetinov, A., Konyagin, E., Demyanova, D., Sorokin, P., Khafizov, R. (2021). Improved 2D Human Pose Tracking Using Optical Flow Analysis. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Intelligent Systems and Applications. IntelliSys 2020. Advances in Intelligent Systems and Computing, vol 1251. Springer, Cham. https://doi.org/10.1007/978-3-030-55187-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-55187-2_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-55186-5
Online ISBN: 978-3-030-55187-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)