Advertisement

An Improved Two-Stage Multi-person Pose Estimation Model

  • Sutong WangEmail author
  • Yanzhang Wang
  • Xuehua Wang
  • Xin Ye
  • Huaiming Li
  • Xuelong Chen
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1103)

Abstract

Generally, multi-person pose estimation plays a crucial role in behavior recognition in images and videos. Previously, pose estimation of a single person is popular and achieves high prediction accuracy with the development of deep learning. However, pose estimation of multi-person remains to be a huge challenge and cannot achieve the same effect as that of a single person. It mainly results from the rare, missing or incorrect location detection and overlap of pose, which are usually caused by incomplete person identification. Therefore, we propose an improved two-stage multi-person pose estimation model (ITMPE) to further improve the performance of multi-person pose estimation. The first stage, Mask R-CNN is used for person identification. The second stage, processed images or videos with identified people only are fed into OpenPose model for multi-person pose estimation. The comparative experiments show that our proposed model achieves a significant improvement than original model. Our proposed model reduces the MSE, MAE by around 27.38%, 21.57% and increases R2, Mean values by 49.80% and 96.91% on average, respectively. The improvement in person identification and misclassification are shown in our comparison images. More people are captured and given the pose estimation, which directly affect the performance of behavior recognition.

Keywords

Pose estimation Multi-person Instance segmentation Complex scenarios 

References

  1. 1.
    Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 (2017)Google Scholar
  2. 2.
    Zhu, X., Jiang, Y., Luo, Z.: Multi-person pose estimation for PoseTrack with enhanced part affinity fields. In: ICCVW (2017)Google Scholar
  3. 3.
    He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: Proceedings of IEEE International Conference on Computational Vision (2017)Google Scholar
  4. 4.
    Yang, Y., Ramanan, D.: Articulated human detection with flexible mixtures of parts. TPAMI 35, 2878–2890 (2013)CrossRefGoogle Scholar
  5. 5.
    Dantone, M., Gall, J., Leistner, C., Van Gool, L.: Human pose estimation using body parts dependent joint regressors. In: CVPR (2013)Google Scholar
  6. 6.
    Chu, X., Yang, W., Ouyang, W., Ma, C., Yuille, A.L., Wang, X.: Multi-context attention for human pose estimation. In: CVPR (2017)Google Scholar
  7. 7.
    Tang, W., Yu, P., Wu, Y.: Deeply learned compositional models for human pose estimation. In: ECCV (2018)Google Scholar
  8. 8.
    Girshick, R.: Fast R-CNN. In: Proceedings of IEEE International Conference on Computer Vision (2015).  https://doi.org/10.1109/iccv.2015.169
  9. 9.
    Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 (2017)Google Scholar
  10. 10.
    Simonyan, K., Zisserman, A.: VGG-16. arXiv Preprint (2014)Google Scholar
  11. 11.
    Wei, S.-E., Ramakrishna, V., Kanade, T., Sheikh, Y.: Convolutional pose machines. In: CVPR (2016)Google Scholar
  12. 12.
    Chen, K., Loy, C.C., Gong, S., Xiang, T.: Feature Mining for Localised Crowd Counting (2012)Google Scholar
  13. 13.
    Idrees, H., Saleemi, I., Seibert, C., Shah, M.: Multi-source multi-scale counting in extremely dense crowd images. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2013)Google Scholar
  14. 14.
    Zhang, Y., Zhou, D., Chen, S., Gao, S., Ma, Y.: Single-image crowd counting via multi-column convolutional neural network. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  15. 15.
    Panteleris, P., Oikonomidis, I., Argyros, A.: Using a single RGB frame for real time 3D hand pose estimation in the wild. In: Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018 (2018)Google Scholar
  16. 16.
    Li, M., Zhou, Z., Liu, X.: Multi-person pose estimation using bounding box constraint and LSTM. IEEE Trans. Multimedia (2019)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Institution of Information and Decision TechnologyDalian University of TechnologyDalianChina

Personalised recommendations