Multimedia Tools and Applications

, Volume 76, Issue 8, pp 10919–10937 | Cite as

Three-dimensional image-based human pose recovery with hypergraph regularized autoencoders

  • Chaoqun Hong
  • Jun Yu
  • You Jane
  • Zhiwen Yu
  • Xuhui Chen


Three-Dimensional image-based human pose recovery tries to retrieves 3D poses with 2D image. Therefore, one of the key problem is how to represent 2D images. However, semantic gap exists for current feature extractors, which limits recovery performance. In this paper, we propose a novel feature extractor with deep neural network. It is based on denoising autoencoders and improves previous autoencoders by adopting locality preserved restriction. To impose this restriction, we introduce manifold regularization with hypergraph learning. Hypergraph Laplacian matrix is constructed with patch alignment framework. In this way, an automatic feature extractor for images is achieved. Experimental results on three datasets show that the recovery error can be reduced by 10 % to 20 %, which demonstrates the effectiveness of the proposed method.


3D human pose recovery Autoencoders Manifold learning Hypergraph Patch alignment framework 


  1. 1.
    Agarwal A, Triggs B (2006) Recovering 3d human pose from monocular images. IEEE Trans Pattern Anal Mach Intell 28(1):44–58CrossRefGoogle Scholar
  2. 2.
    Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522CrossRefGoogle Scholar
  3. 3.
    Brand M (1999) Shadow puppetry. In: Proc IEEE International Conference of Computer Vision, IEEE Press, pp 1237–1244Google Scholar
  4. 4.
    Chen C, Yang Y, Nie F, Odobez JM (2011) 3d human pose recovery from image by efficient visual feature selection. Comput Vis Image Underst 115(3):290–299CrossRefGoogle Scholar
  5. 5.
    Chen M, Weinberger KQ, Sha F, Bengio Y (2014) Marginalized denoising auto-encoders for nonlinear representations. In: IEEE International Conference on Machine Learning, IEEE, pp 1476–1484Google Scholar
  6. 6.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proc IEEE International Conference on Computer Vision and Pattern Recognition, IEEE Press, pp 886–893Google Scholar
  7. 7.
    Fan X, Zheng K, Lin Y, Wang S (2015) Combining local appearance and holistic view: Dual-source deep neural networks for human pose estimation. In: IEEE International Conference on Computer Vision and Pattern Recognition, IEEEGoogle Scholar
  8. 8.
    Gong C, Fu K, Loza A, Wu Q, Liu J, Yang J (2014) Pagerank tracker: From ranking to tracking. IEEE Transactions on Cybernetics 44(6):882–893CrossRefGoogle Scholar
  9. 9.
    Hinton G E, Osindero S, Teh Y W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Hong C, Yu J, Tao D, Wang M (2015a) Image-based three-dimensional human pose recovery by multiview locality-sensitive sparse retrieval. IEEE Trans Ind Electron 62(6):3742–3751Google Scholar
  11. 11.
    Hong C, Yu J, Wan J, Tao D, Wang M (2015b) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24(12):5659–5670MathSciNetCrossRefGoogle Scholar
  12. 12.
    Howe N, Leventon M, Freeman W (1999) Bayesian reconstruction of 3d human motion from single-camera video. In: Neural Information Processing Systems, pp 820–826Google Scholar
  13. 13.
    Ionescu C, Papava D, Olaru V, Sminchisescu C (2014) Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Trans Pattern Anal Mach Intell 36(7):1325–1339CrossRefGoogle Scholar
  14. 14.
    Karasuyama M, Mamitsuka H (2013) Manifold-based similarity adaptation for label propagation. In: Advances in Neural Information Processing Systems, MIT Press, pp 1547–1555Google Scholar
  15. 15.
    Lee H, Grosse R, Ranganath R, Ng AY (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: IEEE Conference on Machine Learning, IEEE, p 609C616Google Scholar
  16. 16.
    Liu L, Shao L, Li X (2013) Building holistic descriptors for scene recognition: A multi-objective genetic programming approach. In: ACM International Conference on Multimedia, ACM, p 997C1006Google Scholar
  17. 17.
    Mori G, Malik J (2002) Estimating human body configurations using shape context matching. In: European Conference of Computer Vision, vol 2, pp 666–680Google Scholar
  18. 18.
    Mori G, Belongie S, Malik J (2005) Efficient shape matching using shape contexts. IEEE Trans Pattern Anal Mach Intell 27(11):1832–1837CrossRefMATHGoogle Scholar
  19. 19.
    Rosales R, Sclaroff S (2000) Inferring body pose without tracking body parts. In: Proc. IEEE International Conference of Computer Vision and Pattern Recognition, IEEE Press, vol 2, pp 721–727Google Scholar
  20. 20.
    Scovanner P, Ali S, Shah M (2007) A 3-dimensional sift descriptor and its application to action recognition. In: ACM Multimedia, ACM, pp 357–360Google Scholar
  21. 21.
    Shakhnarovich G, Viola P, Darrell T (2003) Fast pose estimation with parameter sensitive hashing. In: Proc. IEEE International Conference of Computer Vision, IEEE Press, vol 2, pp 750–757Google Scholar
  22. 22.
    Sigal L, Balan A O, Black M J (2010) Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int J Comput Vis 87(1-2):4–27CrossRefGoogle Scholar
  23. 23.
    Simonyan K, Zisserman A (2014) Two-stream convolutional networks for action recognition in videos. In: Advances in Neural Information Processing Systems, MIT Press, pp 568–576Google Scholar
  24. 24.
    Song M, Tao D, Huang X, Chen C, Bu J (2012) Three-dimensional face reconstruction from a single image by a coupled rbf network. IEEE Trans Image Process 21(5):2887–2897MathSciNetCrossRefGoogle Scholar
  25. 25.
    Song M, Tao D, Sun S, Chen C, Bu J (2013) Joint sparse learning for 3-d facial expression generation. IEEE Trans Image Process 22(8):3283–3295CrossRefGoogle Scholar
  26. 26.
    Song M, Tao D, Sun S, Chen C, Maybank S (2014) Robust 3d face landmark localization based on local coordinate coding. IEEE Trans Image Process 23 (12):5108–5122MathSciNetCrossRefGoogle Scholar
  27. 27.
    Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958MathSciNetMATHGoogle Scholar
  28. 28.
    Toshev A, Szegedy C (2014) Deeppose: Human pose estimation via deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEEGoogle Scholar
  29. 29.
    Yang M, Qiu G, Huang J, Elliman D (2006) Near-duplicate image recognition and content-based image retrieval using adaptive hierarchical geometric centroids. In: Proc IEEE International Conference on Pattern Recognition, IEEE Press, pp 958–961Google Scholar
  30. 30.
    Yoshua B (2009) Learning deep architectures for ai. Foundations and Trends in Machine Learning 2(1):1–127CrossRefMATHGoogle Scholar
  31. 31.
    Yu J, Tao D (2013) Modern machine learning techniques and their applications in Cartoon Animation research. Wiley-IEEE Press, New JorkCrossRefGoogle Scholar
  32. 32.
    Yu J, Hong R, Wang M, You J (2014) Image clustering based on sparse patch alignment framework. Pattern Recogn 47(11):3512–3519CrossRefGoogle Scholar
  33. 33.
    Yu J (2015) Human pose recovery by supervised spectral embedding. Neurocomputing p. doi: 10.1016/j.neucom.2015.04.005
  34. 34.
    Yuan Y (2015) Scene recognition by manifold regularized deep learning architecture. IEEE Transactions on Neural Networks and Learning Systems, Lu XGoogle Scholar
  35. 35.
    Zhang T, Tao D, Li X, Yang J (2009) Patch alignment for dimensionality reduction. IEEE Trans Knowl Data Eng 21:1299–1313CrossRefGoogle Scholar
  36. 36.
    Zhou D, Huang J, Scholkopf B (2007) Learning with hypergraphs: Clustering, classification, and embedding. In: Advances in Neural Information Processing Systems, MIT Press, vol 19, pp 1601–1608Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Chaoqun Hong
    • 1
  • Jun Yu
    • 2
    • 3
  • You Jane
    • 4
  • Zhiwen Yu
    • 5
  • Xuhui Chen
    • 1
  1. 1.School of Computer Science and Information EngineeringXiamen University of TechnologyXiamenChina
  2. 2.School of Computer Science and TechnologyHangzhou Dianzi UniversityHangzhouChina
  3. 3.Key Laboratory of Complex Systems Modeling and SimulationMinistry of Education, Hangzhou Dianzi UniversityHangzhouPeople’s Republic of China
  4. 4.Department of ComputingThe Hong Kong Polytechnic UniversityHung HomHong Kong
  5. 5.School of Computer Science and EngineeringSouth China University of TechnologyGuangzhouChina

Personalised recommendations