Abstract
The neural networks currently outperform earlier approaches to the hand pose estimation. However, to achieve the superior results a large amount of the appropriate training data is desperately needed. But the acquisition of the real hand pose data is a time and resources consuming process. One of the possible solutions uses the synthetic training data. We introduce a method to generate synthetic depth images of the hand closely matching the real images. We extend the approach of the previous works to the modeling of the depth image data using the 3D scan of the subject’s hand and the hand pose prior given by the real data distribution. We found out that combining them with the real training data can result in a better performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
There are 52 basic hand poses in the Czech Sign Language in total.
References
Feix, T., Romero, J., Ek, C.H., Schmiedmayer, H.B., Kragic, D.: A metric for comparing the anthropomorphic motion capability of artificial hands. IEEE Trans. Robot. 29(1), 82–93 (2013)
Ge, L., Liang, H., Yuan, J., Thalmann, D.: 3D convolutional neural networks for efficient and robust hand pose estimation from single depth images. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017
Ge, Y., Liang, H., Yuan, J., Thalmann, D.: Robust 3D hand pose estimation in single depth images: from single-view CNN to multi-view CNNs. In: Proceedings of CVPR 2016, June 2016
Ivanko, D., Karpov, A.: An analysis of perspectives for using high-speed cameras in processing dynamic video information. SPIIRAS Proc. 44(1), 98–113 (2016). https://doi.org/10.15622/sp.44.7
Mueller, F., Mehta, D., Sotnychenko, O., Sridhar, S., Casas, D., Theobalt, C.: Real-time hand tracking under occlusion from an egocentric RGB-D sensor. In: The IEEE International Conference on Computer Vision (ICCV), October 2017
Oberweger, M., Lepetit, V.: Deepprior++: Improving fast and accurate 3D hand pose estimation. In: ICCV Workshops 2017, Venice, Italy, 22–29 October 2017 (2017)
Oberweger, M., Riegler, G., Wohlhart, P., Lepetit, V.: Efficiently creating 3D training data for fine hand pose estimation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016
Oberweger, M., Wohlhart, P., Lepetit, V.: Hands deep in deep learning for hand pose estimation. In: CVWW, February 2015
Sharp, T., Keskin, C., et al.: Accurate, robust, and flexible real-time hand tracking, pp. 3633–3642. ACM, April 2015
Sinha, A., Choi, C., Ramani, K.: Deephand: robust hand pose estimation by completing a matrix imputed with deep features. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016
Sun, X., Wei, Y., Liang, S., Tang, X., Sun, J.: Cascaded hand pose regression. In: CVPR 2015, June 2015
Supancic, J.S., Rogez, G., Yang, Y., Shotton, J., Ramanan, D.: Depth-based hand pose estimation: data, methods, and challenges. In: The IEEE International Conference on Computer Vision (ICCV), December 2015
Tang, D., Jin Chang, H., Tejani, A., Kim, T.K.: Latent regression forest: structured estimation of 3D articulated hand posture. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014
Tang, D., Yu, T.H., Kim, T.K.: Real-time articulated hand pose estimation using semi-supervised transductive regression forests. In: ICCV 2013. IEEE Computer Society, Washington (2013)
Tompson, J., Stein, M., Lecun, Y., Perlin, K.: Real-time continuous pose recovery of human hands using convolutional networks. ACM Trans. Graph. 33(5), 169:1–169:10 (2014)
Vodopivec, T., Lepetit, V., Peer, P.: Fine hand segmentation using convolutional neural networks. CoRR (2016)
Šarić, M.: Libhand: A library for hand articulation (2011). http://www.libhand.org/, version 0.9
Wan, C., Probst, T., Van Gool, L., Yao, A.: Crossing nets: combining GANs and VAEs with a shared latent space for hand pose estimation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017
Yuan, S., Ye, Q., Stenger, B., Jain, S., Kim, T.K.: Bighand2.2m benchmark: hand pose data set and state of the art analysis. In: CVPR, July 2017
Zimmermann, C., Brox, T.: Learning to estimate 3D hand pose from single RGB images. In: ICCV 2017 (2017)
Acknowledgments
This work is financially supported by the Ministry of Education and Science of the Russian Federation, agreement No. 14.616.21.0095 (reference RFMEFI61618-X0095) and the European Regional Development Fund under the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15 003/0000466). This work was supported by the Ministry of Education of the Czech Republic, project No. LTARF18017. Access to computing and storage facilities owned by parties and projects contributing to the National Grid Infrastructure MetaCentrum provided under the programme “Projects of Large Research, Development, and Innovations Infrastructures” (CESNET LM2015042), is greatly appreciated.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Kanis, J., Ryumin, D., Krňoul, Z. (2018). Improvements in 3D Hand Pose Estimation Using Synthetic Data. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2018. Lecture Notes in Computer Science(), vol 11097. Springer, Cham. https://doi.org/10.1007/978-3-319-99582-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-99582-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99581-6
Online ISBN: 978-3-319-99582-3
eBook Packages: Computer ScienceComputer Science (R0)