Advertisement

Model Based Augmentation and Testing of an Annotated Hand Pose Dataset

  • Richárd Bellon
  • Younggeon Choi
  • Nikoletta Ekker
  • Vincent Lepetit
  • L. Mike Olasz
  • Daniel Sonntag
  • Zoltán TősérEmail author
  • Kyounghwan Yoo
  • András Lőrincz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9904)

Abstract

Recent advances of deep learning technology enable one to train complex input-output mappings, provided that a high quality training set is available. In this paper, we show how to extend an existing dataset of depth maps of hand annotated with the corresponding 3D hand poses by fitting a 3D hand model to smart glove-based annotations and generating new hand views. We make available our code and the generated data. Based on the present procedure and our previous results, we suggest a pipeline for creating high quality data.

Notes

Acknowledgments

This work was supported by the EIT Digital grant (Grant No. 16257).

References

  1. 1.
    Bengio, Y., Goodfellow, I.J., Courville, A.: Deep Learning. MIT Press, Cambridge (2015, in preparation). http://www.iro.umontreal.ca/~bengioy/dlbook
  2. 2.
    Bray, M., Koller-Meier, E., Van Gool, L.: Smart particle filtering for 3D hand tracking. In: Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 675–680. IEEE (2004)Google Scholar
  3. 3.
    Cho, M., Sun, J., Duchenne, O., Ponce, J.: Finding matches in a haystack: a max-pooling strategy for graph matching in the presence of outliers. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2083–2090 (2014)Google Scholar
  4. 4.
    Cottrell, G.W., Munro, P., Zipser, D.: Learning internal representations from gray-scale images: an example of extensional programming. In: Ninth Annual Conference of the Cognitive Science Society, pp. 462–473 (1987)Google Scholar
  5. 5.
    Cuturi, M.: Fast global alignment kernels. In: Proceedings of the 28th International Conference on Machine Learning (ICML 2011), pp. 929–936 (2011)Google Scholar
  6. 6.
    Dosovitskiy, A., Tobias Springenberg, J., Brox, T.: Learning to generate chairs with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1538–1546 (2015)Google Scholar
  7. 7.
    Fu, J., Wu, Y., Mei, T., Wang, J., Lu, H., Rui, Y.: Relaxing from vocabulary: robust weakly-supervised deep learning for vocabulary-free image tagging. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1985–1993 (2015)Google Scholar
  8. 8.
    Han, F., Reily, B., Hoff, W., Zhang, H.: Space-time representation of people based on 3D skeletal data: a review. arXiv preprint arXiv:1601.01006 (2016)
  9. 9.
    Jeni, L.A., Lőrincz, A., Szabó, Z., Cohn, J.F., Kanade, T.: Spatio-temporal event classification using time-series kernel based structured sparsity. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 135–150. Springer, Heidelberg (2014)Google Scholar
  10. 10.
    Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
  11. 11.
    Kohonen, T., Lehtio, P., Oja, E., Kortekangas, A., Makisara, K.: Demonstration of pattern processing properties of the optimal associative mappings. In: Proceedings of the International Conference on Cybernetics and Society (1977)Google Scholar
  12. 12.
    Oberweger, M., Riegler, G., Wohlhart, P., Lepetit, V.: Efficiently creating 3D training data for fine hand pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016, accepted)Google Scholar
  13. 13.
    Oberweger, M., Wohlhart, P., Lepetit, V.: Hands deep in deep learning for hand pose estimation. In: Proceedings Computer Vision Winter Workshop (CVWW) (2015)Google Scholar
  14. 14.
    Oberweger, M., Wohlhart, P., Lepetit, V.: Training a feedback loop for hand pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3316–3324 (2015)Google Scholar
  15. 15.
    Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Tracking the articulated motion of two strongly interacting hands. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1862–1869. IEEE (2012)Google Scholar
  16. 16.
    Palotai, Z., Lang, M., Sarkany, A., Toser, Z., Sonntag, D., Toyama, T., Lorincz, A.: Labelmovie: semi-supervised machine annotation tool with quality assurance and crowd-sourcing options for videos. In: 2014 12th International Workshop on Content-Based Multimedia Indexing (CBMI), pp. 1–4. IEEE (2014)Google Scholar
  17. 17.
    Rasmus, A., Berglund, M., Honkala, M., Valpola, H., Raiko, T.: Semi-supervised learning with ladder networks. In: Advances in Neural Information Processing Systems. pp. 3532–3540 (2015)Google Scholar
  18. 18.
    Riegler, G., Ferstl, D., Rüther, M., Bischof, H.: A framework for articulated hand pose estimation and evaluation. In: Paulsen, R.R., Pedersen, K.S. (eds.) SCIA 2015. LNCS, vol. 9127, pp. 41–52. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  19. 19.
    Rogez, G., Supancic, J.S., Ramanan, D.: Understanding everyday hands in action from RGB-D images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3889–3897 (2015)Google Scholar
  20. 20.
    Šaric, M.: Libhand: a library for hand articulation. Version 0.9 (2011)Google Scholar
  21. 21.
    Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)CrossRefGoogle Scholar
  22. 22.
    Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., et al.: Accurate, robust, and flexible real-time hand tracking. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 3633–3642. ACM (2015)Google Scholar
  23. 23.
    Shin, J.H., Kim, M.Y., Lee, J.Y., Jeon, Y.J., Kim, S., Lee, S., Seo, B., Choi, Y.: Effects of virtual reality-based rehabilitation on distal upper extremity function and health-related quality of life: a single-blinded, randomized controlled trial. J. Neuroeng. Rehabil. 13(1), 1 (2016)CrossRefGoogle Scholar
  24. 24.
    Sukhbaatar, S., Bruna, J., Paluri, M., Bourdev, L., Fergus, R.: Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2080 (2014)
  25. 25.
    Supancic, J.S., Rogez, G., Yang, Y., Shotton, J., Ramanan, D.: Depth-based hand pose estimation: data, methods, and challenges. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1868–1876 (2015)Google Scholar
  26. 26.
    Tang, D., Chang, H., Tejani, A., Kim, T.K.: Latent regression forest: structured estimation of 3D articulated hand posture. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3786–3793 (2014)Google Scholar
  27. 27.
    Tompson, J., Stein, M., Lecun, Y., Perlin, K.: Real-time continuous pose recovery of human hands using convolutional networks. ACM Trans. Graph. (TOG) 33(5), 169 (2014)CrossRefGoogle Scholar
  28. 28.
    Xu, C., Cheng, L.: Efficient hand pose estimation from a single depth image. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3456–3462 (2013)Google Scholar
  29. 29.
    Zhu, X., Lei, Z., Yan, J., Yi, D., Li, S.Z.: High-fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 787–796 (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Richárd Bellon
    • 1
  • Younggeon Choi
    • 2
  • Nikoletta Ekker
    • 1
  • Vincent Lepetit
    • 3
  • L. Mike Olasz
    • 1
  • Daniel Sonntag
    • 4
  • Zoltán Tősér
    • 1
    Email author
  • Kyounghwan Yoo
    • 2
    • 5
  • András Lőrincz
    • 1
  1. 1.Faculty of InformaticsEötvös Loránd UniversityBudapestHungary
  2. 2.Department of Applied Computer EngineeringDankook UniversityYong-inKorea
  3. 3.Institute for Computer Vision and GraphicsGraz University of TechnologyGrazAustria
  4. 4.German Research Center for Artificial IntelligenceSaarbrückenGermany
  5. 5.Neofect Co., Ltd.Yong-inKorea

Personalised recommendations