Abstract
In this work we propose, for the first time, to improve the performance of a Hand Pose Reconstruction (HPR) technique from RGBD camera data, which is affected by self-occlusions, leveraging upon postural synergy information, i.e., a priori information on how human most commonly use and shape their hands in everyday life tasks. More specifically, in our approach, we ignore joint angle values estimated with low confidence through a vision-based HPR technique and fuse synergistic information with such incomplete measures. Preliminary experiments are reported showing the effectiveness of the proposed integration.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
Images and depth maps are captured at \(640\times 480@24\) bit and \(640\times 480@16\) bit, respectively.
- 3.
Implementation available online at: http://cvrlcode.ics.forth.gr/handtracking/.
References
Ciotti, S., et al.: A synergy-based optimally designed sensing glove for functional grasp recognition. Sensors 16(6), 811 (2016)
Sturman, D.J., et al.: A survey of glove-based input. IEEE Comput. Graphics Appl. 14(1), 30–39 (1994)
Dipietro, L., et al.: A survey of glove-based systems and their applications. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 38(4), 461–482 (2008)
Bianchi, M., et al.: Synergy-based hand pose sensing: Reconstruction enhancement. Int. J. Robot. Res. 32(4), 396–406 (2013)
Oikonomidis, I., et al.: Efficient model-based 3D tracking of hand articulations using kinect. In: British Machine Vision Conference (BMVC 2011), vol. 1, no. 2, pp. 1–11. BMVA, Dundee (2011)
Muth, J.T., et al.: Embedded 3D printing of strain sensors within highly stretchable elastomers. Adv. Mater. 26(36), 6307–6312 (2014)
Hsiao, P.-C., et al.: Data glove embedded with 9-axis imu and force sensing sensors for evaluation of hand function. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4631–4634. IEEE (2015)
Bianchi, M., et al.: On the use of postural synergies to improve human hand pose reconstruction. In: 2012 IEEE Haptics Symposium (HAPTICS), pp. 91–98. IEEE (2012)
Bianchi, M., et al.: Synergy-based optimal design of hand pose sensing. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3929–3935, October 2012
Bianchi, M., et al.: Synergy-based hand pose sensing: optimal glove design. Int. J. Robot. Res. 32(4), 407–424 (2013)
Bianchi, M., et al.: Exploiting hand kinematic synergies and wearable under-sensing for hand functional grasp recognition. In: 2014 EAI 4th International Conference on Wireless Mobile Communication and Healthcare (Mobihealth), pp. 168–171, November 2014
Bianchi, M., et al.: A multi-modal sensing glove for human manual-interaction studies. Electronics 5(3), 42 (2016)
Santello, M., et al.: Postural hand synergies for tool use. J. Neurosci. 18(23), 10 105–10 115 (1998)
Santello, M., et al.: Hand synergies: integration of robotics and neuroscience for understanding the control of biological and artificial hands. Phys. Life Rev. 17, 1–23 (2016)
Catalano, M.G., et al.: Adaptive synergies for the design and control of the Pisa/IIT softhand. Int. J. Robot. Res. 33(5), 768–782 (2014)
Matrone, G.C., et al.: Principal components analysis based control of a multi-dof underactuated prosthetic hand. J. Neuroeng. Rehabil. 7(1), 1 (2010)
Kennedy, J., et al.: Particle swarm optimization. In: International Conference on Neural Networks, vol. 4, no. 3, pp. 1942–1948. IEEE, January 1995
Sun, X., et al.: Cascaded hand pose regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 824–832 (2015)
Keskin, C., Kıraç, F., Kara, Y.E., Akarun, L.: Hand pose estimation and hand shape classification using multi-layered randomized decision forests. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 852–863. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33783-3_61
Acknowledgment
This work is supported in part by the European Research Council under the Advanced Grant SoftHands (No. ERC-291166), by the EU H2020 projects SoftPro (No. 688857) and SOMA (No. 645599), and by the EU FP7 project WEARHAP (No. 601165).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Ciotti, S. et al. (2017). Synergy-Driven Performance Enhancement of Vision-Based 3D Hand Pose Reconstruction. In: Perego, P., Andreoni, G., Rizzo, G. (eds) Wireless Mobile Communication and Healthcare. MobiHealth 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 192. Springer, Cham. https://doi.org/10.1007/978-3-319-58877-3_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-58877-3_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-58876-6
Online ISBN: 978-3-319-58877-3
eBook Packages: Computer ScienceComputer Science (R0)