Zusammenfassung
Estimation of the hand pose of a surgeon is becoming more and more important in the context of computer assisted surgery. Previous point cloud-based neural network methods for this task usually estimate offset fields to infer the 3D joint positions. Occlusions of important hand parts and inconsistencies in the point clouds, e.g. caused by uneven exposure from the depth sensor, pose a challenge to these methods. We propose to simplify the optimization problem by only estimating a weight for each point of the cloud, such that the inferred joint position is given as the weighted sum over the input points. To better capture the directional information, we define a support point set that expands the convex hull of the hand point set and enter the union of both sets as input to our network. We propose a hierarchical graph CNN, whose graph structure enables optimal information flow between the two point sets. With a mean joint error of 9.43 mm, our approach outperforms most comparable state-of-the-art methods with an average reduction by 19%, while also reducing the computational complexity.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Literatur
Hein J, Seibold M, Bogo F, Farshad M, Pollefeys M, Fürnstahl P et al. Towards markerless surgical tool and hand pose estimation. Int J Comput Assist Radiol Surg. 2021;16:799–808.
Du K, Lin X, Sun Y, Ma X. Crossinfonet: multi-task information sharing based hand pose estimation. Conf Comput Vis Pattern Recognit Workshops. 2019;2019-June:9888–97.
Oberweger M,Wohlhart P, Lepetit V. Hands deep in deep learning for hand pose estimation. CoRR. 2015;abs/1502.06807(July).
Wang Y, Sun Y, Liu Z, Sarma SE, Bronstein MM, Solomon JM. Dynamic graph CNN for learning on point clouds. CoRR. 2018;abs/1801.07829.
Chatzis T, Stergioulas A, Konstantinidis D, Dimitropoulos K, Daras P. A comprehensive study on deep learning-based 3D hand pose estimation methods. Applied Sciences. 2020;10(19).
Ge L, Ren Z, Yuan J. Point-to-point regression pointnet for 3D hand pose estimation. Computer Vision – ECCV 2018. Springer International Publishing, 2018:489–505.
Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P. Geometric deep learning: going beyond Euclidean data. CoRR. 2016.
Qi CR, Su H, Mo K, Guibas LJ. PointNet: deep learning on point sets for 3D classification and segmentation. CoRR. 2016;abs/1612.00593.
Qi CR, Yi L, Su H, Guibas LJ. PointNet++: deep hierarchical feature learning on point sets in a metric space. Adv Neural Inf Process Syst. 2017:5100–9.
Tompson J, Stein M, Lecun Y, Perlin K. Real-time continuous pose recovery of human hands using convolutional networks. ACM Transactions on Graphics. 2014;33.
Corso G, Cavalleri L, Beaini D, Liò P, Velickovic P. Principal neighbourhood aggregation for graph nets. CoRR. 2020;abs/2004.05718.
Srivastav V, Issenhuth T, Kadkhodamohammadi A, De Mathelin M, Gangi A, Padoy N. MVOR: a multi-view RGB-D operating room dataset for 2D and 3D human pose estimation. 2018.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Der/die Autor(en), exklusiv lizenziert an Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature
About this paper
Cite this paper
Hermes, N., Hansen, L., Bigalke, A., Heinrich, M.P. (2022). Support Point Sets for Improving Contactless Interaction in Geometric Learning for Hand Pose Estimation. In: Maier-Hein, K., Deserno, T.M., Handels, H., Maier, A., Palm, C., Tolxdorff, T. (eds) Bildverarbeitung für die Medizin 2022. Informatik aktuell. Springer Vieweg, Wiesbaden. https://doi.org/10.1007/978-3-658-36932-3_19
Download citation
DOI: https://doi.org/10.1007/978-3-658-36932-3_19
Published:
Publisher Name: Springer Vieweg, Wiesbaden
Print ISBN: 978-3-658-36931-6
Online ISBN: 978-3-658-36932-3
eBook Packages: Computer Science and Engineering (German Language)