A New Approach for Hand Augmentation Based on Patch Modelling
In this paper, a novel patch-based modelling approach is proposed to augment the 3D models over the non-planar structure of hand. We present a robust real-time framework which first segment the hand using skin color information. To offer the subjects flexible interface (i.e., no restriction of covered arm), segmented skin-region is refined through distance descriptor which gives the hand’s palm. Next, we measure the geometrical components from the hand’s structure and compute the hand’s curvature to detect the fingertips. The palm centroid and detected fingertips contribute to the patch-based modelling for which optimal path is derived for every detected fingertip. Further, we compute the patches using the path points for every two neighbor detected fingers and then the camera poses are estimated for these patches. These different camera poses are finally combined to generate a single camera pose, and then 3D models are augmented on non-planar hand geometry. The experimental results show that our proposed approach is capable to detect hand, palm centroid and fingertips, as well as integrate the generated patches for augmented reality application in real situations which proves its applicability and usability in the domain of Human Computer Interaction.
KeywordsAugmented Reality Hand and Fingertip Detection Pose Estimation
Unable to display preview. Download preview PDF.
- 3.Buchmann, V., Violich, S., Billinghurst, M., Cockburn, A.: Fingartips: Gesture based direct manipulation in augmented reality. In: 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (GRAPHITE 2004), pp. 212–221 (2004)Google Scholar
- 5.Chun, J.C., Lee, B.S.: Dynamic manipulation of a virtual object in marker-less ar system based on both human hands. Transactions on Internet and Information Systems 4(4), 618–632 (2010)Google Scholar
- 8.Lee, B.S., Chun, J.C.: Interactive manipulation of augmented objects in marker-less ar using vision-based hand mouse. In: International Conference on Information Technology (ITNG), pp. 398–403 (2010)Google Scholar
- 9.Lee, T., Hoellerer, T.: Handy ar: Markerless inspection of augmented reality objects using fingertip tracking. In: ISWC 2007, pp. 83–90 (2007)Google Scholar
- 10.Lee, T., Hoellerer, T.: Hybrid feature tracking and user interaction for markerless augmented reality. In: IEEE Virtual Reality, pp. 145–152 (2008)Google Scholar
- 12.Ng, K.P., Tan, G.Y., Wong, Y.P.: Vision-based hand detection for registration of virtual objects in augmented reality. International Journal of Future Computer and Communication 2(5), 423–427 (2013)Google Scholar
- 13.Noell, T., Pagani, A., Stricker, D.: Markerless camera pose estimation - an overview. In: Visualization of Large and Unstructured Data Sets - Applications in Geospatial Planning, Modeling and Engineering. 4, vol. 19, pp. 45–54 (2011)Google Scholar
- 14.Radkowski, R., Stritzke, C.: Interactive hand gesture-based assembly for augmented reality applications. In: Proceedings of the 2012 International Conference on Advances in Computer-Human Interactions, pp. 303–308. IEEE (2012)Google Scholar