Single Fingertip Detection Using Simple Geometric Properties of the Hand Image: A Case Study for Augmented Reality in an Educational App

  • Nikolaos NomikosEmail author
  • Dimitris Kalles
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11754)


We propose a fingertip detection method suitable for portable devices’ applications where the user can interact with objects or interface elements located in the augmented space. The method has been experimentally implemented in the context of an application that uses a board containing ArUco markers [1]. The user can press virtual buttons laid on the board or drag items along predetermined paths on the board level by extending the index finger (or thumb) and placing its edge on the object of interest, while the other hand holds the device so that both the object and the hand are visible. We present brief but indicative results of our technique.


Fingertip detection Computer vision Augmented reality 



This research has been co-financed by the Operational Program “Human Resources Development, Education and Lifelong Learning” and is co-financed by the European Union (European Social Fund) and Greek national funds.


  1. 1.
    Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F., Marín-Jiménez, M.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014)CrossRefGoogle Scholar
  2. 2.
    Fukumoto, M., Tonomura, Y.: Body coupled FingerRing: wireless wearable keyboard. In: ACM SIGCHI Conference on Human factors in computing systems (CHI ‘97), New York, USA (1997)Google Scholar
  3. 3.
    Kerdvibulvech, C., Saito, H.: Vision-based guitarist fingering tracking using a bayesian classifier and particle filters. In: Mery, D., Rueda, L. (eds.) PSIVT 2007. LNCS, vol. 4872, pp. 625–638. Springer, Heidelberg (2007). Scholar
  4. 4.
    Buchmann, V., Violich, S., Billinghurst, M., Cockburn, A.: FingARtips: gesture based direct manipulation in augmented reality. In: 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (GRAPHITE ‘04). Suntec City, Singapore (2004)Google Scholar
  5. 5.
    Feng, Z., Xu, S., Zhang, X., Jin, L., Ye, Z., Yang, W.: Real-time fingertip tracking and detection using Kinect depth sensor for a new writing-in-the air system. In: 4th International Conference on Internet Multimedia Computing and Service (ICIMCS ‘12), Wuhan, China (2012)Google Scholar
  6. 6.
    Shah, S.A.H., Ahmed, A., Mahmood, I., Khurshid, K.: Hand gesture based user interface for computer using a camera and projector. In: 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia (2011)Google Scholar
  7. 7.
    Letessier, J., Bérard, F.: Visual tracking of bare fingers for interactive surfaces. In: 17th Annual ACM Symposium on User Interface Software and Technology (UIST ‘04), Santa Fe, New Mexico (2004)Google Scholar
  8. 8.
    Malik, S., Laszlo, J.: Visual touchpad: a two-handed gestural input device. In: 6th International Conference on Multimodal Interfaces (ICMI ‘04), State College, PA, USA (2004)Google Scholar
  9. 9.
    Wilson, D.A.: PlayAnywhere: a compact interactive tabletop projection-vision system. In: 18th Annual ACM Symposium on User Interface Software and Technology (UIST ‘05), Seattle, WA, USA (2005)Google Scholar
  10. 10.
    Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality: a class of displays on the reality-virtuality continuum. In: Telemanipulator and Telepresence Technologies, Boston, MA, United States (1995)Google Scholar
  11. 11.
  12. 12.
    El Kabtane, H., El Adnani, M., Sadgal, M., Mourdi, Y.: Toward an occluded augmented reality framework in E-learning platforms for practical activities. J. Eng. Sci. Technol. 13(2), 394–408 (2018)Google Scholar
  13. 13.
    Sinclair, P.A.S., Martinez, K., Millard, D.E., Weal, M.J.: Augmented reality as an interface to adaptive. New Rev. Hypermedia Multimedia 9, 117–136 (2003)CrossRefGoogle Scholar
  14. 14.
    Cai, S., Chiang, F.-K., Sun, Y., Lin, C., Lee, J.J.: Applications of augmented reality-based natural interactive learning in magnetic field instruction. Interact. Learn. Environ. 25, 778–791 (2017)CrossRefGoogle Scholar
  15. 15.
    Martin-Gonzalez, A., Chi-Poot, A., Uc-Cetina, V.: Usability evaluation of an augmented reality system for teaching Euclidean vectors. Innov. Educ. Teach. Int. 53, 627–636 (2016)CrossRefGoogle Scholar
  16. 16.
    OpenCV library.
  17. 17.
    ArUco: a minimal library for Augmented Reality applications based on OpenCV. University of Córdoba.
  18. 18.

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Hellenic Open UniversityPatraGreece

Personalised recommendations