Skip to main content
SpringerLink
Log in

Virtual Keyboard: Real-Time Finger Joints Tracking for Keystroke Detection and Recognition

  • Research Article - Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

This paper presents a finger joints tracking-based keystroke detection and recognition approach for the development of virtual keyboard. Activities of user’s hands are captured in a video sequence for the finger joints localization and tracking. Averaging background subtraction, connected component labeling and contour analysis are performed for hands region detection and finger joints localization. Probabilistic regional density-based kernel tracking estimates finger joint trajectories. Estimated trajectories are further interpreted into respective feature vectors based on spatial and pathic information. These feature vectors are fed into logic-based technique and Dynamic Bayesian Network for the classification of feature vectors leading towards the detection and recognition of keystrokes. Using actual video data set and real-time implementation, experimentation results demonstrate accurate keystroke detection and recognition for twenty eight (28) keys. Successful research results motivate for the inclusion of non-intrusive vision-based virtual keyboard in mobile devices in addition to existing tiny keypads for facilitation of the users.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Goldstein, M.; Chincholle, D.: Finger-joint gesture wearable keypad.In: 2nd Workshop on Human Computer Interaction with Mobile Devices, pp. 9–18 (1999)

  2. Rosenberg, R.: Computing without mice and keyboards: text and graphic input devices for mobile computing. PhD thesis, Department of Computer Science, University College, London (1998)

  3. Fukumoto, M.; Tonomura, Y.: Body coupled FingeRing: wireless wearable keyboard. In: ACM Conference on Human Factors in Computing Systems (1997)

  4. Evans, F.; Skiena, S.; Varshney, A.: VType: entering text in a virtual world. Submitted to Int. J. Hum.–Comput. Stud.

  5. Samsung Scurry: PCWorld article at http://www.pcworld.com/article/70568/virtual_keyboards_let_you_type_in_air.html

  6. Senseboard, Method for virtual input to mobile devices: http://www.senseboard.com (2011). Accessed 15 Dec 2011

  7. Zhang, Z.; Wu, Y.; Shan, Y.; Shafer, S.: Visual panel: virtual mouse, keyboard and 3D controller with an ordinary piece of paper. In: ACM Workshop on Perceptive User Interfaces, November 2001

  8. Virtual Keybord (VKey) by Virtual Devices, Inc.: V’Key. http://www.virtualdevices.net/products.php (2013). Accessed 18 Jul 2013

  9. i-Tech’s Virtual Keyboard at http://www.vkb-support.com

  10. Nadeeka Samanthi, W.; Chanaka, A.: VISTAKEY: a keyboard without a keyboard—a type of virtual keyboard. In: IEE Eleventh Annual Conference, Sri Lanka (2004)

  11. Huan, D.; Thierry, O.; Felix, L.: A virtual keyboard based on true-3D optical ranging. In: British Machine Vision Conference (2005)

  12. Huan, D.; Edoardo, C.: A virtual keyboard system based on multi-level feature matching. In: International Conference on Human Systems Interaction (2008)

  13. Lee, M.; Woo, W.: ARKB: 3D vision-based augmented reality keyboard. In: International Conference on Artificial Reality and Telexisitence, pp. 54–57 (2003)

  14. Bowden, R.; Zisserman, A.; Kadir, T.; Brady, M.: Vision based interpretation of natural sign languages. In: Proceedings of 3rd International Conference Computer Vision Systems, Graz, Austria, pp. 391–401, April (2003)

  15. Yang, L.; Yunde, L.: A robust hand tracking and gesture recognition method for wearable virtual interfaces and its applications. In: 3rd International Conference on Image and Graphics (ICIG’04) (2004)

  16. Luca, L.; Marco, P.: Adding gestures to ordinary mouse use: a new input modality for improved human–computer interaction. In: 14th International Conference on Image Analysis and Processing (ICIAP 2007)

  17. Micah, A.; Katie, M.: Developing a car gesture interface for use as a secondary task. In: ACM Conference on Computer Human Interaction, pp. 32–33 (2003)

  18. Xiaoming, Y.; Ming, X.: Finger identification and hand posture recognition for human–robot interaction. Image Vis. Comput. 25, 1291–1300 (2007)

    Article  Google Scholar 

  19. Habib, H.A.; Mufti, M.: Real time mono vision gesture based virtual keyboard system. IEEE Trans. Consum. Electron. 52(4) (2006)

  20. Erol, A.; Bebis, G.; Nicolescu, M.; Boyle, R.D.; Twombly, X.: Vision based hand pose estimation: a review. Comput. Vis. Image Underst. 108, 52–73 (2007)

    Article  Google Scholar 

  21. Caglar, M.B.; Lobo, N.: Open hand detection in a cluttered single imageusing finger primitives. In: Computer Vision and Pattern Recognition Workshop (CVPRW’06) (2006)

  22. Kolsch, M.; Turk, M.: Robust hand detection. In: 6th IEEE International Conference on Automatic Face and Gesture Recognition (FGR’04) (2004)

  23. Dente, E.; Bharath, A.A.; Ng, J.; Vrij, A.; Mann, S.; Bull, A.: Tracking hand and finger movements for behaviour analysis. Pattern Recognit. Lett. 27, 1797–1808 (2006)

    Article  Google Scholar 

  24. Roh, M.-C.; Huh, S.-J.; Lee, S.-W.: A virtual mouse interface based on two-layered Bayesian network. In: Workshop on Applications of Computer Vision (2009)

  25. Jean, F.; Albu, A.B.: The visual keyboard: real-time feet tracking for the control of musical meta-instruments. J. Signal Process.: Image Commun. 23, 505– 515 (2008)

    Google Scholar 

  26. Chang, F.; Chen, C.-J.; Lu, C.-J.: A linear-time component-labeling algorithm using contour tracing technique. Comput. Vis. Image Underst. 93(2), 206–220 (2004)

    Article  Google Scholar 

  27. Yilmaz, A.; Javed, O.; Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4), Article 13 (2006)

  28. Duda, R.; Hart, P.; Stork, D.: Pattern Classification, 2nd edn. Wiley-Interscience, London. ISBN 0471056693 (2000)

Download references

Author information

Authors and Affiliations

  1. Video and Image Processing Laboratory, Department of Computer Engineering, University of Engineering and Technology, Taxila, Pakistan

    Muhammad Haroon Yousaf & Hafiz Adnan Habib

Authors
  1. Muhammad Haroon Yousaf
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hafiz Adnan Habib
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Haroon Yousaf.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yousaf, M.H., Habib, H.A. Virtual Keyboard: Real-Time Finger Joints Tracking for Keystroke Detection and Recognition. Arab J Sci Eng 39, 923–934 (2014). https://doi.org/10.1007/s13369-013-0909-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-013-0909-2

Keywords

Search

Navigation