Skip to main content
SpringerLink
Log in

Real-time Kinect-based air-writing system with a novel analytical classifier

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

Air-writing is an attractive method of interaction between human and machine due to lack of any interface device on the user side. After removing existing limitations and solving the current challenges, it can be used in many applications in the future. In this paper, using the Kinect depth and color images, an air-writing system is proposed to identify single characters such as digits or letters and connected characters such as numbers or words. In this system, automatic clustering, slope variations detection, and a novel analytical classification are proposed as new approaches to eliminate noise in the trajectory from the depth image and hand segmentation, to extract the feature vector, and to identify the character from the feature vector, respectively. Experimental results show that the proposed system can successfully identify single characters and connected characters with the average recognition rate of 97%. It provides a better result than other similar approaches proposed in the literature. In the proposed system, the character recognition time is quite low, about 3 ms, because of using a novel analytical classifier. Evaluation of 4 classifiers shows that the proposed classifier has a higher speed and precision than the SVM, HMM, and K-nearest neighbors classifiers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Palmondon, R., Srihari, S.N.: On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63–84 (2000)

    Article  Google Scholar 

  2. Saini, R., Kumar, P., Roy, P.P., Dogra, D.P.: A novel framework of continuous human-activity recognition using kinect. Neurocomputing 311, 99–111 (2018)

    Article  Google Scholar 

  3. Kumar, P., Saini, R., Behera, S.K., Dogra, D.P.: Real-time recognition of sign language gesture and air-writing using leap motion. In: 2017 Fifteenth IAPR International Conference on Machine Vision Applications, Nagoya, Japan (2017)

  4. Khan, N.A., Khan, S.M., Abdullah, M., Kanji, S.J., Iltifat, U.: Use hand gesture to write in air recognize with computer vision. IJCSNS Int. J. Comput. Sci. Netw. Secur. 17(5), 51–55 (2017)

    Google Scholar 

  5. Agrawal, S., Constandache, L., Gaonkar, S., Roy, R., Caves, K., Deruyter, F.: Using mobile phones to write in air. In: MobiSys’11 Proceeding of the 9th International Conference on Mobile Systems, ACM (2013)

  6. Xu, S., Xue, Y.: Air-writing characters modelling and recognition on modified CHMM. 2016 IEEE International Conference on Systems, Man and Cybernetics (SMC) (2016)

  7. Beg, S., Khan, M.F., Baig, F.: Text writing in air. J. Inf. Display 14(4), 137–148 (2013)

    Article  Google Scholar 

  8. Amma, C., Georgi, M., Schultz, T.: Air writing: a wearable handwriting recognition system. Pers. Ubiquitous Comput. 18(1), 191–203 (2014)

    Article  Google Scholar 

  9. Kumar, P., Verma, J., Prasad, S.: A wearable real-time device for human–computer interaction. Int. J. Adv. Sci. Technol. 43, 15–26 (2012)

    Google Scholar 

  10. Patil, S., Kim, D., Park, S., Chai, Y.: Handwriting recognition in free space using WIMU-based hand motion analysis. Hindawi J. Sens. 2016, 1–10 (2016)

    Article  Google Scholar 

  11. Islam, R., Mahmud, H., Hasan, M.K., Rubaiyeat, H.A.: Alphabet recognition in air writing using depth information. ACHI. ISBN: 978-1-61208-468-8 (2016)

  12. Chen, M., Alregib, G., Juang, B.H.: Air-writing recognition, Part 1: modeling and recognition of characters, words and connecting motions. IEEE Trans. Hum. Mach. Syst. 46(3), 403–413 (2016)

    Article  Google Scholar 

  13. Jin, X.J., Feng, Q., Hou, X., Liu, C.L.: Visual gesture character string recognition by classification-based segmentation with stroke deletion. In: Second IAPR Asian Conference on Pattern Recognition, ACPR (2011)

  14. Aggarwal, R., Swetha, S.: Online handwriting recognition using depth sensors. In: ICDAR, 13th International Conference (2015)

  15. Murata, T., Shin, J.: Hand gesture and character recognition based on kinect sensor. Int. J. Distrib. Sens. Netw. 10, 278460 (2014)

    Article  Google Scholar 

  16. Deepa, D., Dharmalingam, R.: Feature and processing of recognition of characters, words & connecting motions. IJARIIT 3(2), 699–704 (2017)

    Google Scholar 

  17. Rautaray, S.S., Agarwal, A.: Real time hand gesture recognition system for dynamic applications. Int. J. UbiComp. 3, 21 (2012)

    Article  Google Scholar 

  18. Froba, B., Ernst, A.: Face detection with the modified census transform. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition (2004)

  19. Conseil, S., Bourennane, S.: Comparison of Fourier descriptors and Hu moments for hand posture recognition. In: 15th European Signal Processing Conference (2007)

  20. Zhang, D., Lu, G.: A comparative study of Fourier descriptors for shape representation and retrieval. In: 5th Asian Conference on Computer Vision (ACCV), Melbourne, Australia (2002)

  21. Lockton, R.: Hand Gesture Recognition Using Computer Vision. Oxford University Press, Oxford (2002)

    Google Scholar 

  22. YanuTara, R., Santosa, P., Adji, T.: Sign language recognition in robot teleoperation using centroid distance Fourier descriptors. Int. J. Comput. Appl. 48(2), 8–12 (2012)

    Google Scholar 

  23. Ren, Y., Zhang, F.: Hand gesture recognition based on MEB-SVM. In: International Conference on Embedded Software and Systems, ICESS’09 (2009)

  24. Ratanamahatana, C.H.A., Keogh, E.: Making Time-Series Classification More Accurate Using Learned Constraints. California University Press, Oakland (2004)

    Book  Google Scholar 

  25. Rabiner, L.R.: A tutorial on hidden Markov models and selected application in speech recognition. Proc. IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  26. Kundu, A., He, Y., Bahl, P.: Recognition of hand writing words: first and second order hidden Markov model based approach. Pattern Recogn. 22(3), 283 (1989)

    Article  Google Scholar 

  27. Nyirarugira, C., Kim, T.: Stratified gesture recognition using the normalized longest common subsequence with rough sets. Signal Process. Image Commun. 30, 178–189 (2015)

    Article  Google Scholar 

  28. Hun, J., Shao, L., Xu, D., Shotten, J.: Enhanced computer vision with Microsoft kinect sensor: a review. IEEE Trans. Cybern. 43, 1318–1334 (2013)

    Article  Google Scholar 

  29. Mackay, D.: Information Theory, Inference and Learning Algorithms. MR 2012999. Cambridge University Press, pp 284–292. ISBN: 0-521-64298-1 (2003)

  30. Liu, F., Du, B., Wang, Q., Wang, Y., Zeng, W.: Hand gesture recognition using via deterministic learning. In: 29th Chinese Control and Decision Conference (CCDC) (2017)

  31. Feng, Z., Xu, S., Zhang, X., Jin, L., Ye, Z.: Real-time fingertip tracking and detection using kinect depth sensor for a new writing-in-the air system. In: The 4th International Conference on Internet Multimedia Computing and Service (ICIMCS), China (2012)

  32. Kane, L., Khanna, P.: Vision-based mid-air unistroke character input using polar signatures. IEEE Trans. Hum. Mach. Syst. J. 47(6), 1077–1088 (2017)

    Article  Google Scholar 

  33. Ortiz, L., Cabrera, V., Goncalves, L.: Depth data error modeling of the ZED 3D vision sensor from stereolabs. ELCVIA Electron Lett. Comput. Vis. Image Anal. 17(1), 1–15 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Engineering, University of Zanjan, Zanjan, Iran

    Shahram Mohammadi & Reza Maleki

Authors
  1. Shahram Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reza Maleki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shahram Mohammadi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mohammadi, S., Maleki, R. Real-time Kinect-based air-writing system with a novel analytical classifier. IJDAR 22, 113–125 (2019). https://doi.org/10.1007/s10032-019-00321-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-019-00321-4

Keywords

Search

Navigation