Real-time Kinect-based air-writing system with a novel analytical classifier

  • Shahram MohammadiEmail author
  • Reza Maleki
Original Paper


Air-writing is an attractive method of interaction between human and machine due to lack of any interface device on the user side. After removing existing limitations and solving the current challenges, it can be used in many applications in the future. In this paper, using the Kinect depth and color images, an air-writing system is proposed to identify single characters such as digits or letters and connected characters such as numbers or words. In this system, automatic clustering, slope variations detection, and a novel analytical classification are proposed as new approaches to eliminate noise in the trajectory from the depth image and hand segmentation, to extract the feature vector, and to identify the character from the feature vector, respectively. Experimental results show that the proposed system can successfully identify single characters and connected characters with the average recognition rate of 97%. It provides a better result than other similar approaches proposed in the literature. In the proposed system, the character recognition time is quite low, about 3 ms, because of using a novel analytical classifier. Evaluation of 4 classifiers shows that the proposed classifier has a higher speed and precision than the SVM, HMM, and K-nearest neighbors classifiers.


Kinect Air-writing Slope variations detection Automatic clustering Analytical classification 



  1. 1.
    Palmondon, R., Srihari, S.N.: On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63–84 (2000)CrossRefGoogle Scholar
  2. 2.
    Saini, R., Kumar, P., Roy, P.P., Dogra, D.P.: A novel framework of continuous human-activity recognition using kinect. Neurocomputing 311, 99–111 (2018)CrossRefGoogle Scholar
  3. 3.
    Kumar, P., Saini, R., Behera, S.K., Dogra, D.P.: Real-time recognition of sign language gesture and air-writing using leap motion. In: 2017 Fifteenth IAPR International Conference on Machine Vision Applications, Nagoya, Japan (2017)Google Scholar
  4. 4.
    Khan, N.A., Khan, S.M., Abdullah, M., Kanji, S.J., Iltifat, U.: Use hand gesture to write in air recognize with computer vision. IJCSNS Int. J. Comput. Sci. Netw. Secur. 17(5), 51–55 (2017)Google Scholar
  5. 5.
    Agrawal, S., Constandache, L., Gaonkar, S., Roy, R., Caves, K., Deruyter, F.: Using mobile phones to write in air. In: MobiSys’11 Proceeding of the 9th International Conference on Mobile Systems, ACM (2013)Google Scholar
  6. 6.
    Xu, S., Xue, Y.: Air-writing characters modelling and recognition on modified CHMM. 2016 IEEE International Conference on Systems, Man and Cybernetics (SMC) (2016)Google Scholar
  7. 7.
    Beg, S., Khan, M.F., Baig, F.: Text writing in air. J. Inf. Display 14(4), 137–148 (2013)CrossRefGoogle Scholar
  8. 8.
    Amma, C., Georgi, M., Schultz, T.: Air writing: a wearable handwriting recognition system. Pers. Ubiquitous Comput. 18(1), 191–203 (2014)CrossRefGoogle Scholar
  9. 9.
    Kumar, P., Verma, J., Prasad, S.: A wearable real-time device for human–computer interaction. Int. J. Adv. Sci. Technol. 43, 15–26 (2012)Google Scholar
  10. 10.
    Patil, S., Kim, D., Park, S., Chai, Y.: Handwriting recognition in free space using WIMU-based hand motion analysis. Hindawi J. Sens. 2016, 1–10 (2016)CrossRefGoogle Scholar
  11. 11.
    Islam, R., Mahmud, H., Hasan, M.K., Rubaiyeat, H.A.: Alphabet recognition in air writing using depth information. ACHI. ISBN: 978-1-61208-468-8 (2016)Google Scholar
  12. 12.
    Chen, M., Alregib, G., Juang, B.H.: Air-writing recognition, Part 1: modeling and recognition of characters, words and connecting motions. IEEE Trans. Hum. Mach. Syst. 46(3), 403–413 (2016)CrossRefGoogle Scholar
  13. 13.
    Jin, X.J., Feng, Q., Hou, X., Liu, C.L.: Visual gesture character string recognition by classification-based segmentation with stroke deletion. In: Second IAPR Asian Conference on Pattern Recognition, ACPR (2011)Google Scholar
  14. 14.
    Aggarwal, R., Swetha, S.: Online handwriting recognition using depth sensors. In: ICDAR, 13th International Conference (2015)Google Scholar
  15. 15.
    Murata, T., Shin, J.: Hand gesture and character recognition based on kinect sensor. Int. J. Distrib. Sens. Netw. 10, 278460 (2014)CrossRefGoogle Scholar
  16. 16.
    Deepa, D., Dharmalingam, R.: Feature and processing of recognition of characters, words & connecting motions. IJARIIT 3(2), 699–704 (2017)Google Scholar
  17. 17.
    Rautaray, S.S., Agarwal, A.: Real time hand gesture recognition system for dynamic applications. Int. J. UbiComp. 3, 21 (2012)CrossRefGoogle Scholar
  18. 18.
    Froba, B., Ernst, A.: Face detection with the modified census transform. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition (2004)Google Scholar
  19. 19.
    Conseil, S., Bourennane, S.: Comparison of Fourier descriptors and Hu moments for hand posture recognition. In: 15th European Signal Processing Conference (2007)Google Scholar
  20. 20.
    Zhang, D., Lu, G.: A comparative study of Fourier descriptors for shape representation and retrieval. In: 5th Asian Conference on Computer Vision (ACCV), Melbourne, Australia (2002)Google Scholar
  21. 21.
    Lockton, R.: Hand Gesture Recognition Using Computer Vision. Oxford University Press, Oxford (2002)Google Scholar
  22. 22.
    YanuTara, R., Santosa, P., Adji, T.: Sign language recognition in robot teleoperation using centroid distance Fourier descriptors. Int. J. Comput. Appl. 48(2), 8–12 (2012)Google Scholar
  23. 23.
    Ren, Y., Zhang, F.: Hand gesture recognition based on MEB-SVM. In: International Conference on Embedded Software and Systems, ICESS’09 (2009)Google Scholar
  24. 24.
    Ratanamahatana, C.H.A., Keogh, E.: Making Time-Series Classification More Accurate Using Learned Constraints. California University Press, Oakland (2004)CrossRefGoogle Scholar
  25. 25.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected application in speech recognition. Proc. IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  26. 26.
    Kundu, A., He, Y., Bahl, P.: Recognition of hand writing words: first and second order hidden Markov model based approach. Pattern Recogn. 22(3), 283 (1989)CrossRefGoogle Scholar
  27. 27.
    Nyirarugira, C., Kim, T.: Stratified gesture recognition using the normalized longest common subsequence with rough sets. Signal Process. Image Commun. 30, 178–189 (2015)CrossRefGoogle Scholar
  28. 28.
    Hun, J., Shao, L., Xu, D., Shotten, J.: Enhanced computer vision with Microsoft kinect sensor: a review. IEEE Trans. Cybern. 43, 1318–1334 (2013)CrossRefGoogle Scholar
  29. 29.
    Mackay, D.: Information Theory, Inference and Learning Algorithms. MR 2012999. Cambridge University Press, pp 284–292. ISBN: 0-521-64298-1 (2003)Google Scholar
  30. 30.
    Liu, F., Du, B., Wang, Q., Wang, Y., Zeng, W.: Hand gesture recognition using via deterministic learning. In: 29th Chinese Control and Decision Conference (CCDC) (2017)Google Scholar
  31. 31.
    Feng, Z., Xu, S., Zhang, X., Jin, L., Ye, Z.: Real-time fingertip tracking and detection using kinect depth sensor for a new writing-in-the air system. In: The 4th International Conference on Internet Multimedia Computing and Service (ICIMCS), China (2012)Google Scholar
  32. 32.
    Kane, L., Khanna, P.: Vision-based mid-air unistroke character input using polar signatures. IEEE Trans. Hum. Mach. Syst. J. 47(6), 1077–1088 (2017)CrossRefGoogle Scholar
  33. 33.
    Ortiz, L., Cabrera, V., Goncalves, L.: Depth data error modeling of the ZED 3D vision sensor from stereolabs. ELCVIA Electron Lett. Comput. Vis. Image Anal. 17(1), 1–15 (2018)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of EngineeringUniversity of ZanjanZanjanIran

Personalised recommendations