Abstract
Air-writing through hand or fingertip is a functional and attractive mechanism. Since there is usually no pen-up and pen-down in air-writing, trajectory of numbers and words in the air-writing will be a connected series of characters (ligature Stroke). Identification of legitimate characters such as digits or letters inside a ligature Stroke is one of the most important challenges faced in this area. By solving these challenges, there will be more uses in future. In this work, the color and depth images of the Kinect sensor are used to identify the user’s air-writing, which includes the digits and numbers of the Persian language. To extract a feature vector from the trajectory, we propose a simple but very effective method, called slope variations detection, which is robust to variations of size, translation, and rotation of the trajectory. Also, a novel analytical classifier is proposed to map a vector to a character. This classifier has higher speed and accuracy than traditional classifiers, such as SVM, HMM, and K Nearest Neighbors. Experimental results show that the average recognition rate for digits and numbers of Persian language is 98% which is quite acceptable for a practical system.
Similar content being viewed by others
References
Palmondon, R., Srihari, S.N.: On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63–84 (2000). https://doi.org/10.1109/34.824821
Wu, C.H., Lin, C.H.: Depth-based hand gesture recognition for home appliance control. In: IEEE, 17th International Symposium on Consumer Electronics (ISCE) (2013). https://doi.org/10.1109/ISCE.2013.6570227
Kumar, P., Saini, R., Behera, S.K., Dogra, D.P.: Real-time recognition of sign language gesture and air-writing using leap motion. In: 2017 Fifteenth IAPR International Conference on Machine Vision Applications, Nagoya, Japan (2017). https://doi.org/10.23919/MVA.2017.7986825
Zeng, W., Wang, C., Wang, Q.: Hand gesture recognition using Leap Motion via deterministic learning. Multimed. Tools Appl. 77(21), 28185–28206 (2018). https://doi.org/10.1007/s11042-018-5998-1
Agrawal, S., Constandache, I., Gaonkar, S., Roy Choudhury, R., Caves, K., Deruyter, F.: Using mobile phones to write in air. In: MobiSys’11 Proceeding of the 9th International Conference on Mobile Systems, pp. 15–28 (2011). https://doi.org/10.1145/1999995.1999998
Songbin, X., Yang, X.: Air-writing characters modelling and recognition on modified CHMM. In: 2016 IEEE International Conference on Systems, Man and Cybernetics (SMC) (2016). https://doi.org/10.1109/SMC.2016.7844452
Beg, S., Khan, M.F., Baig, F.: Text writing in air. J. Inf. Display (2013). https://doi.org/10.1080/15980316.2013.860928
Kalra, S., Jain, S., Agarwal, A.: A wearable computing system for wireless communication and gesture based human computer interface. In: 2016 2nd International Conference on Next Generation Computing Technologies (NGCT), pp. 456–462 (2016). https://doi.org/10.1109/NGCT.2016.7877459
Fu, Z., Xu, J., Zhu, Z., Liu, A.X., Sun, X.: Writing in the air with WiFi signals for virtual reality devices. IEEE Trans. Mobile Comput. 18(2), 473–484 (2019). https://doi.org/10.1109/tmc.2018.2831709
Patil, S., Kim, D., Park, S., Chai, Y.: Handwriting recognition in free space using WIMU-based hand motion analysis. Hindawi J. Sens (2016). https://doi.org/10.1155/2016/3692876
Zhang, X., Ye, Z., Jin, L., Feng, Z., Xu, S.: A new writing experience: finger writing in the air using a kinect sensor. IEEE Multimed. 20(4), 85–93 (2013). https://doi.org/10.1109/MMUL.2013.50
Chen, M., Alregib, G., Juang, B.: Air-writing recognition-part I: modeling and recognition of characters, words and connecting motions. IEEE Trans. Hum. Mach. Syst. 46(3), 403–413 (2016). https://doi.org/10.1109/THMS.2015.2492598
Singha, J., Laskar, R.H.: Hand gesture recognition using two-level speed normalization, feature selection and classifier fusion. Multimed. Syst. 23(4), 499–514 (2016). https://doi.org/10.1007/s00530-016-0510-0
Singha, J., Laskar, R.H.: Vision based hand gesture recognition for human. Artif. Intell. Rev. 43(1), 1–54 (2015). https://doi.org/10.1007/s10462-012-9356-9
Stern, H., Shmueli, M., Berman, S.: Most discriminating segment—longest common subsequence (MDSLCS) algorithm for dynamic hand gesture classification. Pattern Recogn. Lett. 34(15), 1980–1989 (2013). https://doi.org/10.1016/j.patrec.2013.02.007
Rautaray, S.S., Agarwal, A.: Real time hand gesture recognition system for dynamic applications. Int. J. UbiComp 1, 21 (2012). https://doi.org/10.5121/iju.2012.3103
Froba, B., Ernst, A.: Face detection with the modified census transform. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition (2004). https://doi.org/10.1109/AFGR.2004.1301514
Dan, Z., Sang, N., Hu, J., Sun, S.: A transfer knowledge framework for object recognition of infrared image. In: Tan, T., Ruan, Q., Chen, X., Ma, H., Wang L. (eds.) Advances in Image and Graphics Technologies. IGTA 2013. Communications in Computer and Information Science, vol. 363. Springer, Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37149-3_25
Zhang, D., Lu, G.: A comparison of shape retrieval using fourier descriptors and short-time fourier descriptors. In: Shum, H.Y., Liao, M., Chang, S.F. (eds.) Lecture Notes in Computer Science: Vol. 2195. Advances in Multimedia Information Processing—PCM 2001. PCM 2001. Springer, Berlin, Heidelberg (2001). https://doi.org/10.1007/3-540-45453-5_111
Trigueiros, P., Ribeiro, F., Reis, L.P.: Hand gesture recognition for human computer interaction: a comparative study of different image features. In: Filipe, J., Fred, A. (eds.) Agents and Artificial Intelligence. ICAART 2013. Communications in Computer and Information Science, vol. 449. Berlin, Heidelberg: Springer (2014). https://doi.org/10.1007/978-3-662-44440-5_10
Wang, S., Kim, H., Chen, H.: Contour tracking using centroid distance signature and dynamic programming method. In: 2009 International Joint Conference on Computational Sciences and Optimization, 2009, vol. 1, pp. 274–278. https://doi.org/10.1109/CSO.2009.201
Jin, X., Wang, Q., Hou, X., Liu, C.: Visual gesture character string recognition by classification-based segmentation with stroke deletion. In: 2013 2nd IAPR Asian Conference on Pattern Recognition, 2013, pp. 120–124. https://doi.org/10.1109/ACPR.2013.7
Aggarwal, R., Swetha, S.: Online handwriting recognition using depth sensors. In: ICDAR, 13th International Conference. https://doi.org/10.1109/ICDAR.2015.7333924. (2015)
Murata, T., Shin, J.: Hand gesture and character recognition based on kinect sensor. Int. J. Distrib. Sensor Netw. (2014). https://doi.org/10.1155/2014/278460
Ren, Y., Zhang, F.: Hand gesture recognition based on MEB-SVM. In: 2009 International Conference on Embedded Software and Systems, 2009, pp. 344–349. https://doi.org/10.1109/ICESS.2009.21
Rabiner, L.R.: A tutorial on hidden markov models and selected application in speech recognition. Proc. IEEE 77(2), 257–286 (1989). https://doi.org/10.1109/5.18626
Saini, R., Roy, P., Dogra, D.P.: A segmental HMM based trajectory classification using genetic algorithm. Expert Syst. Appl. 93, 169–181 (2017). https://doi.org/10.1016/j.eswa.2017.10.021
Nyirarugira, C., Kim, T.: Stratified gesture recognition using the normalized longest common subsequence with rough sets. Sig. Process. Image Commun. 30, 178–189 (2015). https://doi.org/10.1016/j.image.2014.10.008
Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with microsoft Kinect sensor: a review. IEEE Trans. Cybern. 43, 5 (2013). https://doi.org/10.1109/TCYB.2013.2265378
David, D., Thomas, P.: Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Can. Cartogr. 10(2), 112–122 (1973). https://doi.org/10.3138/FM57-6770-U75U-7727
Kane, L., Khanna, P.: Vision-based mid-air unistroke character input using polar signatures. IEEE Trans. Hum. Mach. Syst. J. 1, 21 (2017). https://doi.org/10.1109/THMS.2017.2706695
Liu, F., Du, B., Wang, Q., Wang, Y., Zeng, W.: Hand gesture recognition using via deterministic learning. In: 29th Chinese Control and Decision Conference (CCDC) (2017). https://doi.org/10.1109/CCDC.2017.7978867
Feng, Z., Xu, S., Zhang, X., Jin, L., Ye, Z.: Real-time fingertip tracking and detection using kinect depth sensor for a new writing-in-the air system. In: The 4th International Conference on Internet Multimedia Computing and Service (ICIMCS), China (2012). https://doi.org/10.1145/2382336.2382356
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mohammadi, S., Maleki, R. Air-writing recognition system for Persian numbers with a novel classifier. Vis Comput 36, 1001–1015 (2020). https://doi.org/10.1007/s00371-019-01717-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-019-01717-3