Skip to main content
SpringerLink
Log in

On-air English Capital Alphabet (ECA) recognition using depth information

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

On-air writing can be considered as a time-dependent event where hand gesture is produced in a natural environment through index finger movement. A sequence of such movements containing several time steps in 3D space can be utilized to construct an English Capital Alphabet (ECA). While Previous researches investigated 2D features, we believe that depth information may play a significant role along with other features in recognition of these dynamic gestures. We have captured hand finger motion information using a depth camera and represented them as depth images for each ECA. The hand finger trajectory data were extracted from the depth image, and a combination of depth-based features and non-depth features were generated; depth variation was performed in the depth-based features, and then, all the feature values were converted into time-series data. Dynamic Time Warping distances were determined between a template ECA and a test ECA for each ECA collected from 15 participants. These distance-based features were then fed into a multi-class SVM for training and testing and got the recognition accuracy of \(80.77\%\) without depth and \(88.21\%\) with depth-based features. To cope with the over-fitting problem, we applied the resampling technique and got the highest recognition accuracy of 96.85%, and at last, we applied some feature selection techniques to analyze the recognition results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Agrawal, S., Constandache, I., Gaonkar, S., Roy Choudhury, R., Caves, K., DeRuyter, F.: Using mobile phones to write in air. In: Proceedings of the 9th International Conference on Mobile Systems, Applications, and Services, MobiSys ’11, New York, NY, USA, pp. 15–28. ACM (2011)

  2. Amma, C., Georgi, M., Schultz, T.: Airwriting: a wearable handwriting recognition system. Pers. Ubiquitous Comput. 18(1), 191–203 (2014)

    Article  Google Scholar 

  3. Freeman, D., Barrentine, D.B.: Method and system for operating a multi-function portable electronic device using voice-activation, March 10 . US Patent 8,977,255 (2015)

  4. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. ACM SIGKDD Explor. Newslett. 11(1), 10–18 (2009)

    Article  Google Scholar 

  5. Hunter, J.S.: The exponentially weighted moving average. J. Qual. Technol. 18(4), 203–210 (1986)

    Article  Google Scholar 

  6. Islam, R., Mahmud, H., Hasan, M.K., Rubaiyeat, H.A.: Alphabet recognition in air writing using depth information. In: The Ninth International Conference on Advances in Computer–Human Interactions, pp. 299–301 (2016)

  7. Jalalian, Arash, Chalup, Stephan K.: GDTW-P-SVMS: variable-length time series analysis using support vector machines. Neurocomputing 99, 270–282 (2013)

    Article  Google Scholar 

  8. Jeong, Y.S., Jeong, M.K., Omitaomu, O.A.: Weighted dynamic time warping for time series classification. Pattern Recognit. 44(9), 2231–2240 (2011)

    Article  Google Scholar 

  9. Kim, D.H., Choi, H.I., Kim, J.H.: 3d space handwriting recognition with ligature model. In: Proceedings of the Third International Conference on Ubiquitous Computing Systems, UCS’06, pp. 41–56. Springer, Berlin (2006)

  10. Knowledge Euronews. Future of texting: writing in the air! Apr 8, 2013

  11. Mahmud, H., Hasan, M., Kabir, M., Mottalib, M.A.: Recognition of symbolic gestures using depth information. Adv. Hum. Comput. Interact. (2018). https://doi.org/10.1155/2018/1069823

    Article  Google Scholar 

  12. Mullin, R.: Time warps, string edits, and macromolecules. In: Kruskall, J., Liberman, M. (eds.) The theory and practice of sequence comparison. Advanced book program, Addison-Wesley, Reading, MA, Don mills, ON, 300 pp. us \$31.95. ISBN: 0-201-07809-0 (1983). Can. J. Stat. 13(2):167–168 (1985)

  13. Priya, A., Mishra, S., Raj, S., Mandal, S., Datta, S.: Online and offline character recognition: a survey. In: International Conference on Communication and Signal Processing (ICCSP). IEEE (2016)

  14. Ren, Z., Yuan, J., Meng, J., Zhang, Z.: Robust part-based hand gesture recognition using kinect sensor. IEEE Trans. Multimed. 15(5), 1110–1120 (2013)

    Article  Google Scholar 

  15. Mohammadi, S., Maleki, R.: Air-writing recognition system for Persian numbers with a novel classifier. Vis. Comput. 36, 1001–1015 (2019)

    Article  Google Scholar 

  16. Siena, F.L., Byrom, B., Watts, P., Breedon, P.: Utilising the intel realsense camera for measuring health outcomes in clinical research. J. Med. Syst. 42(3), 53 (2018)

    Article  Google Scholar 

  17. Sreekanth, N.S., Narayanan, N.K.: Dynamic gesture recognition: a machine vision based approach. In: Proceedings of the International Conference on Signal, Networks, Computing, and Systems, pp. 105–115. Springer (2017)

  18. Thomas, M.: A role for “air writing” in second-language learners’ acquisition of Japanese in the age of the word processor. J. Jpn. Linguist. 30(1), 86–106 (2014)

    Article  Google Scholar 

  19. Wexelblat, A.: An approach to natural gesture in virtual environments. ACM Trans. Comput. Hum. Interact. 2(3), 179–200 (1995)

    Article  Google Scholar 

  20. Yin, Y., Xie, L., Gu, T., Lu, Y., Lu, S.: AirContour: building contour-based model for in-air writing gesture recognition. ACM Trans. Sens. Netw. 15(4), 44:1–44:25 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank to all users who have participated voluntarily in the data collection process.

Author information

Authors and Affiliations

  1. Systems and Software Lab, Department of Computer Science and Engineering (CSE), Islamic University of Technology (IUT), Dhaka, Bangladesh

    Hasan Mahmud & Md. Kamrul Hasan

  2. Laboratory of Computational Pragmatics Models and Methods, Higher School of Economic, Pokrovsky Boulevard, 11, Moscow, Russia

    Robiul Islam

Authors
  1. Hasan Mahmud
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robiul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Md. Kamrul Hasan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hasan Mahmud.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mahmud, H., Islam, R. & Hasan, M.K. On-air English Capital Alphabet (ECA) recognition using depth information. Vis Comput 38, 1015–1025 (2022). https://doi.org/10.1007/s00371-021-02065-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02065-x

Keywords

Search

Navigation