Journal of Medical and Biological Engineering

, Volume 38, Issue 2, pp 284–295 | Cite as

A Wrist-Worn Piezoelectric Sensor Array for Gesture Input

Original Article
  • 161 Downloads

Abstract

Accurately estimating hand and finger poses for recognizing gestures helps solve many technical challenges, such as controlling prosthetics, robotic manipulation, and computer input, e.g. for virtual reality. A major challenge is accurately identifying gestures under different conditions—such as mobile or low-light environments—without hindering hand function. This paper describes a low-cost wrist-mounted device that uses piezoelectric sensors to estimate finger gestures. The signals that are recorded are vibrations and shape changes that occur at the wrist due to muscle and tendon motion. An array of six piezoelectric sensors was affixed to the inside of an adjustable wrist strap. A user study was completed. To identify when a subject made a finger tap gesture, a touch graphics tablet recorded when a fingertip contacted the tablet surface. Piezoelectric signal features were computed over timing windows coinciding with a gesture. The features were used in the training of a support vector machine classification model. The results indicate the viability of using piezoelectric sensors to classify finger tap gestures, with a mean classification accuracy of 97% for tap gestures made with each of the five fingers.

Keywords

Piezoelectric sensors Biosignals Surface mechanomyography (sMMG) Human computer interface (HCI) Gesture recognition Wearables 

Notes

Acknowledgements

This research was supported by the Schulich School of Engineering at the University of Calgary.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Ethics approval was obtained from the University of Calgary Conjoint Faculties Research Ethics Board.

References

  1. 1.
    Khan, R. Z. (2012). Hand gesture recognition: A literature review. International Journal of Artificial Intelligence & Applications, 3(4), 161–174.MathSciNetCrossRefGoogle Scholar
  2. 2.
    Nicolau, H. (2012). Touch typing using thumbs: Understanding the effect of mobility and hand posture. In proceedings 2012 ACM annual conference on human factors in computing systems. (pp. 2683–2686).Google Scholar
  3. 3.
    Sharan, D., Mohandoss, M., Ranganathan, R., & Jose, J. (2014). Musculoskeletal disorders of the upper extremities due to extensive usage of hand held devices. Annals of Occupational and Environmental Medicine, 26, 22.CrossRefGoogle Scholar
  4. 4.
    Coggon, D., Ntani, G., Palmer, K. T., Felli, V. E., Harari, R., Barrero, L. H., et al. (2013). Disabling musculoskeletal pain in working populations: Is it the job, the person, or the culture? Pain, 154(6), 856–863.CrossRefGoogle Scholar
  5. 5.
    March, L., Smith, E. U. R., Hoy, D. G., Cross, M. J., Sanchez-Riera, L., Blyth, F., et al. (2014). Burden of disability due to musculoskeletal (MSK) disorders. Best Practice & Research Clinical Rheumatology, 28(3), 353–366.CrossRefGoogle Scholar
  6. 6.
    Morris, D. (2011). Emerging input technologies for always-available mobile interaction. Foundations and Trends Human–Computer Interaction, 4(4), 245–316.CrossRefGoogle Scholar
  7. 7.
    Maciejasz, P., Eschweiler, J., Gerlach-Hahn, K., Jansen-Troy, A., & Leonhardt, S. (2014). A survey on robotic devices for upper limb rehabilitation. Journal of NeuroEngineering and Rehabilitation, 11(3), 1–29.Google Scholar
  8. 8.
    Hrabia, C.-E., Wolf, K., & Wilhelm, M. (2013). Whole hand modeling using 8 wearable sensors: Biomechanics for hand pose prediction. In proceedings of the 4th augmented human international conference - AH’13. (pp. 21–28).Google Scholar
  9. 9.
    Lobo-Prat, J., Kooren, P. N., Stienen, A. H., Herder, J. L., Koopman, B. F., & Veltink, P. H. (2014). Non-invasive control interfaces for intention detection in active movement-assistive devices. Journal of Neuroengineering and Rehabilitation, 11(1), 168.CrossRefGoogle Scholar
  10. 10.
    Yoshikawa, M., Mikawa, M., & Tanaka, K. (2007). Hand Pose Estimation Using EMG Signals. In Proceedings 29th annual international conference IEEE EMBS. (pp. 1–2).Google Scholar
  11. 11.
    Costanza, E., Inverso, S. A., Allen, R., & Maes, P. (2007). Intimate interfaces in action: Assessing the usability and subtlety of EMG-based motionless gestures. In CHI’07 proceedings of the sigchi conference on human factors in computing systems. (pp. 819–828).Google Scholar
  12. 12.
    Boostani, R., & Moradi, M. H. (2003). Evaluation of the forearm EMG signal features for the control of a prosthetic hand. Physiological Measurement, 24(2), 309–319.CrossRefGoogle Scholar
  13. 13.
    McIntosh, J., McNeill, C., Fraser, M., Kerber, F., Löchtefeld, M., & Krüger, A. (2016). EMPress: Practical hand gesture classification with wrist-mounted EMG and pressure sensing. In proceedings of the 2016 CHI conference on human factors in computing systems, May 912, San Jose, CA, USA. (pp. 2332–2342).Google Scholar
  14. 14.
    Nuwer, R. (2013). Armband adds a twitch to gesture control. London: New Scientist.Google Scholar
  15. 15.
    Abbas, B., Farooq, O., Uzzaman, Y., Khan, A. A., & Vyas, a. L. (2013). Enhancing classification accuracy of wrist movement by denoising sEMG signals. Annual International Conference of the IEEE Engineering in Medicine and Biological Society. EMBS. (pp. 5762–5764).Google Scholar
  16. 16.
    Castro, M. C., Arjunan, S. P., & Kumar, D. K. (2015). Selection of suitable hand gestures for reliable myoelectric human computer interface. BioMedical Engineering Online, 14(1), 1–11.CrossRefGoogle Scholar
  17. 17.
    Irfan, M. R. M., Sudharsan, N., Santhanakrishnan, S., & Geethanjali, B. (2011). A Comparative Study of EMG and MMG Signals for Practical Applications. In proceedings of the international conference on signal, image processing and applications with workshop of ICEEA. (Vol. 21, pp. 106–110).Google Scholar
  18. 18.
    Park, S. H., & Lee, S. P. (1998). EMG pattern recognition based on artificial intelligence techniques. IEEE Transactions on Rehabilitation Engineering: A Publication of the IEEE Engineering in Medicine and Biology Society, 6(4), 400–405.CrossRefGoogle Scholar
  19. 19.
    Orizio, C., & Gobbo, M. (2006). Mechanomyography. In Wiley encyclopedia of biomedical engineering (pp. 1–11). Hoboken: John Wiley & Sons Inc.Google Scholar
  20. 20.
    Murphy, C., Campbell, N., Caulfield, B., Ward, T., & Deegan, C. (2008). Micro Electro Mechanical Systems Based Sensor for Mechanomyography. In 19th Int. Conf. Biosignal.Google Scholar
  21. 21.
    Gavriel, C., & Faisal, A. A. (2014). A comparison of day-long recording stability and muscle force prediction between bsn-based mechanomyography and electromyography. In wearable and implantable body sensor networks (BSN), 2014 11th international conference on.(pp. 69–74).Google Scholar
  22. 22.
    Laput, G., Xiao, R., & Harrison, C. (2016). ViBand: High-fidelity bio-acoustic sensing using commodity smartwatch accelerometers. In proceedings of the 29th annual symposium on user interface software and technology - UIST’16 (pp. 321–333).Google Scholar
  23. 23.
    Krueger, E., Scheeren, E. M., Nogueira-neto, G. N., Lúcia, V., Button, N., & Nohama, P. (2014). Advances and perspectives of mechanomyography. Revista Brasileira de Engenharia Biomédica, 30(4), 384–401.CrossRefGoogle Scholar
  24. 24.
    Kaneko, T., Minh-Dung, N., Aoki, R., Takahata, T., Matsumoto, K., & Shimoyama, I. (2014). Measurement of mechanomyogram. In micro electro mechanical systems (MEMS), 2014 IEEE 27th international conference on. (pp. 845–848).Google Scholar
  25. 25.
    Fara, S., Vikram, C. Sen, Gavriel, C., & Faisal, a. A. (2013). Robust, ultra low-cost MMG system with brain-machine-interface applications. In 6th Annu. Int. IEEE EMBS Conf. Neural Eng. (pp. 723–726).Google Scholar
  26. 26.
    Harrison, C., Tan, D., & Morris, D. (2010). Skinput: Appropriating the Body as an Input Surface. In proceedings of the SIGCHI conference on human factors in computing systems - CHI’10. (p. 453).Google Scholar
  27. 27.
    Deyle, T., Palinko, S., Poole, E. S., & Starner, T. (2007). Hambone: A bio-acoustic gesture interface. In wearable computers, 2007 11th IEEE international symposium on. IEEE. (pp. 3–10).Google Scholar
  28. 28.
    Bu, N., Tsukamoto, J., Ueno, N., Shima, K., & Tsuji, T. (2008). Measuring muscle movements for human interfaces using a flexible piezoelectric thin film sensor. In engineering in medicine and biology society, 2008. EMBS 2008. 30th annual international conference of the IEEE. (Vol. 2008, pp. 112–116).Google Scholar
  29. 29.
    Amento, B., Hill, W., & Terveen, L. (2002). The sound of one hand. CHI’02 extended abstracts on Human factors in computing systems - CHI’02, 724.Google Scholar
  30. 30.
    Dementyev, A., & Paradiso, J. A. (2014). WristFlex: Low-Power Gesture Input with Wrist-Worn Pressure Sensors. In Proceedings of the 27th annual ACM symposium on user interface software and technology - UIST’14. (pp. 161–166).Google Scholar
  31. 31.
    Kadkhodayan, A., Jiang, X., & Menon, C. (2016). Continuous prediction of finger movements using force myography. Journal of Medical and Biological Engineering, 36(4), 1–11.CrossRefGoogle Scholar
  32. 32.
    Bu, N., Fukuda, O., Ueno, N., & Inoue, M. (2009). A preliminary study on detection of muscle activity using a flexible AlN piezoelectric thin film sensor. In robotics and biomimetics (ROBIO), 2009 IEEE international conference on IEEE. (pp. 944–948).Google Scholar
  33. 33.
    Ortega-Avila, S., Rakova, B., Sadi, S., & Mistry, P. (2015). Non-invasive optical detection of hand gestures. In proceedings of the 6th augmented human international conference. (pp. 179–180).Google Scholar
  34. 34.
    Jung, P. G., Lim, G., Kim, S., & Kong, K. (2015). A wearable gesture recognition device for detecting muscular activities based on air-pressure sensors. IEEE Transactions on Industrial Informatics, 11(2), 485–494.Google Scholar
  35. 35.
    Tiwana, M., Redmond, S., & Lovell, N. (2012). A review of tactile sensing technologies with applications in biomedical engineering. Sensors and Actuators, A: Physical, 179, 17–31.CrossRefGoogle Scholar
  36. 36.
    Booth, R., & Goldsmith, P. (2016). Validation of a piezo-electric sensor array for a wrist-worn muscle-computer interface. In Proc. Can. Med. Biol. Eng. Soc. Google Scholar
  37. 37.
    Dahiya, R. S., Metta, G., Valle, M., & Sandini, G. (2010). Tactile sensing—from humans to humanoids. IEEE Transactions on Robotics, 26(1), 1–20.CrossRefGoogle Scholar
  38. 38.
    Sirohi, J., & Chopra, I. (2000). Fundamental understanding of piezoelectric strain sensors. Journal of Intelligent Materials Systems and Structures, 11(4), 246–257.CrossRefGoogle Scholar
  39. 39.
    Chu, J. U., Moon, I., & Mun, M. S. (2006). A real-time EMG pattern recognition system based on linear-nonlinear feature projection for a multifunction myoelectric hand. IEEE Transactions on Biomedical Engineering, 53(11), 2232–2239.CrossRefGoogle Scholar
  40. 40.
    Alves, N., & Chau, T. (2010). Recognition of forearm muscle activity by continuous classification of multi-site mechanomyogram signals. In 32nd annual international conference of the IEEE EMBS. (pp. 3531–3534).Google Scholar
  41. 41.
    Zhang, Y., & Harrison, C. (2015). Tomo: Wearable, low-cost, electrical impedance tomography for hand gesture recognition. In proceedings of the 28th annual ACM symposium on user interface software & technology - UIST’15. (pp. 167–173).Google Scholar
  42. 42.
    Ogata, M., & Imai, M. (2015). SkinWatch: Skin gesture interaction for smart watch. In Proceedings of the 6th Augmented Human International Conference. ACM. (pp. 21–24).Google Scholar

Copyright information

© Taiwanese Society of Biomedical Engineering 2017

Authors and Affiliations

  1. 1.Biomedical EngineeringUniversity of CalgaryCalgaryCanada
  2. 2.Mechanical EngineeringUniversity of CalgaryCalgaryCanada

Personalised recommendations