Hand Posture Detection of Smartphone Users Using LSTM Networks

  • Song Lim Tan
  • Hui Fuang NgEmail author
  • Boon Yaik Ooi
  • Hung Khoon Tan
  • Jacqueline Lee Fang Ang
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 547)


Automatic hand posture detection of smartphone users is important for adaptive user interface design, context aware application development, and activity analysis. This paper presents a method for hand posture and phone placement detection from data produced by accelerometer, magnetometer and gyroscope of a smartphone using LSTM networks. Real-time testing results indicated that LSTM network is effective in hand posture and phone placement prediction, and the proposed method outperformed existing methods by significant margins.


Hand posture Smartphone Accelerometer LSTM 


  1. 1.
    Zhang, C., Guo, A., Zhang, D., Southern, C., Arriaga, R., Abowd, G.: BeyondTouch: Extending the input language with built-in sensors on commodity smartphones. In: The 20th International Conference on Intelligent User Interfaces, pp. 67–77 (2015)Google Scholar
  2. 2.
    Coskun, D., Incel, D., Ozgovde, A.: Phone position/placement detection using accelerometer: impact on activity recognition. In: The IEEE Tenth International Conference on Intelligent Sensors, Sensor Networks and Information Processing, pp. 1–6 (2015)Google Scholar
  3. 3.
    Zhang, S., McCullagh, P., Zheng, H., Nugent, C.: Situation awareness inferred from posture transition and location: derived from smartphone and smart home sensors. IEEE Trans. Hum.-Mach. Syst. 47(6), 814–821 (2017)CrossRefGoogle Scholar
  4. 4.
    Lee, Y., Yeh, H., Kim, K.H., Choi, O.: A real-time fall detection system based on the acceleration sensor of smartphone. Int. J. Eng. Bus. Manag. (2018).
  5. 5.
    Kang, X., Huang, B., Qi, G.: A novel walking detection and step counting algorithm using unconstrained. Sensors (Basel) 18(1) (2018).
  6. 6.
    Allen, R., Ambikairajah, E., Lovell, H., Celler. G.: Classification of a known sequence of motions and postures from accelerometry data using adapted Gaussian mixture models. Physiol. Meas. 27(10), 935–951 (2006)Google Scholar
  7. 7.
    Anguita, D., Ghio, A., Oneto, L., Parra, X., Reyes-Ortiz, JL.: A public domain dataset for human activity recognition using smartphones. In: 21th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pp. 24–26 (2013)Google Scholar
  8. 8.
    Yang, J.Y., Wang, J.S., Chen, Y.P.: Using acceleration measurements for activity recognition: an effective learning algorithm for constructing neural classifiers. Pattern Recogn. Lett. 29(16), 2213–2220 (2008)CrossRefGoogle Scholar
  9. 9.
    Gammulle, H., Denman, S., Sridharan, S., Fookes, C.: Two stream LSTM: A deep fusion framework for human action recognition. In: 2017 IEEE Winter Conference on Applications of Computer Vision, pp. 177–186 (2017)Google Scholar
  10. 10.
    Zeyer, A., Doetsch, P., Voigtlaender, P., Schlüter, R., Ney, H.: A comprehensive study of deep bidirectional LSTM RNNS for acoustic modeling in speech recognition. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (2017).
  11. 11.
    Ordóñez, F.J., Roggen, D.: Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1) (2016).

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Song Lim Tan
    • 1
  • Hui Fuang Ng
    • 1
    Email author
  • Boon Yaik Ooi
    • 1
  • Hung Khoon Tan
    • 1
  • Jacqueline Lee Fang Ang
    • 1
  1. 1.Universiti Tunku Abdul RahmanKamparMalaysia

Personalised recommendations