Hand Pose Estimation System Based on a Cascade Approach for Mobile Devices

  • Houssem Lahiani
  • Monji Kherallah
  • Mahmoud Neji
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 736)


The rise in the use of mobile devices requires finding new ways to interact with this type of devices. Gestures are an effective way to interact with the mobile device and to place order to it. However, gesture recognition in this context constitute a challenging task due the limited computational capacities of this type of devices. In this work, we present a hand pose estimation system for mobile device. The gesture is recognized by using a boosting algorithm and Haar-like features. The system is designed for Android devices. The method used consists of capturing gestures by a smartphone’s camera to recognize the hand sign. It presents a system based on a real-time hand posture recognition algorithm for mobile devices. The aim of this system is to allow the mobile device interpreting hand signs made by users without the need to touch the screen.


Hand gesture recognition Android Haar-like features AdaBoost HCI 


  1. 1.
    Manjoo, F.: A murky road ahead for android, despite market dominance. The New York Times (2015). ISSN 0362-4331. Accessed 27 May 2015Google Scholar
  2. 2.
  3. 3.
  4. 4.
    Cobârzan, C., Hudelist, M.A., Schoeffmann, K., Primus, M.J.: Mobile image analysis: android vs. iOS. In: 21st International Conference on MultiMedia Modelling (MMM), pp. 99–110 2015Google Scholar
  5. 5.
    Seymour, M., Tšoeu, M., A Mobile application for South African Sign Language (SASL) recognition. In: IEEE AFRICON 2015, pp. 281–285 (2015)Google Scholar
  6. 6.
    Xie, C., Luan, S., Wang, H., Zhang, B.: Gesture recognition benchmark based on mobile phone. In: You, Z., Zhou, J., Wang, Y., Sun, Z., Shan, S., Zheng, W., Feng, J., Zhao, Q. (eds.) CCBR 2016. LNCS, vol. 9967, pp. 432–440. Springer, Cham (2016). CrossRefGoogle Scholar
  7. 7.
    Lahiani, H., Elleuch, M., Kherallah, M.: Real time hand gesture recognition system for android devices. In: 15th International Conference on Intelligent Systems Design and Applications (ISDA), pp. 592–597 (2015)Google Scholar
  8. 8.
    Lahiani, H., Elleuch, M., Kherallah, M.: Real time static hand gesture recognition system for mobile devices. J. Inf. Assur. Secur. 11, 067–076 (2016). ISSN 1554-1010Google Scholar
  9. 9.
    Lahiani, H., Kherallah, M., Neji, M.: Hand pose estimation system based on Viola-Jones algorithm for android devices. In: 13th ACS/IEEE International Conference on Computer Systems and Applications, (AICCSA) (2016)Google Scholar
  10. 10.
    Lahiani, H., Kherallah, M., Neji, M.: Vision based hand gesture recognition for mobile devices: a review. In: Abraham, A., Haqiq, A., Alimi, Adel M., Mezzour, G., Rokbani, N., Muda, A.K. (eds.) HIS 2016. AISC, vol. 552, pp. 308–318. Springer, Cham (2017). CrossRefGoogle Scholar
  11. 11.
    Viola, P., Jones, M.: Rapid objet detection using a boosted cascade of simple features. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 511–518 (2001)Google Scholar
  12. 12.
    Song, J., Sörös, G., Pece, F., Fanello, S.R., Izadi, S., Keskin, C., Hilliges, O.: In-air gestures around unmodified mobile devices. In: 27TH ACM User Interface Software and Technology Symposium (UIST 2014), pp. 319–129 (2014)Google Scholar
  13. 13.
    Jin, C., Omar, Z., Jaward, M.H.: A mobile application of american sign language translation via image processing algorithms. In: 2016 IEEE Region 10 Symposium (TENSYMP), pp. 104–109 (2016)Google Scholar
  14. 14.
    Guerra-Casanova, J., Sánchez-Ávila, C., Bailador, G., de Santos Sierra, A.: Authentication in mobile devices through hand gesture recognition. Int. J. Inf. Secur. 11(2), 65–83 (2012)CrossRefGoogle Scholar
  15. 15.
    Pouke, M., Karhu, A., Hickey, S., Arhippainen, L.: Gaze tracking and non-touch gesture based interaction method for mobile 3D virtual spaces. In: Proceedings of OzCHI, pp. 505–512 (2012)Google Scholar
  16. 16.
    Prasuhn, L., Oyamada, Y., Mochizuki, Y., Ishikawa, H.: A HOG-based hand gesture recognition system on a mobile device. In: IEEE International Conference on Image Processing (ICIP), pp. 3973–3977Google Scholar
  17. 17.
    Harwood, D., Ojala, T., Pietikäinen, M., Kelman, S., Davis, S.: Texture classification by center-symmetric auto-correlation, using Kullback discrimination of distributions. Technical report CAR-TR-678, Computer Vision Laboratory, Center for Automation Research, University of Maryland, College Park, Maryland (1993)Google Scholar
  18. 18.
    Dixit, V., Agrawal, A.: Real time hand detection & tracking for dynamic gesture recognition. Int. J. Intell. Syst. Appl. 08, 38–44 (2015)Google Scholar
  19. 19.
    Tresadern, P.A., Ionita, M.C., Cootes, T.F.: Real-time facial feature tracking on a mobile device. Int. J. Comput. Vis. 96(3), 280–289 (2012)CrossRefGoogle Scholar
  20. 20.
    Lienhart, R., Maydt, J.: An extended set of Haar-like features for rapid object detection. In: Proceedings of IEEE Conference on Image Processing (ICIP 2002), New York, USA, pp. 155–162, September 2002Google Scholar
  21. 21.
    Ferreira, A., Figueiredo, M.: Boosting algorithms: a review of methods, theory, and applications. In: Zhang, C., Ma, Y. (eds.) Ensemble Machine Learning: Methods and Applications, pp. 35–85. Springer, New York (2012). CrossRefGoogle Scholar
  22. 22.
    Frejlichowski, D., Gościewska, K., Forczmański, P., Nowosielski, A., Hofman, R.: Applying image features and AdaBoost classification for vehicle detection in the ‘SM4Public’ system. In: Choraś, Ryszard S. (ed.) Image Processing and Communications Challenges 7. AISC, vol. 389, pp. 81–88. Springer, Cham (2016). CrossRefGoogle Scholar
  23. 23.
  24. 24.
    Lahiani, H., Kherallah, M., Neji, M.: Hand gesture recognition system based on Local Binary Pattern approach for mobile devices. In: 17th International Conference on Intelligent Systems Design and Applications (ISDA) (2017)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Houssem Lahiani
    • 1
    • 3
    • 4
  • Monji Kherallah
    • 2
  • Mahmoud Neji
    • 3
    • 4
  1. 1.National School of Electronics and TelecommunicationsUniversity of SfaxSfaxTunisia
  2. 2.Faculty of SciencesUniversity of SfaxSfaxTunisia
  3. 3.Faculty of Economics and ManagementUniversity of SfaxSfaxTunisia
  4. 4.Multimedia Information Systems and Advanced Computing LaboratorySfaxTunisia

Personalised recommendations