Neural Computing and Applications

, Volume 31, Issue 12, pp 9349–9361 | Cite as

Real-time Yoga recognition using deep learning

  • Santosh Kumar YadavEmail author
  • Amitojdeep Singh
  • Abhishek Gupta
  • Jagdish Lal Raheja
Original Article


An approach to accurately recognize various Yoga asanas using deep learning algorithms has been presented in this work. A dataset of six Yoga asanas (i.e. Bhujangasana, Padmasana, Shavasana, Tadasana, Trikonasana, and Vrikshasana) has been created using 15 individuals (ten males and five females) with a normal RGB webcam and is made publicly available. A hybrid deep learning model is proposed using convolutional neural network (CNN) and long short-term memory (LSTM) for Yoga recognition on real-time videos, where CNN layer is used to extract features from keypoints of each frame obtained from OpenPose and is followed by LSTM to give temporal predictions. To the best of our knowledge, this is the first study using an end-to-end deep learning pipeline to detect Yoga from videos. The system achieves a test accuracy of 99.04% on single frames and 99.38% accuracy after polling of predictions on 45 frames of the videos. Using a model with temporal data leverages the information from previous frames to give an accurate and robust result. We have also tested the system in real time for a different set of 12 persons (five males and seven females) and achieved 98.92% accuracy. Experimental results provide a qualitative assessment of the method as well as a comparison to the state-of-the-art.


Activity recognition OpenPose Posture analysis Sports training Yoga 



The work is carried out at CSIR-CEERI, Pilani and authors would like to thank Director, CSIR-CEERI, Pilani for providing the infrastructure and technical support and also, we appreciate the assistance provided by CSIR, India.


  1. 1.
    Gao Z, Zhang H, Liu AA et al (2016) Human action recognition on depth dataset. Neural Comput Appl 27:2047–2054. CrossRefGoogle Scholar
  2. 2.
    Poppe R (2010) A survey on vision-based human action recognition. Image Vis Comput 28:976–990. CrossRefGoogle Scholar
  3. 3.
    Weinland D, Ronfard R, Boyer E (2011) A survey of vision-based methods for action representation, segmentation and recognition. Comput Vis Image Underst 115:224–241. CrossRefGoogle Scholar
  4. 4.
    Ladjailia A, Bouchrika I, Merouani HF et al (2019) Human activity recognition via optical flow: decomposing activities into basic actions. Neural Comput Appl. CrossRefGoogle Scholar
  5. 5.
    Suto J (2018) Comparison of offline and real-time human activity recognition results using machine learning techniques. Neural Comput Appl. CrossRefGoogle Scholar
  6. 6.
    Guddeti RR, Dang G, Williams MA, Alla VM (2018) Role of Yoga in cardiac disease and rehabilitation. J Cardiopulm Rehabil Prev. CrossRefGoogle Scholar
  7. 7.
    Neumark-Sztainer D, Watts AW, Rydell S (2018) Yoga and body image: how do young adults practicing yoga describe its impact on their body image? Body Image 27:156–168. CrossRefGoogle Scholar
  8. 8.
    Halliwell E, Dawson K, Burkey S (2019) A randomized experimental evaluation of a yoga-based body image intervention. Body Image 28:119–127. CrossRefGoogle Scholar
  9. 9.
    Sathyanarayanan G, Vengadavaradan A, Bharadwaj B (2019) Role of yoga and mindfulness in severe mental illnesses: a narrative review. Int J Yoga 12:3–28. CrossRefGoogle Scholar
  10. 10.
    Patil S, Pawar A, Peshave A et al (2011) Yoga tutor: visualization and analysis using SURF algorithm. In: Proceedings of 2011 IEEE control system graduate research colloquium, ICSGRC 2011, pp 43–46Google Scholar
  11. 11.
    Chen HT, He YZ, Hsu CC et al (2014) Yoga posture recognition for self-training. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), pp 496–505CrossRefGoogle Scholar
  12. 12.
    Chen HT, He YZ, Chou CL et al (2013) Computer-assisted self-training system for sports exercise using kinects. In: Electronic proceedings of 2013 IEEE international conference multimedia and expo work ICMEW 2013 3–6.
  13. 13.
    Schure MB, Christopher J, Christopher S (2008) Mind–body medicine and the art of self-care: teaching mindfulness to counseling students through yoga, meditation, and qigong. J Couns Dev. CrossRefGoogle Scholar
  14. 14.
    Lim S-A, Cheong K-J (2015) Regular Yoga practice improves antioxidant status, immune function, and stress hormone releases in young healthy people: a randomized, double-blind, controlled pilot study. J Altern Complement Med 1:1. CrossRefGoogle Scholar
  15. 15.
    Chen HT, He YZ, Hsu CC (2018) Computer-assisted yoga training system. Multimed Tools Appl 77:23969–23991. CrossRefGoogle Scholar
  16. 16.
    Maanijou R, Mirroshandel SA (2019) Introducing an expert system for prediction of soccer player ranking using ensemble learning. Neural Comput Appl. CrossRefGoogle Scholar
  17. 17.
    Nordsborg NB, Espinosa HG, Thiel DV (2014) Estimating energy expenditure during front crawl swimming using accelerometers. Procedia Eng 72:132–137. CrossRefGoogle Scholar
  18. 18.
    Connaghan D, Kelly P, O’Connor NE et al (2011) Multi-sensor classification of tennis strokes. Proc IEEE Sens. CrossRefGoogle Scholar
  19. 19.
    Shan CZ, Su E, Ming L (2015) Investigation of upper limb movement during badminton smash. In: 2015 10th Asian Control conference, pp 1–6.
  20. 20.
    Waldron M, Twist C, Highton J et al (2011) Movement and physiological match demands of elite rugby league using portable global positioning systems. J Sports Sci 29:1223–1230. CrossRefGoogle Scholar
  21. 21.
    Pai PF, ChangLiao LH, Lin KP (2017) Analyzing basketball games by a support vector machines with decision tree model. Neural Comput Appl 28:4159–4167. CrossRefGoogle Scholar
  22. 22.
    Bai L, Efstratiou C, Ang CS (2016) WeSport: utilising wrist-band sensing to detect player activities in basketball games. In: 2016 IEEE international conference on pervasive computing and communication workshops, PerCom workshops 2016. IEEE, pp 1–6Google Scholar
  23. 23.
    Yahya U, Arosha Senanayake SMN, Naim AG (2018) A database-driven neural computing framework for classification of vertical jump patterns of healthy female netballers using 3D kinematics—EMG features. Neural Comput Appl. CrossRefGoogle Scholar
  24. 24.
    Przednowek K, Wiktorowicz K, Krzeszowski T, Iskra J (2018) A web-oriented expert system for planning hurdles race training programmes. Neural Comput Appl 1:1–17. CrossRefGoogle Scholar
  25. 25.
    Luo Z, Yang W, Ding ZQ et al (2011) “Left arm up!” interactive Yoga training in virtual environment. In: 2011 IEEE virtual real conference, pp 261–262.
  26. 26.
    Wu W, Yin W, Guo F (2010) Learning and self-instruction expert system for Yoga. In: Proceedings of 2010 2nd International Work Intelligent System Application: ISA, pp 2–5.
  27. 27.
    Trejo EW, Yuan P (2018) Recognition of Yoga poses through an interactive system with kinect device. In: 2018 2nd international conference robotics and automation science: ICRAS, pp 12–17.
  28. 28.
    Mohanty A, Ahmed A, Goswami T et al (2017) Robust pose recognition using deep learning. In: Raman B, Kumar S, Roy PP, Sen D (eds) Advances in intelligent systems and computing. Springer, Singapore, pp 93–105. CrossRefGoogle Scholar
  29. 29.
    Toshev A, Szegedy C (2013) DeepPose: human pose estimation via deep neural networks.
  30. 30.
    Cao Z, Simon T, Wei SE, Sheikh Y (2017) Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of 30th IEEE conference computer vision and pattern recognition, CVPR 2017, 2017 January, pp 1302–1310.
  31. 31.
    Qiao S, Wang Y, Li J (2017) Real time human gesture grading based on OpenPose. International Congress Image Signal ProcessGoogle Scholar
  32. 32.
    Luo G, Sun G, Wang K et al (2016) A novel left ventricular volumes prediction method based on deep learning network in cardiac MRI, pp 1604–1610.
  33. 33.
    Kiros R, Zhu Y, Salakhutdinov R et al (2015) Skip-thought vectors, pp 1–11.
  34. 34.
    Li J, Luong M-T, Jurafsky D (2015) A hierarchical neural autoencoder for paragraphs and documents.
  35. 35.
    Grushin A, Monner DD, Reggia JA, Mishra A (2013) Robust human action recognition via long short-term memory. In: International joint conference on neural networks (IJCNN), pp 1–8Google Scholar
  36. 36.
    Wang J, Yu L-C, Lai KR, Zhang X (2016) Dimensional sentiment analysis using a regional CNN-LSTM model, pp 225–230.
  37. 37.
    Zhou C, Sun C, Liu Z, Lau FCM (2015) A C-LSTM neural network for text classification. CrossRefGoogle Scholar
  38. 38.
    Oh SL, Ng EYK, Tan RS, Acharya UR (2018) Automated diagnosis of arrhythmia using combination of CNN and LSTM techniques with variable length heart beats. Comput Biol Med 102:278–287. CrossRefGoogle Scholar
  39. 39.
    Xu Z, Li S, Deng W (2016) Learning temporal features using LSTM-CNN architecture for face anti-spoofing. In: Proceedings of 3rd IAPR Asian conference pattern recognition, ACPR 2015, pp 141–145Google Scholar
  40. 40.
    Wang X, Gao L, Song J, Shen H (2017) Beyond frame-level CNN: saliency-Aware 3-D CNN with LSTM for video action recognition. IEEE Signal Process Lett 24:510–514. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Santosh Kumar Yadav
    • 1
    Email author
  • Amitojdeep Singh
    • 2
  • Abhishek Gupta
    • 2
  • Jagdish Lal Raheja
    • 1
  1. 1.Cyber Physical SystemCSIR – Central Electronics Engineering Research InstitutePilaniIndia
  2. 2.Department of Computer ScienceBirla Institute of Technology and Science (BITS)PilaniIndia

Personalised recommendations