Abstract
In this paper, the method to overcome the limitations of the existing three-dimensional golf swing analysis system by using deep learning technology, and analyze the three-dimensional quantitative information through sequence images acquired with a single camera is studied. In this paper, CNN was used to extract the appropriate features from the image of the golf frontal swing sequence, and a regression model based on Bi-LSTM was used to predict the correct information in each sequence. This classifies the major swing section, and analyzes the quantitative status of the twisting angles of the upper body, head, shoulder and pelvis for body-sway, head-up and X-factor analysis. For the experiment, in this paper, a total of 520 times swing data were obtained using no. 1 wood club and no. 7 iron club from five subjects. In the major swing section classification experiment, each swing section was classified with an average accuracy of about 95.44%. Quantitative analysis results from each analysis model showed that the upper body motion prediction RMSE averaged 4.23 degrees, the head motion prediction RMSE averaged 5.18 degrees, and the shoulder and pelvis twisting angle prediction RMSE averaged 3.86 degrees. As a result, it was confirmed that a three-dimensional quantitative analysis based on sequence images is possible.
Similar content being viewed by others
References
An J, Wulf G, Kim S (2013) Increased carry distance and X-factor stretch in golf through an external focus of attention. J Mot Learn Dev 1(1):2–11
Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166
Bourgain M, Hybois S, Thoreux P, Rouillon O, Rouch P, Sauret C (2018) Effect of shoulder model complexity in upper-body kinematics analysis of the golf swing. J Biomech 25(75):154–158. https://doi.org/10.1016/j.jbiomech.2018.04.025
Chen HT, Huang TW, Chou CL, Tsai HC, Lee SY (2015) Improving golf swing skills using intelligent glasses, In Proceedings of IEEE Visual Communications and Image Processing, pp. 1–4. https://doi.org/10.1109/VCIP.2015.7457819
Chiu JPC, Nichols E (2016) Named entity recognition with bidirectional LSTM-CNNs. Trans Assoc Comput Linguist 4:357–370
Chotimanus P, Cooharojananone N, Phimoltares S (2012) Real swing extraction for video indexing in golf practice video, In Proceedings of the Computing Communications and Applications Conference, pp. 420–425. https://doi.org/10.1109/ComComAp.2012.6154885
Fradkin A (2017) Routledge International Handbook of Golf Science(1st Edition), Routledge, pp. 490
Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning (vol. 1). Cambridge: MIT press.
Ha CS (2008) The kinematic and kinetic variables of head up putting involved in golf. Korea J Sports Sci 17(1):675–684
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Hsu YL, Chen YT, Chou PH, Kou YC, Chen YC, Su HY (2016) Golf swing motion detection using an inertial-sensor-based portable instrument. In Proceedings of the 2016 IEEE International Conference on Consumer Electronics-Taiwan, pp. 1–2. https://doi.org/10.1109/ICCE-TW.2016.7521016
Hua Y, Mou L, Zhu XX (2019) Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional LSTM network for multilabel aerial image classification. ISPRS J Photogramm Remote Sens 149:188–199
Huang SY, Kuo KP, Lin YH (2015) A golf swing analysis system using wii balance board and kinect sensors for novice players. Multimed Tools Appl 74(23):10679–10696
Ji S, Zhang C, Xu A, Shi Y, Duan Y (2018) 3D convolutional neural networks for crop classification with multi-temporal remote sensing images, Remote Sensing, vol. 10, no. 1, 75
Jiao L, Bie R, Wu H, Wei Y, Kos A, Umek A (2018) Golf swing data classification with deep convolutional neural network. The IPSI BgD Trans Int Res 14(1):29–34
Jin S, Lee W, Kim Y, Lee D, Jung S (2017) Deep-learning based adaptive power control on common data link system. J Inst Electron Inf Engin 54(11):33–44
Kelly P, Healy A, Moran K, O’Connor NE (2010) A virtual coaching environment for improving golf swing technique, In Proceedings of the 2010 ACM Workshop on Surreal Media and Virtual Cloning, pp. 51-56
Kim PK (2014) Automatic extraction of golf swing features using a single Kinect. J Korea Soc Comput Inf 19(12):197–207
Kim HJ (2017) Deep learning city: a big data analytics framework for smart cities. Inf policy 24(4):79–92
Kim SJ, Shin YS, Choi JY (2002) A comparative study of kinematic variables related with address and impact in golf driver swing. J Spor Leis Stud 17:145–158
Kim YJ, Kim KD, Kim SH, Lee S, Lee HS (2017) Golf swing analysis system with a dual band and motion analysis algorithm. IEEE Trans Consum Electron 63(3):309–317
Ko KR (2018) A study on golf swing analysis based on sequence images using CNN and LSTM, PhD diss., Graduate School of Chosun University
Ko KR, Pan SB (2015) Feature Extraction and Classification of Posture for Four-Joint based Human Motion Data Analysis, J Ins Elec Inf Eng 52(6):117–125. https://doi.org/10.5573/ieie.2015.52.6.117
Ko KR, Pan SB (2018) Automatic classification of golf swing section based on single camera using transfer learning. In Proceedings of the KIIT Summer Conference, pp. 37–39
Ko KR, Pan SB (2018) Golf swing motion analysis based on inertial sensors. J Korean Inst Inf Tech 16(1):125–131
Lee SH, Cho HY (2009) An condition analysis about the exercise injury factors and treatment of golf club members. J Spor Leis Stud 37(2):1303–1314
Lee KI, Park JJ (2006) The kinematic analysis of golf swing by amateur female golfers. J Spor Leis Stud 28:349–362
Lindsay DM, Vandervoort AA (2014) Golf-related low back pain: a review of causative factors and prevention strategies. Asian J Sports Med 5(4):e24289. https://doi.org/10.5812/asjsm.24289
Liu Q, Zhou F, Hang R, Yuan X (2017) Bidirectional-convolutional LSTM based spectralspatial feature learning for hyperspectral image classification, Remote Sensing, vol. 9, no. 12, 1330
McGill S (2010) Core training: evidence translating to better performance and injury prevention. Strength Cond J 32(3):33–46
McNally W, Vats K, Pinto T, Dulhanty C, McPhee J, Wong A (2019) “GolfDB: a video database for golf swing sequencing,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops
Mun F, Suh SW, Park HJ, Choi A (2015) Kinematic relationship between rotation of lumbar spine and hip joints during golf swing in professional golfers. Biomed Eng Online 14(14):41. https://doi.org/10.1186/s12938-015-0041-5
Najafi B, Lee-Eng J, Wrobel JS, Goebel R (2015) Estimation of center of mass trajectory using wearable sensors during golf swing. J Sports Sci Med 14(2):354–363
Nam CNK, Kang HJ, Suh YS (2014) Golf swing motion tracking using inertial sensors and a stereo camera. IEEE Trans Instrum Meas 63(4):943–952
Nike Sport Research Lab (n.d.), [online] Available at: http://news.nike.com/news/a-look-inside-nike-s-sport-research-lab
O’Donoghue P (2014) An introduction to performance analysis of sport, Routledge, pp. 274
Park J, Kwak S (2013) Golf swing classification using fuzzy system. J Broadcast Eng 18(3):380–392
Park S, Chang JY, Jeong H, Lee JH, Park J (2017) Accurate and efficient 3D human pose estimation algorithm using single depth images for pose analysis in golf. In Proceedings of 2017 IEEE Conf Comput Vis Pattern Recogn Workshops, pp. 105-113. https://doi.org/10.1109/CVPRW.2017.19
Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681
Sim FK, Sundaraj K (2010) Human motion tracking on broadcast golf swing video using optical flow and template matching, Comput Appl Ind Electron, pp. 169–173
Smith AC, Roberts JR, Wallace ES, Kong P, Forrester SE (2016) Comparison of two-and three-dimensional methods for analysis of trunk kinematic variables in the golf swing. J Appl Biomech 32(1):23–31
Smith AC, Roberts JR, Kong PW, Forrester SE (2017) Comparison of center of gravity and center of pressure patterns in the golf swing. Eur J Sport Sci 17(2):168–178
Tucker CJ (2016) Golf injuries. Sports Med Update:2–6
Ueda M, Negoro H, Kurihara Y, Watanabe K (2013) Measurement of angular motion in golf swing by a local sensor at the grip end of a golf club. IEEE Trans Hum Mach Syst 43(4):398–404
Ullah A, Ahmad J, Muhammad K, Sajjad M, Baik SW (2017) Action recognition in video sequences using deep bi-directional LSTM with CNN features. IEEE Access 6:1155–1166
Watanabe K, Hokari M (2006) Kinematical analysis and measurement of sports form. IEEE Trans Syst Man Cybern Syst Hum 36(3):549–557
Zhang Z, Zhang Y, Kos A, Umek A (2017) A sensor-based golfer-swing signature recognition method using linear support vector machine. Elektrotehniski Vestnik 84(5):247–252
Zhang Z, Zhang Y, Kos A, Umek A (2018) Strain gage sensor based golfer identification using machine learning algorithms. Procedia Compu Sci 129:135–140
Zheng N, Barrentine SW, Fleisig GS, Andrews JR (2008) Kinematic analysis of swing in pro and amateur golfers. Int J Sports Med 29(6):487–493
Zhong L, Hu L, Zhou H (2019) Deep learning based multi-temporal crop classification. Remote Sens Environ 221:430–443
Acknowledgements
This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (No. 2017R1A6A1A03015496) and by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. NRF-2018R1A2B6001984).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ko, KR., Pan, S.B. CNN and bi-LSTM based 3D golf swing analysis by frontal swing sequence images. Multimed Tools Appl 80, 8957–8972 (2021). https://doi.org/10.1007/s11042-020-10096-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-10096-0