Activity recognition in beach volleyball using a Deep Convolutional Neural Network

Leveraging the potential of Deep Learning in sports
  • Thomas Kautz
  • Benjamin H. Groh
  • Julius Hannink
  • Ulf Jensen
  • Holger Strubberg
  • Bjoern M. Eskofier
Article
Part of the following topical collections:
  1. Sports Analytics

Abstract

Many injuries in sports are caused by overuse. These injuries are a major cause for reduced performance of professional and non-professional beach volleyball players. Monitoring of player actions could help identifying and understanding risk factors and prevent such injuries. Currently, time-consuming video examination is the only option for detailed player monitoring in beach volleyball. The lack of a reliable automatic monitoring system impedes investigations about the risk factors of overuse injuries. In this work, we present an unobtrusive automatic monitoring system for beach volleyball based on wearable sensors. We investigate the possibilities of Deep Learning in this context by designing a Deep Convolutional Neural Network for sensor-based activity classification. The performance of this new approach is compared to five common classification algorithms. With our Deep Convolutional Neural Network, we achieve a classification accuracy of 83.2%, thereby outperforming the other classification algorithms by 16.0%. Our results show that detailed player monitoring in beach volleyball using wearable sensors is feasible. The substantial performance margin between established methods and our Deep Neural Network indicates that Deep Learning has the potential to extend the boundaries of sensor-based activity recognition.

Keywords

Deep Learning Convolutional Neural Network Activity classification Wearable sensors Sports 

References

  1. Aagaard H, Scavenius M, Jørgensen U (1997) An epidemiological analysis of the injury pattern in indoor and in beach volleyball. Int J Sports Med 18(3):217–221CrossRefGoogle Scholar
  2. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: Large-scale machine learning on heterogeneous systems. http://download.tensorflow.org/paper/whitepaper2015.pdf. Accessed 2 Dec 2015
  3. Bahr R, Holme I (2003) Risk factors for sports injuries—a methodological approach. Br J Sports Med 37(5):384–392CrossRefGoogle Scholar
  4. Bahr R, Reeser JC (2003) Injuries among world-class professional beach volleyball players. The Fédération Internationale de Volleyball beach volleyball injury study. Am J Sports Med 31(1):119–125Google Scholar
  5. Bailador G, Roggen D, Tröster G, Trivino G (2007) Real time gesture recognition using continuous time recurrent neural networks. In: Proceedings of the international conference on body area networksGoogle Scholar
  6. Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In: Proceedings of the 2nd international ICST conference on body area networks (BODYNETS-07), Florence, ItalyGoogle Scholar
  7. Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkMATHGoogle Scholar
  8. Blank P, Hoßbach J, Schuldhaus D, Eskofier BM (2015) Sensor-based stroke detection and stroke type classification in table tennis. In: Proceedings of the ACM international symposium on wearable computers, Osaka, Japan, pp 93–100Google Scholar
  9. Bosch Sensortec (2014a) BMA280—digital, triaxial acceleration sensor, Data Sheet, Reutlingen, Germany. http://ae-bst.resource.bosch.com/media/_tech/media/datasheets/BST-BMA280-DS000-11_published.pdf. Accessed 16 Nov 2015
  10. Bosch Sensortec (2014b) BMG160—digital, triaxial gyroscope sensor, data sheet, Reutlingen, Germany. http://ae-bst.resource.bosch.com/media/_tech/media/datasheets/BST-BMG160-DS000-09.pdf. Accessed 16 Nov 2015
  11. Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: 19th international conference on computational statistics, Physica-Verlag, Paris, France, pp 177–186Google Scholar
  12. Boureau Y, Ponce J, LeCun Y (2010) A theoretical analysis of feature pooling in visual recognition. In: Proceedings of the 27th international conference on machine learning (ICML-10)Google Scholar
  13. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140MATHGoogle Scholar
  14. Breiman L (2001) Random forests. Mach Learn 45(1):5–32CrossRefMATHGoogle Scholar
  15. Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression trees. CRC Press, Boca RatonMATHGoogle Scholar
  16. Bridle JS (1990) Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In: Fogleman-Soulie F, Herault J (eds) Neurocomputing: algorithms, architectures and applications. Springer-Verlag Berlin, pp 227–236Google Scholar
  17. Chatfield K, Simonyan K, Vedaldi A, Zisserman A (2014) Return of the devil in the details: delving deep into convolutional nets. arXiv:1405.3531v4
  18. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16(3):321–357MATHGoogle Scholar
  19. Contreras I (2015) Qualcomm research brings server-class machine learning to everyday devicesmaking them smarte. Technical report, Qualcomm Technologies, Inc., San Diego, CA. https://www.qualcomm.com/news/onq/2015/10/01/qualcomm-research-brings-server-class-machine-learning-everyday-devices-making. Accessed 12 Jan 2016
  20. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297MATHGoogle Scholar
  21. Covassin T, Cheng G, Nayar S, Heiden E (2012) Epidemiology of overuse and acute injuries among competitive collegiate athletes. J Athl Train 47(2):198–204Google Scholar
  22. Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27CrossRefMATHGoogle Scholar
  23. Cuspinera LP, Uetsuji S, Ordonez J, Roggen D (2016) Wearable beach volleyball serve type recognition. In: Proceedings of the 20th international symposium on wearable computersGoogle Scholar
  24. Deng L, Li J, Huang J-T, Yao K, Yu D, Seide F, Seltzer M, Zweig G, He X, Williams J, Gong Y, Acero A (2013) Recent advances in deep learning for speech research at Microsoft. In: Proceedings of the international conference on acoustics, speech and signal processing (ICASSP), Vancouver, Canada, pp 8604–8608Google Scholar
  25. Domingos P (1999) MetaCost: a general method for making classifiers cost-sensitive. In: Proceedings of the 5th ACM SIGKDD international conference on knowledge discovery and data mining, San Diego, USA, pp 155–164. ISBN 1581131437. doi:10.1145/312129.312220
  26. Gomez G, Linarth A, Link D, Eskofier BM (2012) Semi-automatic tracking of beach volleyball players. In: 9. Symposium der Sektion Sportinformatik der Deutschen Vereinigung fr Sportwissenschaft, p 22Google Scholar
  27. Groh BH, Kautz T, Schuldhaus D, Eskofier BM (2015) Imu-based trick classification in skateboarding. In: Proceedings of the KDD workshop on large-scale sports analytics, Sydney, AustraliaGoogle Scholar
  28. Groh BH, Fleckenstein M, Eskofier B (2016) Wearable trick classification in freestyle snowboarding. In: Proceedings of the 13th annual international body sensor networks conference, San Francisco, USA, pp 89–93Google Scholar
  29. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. arXiv:1502.01852
  30. Hinton G, Deng L, Yu D, Dahl GE, Mohamed A-Rahman, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B (2012a) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97CrossRefGoogle Scholar
  31. Hinton G, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012b) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580
  32. Ho TK, Hull JJ, Srihari SN (1994) Decision combination in multiple classifier systems. IEEE Trans Pattern Anal Mach Intell 16(1):66–75CrossRefGoogle Scholar
  33. Holleczek T, Schoch J, Arnrich B, Troster G (2010) Recognizing turns and other snowboarding activities with a gyroscope. In: Proceedings of the international symposium on wearable computers (ISWC), Seoul, South Korea, pp 1–8Google Scholar
  34. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167
  35. Jarning JM, Mok K-M, Hansen BH, Bahr R (2015) Application of a tri-axial accelerometer to estimate jump frequency in volleyball. Sports Biomech 14(1):95–105CrossRefGoogle Scholar
  36. Jarrett K, Kavukcuoglu K, Ranzato M, LeCun Y (2009) What is the best multi-stage architecture for object recognition? In: Proceedings of the 12th international conference on computer vision, Kyoto, Japan, pp 2146–2153Google Scholar
  37. Jordan MI (1994) A statistical approach to decision tree modeling. In: Proceedings of the 7th annual conference on computational learning theoryGoogle Scholar
  38. Kautz T, Groh BH, Eskofier BM (2015) Sensor fusion for multi-player activity recognition in game sports. In: Proceedings of the KDD workshop on large-scale sports analytics, Sydney, AustraliaGoogle Scholar
  39. Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of the international conference on learning representations, San Diego, USA, pp 1–13Google Scholar
  40. Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1):273–324CrossRefMATHGoogle Scholar
  41. Kotsiantis S, Kanellopoulos D, Pintelas P (2006) Handling imbalanced datasets: a review. Science 30:25–36Google Scholar
  42. Lane ND, Georgiev P (2015) Can deep learning revolutionize mobile sensing? In: Proceedings of the 16th international workshop on mobile computing systems and applications, Santa Fe, USA, pp 117–122Google Scholar
  43. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1990) Handwritten digit recognition with a back-propagation network. In: Touretzky D (ed) Advances in neural information processing systems. Morgan Kaufmann, San Francisco, vol 2, pp 396–404Google Scholar
  44. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444CrossRefGoogle Scholar
  45. Leutheuser H, Schuldhaus D, Eskofier BM (2013) Hierarchical, multi-sensor based classification of daily life activities: comparison with state-of-the-art algorithms using a benchmark dataset. PLoS ONE 8(10):e75196CrossRefGoogle Scholar
  46. Lewis DD (1998) Naive bayes at forty: the independence assumption in information retrieval. In: Proceedings of the 10th European conference on machine learning, Chemnitz, Germany, pp 4–15Google Scholar
  47. Link D, Haag T, Rau C, Lames M (2010) Wettkampfanalyse im Beachvolleyball mittels Positionsdaten. BISp Jahrb Forsch 11:171–174Google Scholar
  48. Meir R, Rätsch G (2003) An introduction to boosting and leveraging. In: Mendelson S, Smola AJ (eds) Advanced lectures on machine learning. Springer, New York, pp 118–183Google Scholar
  49. Mitchell E, Monaghan D, O’Connor NE (2013) Classification of sporting activities using smartphone accelerometers. Sensors 13(4):5317–5337CrossRefGoogle Scholar
  50. Nguyen LNN, Rodríguez-Martín D, Català A, Pérez-López C, Samà A, Cavallaro A (2015) Basketball activity recognition using wearable inertial measurement units. In: Proceedings of the 16th international conference on human computer interaction, Heraklion, Greece, p 60Google Scholar
  51. Nguyen-Dinh L.-V, Roggen D, Calatroni A, Tröster G (2012) Improving online gesture recognition with template matching methods in accelerometer data. In: Proceedings of the 12th international conference on intelligent systems design and applications (ISDA), pp 831–836Google Scholar
  52. Ordóñez FJ, Roggen D (2016) Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1):115CrossRefGoogle Scholar
  53. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830MathSciNetMATHGoogle Scholar
  54. Ravi N, Dandekar N, Mysore P, Littman ML (2005) Activity recognition from accelerometer data. In: Proceedings of the 17th national conference on innovative applications of artificial intelligence, Pittsburgh, USA, vol 5, pp 1541–1546Google Scholar
  55. Rawashdeh SA, Rafeldt DA, Uhl TL, Lumpp JE (2015) Wearable motion capture unit for shoulder injury prevention. In: Proceedings of the IEEE 12th international conference on wearable and implantable body sensor networks (BSN), London, UK, pp 1–6Google Scholar
  56. Reeser JC, Verhagen E, Brinner WW, Askeland I, Bahr R (2006) Strategies for the prevention of volleyball related injuries. Br J Sports Med 40(7):594–600CrossRefGoogle Scholar
  57. Roggen D, Cuspinera LP, Pombo G, Ali F, Nguyen-Dinh L-V (2015) Limited-memory warping LCSS for real-time low-power pattern recognition in wireless nodes. In: Abdelzaher T, Pereira N, Tovar E (eds) Wireless sensor networks. Springer, Cham, pp 151–167Google Scholar
  58. Santos JM, Embrechts M (2009) On the use of the adjusted rand index as a metric for evaluating supervised classification. In: 19th international conference on artificial neural networks (ICANN 2009), Limassol, Cyprus, pp 175–184Google Scholar
  59. Schuldhaus D, Zwick C, Körger H, Dorschky E, Kirk R, Eskofier BM (2015) Inertial sensor-based approach for shot/pass classification during a soccer match. In: Proceedings of the KDD workshop on large-scale sports analytics, Sydney, AustraliaGoogle Scholar
  60. Shore JE, Johnson RW (1981) Properties of cross-entropy minimization. IEEE Trans Inf Theory 27(4):472–482MathSciNetCrossRefMATHGoogle Scholar
  61. Stiefmeier T, Roggen D, Ogris G, Lukowicz P, Tröster G (2008) Wearable activity tracking in car manufacturing. IEEE Pervasive Comput 2:42–50CrossRefGoogle Scholar
  62. STMicroelectronics (2015) STM32L151CC microcontroller, data sheet, Geneva, Switzerland. http://www.st.com/st-web-ui/static/active/en/resource/technical/document/datasheet/DM00048356.pdf. Accessed 16 Nov 2015
  63. van Mechelen W, Hynek H, Kemper HCG (1992) Incidence, severity, aetiology and prevention of sports injuries. Sports Med 14(2):82–99CrossRefGoogle Scholar
  64. Zeiler MD, Ranzato M, Monga R, Mao M, Yang K, Le QV, Nguyen P, Senior A, Vanhoucke V, Dean J, et al (2013) On rectified linear units for speech processing. In: Proceedings of the IEEE international conference on acoustics, speech and signal processing (ICASSP), Vancouver, Canada, pp 3517–3521Google Scholar
  65. Zeng M, Nguyen LT, Yu B, Mengshoel OJ, Zhu J, Wu P, Zhang J (2014) Convolutional neural networks for human activity recognition using mobile sensors. In: Proceedings of the 6th international conference on mobile computing, applications and services (MobiCASE), Austin, USA, pp 197–205Google Scholar
  66. Zheng Y, Liu Q, Chen E, Ge Y, Zhao JL (2014) Time series classification using multi-channels deep convolutional neural networks. In: Li F, Li G, Hwang S, Yao B, Zhang Z (eds) Web-age information management. Springer, Cham, pp 298–310Google Scholar

Copyright information

© The Author(s) 2017

Authors and Affiliations

  • Thomas Kautz
    • 1
  • Benjamin H. Groh
    • 1
  • Julius Hannink
    • 1
  • Ulf Jensen
    • 1
  • Holger Strubberg
    • 2
  • Bjoern M. Eskofier
    • 3
  1. 1.Digital Sports GroupFriedrich-Alexander-Universität Erlangen-Nürnberg (FAU)ErlangenGermany
  2. 2.Zentrum für Hochschulsport (ZfH)Universität LeipzigLeipzigGermany
  3. 3.ErlangenGermany

Personalised recommendations