Skip to main content
SpringerLink
Log in

Activity recognition in beach volleyball using a Deep Convolutional Neural Network

Leveraging the potential of Deep Learning in sports

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

Many injuries in sports are caused by overuse. These injuries are a major cause for reduced performance of professional and non-professional beach volleyball players. Monitoring of player actions could help identifying and understanding risk factors and prevent such injuries. Currently, time-consuming video examination is the only option for detailed player monitoring in beach volleyball. The lack of a reliable automatic monitoring system impedes investigations about the risk factors of overuse injuries. In this work, we present an unobtrusive automatic monitoring system for beach volleyball based on wearable sensors. We investigate the possibilities of Deep Learning in this context by designing a Deep Convolutional Neural Network for sensor-based activity classification. The performance of this new approach is compared to five common classification algorithms. With our Deep Convolutional Neural Network, we achieve a classification accuracy of 83.2%, thereby outperforming the other classification algorithms by 16.0%. Our results show that detailed player monitoring in beach volleyball using wearable sensors is feasible. The substantial performance margin between established methods and our Deep Neural Network indicates that Deep Learning has the potential to extend the boundaries of sensor-based activity recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. According to the sensor data sheet (Bosch Sensortec 2014a), the sampling rate of the acceleration sensors in low-power mode is 40 Hz. However, our tests showed, that the actual sampling rate was only approximately 39 Hz.

References

  • Aagaard H, Scavenius M, Jørgensen U (1997) An epidemiological analysis of the injury pattern in indoor and in beach volleyball. Int J Sports Med 18(3):217–221

    Article  Google Scholar 

  • Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: Large-scale machine learning on heterogeneous systems. http://download.tensorflow.org/paper/whitepaper2015.pdf. Accessed 2 Dec 2015

  • Bahr R, Holme I (2003) Risk factors for sports injuries—a methodological approach. Br J Sports Med 37(5):384–392

    Article  Google Scholar 

  • Bahr R, Reeser JC (2003) Injuries among world-class professional beach volleyball players. The Fédération Internationale de Volleyball beach volleyball injury study. Am J Sports Med 31(1):119–125

    Article  Google Scholar 

  • Bailador G, Roggen D, Tröster G, Trivino G (2007) Real time gesture recognition using continuous time recurrent neural networks. In: Proceedings of the international conference on body area networks

  • Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In: Proceedings of the 2nd international ICST conference on body area networks (BODYNETS-07), Florence, Italy

  • Bishop CM (2006) Pattern recognition and machine learning. Springer, New York

    MATH  Google Scholar 

  • Blank P, Hoßbach J, Schuldhaus D, Eskofier BM (2015) Sensor-based stroke detection and stroke type classification in table tennis. In: Proceedings of the ACM international symposium on wearable computers, Osaka, Japan, pp 93–100

  • Bosch Sensortec (2014a) BMA280—digital, triaxial acceleration sensor, Data Sheet, Reutlingen, Germany. http://ae-bst.resource.bosch.com/media/_tech/media/datasheets/BST-BMA280-DS000-11_published.pdf. Accessed 16 Nov 2015

  • Bosch Sensortec (2014b) BMG160—digital, triaxial gyroscope sensor, data sheet, Reutlingen, Germany. http://ae-bst.resource.bosch.com/media/_tech/media/datasheets/BST-BMG160-DS000-09.pdf. Accessed 16 Nov 2015

  • Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: 19th international conference on computational statistics, Physica-Verlag, Paris, France, pp 177–186

  • Boureau Y, Ponce J, LeCun Y (2010) A theoretical analysis of feature pooling in visual recognition. In: Proceedings of the 27th international conference on machine learning (ICML-10)

  • Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    MATH  Google Scholar 

  • Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  • Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression trees. CRC Press, Boca Raton

    MATH  Google Scholar 

  • Bridle JS (1990) Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In: Fogleman-Soulie F, Herault J (eds) Neurocomputing: algorithms, architectures and applications. Springer-Verlag Berlin, pp 227–236

  • Chatfield K, Simonyan K, Vedaldi A, Zisserman A (2014) Return of the devil in the details: delving deep into convolutional nets. arXiv:1405.3531v4

  • Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16(3):321–357

    MATH  Google Scholar 

  • Contreras I (2015) Qualcomm research brings server-class machine learning to everyday devicesmaking them smarte. Technical report, Qualcomm Technologies, Inc., San Diego, CA. https://www.qualcomm.com/news/onq/2015/10/01/qualcomm-research-brings-server-class-machine-learning-everyday-devices-making. Accessed 12 Jan 2016

  • Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  • Covassin T, Cheng G, Nayar S, Heiden E (2012) Epidemiology of overuse and acute injuries among competitive collegiate athletes. J Athl Train 47(2):198–204

    Article  Google Scholar 

  • Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27

    Article  MATH  Google Scholar 

  • Cuspinera LP, Uetsuji S, Ordonez J, Roggen D (2016) Wearable beach volleyball serve type recognition. In: Proceedings of the 20th international symposium on wearable computers

  • Deng L, Li J, Huang J-T, Yao K, Yu D, Seide F, Seltzer M, Zweig G, He X, Williams J, Gong Y, Acero A (2013) Recent advances in deep learning for speech research at Microsoft. In: Proceedings of the international conference on acoustics, speech and signal processing (ICASSP), Vancouver, Canada, pp 8604–8608

  • Domingos P (1999) MetaCost: a general method for making classifiers cost-sensitive. In: Proceedings of the 5th ACM SIGKDD international conference on knowledge discovery and data mining, San Diego, USA, pp 155–164. ISBN 1581131437. doi:10.1145/312129.312220

  • Gomez G, Linarth A, Link D, Eskofier BM (2012) Semi-automatic tracking of beach volleyball players. In: 9. Symposium der Sektion Sportinformatik der Deutschen Vereinigung fr Sportwissenschaft, p 22

  • Groh BH, Kautz T, Schuldhaus D, Eskofier BM (2015) Imu-based trick classification in skateboarding. In: Proceedings of the KDD workshop on large-scale sports analytics, Sydney, Australia

  • Groh BH, Fleckenstein M, Eskofier B (2016) Wearable trick classification in freestyle snowboarding. In: Proceedings of the 13th annual international body sensor networks conference, San Francisco, USA, pp 89–93

  • He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. arXiv:1502.01852

  • Hinton G, Deng L, Yu D, Dahl GE, Mohamed A-Rahman, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B (2012a) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97

    Article  Google Scholar 

  • Hinton G, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012b) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580

  • Ho TK, Hull JJ, Srihari SN (1994) Decision combination in multiple classifier systems. IEEE Trans Pattern Anal Mach Intell 16(1):66–75

    Article  Google Scholar 

  • Holleczek T, Schoch J, Arnrich B, Troster G (2010) Recognizing turns and other snowboarding activities with a gyroscope. In: Proceedings of the international symposium on wearable computers (ISWC), Seoul, South Korea, pp 1–8

  • Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

  • Jarning JM, Mok K-M, Hansen BH, Bahr R (2015) Application of a tri-axial accelerometer to estimate jump frequency in volleyball. Sports Biomech 14(1):95–105

    Article  Google Scholar 

  • Jarrett K, Kavukcuoglu K, Ranzato M, LeCun Y (2009) What is the best multi-stage architecture for object recognition? In: Proceedings of the 12th international conference on computer vision, Kyoto, Japan, pp 2146–2153

  • Jordan MI (1994) A statistical approach to decision tree modeling. In: Proceedings of the 7th annual conference on computational learning theory

  • Kautz T, Groh BH, Eskofier BM (2015) Sensor fusion for multi-player activity recognition in game sports. In: Proceedings of the KDD workshop on large-scale sports analytics, Sydney, Australia

  • Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of the international conference on learning representations, San Diego, USA, pp 1–13

  • Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1):273–324

    Article  MATH  Google Scholar 

  • Kotsiantis S, Kanellopoulos D, Pintelas P (2006) Handling imbalanced datasets: a review. Science 30:25–36

    Google Scholar 

  • Lane ND, Georgiev P (2015) Can deep learning revolutionize mobile sensing? In: Proceedings of the 16th international workshop on mobile computing systems and applications, Santa Fe, USA, pp 117–122

  • LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1990) Handwritten digit recognition with a back-propagation network. In: Touretzky D (ed) Advances in neural information processing systems. Morgan Kaufmann, San Francisco, vol 2, pp 396–404

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  • Leutheuser H, Schuldhaus D, Eskofier BM (2013) Hierarchical, multi-sensor based classification of daily life activities: comparison with state-of-the-art algorithms using a benchmark dataset. PLoS ONE 8(10):e75196

    Article  Google Scholar 

  • Lewis DD (1998) Naive bayes at forty: the independence assumption in information retrieval. In: Proceedings of the 10th European conference on machine learning, Chemnitz, Germany, pp 4–15

  • Link D, Haag T, Rau C, Lames M (2010) Wettkampfanalyse im Beachvolleyball mittels Positionsdaten. BISp Jahrb Forsch 11:171–174

    Google Scholar 

  • Meir R, Rätsch G (2003) An introduction to boosting and leveraging. In: Mendelson S, Smola AJ (eds) Advanced lectures on machine learning. Springer, New York, pp 118–183

  • Mitchell E, Monaghan D, O’Connor NE (2013) Classification of sporting activities using smartphone accelerometers. Sensors 13(4):5317–5337

    Article  Google Scholar 

  • Nguyen LNN, Rodríguez-Martín D, Català A, Pérez-López C, Samà A, Cavallaro A (2015) Basketball activity recognition using wearable inertial measurement units. In: Proceedings of the 16th international conference on human computer interaction, Heraklion, Greece, p 60

  • Nguyen-Dinh L.-V, Roggen D, Calatroni A, Tröster G (2012) Improving online gesture recognition with template matching methods in accelerometer data. In: Proceedings of the 12th international conference on intelligent systems design and applications (ISDA), pp 831–836

  • Ordóñez FJ, Roggen D (2016) Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1):115

    Article  Google Scholar 

  • Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  • Ravi N, Dandekar N, Mysore P, Littman ML (2005) Activity recognition from accelerometer data. In: Proceedings of the 17th national conference on innovative applications of artificial intelligence, Pittsburgh, USA, vol 5, pp 1541–1546

  • Rawashdeh SA, Rafeldt DA, Uhl TL, Lumpp JE (2015) Wearable motion capture unit for shoulder injury prevention. In: Proceedings of the IEEE 12th international conference on wearable and implantable body sensor networks (BSN), London, UK, pp 1–6

  • Reeser JC, Verhagen E, Brinner WW, Askeland I, Bahr R (2006) Strategies for the prevention of volleyball related injuries. Br J Sports Med 40(7):594–600

    Article  Google Scholar 

  • Roggen D, Cuspinera LP, Pombo G, Ali F, Nguyen-Dinh L-V (2015) Limited-memory warping LCSS for real-time low-power pattern recognition in wireless nodes. In: Abdelzaher T, Pereira N, Tovar E (eds) Wireless sensor networks. Springer, Cham, pp 151–167

  • Santos JM, Embrechts M (2009) On the use of the adjusted rand index as a metric for evaluating supervised classification. In: 19th international conference on artificial neural networks (ICANN 2009), Limassol, Cyprus, pp 175–184

  • Schuldhaus D, Zwick C, Körger H, Dorschky E, Kirk R, Eskofier BM (2015) Inertial sensor-based approach for shot/pass classification during a soccer match. In: Proceedings of the KDD workshop on large-scale sports analytics, Sydney, Australia

  • Shore JE, Johnson RW (1981) Properties of cross-entropy minimization. IEEE Trans Inf Theory 27(4):472–482

    Article  MathSciNet  MATH  Google Scholar 

  • Stiefmeier T, Roggen D, Ogris G, Lukowicz P, Tröster G (2008) Wearable activity tracking in car manufacturing. IEEE Pervasive Comput 2:42–50

    Article  Google Scholar 

  • STMicroelectronics (2015) STM32L151CC microcontroller, data sheet, Geneva, Switzerland. http://www.st.com/st-web-ui/static/active/en/resource/technical/document/datasheet/DM00048356.pdf. Accessed 16 Nov 2015

  • van Mechelen W, Hynek H, Kemper HCG (1992) Incidence, severity, aetiology and prevention of sports injuries. Sports Med 14(2):82–99

    Article  Google Scholar 

  • Zeiler MD, Ranzato M, Monga R, Mao M, Yang K, Le QV, Nguyen P, Senior A, Vanhoucke V, Dean J, et al (2013) On rectified linear units for speech processing. In: Proceedings of the IEEE international conference on acoustics, speech and signal processing (ICASSP), Vancouver, Canada, pp 3517–3521

  • Zeng M, Nguyen LT, Yu B, Mengshoel OJ, Zhu J, Wu P, Zhang J (2014) Convolutional neural networks for human activity recognition using mobile sensors. In: Proceedings of the 6th international conference on mobile computing, applications and services (MobiCASE), Austin, USA, pp 197–205

  • Zheng Y, Liu Q, Chen E, Ge Y, Zhao JL (2014) Time series classification using multi-channels deep convolutional neural networks. In: Li F, Li G, Hwang S, Yao B, Zhang Z (eds) Web-age information management. Springer, Cham, pp 298–310

Download references

Acknowledgements

We would like to thank Markus N. Streicher, Alexander Stolpe, Christoph Eckelmann, Adib Taraben and David Reck for their valuable suggestions and their contributions to the development of the sensor hardware as well as the recording and labeling of the data. Bjoern Eskofier gratefully acknowledges the support of the German Research Foundation (DFG) within the framework of the Heisenberg professorship programme (grant number ES 434/8-1).

Author information

Authors and Affiliations

  1. Digital Sports Group, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Martensstraße 3, 91058, Erlangen, Germany

    Thomas Kautz, Benjamin H. Groh, Julius Hannink & Ulf Jensen

  2. Zentrum für Hochschulsport (ZfH), Universität Leipzig, Jahnallee 59, 04109, Leipzig, Germany

    Holger Strubberg

  3. Immerwahrstraße 2a, 91058, Erlangen, Germany

    Bjoern M. Eskofier

Authors
  1. Thomas Kautz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin H. Groh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Julius Hannink
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ulf Jensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Holger Strubberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bjoern M. Eskofier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas Kautz.

Additional information

Responsible editor: Johannes Fuernkranz.

Appendices

Appendix 1: Generic features

The following generic features were calculated for the classification with the NB, kNN, CART, SVM, RF and VOTE classifiers. The features with the highest ARI values that were used for the experiments with feature selection are printed in bold type.

  1. 1.

    standard deviation x-axis

  2. 2.

    standard deviation y-axis

  3. 3.

    standard deviation z-axis

  4. 4.

    mean x-axis

  5. 5.

    mean y-axis

  6. 6.

    mean z-axis

  7. 7.

    median x-axis

  8. 8.

    median y-axis

  9. 9.

    median z-axis

  10. 10.

    skewness x-axis

  11. 11.

    skewness y-axis

  12. 12.

    skewness z-axis

  13. 13.

    kurtosis x-axis

  14. 14.

    kurtosis y-axis

  15. 15.

    kurtosis z-axis

  16. 16.

    dominant frequency x-axis

  17. 17.

    dominant frequency y-axis

  18. 18.

    dominant frequency z-axis

  19. 19.

    amplitude of spectrum at dominant frequency x-axis

  20. 20.

    amplitude of spectrum at dominant frequency y-axis

  21. 21.

    amplitude of spectrum at dominant frequency z-axis

  22. 22.

    minimum x-axis

  23. 23.

    minimum y-axis

  24. 24.

    minimum z-axis

  25. 25.

    maximum x-axis

  26. 26.

    maximum y-axis

  27. 27.

    maximum z-axis

  28. 28.

    position of the maximum x-axis

  29. 29.

    position of the maximum y-axis

  30. 30.

    position of the maximum z-axis

  31. 31.

    position of the minimum x-axis

  32. 32.

    position of the minimum y-axis

  33. 33.

    position of the minimum z-axis

  34. 34.

    energy x-axis

  35. 35.

    energy y-axis

  36. 36.

    energy z-axis

  37. 37.

    correlation between x-axis and y-axis

  38. 38.

    correlation between x-axis and z-axis

  39. 39.

    correlation between y-axis and z-axis

Appendix 2: Classification accuracy

The performance of the compared classification approaches is summarized in Table 4.

Table 4 Sample accuracy (SAC) and balanced accuracy (BAC) for all compared classifiers with and without feature selection (%)
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kautz, T., Groh, B.H., Hannink, J. et al. Activity recognition in beach volleyball using a Deep Convolutional Neural Network. Data Min Knowl Disc 31, 1678–1705 (2017). https://doi.org/10.1007/s10618-017-0495-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-017-0495-0

Keywords

Search

Navigation