Skip to main content
SpringerLink
Log in

Camera-based Basketball Scoring Detection Using Convolutional Neural Network

  • Research Article
  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

Abstract

Recently, deep learning methods have been applied in many real scenarios with the development of convolutional neural networks (CNNs). In this paper, we introduce a camera-based basketball scoring detection (BSD) method with CNN based object detection and frame difference-based motion detection. In the proposed BSD method, the videos of the basketball court are taken as inputs. Afterwards, the real-time object detection, i.e., you only look once (YOLO) model, is implemented to locate the position of the basketball hoop. Then, the motion detection based on frame difference is utilized to detect whether there is any object motion in the area of the hoop to determine the basketball scoring condition. The proposed BSD method runs in real-time with satisfactory basketball scoring detection accuracy. Our experiments on the collected real scenario basketball court videos show the accuracy of the proposed BSD method. Furthermore, several intelligent basketball analysis systems based on the proposed method have been installed at multiple basketball courts in Beijing, and they provide good performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. G. Thomas, R. Gade, T. B. Moeslund, P. Carr, A. Hilton. Computer vision for sports: Current applications and research topics. Computer Vision and Image Understanding, vol. 159, pp. 3–18, 2017. DOI: https://doi.org/10.1016/j.cviu.2017.04.011.

    Article  Google Scholar 

  2. T. B. Moeslund, G. Thomas, A. Hilton. Computer Vision in Sports, Cham, Germany: Springer, 2014. DOI: https://doi.org/10.1007/978-3-319-09396-3.

    Book  Google Scholar 

  3. R. P. Schumaker, O. K. Solieman, H. Chen. Sports Data Mining, Boston, USA: Springer, 2010. DOI: https://doi.org/10.1007/978-1_4419_6730-5.

    Book  Google Scholar 

  4. M. Haghighat, H. Rastegari, N. Nourafza. A review of data mining techniques for result prediction in sports. Advances in Computer Science: An International Journal, vol. 2, no. 5, pp. 7–12, 2013.

    Google Scholar 

  5. C. K. Leung and K. W. Joseph. Sports data mining: Predicting results for the college football games. Procedia Computer Science, vol. 35, pp. 710–719, pp. 2014. DOI: https://doi.org/10.1016/j.procs.2014.08.153.

    Article  Google Scholar 

  6. A. McCabe and J. Trevathan. Artificial intelligence in sports prediction. In Proceedings of the 5th International Conference on Information Technology: New Generations, IEEE, Las Vegas, USA, pp. 1194–1197, 2008. DOI: https://doi.org/10.1109/ITNG.2008.203.

    Google Scholar 

  7. H. Novatchkov and A. Baca. Artificial intelligence in sports on the example of weight training. Journal of Sports Science & Medicine, vol. 12, no. 1, pp. 27–37, 2013.

    Google Scholar 

  8. F. Owramipur, P. Eskandarian, F. S. Mozneb. Football result prediction with Bayesian network in Spanish League-Barcelona team. International Journal of Computer Theory and Engineering, vol. 5, no. 5, pp. 812–815, 2013. DOI: https://doi.org/10.7763/IJCTE.2013.V5.802.

    Article  Google Scholar 

  9. D. Miljkovic, L. Gajic, A. Kovacevic, Z. Konjovic. The use of data mining for basketball matches outcomes prediction. In Proceedings of the 8th IEEE International Symposium on Intelligent Systems and Informatics, IEEE, Subotica, Serbia, pp. 309–312, 2010. DOI: https://doi.org/10.1109/SISY.2010.5647440.

    Chapter  Google Scholar 

  10. M. Nakai, Y. Tsunoda, H. Hayashi, H. Murakoshi. Prediction of basketball free throw shooting by openpose. In Proceedings of New Frontiers in Artificial Intelligence, Springer, Yokohama, Japan, pp. 435–446, 2018.

    Google Scholar 

  11. Z. Cao, G. H. Martinez, T. Simon, S. E. Wei, Y. A. Sheikh. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. DOI: https://doi.org/10.1109/TPAMI.2019.2929257.

  12. J. C. Liu and P. Carr. Detecting and tracking sports players with random forests and context-conditioned motion models. Computer Vision in Sports, T. B. Moeslund, G. Thomas, A. Hilton, Eds., Cham, Germany: Springer, pp. 113–132, 2014. DOI: https://doi.org/10.1007/978-3-319-09396-3_6.

    Chapter  Google Scholar 

  13. R. Girshick, J. Donahue, T. Darrell, J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Columbus, USA, pp. 580–587, 2014. DOI: https://doi.org/10.1109/CVPR.2014.81.

    Google Scholar 

  14. S. Q. Ren, K. M. He, R. Girshick, J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of Advances in Neural Information Processing Systems, Montreal, Canada, pp. 91–99, 2015.

  15. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, A. C. Berg. SSD: Single shot MultiBox detector. In Proceedings of the 14th European Conference on Computer Vision, Springer, Amsterdam, The Netherlands, pp. 21–37, 2016. DOI: https://doi.org/10.1007/978-3-319-46448-0_2.

    Google Scholar 

  16. J. Redmon, S. Divvala, R. Girshick, A. Farhadi. You only look once: Unified, real-time object detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 779–788, 2016. DOI: https://doi.org/10.1109/CVPR.2016.91.

    Google Scholar 

  17. J. Redmon and A. Farhadi. YOLOv3: An incremental improvement, [Online], Available: https://arxiv.org/abs/1804.02767, 2018.

  18. A. Krizhevsky, I. Sutskever, G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, USA, pp. 1097–1105, 2012.

  19. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 3rd International Conference on Learning Representations, San Diego, USA, 2014.

  20. C. Szegedy, W. Liu, Y. Q. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich. Going deeper with convolutions. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 1–9, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298594.

    Google Scholar 

  21. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna. Rethinking the inception architecture for computer vision. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 2818–2826, 2016. DOI: https://doi.org/10.1109/CVPR.2016.308.

    Google Scholar 

  22. K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CVPR.2016.90.

    Google Scholar 

  23. N. Singla. Motion detection based on frame difference method. International Journal of Information & Computation Technology, vol. 4, no. 15, pp. 1559–1565, 2014.

    Google Scholar 

  24. K. Aukkapinyo, S. Sawangwong, P. Pooyoi, W. Kusakunniran. Localization and classification of rice-grain images using region proposals-based convolutional neural network. International Journal of Automation and Computing, vol. 17, no. 2, pp. 233–246, 2020. DOI: https://doi.org/10.1007/s11633-019-1207-6.

    Article  Google Scholar 

  25. A. X. Li, K. X. Zhang, L. W. Wang. Zero-shot fine-grained classification by deep feature learning with semantics. International Journal of Automation and Computing, vol. 16, no. 5, pp. 563–574, 2020. DOI: https://doi.org/10.1007/s11633-019-1177-8.

    Article  Google Scholar 

  26. C. Cortes, V. Vapnik. Support-vector networks. Machine Learning, vol. 20, no. 3, pp. 273–297, 1995. DOI: https://doi.org/10.1007/BF00994018.

    MATH  Google Scholar 

  27. T. Y. Lin, P. Goyal, R. Girshick, K. M. He, P. Dollar. Focal loss for dense object detection. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Venice, Italy, pp. 2999–3007, 2017. DOI: https://doi.org/10.1109/ICCV.2017.324.

    Google Scholar 

  28. S. F. Zhang, L. Y. Wen, X. Bian, Z. Lei, S. Z. Li. Single-shot refinement neural network for object detection. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake City, USA, pp. 4203–4212, 2018. DOI: https://doi.org/10.1109/CVPR.2018.00442.

    Google Scholar 

  29. Z. W. Cai and N. Vasconcelos. Cascade R-CNN: Delving into high quality object detection. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake City, USA, pp. 6154–6162, 2018. DOI: https://doi.org/10.1109/CVPR.2018.00644.

    Google Scholar 

  30. H. Law and J. Deng. CornerNet: Detecting objects as paired keypoints. In Proceedings of the 15th European Conference on Computer Vision, Springer, Munich, Germany, pp. 765–781, 2018. DOI: https://doi.org/10.1007/978-3-030-01264-9_45.

    Google Scholar 

  31. K. W. Duan, S. Bai, L. X. Xie, H. G. Qi, Q. M. Huang, Q. Tian. Centernet: Keypoint triplets for object detection. In Proceedings of IEEE/CVF International Conference on Computer Vision, IEEE, Seoul, South Korea, pp. 6568–6577, 2019. DOI: https://doi.org/10.1109/ICCV.2019.00667.

    Google Scholar 

  32. G. Ghiasi, T. Y. Lin, Q. V. Le. NAS-FPN: Learning scalable feature pyramid architecture for object detection. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Long Beach, USA, pp. 7029–7038, 2019. DOI: https://doi.org/10.1109/CVPR.2019.00720.

    Google Scholar 

  33. M. X. Tan and Q. V. Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, ICML, Long Beach, USA, 2019.

    Google Scholar 

  34. C. C. Zhu, Y. H. He, M. Savvides. Feature selective anchor-free module for single-shot object detection. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Long Beach, USA, pp. 840–849, 2019. DOI: https://doi.org/10.1109/CVPR.2019.00093.

    Google Scholar 

  35. Q. Fan, W. Zhuo, C. K. Tang, Y. W. Tai. Few-shot object detection with attention-RPN and multi-relation detector. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Seattle, USA, 2019. DOI: https://doi.org/10.1109/CVPR42600.2020.00407.

    Google Scholar 

  36. Z. W. Dong, G. X. Li, Y. Liao, F. Wang, P. J. Ren, C. Qian. CentripetalNet: Pursuing high-quality keypoint pairs for object detection. [Online], Available: https://arxiv.org/abs/2003.09119, 2020.

  37. C. H. Zhan, X. H. Duan, S. Y. Xu, Z. Song, M. Luo. An improved moving object detection algorithm based on frame difference and edge detection. In Proceedings of the 4th International Conference on Image and Graphics, IEEE, Sichuan, China, pp. 519–523, 2007. DOI: https://doi.org/10.1109/ICIG.2007.153.

    Google Scholar 

  38. D. A. Migliore, M. Matteucci, M. Naccari. A revaluation of frame difference in fast and robust motion detection. In Proceedings of the 4th ACM International Workshop on Video Surveillance and Sensor Networks, Santa Barbara, USA, pp. 215–218, 2006. DOI: https://doi.org/10.1145/1178782.1178815.

  39. Y. Li, Z. X. Sun, B. Yuan, Y. Zhang. An improved method for motion detection by frame difference and background subtraction. Journal of Image and Graphics, vol. 14, no. 6, pp. 1162–1168, 2009. (in Chinese)

    Google Scholar 

  40. X. F. Ji, Q. Q. Wu, Z. J. Ju, Y. Y. Wang. Study of human action recognition based on improved spatio-temporal features. International Journal of Automation and Computing, vol. 11, no. 5, pp. 500–509, 2014. DOI: https://doi.org/10.1007/s11633-014-0831-4.

    Article  Google Scholar 

  41. M. Everingham, S. A. Eslami, L. Van Gool, C. K. I. Wilhams, J. Winn, A. Zisserman. The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision, vol. 111, no. 1, pp. 98–136, 2015. DOI: https://doi.org/10.1007/s11263-014-0733-5.

    Article  Google Scholar 

  42. T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, C. L. Zitnick. Microsoft coco: Common objects in context. In Proceedings of the 13th European Conference on Computer Vision, Springer, Zurich, Switzerland, pp. 740–755, 2014. DOI: https://doi.org/10.1007/978-3-319-10602-1_48.

    Google Scholar 

  43. A. Neubeck and L. van Gool. Efficient non-maximum suppression. In Proceedings of the 18th International Conference on Pattern Recognition, IEEE, Hong Kong, China, pp. 850–855, 2006. DOI: https://doi.org/10.1109/ICPR.2006.479.

    Google Scholar 

  44. Q. Y. Gu and I. Ishii. Review of some advances and applications in real-time high-speed vision: Our views and experiences. International Journal of Automation and Computing, vol. 13, no. 4, pp. 305–318, 2016. DOI: https://doi.org/10.1007/s11633-016-1024-0.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Research on Educational Science Planning in Zhejiang Province (No. 2019SCG195), “13th Five Year Plan” Teaching Reform Project of Zhejiang University and Shandong Provincial Key Research and Development Program (Major Scientific and Technological Innovation Project) (No. 2019JZZY010119).

Author information

Authors and Affiliations

  1. Department of Public Physical and Art Education, Zhejiang University, Hangzhou, 310058, China

    Xu-Bo Fu & De-Yun Pan

  2. North China Electric Power University, Beijing, 102206, China

    Shao-Long Yue

Authors
  1. Xu-Bo Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shao-Long Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. De-Yun Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xu-Bo Fu.

Additional information

Colored figures are available in the online version at https://link.springer.com/journal/11633

Xu-Bo Fu received the B.Sc. degree in physical education and training from the Physical Education Department, Zhejiang University, China in 2006, the M.Sc. degree in physical education and training from the Physical Education Department, Zhejiang University, China in 2008. He is an associate professor in the Public Sports and Art Department, Zhejiang University, China. He offers junior and football intermediate courses for undergraduate students. He also published five papers in related fields. He hosted or participated in 4 types of projects. Among them, he presided over the “Regional Sports Economic Industry Layout and Institutional Adjustment Research: Taking the Yangtze River Delta as an Example”. He participated in the “Practical Research on Public Physical Education Model with Non-Austrian Projects” project and won the second prize of Zhejiang Teaching Achievements. In 2010, he got the “Sunshine Sports Award” by the Sports Association of the Higher Industri- al School directly under the Ministry of Education. In 2010, he was awarded as the “Outstanding Physical Education Teacher” by the Hangzhou Sports Association. In 2013, he won the second prize of Zhejiang University Quality Teaching Award.

His research interest is school physical education.

Shao-Long Yue received the B.Sc. degree in automation from School of Electrical and Electronic Engineering, Shandong University of Technology, China in 2018. He is currently a master student in control engineering at School of Control and Computer Engineering, North China Electric Power University, China.

His research interests include pattern recognition, computer vision and machine learning.

De-Yun Pan received the B. Sc. degree in physical education basketball from Beijing Sport University, China in 1999. He also is a national basketball referee. He is currently the director of the Sports Training Center of the Public Sports and Art Department of Zhejiang University, China. He is the deputy director of the Basketball Committee of the Chinese University Sports Association, and the director of the Coaching Committee of the Zhejiang University Sports Association. He is the director of the Coaching Committee of the Zhejiang University Sports Association, the deputy director of the Zhejiang Basketball Association Coaching Committee and member of the Youth Committee of the Chinese Basketball Association.

His research interest is physical education.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, XB., Yue, SL. & Pan, DY. Camera-based Basketball Scoring Detection Using Convolutional Neural Network. Int. J. Autom. Comput. 18, 266–276 (2021). https://doi.org/10.1007/s11633-020-1259-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-020-1259-7

Keywords

Search

Navigation