Abstract
To build a smart basketball court, one basic task is to track players with the aid of the basketball court monitoring. This task can be regarded as a special case of the multiple object tracking (MOT) problem. But different from it, the task puts request to the existing MOT methods with both good accuracy and operational efficiency (toward real time) in the new basketball scenario. To deal with this task, we make the following attempts: (1) Considering the differences between pedestrians and basketball players and the lack of corresponding dataset for basketball players tracking, we construct a new MOT dataset under the basketball court monitoring scene, to better evaluate MOT methods in our task and to help promote future research in the related task, (2) evaluating the performance of the several candidate MOT methods on the new dataset, and (3) proposing the issues to be further addressed in this specific scenario.
Similar content being viewed by others
References
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., Savarese, S.: Social LSTM: Human trajectory prediction in crowded spaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 961–971 (2016)
Andriyenko, A., Schindler, K.: Multi-target tracking by continuous energy minimization. In: CVPR 2011, pp. 1265–1272. IEEE (2011)
Ba, S., Alameda-Pineda, X., Xompero, A., Horaud, R.: An on-line variational Bayesian model for multi-person tracking from cluttered scenes. Comput. Vis. Image Underst. 153, 64–76 (2016)
Bae, S.H., Yoon, K.J.: Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 595–610 (2017)
Ban, Y., Ba, S., Alameda-Pineda, X., Horaud, R.: Tracking multiple persons based on a variational Bayesian model. In: European Conference on Computer Vision, pp. 52–67. Springer (2016)
Berclaz, J., Fleuret, F., Turetken, E., Fua, P.: Multiple object tracking using k-shortest paths optimization. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1806–1819 (2011)
Bergmann, P., Meinhardt, T., Leal-Taixe, L.: Tracking Without Bells and Whistles (2019). arXiv preprint arXiv:1903.05625
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the clear mot metrics. J. Image Video Process. 2008, 1 (2008)
Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and real-time tracking. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3464–3468. IEEE (2016)
Chen, B., Wang, D., Li, P., Wang, S., Lu, H.: Real-time ’actor-critic’ tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 318–334 (2018)
Choi, W.: Near-online multi-target tracking with aggregated local flow descriptor. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3029–3037 (2015)
Choi, W., Savarese, S.: Multiple target tracking in world coordinate with single, minimally calibrated camera. In: European Conference on Computer Vision, pp. 553–567. Springer (2010)
Choi, W., Savarese, S.: A unified framework for multi-target tracking and collective activity recognition. In: European Conference on Computer Vision, pp. 215–230. Springer (2012)
Han, J., Farin, D., de With, P.: A mixed-reality system for broadcasting sports video to mobile devices. IEEE Multimed. 18(2), 72–84 (2010)
Han, J., Farin, D., de With, P.H.: Broadcast court-net sports video analysis using fast 3-d camera modeling. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1628–1638 (2008)
Henschel, R., Leal-Taixe, L., Cremers, D., Rosenhahn, B.: Fusion of head and full-body detectors for multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1428–1437 (2018)
Jiang, H., Fels, S., Little, J.J.: A linear programming approach for multiple object tracking. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)
Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)
Keuper, M., Tang, S., Andres, B., Brox, T., Schiele, B.: Motion segmentation & multiple object tracking by correlation co-clustering. IEEE Trans. Pattern Anal. Mach. Intell. 42, 140–153 (2018)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Kuhn, H.W.: The Hungarian method for the assignment problem. Nav. Res. Logist. Q. 2(1–2), 83–97 (1955)
Leal-Taixé, L., Canton-Ferrer, C., Schindler, K.: Learning by tracking: Siamese CNN for robust target association. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 33–40 (2016)
Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: Motchallenge 2015: Towards a Benchmark for Multi-target tracking (2015). arXiv preprint arXiv:1504.01942
Leal-Taixé, L., Pons-Moll, G., Rosenhahn, B.: Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 120–127. IEEE (2011)
Leal-Taixe, L., Pons-Moll, G., Rosenhahn, B.: Branch-and-price global optimization for multi-view multi-target tracking. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1987–1994. IEEE (2012)
Li, T., Chang, H., Wang, M., Ni, B., Hong, R., Yan, S.: Crowded scene analysis: a survey. IEEE Trans. Circuits Syst. Video Technol. 25(3), 367–386 (2014)
Li, B., Yan, J., Wu, W., Zhu, Z., Hu, X.: High performance visual tracking with Siamese region proposal network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8971–8980 (2018)
Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Zhao, X., Kim, T.K.: Multiple Object Tracking: A Literature Review (2014). arXiv preprint arXiv:1409.7618
Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: Mot16: A Benchmark for Multi-object Tracking (2016). arXiv preprint arXiv:1603.00831
Milan, A., Leal-Taixé, L., Schindler, K., Reid, I.: Joint tracking and segmentation of multiple targets. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5397–5406 (2015)
Pellegrini, S., Ess, A., Schindler, K., Van Gool, L.: You’ll never walk alone: Modeling social behavior for multi-target tracking. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 261–268. IEEE (2009)
Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: CVPR 2011, pp. 1201–1208. IEEE (2011)
Redmon, J., Farhadi, A.: Yolov3: An Incremental Improvement (2018). arXiv preprint arXiv:1804.02767
Ren, L., Lu, J., Wang, Z., Tian, Q., Zhou, J.: Collaborative deep reinforcement learning for multi-object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 586–602 (2018)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
Robicquet, A., Sadeghian, A., Alahi, A., Savarese, S.: Learning social etiquette: human trajectory understanding in crowded scenes. In: European Conference on Computer Vision, pp. 549–565. Springer (2016)
Schenk, M.J., Reed, D.D.: Experimental evaluation of matching via a commercially available basketball video game. J. Appl. Behav. Anal. 53, 209–221 (2019)
Scovanner, P., Tappen, M.F.: Learning pedestrian dynamics from the real world. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 381–388. IEEE (2009)
Son, J., Baek, M., Cho, M., Han, B.: Multi-object tracking with quadruplet convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5620–5629 (2017)
Tang, S., Andres, B., Andriluka, M., Schiele, B.: Multi-person tracking by multicut and deep matching. In: European Conference on Computer Vision, pp. 100–111. Springer (2016)
Wang, Z., Zheng, L., Liu, Y., Wang, S.: Towards Real-Time Multi-object Tracking (2019). arXiv preprint arXiv:1909.12605
Wojke, N., Bewley, A., Paulus, D.: Simple online and real-time tracking with a deep association metric. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3645–3649. IEEE (2017)
Wu, Z., Kunz, T.H., Betke, M.: Efficient track linking methods for track graphs using network-flow and set-cover techniques. In: CVPR 2011, pp. 1185–1192. IEEE (2011)
Xiang, Y., Alahi, A., Savarese, S.: Learning to track: online multi-object tracking by decision making. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4705–4713 (2015)
Xu, Y., Ban, Y., Alameda-Pineda, X., Horaud, R.: Deepmot: A Differentiable Framework for Training Multiple Object Trackers (2019). arXiv preprint arXiv:1906.06618
Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are you with and where are you going? In: CVPR 2011, pp. 1345–1352. IEEE (2011)
Yang, F., Choi, W., Lin, Y.: Exploit all the layers: fast and accurate CNN object detector with scale dependent pooling and cascaded rejection classifiers. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2129–2137 (2016)
Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)
Zhu, J., Yang, H., Liu, N., Kim, M., Zhang, W., Yang, M.H.: Online multi-object tracking with dual matching attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 366–382 (2018)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fu, X., Zhang, K., Wang, C. et al. Multiple player tracking in basketball court videos. J Real-Time Image Proc 17, 1811–1828 (2020). https://doi.org/10.1007/s11554-020-00968-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-020-00968-x