Skip to main content

Online multi-object tracking with pedestrian re-identification and occlusion processing

The Visual Computer volume 37, pages 1089–1099 (2021)Cite this article

Abstract

Tracking-by-detection is a common approach for online multi-object tracking problem. At present, the following challenges still exist in the multi-object tracking scenarios: (1) The result of object re-tracking after full occlusion is not ideal; (2) The predicted position of object is not accurate enough in the complicated video scenarios. Aiming at these two problems, this paper proposes a multi-object tracking framework called DROP (Deep Re-identification Occlusion Processing). The framework consists of object detection, fast pedestrian re-identification, and a confidence-based data association algorithm. A lightweight convolutional neural network that can solve the re-tracking problem is constructed by increasing and learning the affinity of appearance features of the same object in different frames. And this paper proposes to judge the occlusion of the object that can solve inaccurate position predicted by Kalman filter by using the data association result of the appearance features of the object, and to reduce the matching error by improving the data association formula. The experimental results on the multi-object tracking datasets MOT15 and MOT16 show that the proposed method can improve the precision while ensure the real-time tracking performance.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Rehman, B., Ong, W.H., Tan, A.C.H., et al.: Face detection and tracking using hybrid margin-based ROI techniques. Vis. Comput. 36(3), 633–647 (2020)

    Article  Google Scholar 

  2. Li, Z., Yu, X., Li, P., Hashem, M.: Moving object tracking based on multi-independent features distribution fields with comprehensive spatial feature similarity. Vis. Comput. 31(12), 1633–1651 (2015)

    Article  Google Scholar 

  3. Camgoz, N. C., Hadfield, S., Koller, O., Bowden, R.: Using convolutional 3d neural networks for user-independent continuous gesture recognition. In: 23rd International Conference on Pattern Recognition (ICPR), pp. 49–54 (2016)

  4. Wang, Y., Hu, S., Wu, S.: Object tracking based on huber loss function. Vis. Comput. 35(11), 1641–1654 (2019)

    Article  Google Scholar 

  5. Rasekhipour, Y., Khajepour, A., Chen, S.K., Litkouhi, B.: A potential field-based model predictive path-planning controller for autonomous road vehicles. IEEE Trans. Intell. Transp. Syst. 18(5), 1255–1267 (2016)

    Article  Google Scholar 

  6. Choi, W., Savarese, S.: A unified framework for multi-target tracking and collective activity recognition. In: European Conference on Computer Vision, pp. 215–230 (2012)

  7. Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., Van Gool, L.: Robust tracking-by-detection using a detector confidence particle filter. In: 12th IEEE International Conference on Computer Vision, pp. 1515–1522 (2009)

  8. Zhang, H., Liu, G.: Coupled-layer based visual tracking via adaptive kernelized correlation filters. Vis. Comput. 34(1), 41–54 (2018)

    MathSciNet  Article  Google Scholar 

  9. Fazl-Ersi, E., Nooghabi, M.K.: Revisiting correlation-based filters for low-resolution and long-term visual tracking. Vis. Comput. 35(10), 1447–1459 (2019)

    Article  Google Scholar 

  10. Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 58–72 (2013)

    Article  Google Scholar 

  11. Qin, Z., Shelton, C.R.: Improving multi-target tracking via social grouping. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1972–1978 (2012)

  12. Xiang, Y., Alahi, A., Savarese, S.: Learning to track: online multi-object tracking by decision making. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4705–4713 (2015)

  13. Zhang, L., Van Der Maaten, L.: Preserving structure in model-free tracking. IEEE Trans. Pattern Anal. Mach. Intell. 36(4), 756–769 (2013)

    Article  Google Scholar 

  14. Izadinia, H., Saleemi, I., Li, W., Shah, M.: 2T: multiple people multiple parts tracker. In: European Conference on Computer Vision, Springer, Berlin, pp. 100–114 (2012)

  15. Butt, A.A., Collins, R.T.: Multi-target tracking by lagrangian relaxation to min-cost network flow. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1846–1853 (2013)

  16. Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., Yan, J.: Poi: multiple object tracking with high performance detection and appearance feature. In: European Conference on Computer Vision, Springer, Berlin, pp. 36–42 (2016)

  17. Guan, H., Cheng, B.: How do deep convolutional features affect tracking performance: an experimental study. Vis. Comput. 34(12), 1701–1711 (2018)

    Article  Google Scholar 

  18. Wu, Y., Jia, N., Sun, J.: Real-time multi-scale tracking based on compressive sensing. Vis. Comput. 31(4), 471–484 (2015)

    Article  Google Scholar 

  19. Milan, A., Rezatofighi, S. H., Dick, A., Reid, I., Schindler, K.: Online multi-target tracking using recurrent neural networks. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)

  20. Yoon, K., Kim, D.Y., Yoon, Y.C., Jeon, M.: Data association for multi-object tracking via deep neural networks. Sensors 19(3), 559 (2019)

    Article  Google Scholar 

  21. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: IEEE International Conference on Image Processing (ICIP), pp. 3464–3468 (2016)

  22. Wojke, N., Bewley, A., Paulus, D.: Simple online and realtime tracking with a deep association metric. In: IEEE International Conference on Image Processing (ICIP), pp. 3645–3649 (2017)

  23. Yoon, Y.C., Song, Y.M., Yoon, K., Jeon, M.: Online multi-object tracking using selective deep appearance matching. In: IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), pp. 206–212 (2018)

  24. Xu, J., Cao, Y., Zhang, Z., Hu, H.: Spatial-temporal relation networks for multi-object tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3988–3998 (2019)

  25. Mahmoudi, N., Ahadi, S.M., Rahmati, M.: Multi-target tracking using CNN-based features: CNNMTT. Multimed. Tools Appl. 78(6), 7077–7096 (2019)

    Article  Google Scholar 

  26. Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., Yu, N.: Online multi-object tracking using CNN-based single object tracker with spatial-temporal attention mechanism. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4836–4845 (2017)

  27. Sadeghian, A., Alahi, A., Savarese, S.: Tracking the untrackable: learning to track multiple cues with long-term dependencies. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 300–311 (2017)

  28. Zhu, J., Yang, H., Liu, N., Kim, M., Zhang, W., Yang, M.H.: Online multi-object tracking with dual matching attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 366–382 (2018)

  29. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)

  30. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  31. Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. arXiv Prepr. arXiv:1703.07737. (2017)

  32. Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: Motchallenge 2015: Towards a benchmark for multi-target tracking (2015). arXiv Prepr. arXiv:1504.01942

  33. Sanchez-Matilla, R., Poiesi, F., Cavallaro, A.: Online multi-target tracking with strong and weak detections. In: European Conference on Computer Vision, pp. 84–99 (2016)

  34. Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C.: Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision, pp. 17–35 (2016)

  35. Fang, K., Xiang, Y., Li, X., Savarese, S.: Recurrent autoregressive networks for online multi-object tracking. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 466–475 (2018)

  36. Bochinski, E., Eiselein, V., Sikora, T.: High-speed tracking-by-detection without using image information. In: 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6 (2017)

  37. Yoon, Y.C., Boragule, A., Song, Y.M., Yoon, K., Jeon, M.: Online multi-object tracking with historical appearance matching and scene adaptive detection filtering. In: 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6 (2018)

  38. Boragule, A., Jeon, M.: Joint cost minimization for multi-object tracking. In: 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6 (2017)

  39. Anh, N.T.L., Khan, F.M., Negin, F., Bremond, F.: Multi-object tracking using multi-channel part appearance representation. In: 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6 (2017)

Download references

Funding

Funding was provided by: Shanghai Aerospace Science and Technology Innovation Fund (SAST2018086)

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, East China University of Science and Technology, Meilong Road 130, Xuhui District, Shanghai, China

    Xueqin Zhang, Xiaoxiao Wang & Chunhua Gu

Authors
  1. Xueqin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoxiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chunhua Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xueqin Zhang.

Ethics declarations

Conflict of interest

Author XueQin Zhang declares that she has no conflict of interest. Author Xiaoxiao Wang declares that she has no conflict of interest. Author Chunhua Gu declares that he has no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Wang, X. & Gu, C. Online multi-object tracking with pedestrian re-identification and occlusion processing. Vis Comput 37, 1089–1099 (2021). https://doi.org/10.1007/s00371-020-01854-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01854-0

Keywords

  • Multi-object tracking
  • Tracking-by-detection
  • Pedestrian re-identification
  • Data association