Parcel Tracking by Detection in Large Camera Networks

  • Sascha ClausenEmail author
  • Claudius Zelenka
  • Tobias Schwede
  • Reinhard Koch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11269)


Inside parcel distribution hubs, several tenth of up 100 000 parcels processed each day get lost. Human operators have to tediously recover these parcels by searching through large amounts of video footage from the installed large-scale camera network. We want to assist these operators and work towards an automatic solution. The challenge lies both in the size of the hub with a high number of cameras and in the adverse conditions. We describe and evaluate an industry scale tracking framework based on state-of-the-art methods such as Mask R-CNN. Moreover, we adapt a siamese network inspired feature vector matching with a novel feature improver network, which increases tracking performance. Our calibration method exploits a calibration parcel and is suitable for both overlapping and non-overlapping camera views. It requires little manual effort and needs only a single drive-by of the calibration parcel for each conveyor belt. With these methods, most parcels can be tracked start-to-end.


Multi-object tracking Tracking by Detection Instance segmentation Camera network calibration 



This work was supported by the Central Innovation Programme for SMEs of the Federal Ministry for Economic Affairs and Energy of Germany under grant agreement number 16KN044302.

Supplementary material

Supplementary material 1 (mp4 6877 KB)

Supplementary material 2 (mp4 11745 KB)

480455_1_En_7_MOESM3_ESM.txt (0 kb)
Supplementary material 3 (txt 1 KB)


  1. 1.
    Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the CLEAR MOT metrics. EURASIP J. Image Video Process. 2008 (2008). Scholar
  2. 2.
    Bewley, A., Ge, Z., Ott, L., Ramos, F.T., Upcroft, B.: Simple online and realtime tracking. In: 2016 IEEE International Conference on Image Processing, pp. 3464–3468 (2016).
  3. 3.
    Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition, pp. 2544–2550 (2010).
  4. 4.
    Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a Siamese time delay neural network. In: Advances in Neural Information Processing Systems, vol. 6, pp. 737–744 (1993)Google Scholar
  5. 5.
    Chahyati, D., Fanany, M.I., Arymurthy, A.M.: Tracking people by detection using CNN features. Proc. Comput. Sci. 124, 167–172 (2017). Scholar
  6. 6.
    Danelljan, M., Khan, F.S., Felsberg, M., van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1090–1097 (2014).
  7. 7.
    Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014). Scholar
  8. 8.
    Grabner, H., Grabner, M., Bischof, H.: Real-time tracking via on-line boosting. In: Proceedings of the British Machine Vision Conference 2006, pp. 47–56 (2006).
  9. 9.
    He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. In: IEEE International Conference on Computer Vision, pp. 2980–2988 (2017).
  10. 10.
    Held, D., Thrun, S., Savarese, S.: Learning to track at 100 FPS with deep regression networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 749–765. Springer, Cham (2016). Scholar
  11. 11.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 702–715. Springer, Heidelberg (2012). Scholar
  12. 12.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Forward-backward error: automatic detection of tracking failures. In: 20th International Conference on Pattern Recognition, pp. 2756–2759 (2010).
  13. 13.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012). Scholar
  14. 14.
    Kang, K., Ouyang, W., Li, H., Wang, X.: Object detection from video tubelets with convolutional neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 817–825 (2016).
  15. 15.
    Karaca, H.N., Akınlar, C.: A multi-camera vision system for real-time tracking of parcels moving on a conveyor belt. In: Yolum, I., Güngör, T., Gürgen, F., Özturan, C. (eds.) ISCIS 2005. LNCS, vol. 3733, pp. 708–717. Springer, Heidelberg (2005). Scholar
  16. 16.
    Kroeger, T., Timofte, R., Dai, D., Van Gool, L.: Fast optical flow using dense inverse search. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 471–488. Springer, Cham (2016). Scholar
  17. 17.
    Kuhn, H.W., Yaw, B.: The Hungarian method for the assignment problem. Naval Res. Logist. Q. 2, 83–97 (1955)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Leal-Taixé, L., Canton-Ferrer, C., Schindler, K.: Learning by tracking: Siamese CNN for robust target association. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 418–425 (2016).
  19. 19.
    Li, Y., Qi, H., Dai, J., Ji, X., Wei, Y.: Fully convolutional instance-aware semantic segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 4438–4446 (2017).
  20. 20.
    Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). Scholar
  21. 21.
    Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). Scholar
  22. 22.
    Lukezic, A., Vojír, T., Zajc, L.C., Matas, J., Kristan, M.: Discriminative correlation filter tracker with channel and spatial reliability. Int. J. Comput. Vis. 126(7), 671–688 (2018). Scholar
  23. 23.
    Matterport: Mask R-CNN for object detection and segmentation.
  24. 24.
    Milan, A., Leal-Taixé, L., Reid, I.D., Roth, S., Schindler, K.: MOT16: a benchmark for multi-object tracking (2016).
  25. 25.
    Milan, A., Rezatofighi, S.H., Dick, A.R., Reid, I.D., Schindler, K.: Online multi-target tracking using recurrent neural networks. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp. 4225–4232 (2017)Google Scholar
  26. 26.
    Radke, R.J., Andra, S., Al-Kofahi, O., Roysam, B.: Image change detection algorithms: a systematic survey. IEEE Trans. Image Process. 14(3), 294–307 (2005). Scholar
  27. 27.
    Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement (2018).
  28. 28.
    Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017). Scholar
  29. 29.
    Shin, I.S., Nam, S.H., Yu, H.G., Roberts, R.G., Moon, S.B.: Conveyor visual tracking using robot vision. In: Proceedings of 2006 Florida Conference on Recent Advances in Robotics, pp. 1–5. Citeseer (2006)Google Scholar
  30. 30.
    Tang, Z., Miao, Z., Wan, Y.: Background subtraction using running Gaussian average and frame difference. In: Ma, L., Rauterberg, M., Nakatsu, R. (eds.) ICEC 2007. LNCS, vol. 4740, pp. 411–414. Springer, Heidelberg (2007). Scholar
  31. 31.
    Tomasi, C., Kanade, T.: Detection and tracking of feature points. Technical report. Carnegie Mellon University, Technical Report CMU-CS-91-132 (1991)Google Scholar
  32. 32.
    Wang, X., Türetken, E., Fleuret, F., Fua, P.: Tracking interacting objects optimally using integer programming. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 17–32. Springer, Cham (2014). Scholar
  33. 33.
    Weinberger, K.Q., Blitzer, J., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. In: Advances in Neural Information Processing Systems, vol. 18, pp. 1473–1480 (2005)Google Scholar
  34. 34.
    Zeiler, M.D.: ADADELTA: an adaptive learning rate method (2012).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceKiel UniversityKielGermany

Personalised recommendations