Skip to main content
SpringerLink
Log in

Efficient vehicle detection and tracking strategy in aerial videos by employing morphological operations and feature points motion analysis

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Real-time automatic detection and tracking of moving vehicles in videos acquired by airborne cameras is a challenging problem due to vehicle occlusion, camera movement, and high computational cost. This paper presents an efficient and robust real-time approach for automatic vehicle detection and tracking in aerial videos that employ both detections and tracking features to enhance the final decision. The use of Top-hat and Bottom-hat transformation aided by the morphological operation in the detection phase has been adopted. After detection, background regions are eliminated by motion feature points’ analysis of the obtained object regions using a combined technique between KLT tracker and K-means clustering. Obtained object features are clustered into separate objects based on their motion characteristic. Finally, an efficient connecting algorithm is introduced to assign the vehicle labels with their corresponding cluster trajectories. The proposed method was tested on videos taken in different scenarios. The experimental results showed that the recall, precision, and tracking accuracy of the proposed method were about 95.1 %, 97.5%, and 95.2%, respectively. The method also achieves a fast processing speed. Thus, the proposed approach has superior overall performance compared to newly published approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Abdelwahab M A, Abdelwahab M M (2015) A novel algorithm for vehicle detection and tracking in airborne videos. In: Multimedia (ISM), 2015 IEEE International Symposium on, Miami, FL, USA, pp 65–68

  2. Al-Kaff A, de la Escalera A, Armingol J M (2015) Sift and surf performance evaluation and the effect of freak descriptor in the context of visual odometry for unmanned aerial vehicles. In: International Conference on Computer Aided Systems Theory, Las Palmas de Gran Canaria, Spain, pp 739–747

  3. Alkanat T, Tunali E, Öz S (2015) Fully-automatic target detection and tracking for real-time, airborne imaging applications. In: International Joint Conference on Computer Vision, Imaging and Computer Graphics, Berlin, Germany, pp 240–255

  4. Bar-Shalom Y, Fortmann T, Scheffe M, et al. (1980) Joint probabilistic data association for multiple targets in clutter. In: Prof. conf. on information sciences and systems, Albuquerque, pp 404–409

  5. Barth A, Franke U (2009) Estimating the driving state of oncoming vehicles from a moving platform using stereo vision. IEEE Trans Intell Transp Syst 10 (4):560–571

    Article  Google Scholar 

  6. Bouguet J-Y (2001) Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corporation 5(1-10):4

    Google Scholar 

  7. Cao X, Jiang X, Li X, Yan P (2018) Correlation-based tracking of multiple targets with hierarchical layered structure. IEEE transactions on cybernetics 48(1):90–102

    Article  Google Scholar 

  8. Cao X, Shi Z, Yan P, Li X (2013) Tracking vehicles as groups in airborne videos. Neurocomputing 99:38–45

    Article  Google Scholar 

  9. Collins R, Zhou X, Teh S K (2005) An open source tracking testbed and evaluation web site. In: IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS 2005), January 2005, Breckenridge, Colorado

  10. Gao Z, Gao L-S, Zhang H, Cheng Z, Hong R (2019) Deep spatial pyramid features collaborative reconstruction for partial person reid. In: Proceedings of the 27th ACM International Conference on Multimedia, pp 1879–1887

  11. Gao T, Li G, Lian S, Zhang J (2012) Tracking video objects with feature points based particle filtering. Multimedia Tools and Applications 58(1):1–21

    Article  Google Scholar 

  12. Gao Z, Xu C, Zhang H, Li S, de Albuquerque V H C (2020) Trustful internet of surveillance things based on deeply-represented visual co-saliency detection. IEEE Internet Things J 7(5):4092–4100

    Article  Google Scholar 

  13. Gomaa A, Abdelwahab M M, Abo-Zahhad M (2018) Real-time algorithm for simultaneous vehicle detection and tracking in aerial view videos. In: 2018 IEEE 61st International Midwest Symposium on Circuits and Systems (MWSCAS), IEEE, pp 222–225

  14. Gomaa A, Abdelwahab M M, Abo-Zahhad M, Minematsu T, Taniguchi R- (2019) Robust vehicle detection and counting algorithm employing a convolution neural network and optical flow. Sensors 19(20):4588

    Article  Google Scholar 

  15. Hess R, Fern A (2009) Discriminatively trained particle filters for complex multi-object tracking. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, IEEE, Miami, FL, USA, pp 240–247

  16. Jiang X, Cao X (2016) Surveillance from above: A detection-and-prediction based multiple target tracking method on aerial videos. In: Integrated Communications Navigation and Surveillance (ICNS), 2016, IEEE, pp 4D2–1

  17. Kalantar B, Mansor S B, Halin A A, Shafri H Z M, Zand M (2017) Multiple moving object detection from uav videos using trajectories of matched regional adjacency graphs. IEEE Trans Geosci Remote Sens 55(9):5198–5213

    Article  Google Scholar 

  18. Kanistras K, Martins G, Rutherford M J, Valavanis K P (2015) Survey of unmanned aerial vehicles (uavs) for traffic monitoring. In: Handbook of unmanned aerial vehicles. Springer, pp 2643–2666

  19. Kasturi R, Goldgof D, Soundararajan P, Manohar V, Garofolo J, Bowers R, Boonstra M, Korzhova V, Zhang J (2008) Framework for performance evaluation of face, text, and vehicle detection and tracking in video: Data, metrics, and protocol. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(2):319–336

    Article  Google Scholar 

  20. Ke R, Li Z, Kim S, Ash J, Cui Z, Wang Y (2017) Real-time bidirectional traffic flow parameter estimation from aerial videos. IEEE Trans Intell Transp Syst 18(4):890–901

    Article  Google Scholar 

  21. Kent P, Maskell S, Payne O, Richardson S, Scarff L (2012) Robust background subtraction for automated detection and tracking of targets in wide area motion imagery. In: Optics and Photonics for Counterterrorism, Crime Fighting, and Defence VIII, vol 8546, International Society for Optics and Photonics, p 85460Q

  22. Nguyen V D, Tran D T, Byun J Y, Jeon J W (2018) Real-time vehicle detection using an effective region proposal-based depth and 3-channel pattern. IEEE Trans Intell Transp Syst 20(10):3634–3646

    Article  Google Scholar 

  23. Niknejad H T, Takeuchi A, Mita S, McAllester D (2012) On-road multivehicle tracking using deformable object model and particle filter with improved likelihood estimation. IEEE Trans Intell Transp Syst 13(2):748–758

    Article  Google Scholar 

  24. Noh S, Shim D, Jeon M (2016) Adaptive sliding-window strategy for vehicle detection in highway environments. IEEE Trans Intell Transp Syst 17 (2):323–335

    Article  Google Scholar 

  25. Rad R, Jamzad M (2005) Real time classification and tracking of multiple vehicles in highways. Pattern Recogn Lett 26(10):1597–1607

    Article  Google Scholar 

  26. Reid D, et al. (1979) An algorithm for tracking multiple targets. IEEE transactions on Automatic Control 24(6):843–854

    Article  Google Scholar 

  27. Rosenbaum D, Leitloff J, Kurz F, Meynberg O, Reize T (2010) Real-time image processing for road traffic data extraction from aerial images. Proc. of ISPRS TC VII Symposium IAPRS XXXVIII:469–474

    Google Scholar 

  28. Su A, Sun X, Liu H, Zhang X, Yu Q (2015) Online cascaded boosting with histogram of orient gradient features for car detection from unmanned aerial vehicle images. J Appl Remote Sens 9(1):096063

    Article  Google Scholar 

  29. Tang Y, Zhang C, Gu R, Li P, Yang B (2017) Vehicle detection and recognition for intelligent traffic surveillance system. Multimedia tools and applications 76(4):5817–5832

    Article  Google Scholar 

  30. Teutsch M, Kruger W (2015) Robust and fast detection of moving vehicles in aerial videos using sliding windows. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA, pp 26–34

  31. Teutsch M, Krüger W (2012) Spatio-temporal fusion of object segmentation approaches for moving distant targets. In: Information Fusion (FUSION), 2012 15th International Conference on, IEEE, pp 1988–1995

  32. Teutsch M, Krüger W, Beyerer J (2017) Moving object detection in top-view aerial videos improved by image stacking. Opt Eng 56(8):083102

    Article  Google Scholar 

  33. Wan E A, Van Der Merwe R (2001) The unscented Kalman filter. In: Haykin S. (ed) Kalman filtering and neural networks. ch. 7. Wiley, New York

  34. Yang S, Xu J, Chen Y, Wang M (2014) On-road vehicle tracking using keypoint-based representation and online co-training. Multimedia tools and applications 72(2):1561–1583

    Article  Google Scholar 

  35. Yao Y, Li J, Wu T, Zhang L (2017) Retracted article: Moving object surveillance using object proposals and background prior prediction. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-017-4820-9

  36. Yu X, Shi Z (2015) Vehicle detection in remote sensing imagery based on salient information and local shape feature. Optik-International Journal for Light and Electron Optics 126(20):2485–2490

    Article  Google Scholar 

  37. Zhao Z-Q, Zheng P, Xu S-, Wu X (2019) Object detection with deep learning: A review. IEEE transactions on neural networks and learning systems 30 (11):3212–3232

    Article  Google Scholar 

  38. Zheng Z, Zhou G, Wang Y, Liu Y, Li X, Wang X, Jiang L (2013) A novel vehicle detection method with high resolution highway aerial image. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 6(6):2338–2343

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Egyptian Ministry of Higher Education (MoHE), Cairo, Egypt, and in part by the Egypt Japan University of Science and Technology (E-JUST), Alexandria, Egypt.

Author information

Authors and Affiliations

  1. National Research Institute of Astronomy and Geophysics (NRIAG), Helwan, 11731, Egypt

    Ahmed Gomaa

  2. Electronics and Communications Engineering Department, Egypt-Japan University of Science and Technology, Alexandria, 21934, Egypt

    Ahmed Gomaa, Moataz M. Abdelwahab & Mohammed Abo-Zahhad

  3. Center for Japan-Egypt Cooperation in Science and Technology, Kyushu University, Nishiku, Fukuoka, 819-0395, Japan

    Ahmed Gomaa

  4. Electrical and Electronics Engineering Department, Faculty of Engineering, Assiut University, Assiut, 71511, Egypt

    Mohammed Abo-Zahhad

Authors
  1. Ahmed Gomaa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Moataz M. Abdelwahab
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed Abo-Zahhad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed Gomaa.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gomaa, A., Abdelwahab, M.M. & Abo-Zahhad, M. Efficient vehicle detection and tracking strategy in aerial videos by employing morphological operations and feature points motion analysis. Multimed Tools Appl 79, 26023–26043 (2020). https://doi.org/10.1007/s11042-020-09242-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09242-5

Keywords

Search

Navigation