Skip to main content
SpringerLink
Log in

A video-based object detection and tracking system for weight sensitive UAVs

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The capability of detecting and tracking targets can play a significant role in mobile robot navigation systems. Visual tracking systems may control the direction and speed of motion of a robot to keep the target in its field of view either by moving the robot itself or the vision sensors. In this work, a compact size tracking and guiding system that could be mounted on weight-sensitive UAV platforms is presented. The system combines a motion detection technique that could be used with non-static cameras in addition to color filtering to detect and track objects in the field of view of a UAV. This hybrid system provides a reliable tracking system for low resolution images taken by a UAV camera. The proposed system implements keypoint detection algorithms including SIFT, SURF and FAST, a motion detection method using frame subtraction and object detection algorithms using color back projection in a hybrid approach that utilizes the best of each algorithm and avoids heavy usage of computing resources. Keypoint detectors SURF, SIFT and FAST are tested and implemented for the purpose of image alignment and frame subtractions. Experimental tests showed the system’s ability to detect and track low detailed targets. The system is tested on a UAV using a Raspberry Pi 2 mini-computer running OpenCV libraries and was able to process eleven frames per second implementing object detection and tracking. The test objects were mainly cars monitored from different altitudes through a UAV downward pointing camera.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. 3 Support, “IRIS+ Portal,” 3D Robotics, [Online]. Available: https://3dr.com/support/sections/200257165/iris/

  2. Abughalieh K, Qadi W, Melkon K, Fakes B, Sababha B, Al-Mousa A (2014) A compact portable object tracking system. Inform Commun Syst (ICICS), 01 5th Int Conf IEEE

  3. Ali S, Shah M (2006) COCOA: tracking in aerial imagery. Defense Sec Symp Int Soc Optics Photo

  4. H. Bay, T. Tuytelaars and L. V. Gool, (2006) SURF: speeded up robust features. Comput Vision – ECCV 2006, Springer Berlin Heidelberg: 404–417

    Chapter  Google Scholar 

  5. Boudjit K, Larbes C (2015) Detection and implementation autonomous target tracking with a Quadrotor AR.Drone. Inform Contrl Auto Robot (ICINCO). 2015 12th Int Conf

  6. Bradski GR (1998) Computer vision face tracking for use in a perceptual user interface. Intel Technol J

  7. Chen P, Dang Y, Liang R, Zhu W, He X (2018) Real-time object tracking on a drone with multi-inertial sensing data. IEEE Trans Intell Transp Syst 19(1):131–139

    Article  Google Scholar 

  8. Chi Z, Li H, Lu H, Yang M-H (2017) Dual deep network for visual tracking. IEEE Trans Image Process 26(4):2005–2015

    Article  MathSciNet  Google Scholar 

  9. Collins R, Zhou X, Teh SK (2005) An open source tracking testbed and evaluation web site. IEEE Int Workshop Perform Eval Track Surveill 2:35

    Google Scholar 

  10. Coşkun M, Ünal S (2015) Implementation of Tracking of a Moving Object Based on Camshift Approach with a UAV. 9th Int Conf Interdisciplin Eng, INTER-ENG 2015, Tirgu-Mures, Romania

  11. Fabian J, Young T, Jones JCP, Clayton GM (2014) Integrating the microsoft kinect with simulink: real-time object tracking example. IEEE/ASME Trans Mechatron 19(1):249–257

    Article  Google Scholar 

  12. Gil A, Mozos OM, Ballesta M, Reinoso O (2010) A comparative evaluation of interest point detectors and local descriptors for visual SLAM. Mach Vis Appl 21(6):905–920

    Article  Google Scholar 

  13. Ha S-W, Moon Y-H (2011) Multiple object tracking using SIFT features and location matching. Int J Smart Home 5(4):17–26

    Google Scholar 

  14. Kalantar B, Mansor SB, Halin AA, Shafri HZM, Zand M (2017) Multiple moving object detection from UAV videos using trajectories of matched regional adjacency graphs. IEEE Trans Geosci Remote Sens 55(9):5198–5213

    Article  Google Scholar 

  15. Kim B-G, Park D-J (2006) Novel target segmentation and tracking based on fuzzy membership distribution for vision-based target tracking system. Image Vis Comput 24(2006):1319–1331

    Article  Google Scholar 

  16. Lang H, Wang Y, Silva CD (2010) Vision based object identification and tracking for mobile robot visual servo control. 8th IEEE Int Conf Contrl Auto (ICCA)

  17. Li Z, Liu Y, Walker R, Hayward R, Zhang J (2010) Towards automatic power line detection for a UAV surveillance system using pulse coupled neural filter and an improved Hough transform. Mach Vis Appl 21(5):677–686

    Article  Google Scholar 

  18. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  19. Luis M, Srikanth S, Pascual C, Gaurav S (2006) A visual servoing approach for tracking features in urban areas using an autonomous helicopter. Robot Auto 2006. ICRA 2006. Proc 2006 IEEE Int Conf Orlando, FL

  20. Maggio E, Cavallaro A (2011) Video tracking: theory and practice. John Wiley & Sons

  21. Martínez C, Campoy P, Mondragón IF, Sánchez-Lopez JL, Olivares-Méndez MA (2014) HMPMR strategy for real-time tracking in aerial images, using direct methods. Mach Vis Appl 25(5):1283–1308

    Article  Google Scholar 

  22. Minaeian S, Liu J, Son Y-J (2018) Effective and efficient detection of moving targets from a UAV’s camera. IEEE Trans Intell Transp Syst 19(2):497–506

    Article  Google Scholar 

  23. R. P. FOUNDATION, RASPBERRY PI 2 MODEL B,” [Online]. Available: https://www.raspberrypi.org/products/raspberry-pi-2-model-b/

  24. Rawashdeh NA, Rawashdeh OA, Sababha BH (2017) Vision-based sensing of UAV attitude and altitude from downward in-flight images. J Vib Control 23(5):827–841

    Article  Google Scholar 

  25. Richards B, Gan M, Dayton J, Quintana J, Liu J, Enriquez M (2014) Obstacle avoidance system for UAVs using computer vision. IEEE Sens J

  26. Rosten E, Porter R, Drummond T (2010) Faster and better: a machine learning approach to corner detection. Pattern Anal Mach Intell IEEE Trans 32(1):105–119

    Article  Google Scholar 

  27. Saripalli S, Montgomery JF, Sukhatme GS (2003) Visually guided landing of an unmanned aerial vehicle. Robot Auto IEEE Trans 19(3):371–380

    Article  Google Scholar 

  28. Sirmacek B, Cem U (2009) Urban-area and building detection using SIFT keypoints and graph theory. IEEE Trans Geosci Remote Sens 47(4):1156–1167

    Article  Google Scholar 

  29. Snavely N, Seitz SM, Szeliski R (2008) Skeletal graphs for efficient structure from motion. CVPR 1

  30. Szeliski R (2010) Computer vision: algorithms and applications, Springer Science & Business Media

  31. Thumser P, Haas C, Tuhtan JA, Fuentes-Pérez JF, Toming G (2017) RAPTOR-UAV: real-time particle tracking in rivers using an unmanned aerial vehicle. Earth Surf Process Landf 42(14):2439–2446

    Article  Google Scholar 

  32. Valasek J, Kirkpatrick K, May J, Harris J (2016) Intelligent motion video guidance for unmanned air system ground target surveillance. J Aerospace Inform Syst 13(1):10–26

    Article  Google Scholar 

  33. Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv (CSUR) 38(4)

    Article  Google Scholar 

  34. Zhang S (2005) Object tracking in unmanned aerial vehicle (UAV) videos using a combined approach. Acoust Speech Signal Process 2005. Proc (ICASSP '05) IEEE Int Conf

  35. Zhang S, Wang C, Chan S-C, Wei X, Ho C-H (2015) New object detection, tracking, and recognition approaches for video surveillance over camera network. IEEE Sensors J 15(5):2679–2691

    Article  Google Scholar 

  36. The Robotics Institute, School of Computer Science, Carnegie Mellon University, “VIVID Tracking Evaluation Web Site,” [Online]. Available: http://vision.cse.psu.edu/data/vividEval/main.html. Accessed 3 May 2018.

Download references

Funding

This work was funded by King Abdullah I School of Graduate Studies and Scientific Research at Princess Sumaya University for Technology (PSUT).

Author information

Authors and Affiliations

  1. Computer Engineering Department, Princess Sumaya University for Technology, Amman, 11941, Jordan

    Karam M. Abughalieh & Belal H. Sababha

  2. Department of Mechatronics Engineering, German Jordanian University, Amman, 11180, Jordan

    Nathir A. Rawashdeh

Authors
  1. Karam M. Abughalieh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Belal H. Sababha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nathir A. Rawashdeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Belal H. Sababha.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abughalieh, K.M., Sababha, B.H. & Rawashdeh, N.A. A video-based object detection and tracking system for weight sensitive UAVs. Multimed Tools Appl 78, 9149–9167 (2019). https://doi.org/10.1007/s11042-018-6508-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6508-1

Keywords

Search

Navigation