AURORA: autonomous real-time on-board video analytics

Abstract

In this paper, we describe the design and implementation of a computationally efficient system for detecting moving objects on a moving platform which can be deployed on small, lightweight, low-cost and power-efficient hardware. The primary application of the payload system is that of performing real-time on-board autonomous object detection of moving objects from videos stream taken from a camera mounted to an unmanned aerial vehicle (UAV). The implemented object detection algorithms utilise recursive density estimation and evolving local means clustering to perform change and object detection of moving objects without prior knowledge. Furthermore, experiments are presented which demonstrate that the introduced system is able to detect, by on-board processing, any moving objects from a UAV in real time while at the same time sending only important data to a control station located on the ground with minimal time to set up and become operational.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. 1.

    Angelov P (2012) Anomalous system state identification. Patent application, GB1208542.9

  2. 2.

    Angelov P (2012) Autonomous learning systems: from data streams to knowledge in real time. Wiley, New York

    Google Scholar 

  3. 3.

    Angelov P, Sadeghi-Tehran P, Ramezani R (2011) A real-time approach to autonomous novelty detection and object tracking in video stream. Int J Intell Syst 26:189–205

    Article  MATH  Google Scholar 

  4. 4.

    Angelov P, Wilding A (2015) RTSDE: recursive total-sum-distances-based density estimation approach and its application for autonomous real-time video analytics. In: Symposium Series on Computational Intelligence

  5. 5.

    Baruah RD, Angelov P (2012) Evolving local means method for clustering of streaming data. In: IEEE World congress on computational intelligence, pp 2161–2168

  6. 6.

    Bouguet JY (2001) Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corp 5(1–10):4

    Google Scholar 

  7. 7.

    Clarke C (2014) Smartphone application for real-time object detection. Master’s thesis, School of Computing and Communications, Lancaster University

  8. 8.

    Farin D, de With PHN, Effelsberg WA (2004) Video-object segmentation using multi-sprite background subtraction. IEEE Int Conf Multimed Expo, ICME ’04 1:343–346

    Google Scholar 

  9. 9.

    Fauske E, Eliassen LM, Bakken RH (2009) A comparison of learning based background subtraction techniques implemented in CUDA. In: NAIS, pp 181–192

  10. 10.

    Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(11):381–395

    MathSciNet  Article  Google Scholar 

  11. 11.

    Gao T, Liu ZG, Yue SH, Mei JQ (2009) Traffic video-based moving vehicle detection and tracking in the complex environment. Int J Cybern Syst 40(7):569–588

    Article  Google Scholar 

  12. 12.

    Gelgon M, Bouthemy PA (2000) A region-level motion based graph representation and labeling for tracking a spatial image partition. Pattern Recognit 33:725–740

    Article  Google Scholar 

  13. 13.

    Hayman E, Eklundh JO (2003) Statistical background subtraction for a mobile observer. In: IEEE international conference on computer vision. IEEE, pp 67–74

  14. 14.

    Huwer S, Niemann H (2000) Adaptive change detection for real-time surveillance applications. In: Third IEEE international workshop on visual surveillance. IEEE, pp 37–46

  15. 15.

    Kim K, Chalidabhongse TH, Harwood D, Davis L (2005) Real-time foreground–background segmentation using codebook model. Real-Time Imaging 11(3):172–185

    Article  Google Scholar 

  16. 16.

    Li L, Huang W, Tian Q (2004) A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans Image Process 13(11):1459–1472

    Article  Google Scholar 

  17. 17.

    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  18. 18.

    Morris G., Angelov P (2014) Real-time novelty detection in video using background subtraction techniques: state of the art. In: IEEE international conference on systems, man and cybernetics

  19. 19.

    Murray D, Basu A (1994) Motion tracking with an active camera. IEEE Trans Pattern Anal Mach Intell 16:449–459

    Article  Google Scholar 

  20. 20.

    Price A, Pyke J, Ashiri D, Cornall T (2006) Real time object detection for an unmanned aerial vehicle using an FPGA based vision system. In: IEEE international conference on robotics and automation (ICRA), pp 2854–2859

  21. 21.

    Sadeghi-Tehran P, Angelov P (2015) ARTOD: autonomous real time objects detection by a moving camera using recursive density estimation. In: Novel applications of intelligent systems, vol 586. Springer, Berlin, pp 123–138

  22. 22.

    Sadlier DA, O’Connor NE (2010) Evaluation of a vehicle tracking system for multi-modal UAV-captured video data. SPIE Defense, Security, and Sensing 7668

  23. 23.

    Shi J, Tomasi C (1994) Good features to track. In: IEEE computer society conference on computer vision and pattern recognition, CVPR ’94. IEEE Computer Society Press, pp 593–600

  24. 24.

    Stauffer C, Grimson, WEL (1999) Adaptive background mixture models for real-time tracking. In: IEEE computer society conference on computer vision and pattern recognition

  25. 25.

    Sugaya Y, Kanatani K (2004) Extracting moving objects from a moving camera video sequence. In: Symposium on sensing via image information, pp 279–284

  26. 26.

    Thornton S, Hoffelder M, Morris D (2008) Multi-sensor detection and tracking of humans for safe operations with unmanned ground vehicles. In: Proceedings of the 1st IEEE workshop on human detection from mobile platforms, 20 May 2008, Pasadena, CA, USA

  27. 27.

    Tsinko E (2010) Background subtraction with a Pan/Tilt Camera. Ph.D. thesis, The University of British Columbia

  28. 28.

    Watanabe Y, Fabiani P, Le Besnerais G (2009) Simultaneous visual target tracking and navigation in a GPS-denied environment. Int Conf Adv Robot ICAR 2009:1–6

    Google Scholar 

  29. 29.

    Zhang G, Jia J, Xiong W, Wong TT, Heng PA, Bao H (2007) Moving object extraction with a hand-held camera. In: IEEE 11th international conference on computer vision, ICCV 2007, pp 1–8

Download references

Acknowledgments

This work was funded under the MODs Centre for Defence Enterprise themed call for Generic Enablers for Low Size, Weight, Power and Cost (SWAPC), contact reference DSTLX1000082760. RTSDE was developed in EntelSenSys Ltd as a subcontractor in this work. The authors would like to thank Dr. Asmar Khan (C code and first experiments) and Mr. Ashley Wilding (Beagle board implementation) for their contribution to AURORA Project.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Plamen Angelov.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Angelov, P., Sadeghi-Tehran, P. & Clarke, C. AURORA: autonomous real-time on-board video analytics. Neural Comput & Applic 28, 855–865 (2017). https://doi.org/10.1007/s00521-016-2315-7

Download citation

Keywords

  • Autonomous objects detection
  • Unmanned aerial vehicle
  • Evolving clustering
  • Video analytic
  • Linear motion model