Convolutional Neural Network-Based Multi-Target Detection and Recognition Method for Unmanned Airborne Surveillance Systems

  • Sang-Hyeon Kim
  • Han-Lim ChoiEmail author
Original Paper


This paper proposes the convolutional neural network (CNN)-based multiple targets detection and recognition method for unmanned airborne surveillance systems. The proposed method is capable of recognizing the target’s type, position and bearing angle. Recently, deep learning approaches using convolutional neural networks (CNNs) have significantly improved the object detection accuracy on benchmark datasets such as Pascal visual object classes (VOC) and common objects in context (COCO) data sets. Typical CNN-based object detection technologies are designed to recognize regions of interest (RoI) and object classes based on VOC or COCO data set criteria only. However, in many surveillance missions, the bearing angle of the object is also an important entity to infer in addition to the RoI and the vehicle-type. This paper proposes a CNN-based object recognition technique called airborne surveillance neural network (ASNet) that can recognize this additional bearing angle information. Indoor experiments demonstrate the validity of the proposed method.


Convolutional neural network (CNN) Multi-target detection and recognition Unmanned airborne surveillance Bearing angle Airborne surveillance neural network (ASNet) 

List of symbols

\( \left( {x,y} \right) \)

Center coordinates of bounding box

\( \left( {w, h} \right) \)

Width and height of bounding box

\( \theta \)

Bearing angle of object

\( C \)

Conditional probability of class

\( p^{\text{obj}} \)

Probability of objectiveness



This work was supported in part by Institute for Information & communications Technology Promotion (IITP) Grant funded by the Korea Government (MSIT) (#20150002130042002).


  1. 1.
    LeCun Y, Bottou L, Bengio Y, Haffener P (1998) Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, 1998Google Scholar
  2. 2.
    Karpathy A, Toderici G, Shetty S, Leung S, Sukthankar T, Fei-Fei RL (2014) Large-scale video classification with convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2014Google Scholar
  3. 3.
    Tuermer S, Kurz F, Reinartz P, Stilla U (2013) Airborne vehicle detection in dense urban areas using HoG features and disparity maps. IEEE J Sel Topics Appl Earth Observ Remote Sens 6(6):2327–2337CrossRefGoogle Scholar
  4. 4.
    Moranduzzo T, Melgani F (2014) Detecting cars in UAV images with a catalog-based approach. IEEE Trans Geosci Remote Sens 52(10):6356–6367CrossRefGoogle Scholar
  5. 5.
    Liu K, Mattyus G (2015) Fast multiclass vehicle detection on aerial images. IEEE Geosci Remote Sens Lett 12(9):1938–1942CrossRefGoogle Scholar
  6. 6.
    Teutsch M, Kruger W (2015) Robust and fast detection of moving vehicles in aerial videos using sliding windows. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2015Google Scholar
  7. 7.
    Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, 2015Google Scholar
  8. 8.
    Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016Google Scholar
  9. 9.
    Redmon J, Farhadi A (2016) YOLO9000: better, faster, stronger. arXiv preprint arXiv:1612.08242, 2016
  10. 10.
    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. In: European conference on computer vision, 2016Google Scholar
  11. 11.
    Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, 2015Google Scholar
  12. 12.
    Ammour N, Alhichri H, Bazi Y, Benjdira B, Alajlan N, Zuair M (2017) Deep learning approach for car detection in UAV imagery. Remote Sens 9(4):2017CrossRefGoogle Scholar
  13. 13.
    Tang T, Zhou S, Deng Z, Zou H, Lei L (2017) Vehicle detection in aerial images based on region convolutional neural networks and hard negative example mining. Sensors 17(2):2017Google Scholar
  14. 14. Accessed 17 July 2017
  15. 15.
    Kim S-H, Choi H-L (2017) Moving target tracking and recognition method for unmanned airborne surveillance systems. J Inst Control Robot Syst 23(3):157–164CrossRefGoogle Scholar
  16. 16.
  17. 17.
  18. 18.

Copyright information

© The Korean Society for Aeronautical & Space Sciences 2019

Authors and Affiliations

  1. 1.Department of Aerospace EngineeringKorea Advanced Institute of Science and TechnologyDaejeonRepublic of Korea
  2. 2.Department of Aerospace Engineering and KI for RoboticsKorea Advanced Institute of Science and TechnologyDaejeonRepublic of Korea

Personalised recommendations