Skip to main content
SpringerLink
Log in

An Upgraded-YOLO with Object Augmentation: Mini-UAV Detection Under Low-Visibility Conditions by Improving Deep Neural Networks

  • Original Research
  • Published:
Operations Research Forum Aims and scope Submit manuscript

Abstract

Over the last few years, the manufacturing technology of mini-unmanned aerial vehicles (mini-UAVs), also known as mini-drones, has been experiencing a significant evolution. Thus, the early warning optical drone detection, as an important part of intelligent surveillance, is becoming a global research hotspot. In this article, the authors provide a prospective study to prevent any potential hazards that mini-UAVs may cause, especially those that can carry payloads. Subsequently, we regarded the problem of detecting and locating mini-UAVs in different environments as the problem of detecting tiny and very small objects from an aerial perspective. However, the accuracy and speed of existing detection algorithms do not meet the requirements of real-time detection. For solving this problem, we developed a mini-UAV detection model called Upgraded-YOLO based on the state-of-the-art object detection method of YOLOv5. The proposed model is able to perform real-time tiny/small flying object detection. The main contributions of this research are as follows: firstly, an air image dataset of mini-UAVs was built using a Dahua multisensor camera. Secondly, a strategy of instance augmentation is proposed, in which we added small appearance of mini-drones to samples of the custom air image dataset. Thirdly, in addition to hyperparameter tuning and optimization operations, shallow layers are added to improve the model’s ability to detect mini-UAVs. A comparative study with several contemporary object detectors proved that the Upgraded-YOLO performed better. Therefore, the proposed mini-UAV detection technology can be deployed in a monitoring center in order to protect a strategic installation even in low-visibility conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data Availability

The data presented in this study are available on request from the corresponding author.

References

  1. USAF (2009) Unmanned Aircraft Systems Flight Plan 2009-2047, Technical report, unclassified, United States Air Force, Washington DC, pp. 24-27

  2. Doyle DD (2013) Real-time, multiple, pan/tilt/zoom, computer vision tracking, and 3D position estimating system for small unmanned aircraft system metrology. DEPARTMENT OF THE AIR FORCE, AIR UNIVERSITY, Wright-Patterson Air Force Base, Ohio, USA. Jeffrey Maddalon

  3. Maddalon J, Hayhurst KJ, Koppen DM, Upchurch JM (2013) Perspectives on unmanned aircraft classification for civil airworthiness standards. Langley Research Center, Hampton, Virginia. NASA/TM–2013-217969

  4. Lykou G, Moustakas D, Gritzalis D (2020) Defending airports from UAS: a survey on cyber-attacks and counter-drone sensing technologies. Sensors 20(12):3537

  5. Official DJI website: https://www.dji.com/matrice600-pro/info (last Accessed on 23 Mar 2021)

  6. Seidaliyeva U, Akhmetov D, Ilipbayeva L, Matson ET (2020) Real-time and accurate drone detection in a video with a static background. Sensors 20(14):3856

  7. Official DJI website: https://www.dji.com/t16/info#downloads (Last Accessed on 23 Jun 2021)

  8. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497

  9. He K, Gkioxari G, Dollár P, Girshick R (2018) Mask R-CNN. arXiv preprint arXiv:1703.06870

  10. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Alexander C (2015) Berg. SSD: single shot MultiBox detector. arXiv preprint arXiv:1512.02325

  11. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.779-788

  12. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. arXiv preprint arXiv:1612.08242

  13. Everingham M, Ali Eslami SM, Van Gool L, Williams CKI, Winn J, Zisserman A (2015) The Pascal Visual Object Classes challenge: a retrospective. Int J Comput Vis 111:98–136

  14. Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The PASCAL Visual Object Classes (VOC) challenge. Int J Comput Vis 88:303–338

  15. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767

  16. Lawal MO (2021) Tomato detection based on modified YOLOv3 framework. Sci Rep Jan 14;11(1):1447. https://doi.org/10.1038/s41598-021-81216-5. PMID: 33446897; PMCID: PMC7809275

  17. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Lawrence Zitnick C (2014) Microsoft COCO: common objects in context. in 13th European Conference on Computer Vision, pp. 740–755

  18. Kim D-H (2019) Evaluation of COCO Validation 2017 Dataset with YOLOv3. Journal of Multidisciplinary Engineering Science and Technology 6(7)

  19. Bochkovskiy A, Wang CY, Liao HYM (2020) YOLOv4: optimal speed and accuracy of object detection, arXiv preprint arXiv: 2004.10934

  20. Wang Z, Wu Y, Yang L, Thirunavukarasu A, Evison C, Zhao Y (2021) Fast personal protective equipment detection for real construction sites using deep learning approaches Sensors 21(10):3478

  21. Ultralytics YOLOv5 and Vision AI, Madrid, Spain. Available online: http://www.ultralytics.com (Last Accessed on 03 Aug 2021)

  22. Kharel S, Ahmed KR (2021) Potholes detection using deep learning and area estimation using image processing, Proceedings of SAI Intelligent Systems Conference, IntelliSys 2021: Intelligent Systems and Applications 296:373-388

  23. Wang X, Wei J, Liu Y, Li J, Zhang Z, Chen J, Jiang B (2021) Research on morphological detection of FR I and FR II radio galaxies based on improved YOLOv5. Universe 7(7):211

  24. Yan B, Fan P, Lei X, Liu Z, Yang F (2021) A real-time apple targets detection method for picking robot based on improved YOLOv5. Remote Sensing 13(9):1619

  25. Yang G, Feng W, Jin J, Lei Q, Li X, Gui G, Wang W (2020) Face mask recognition system with YOLOV5 based on image recognition. Proceedings of 2020 IEEE 6th International Conference on Computer and Communications, IEEE Xplore , pp. 1398-1404

  26. COCO dataset. Available online: https://cocodataset.org/#home (Last Accessed on 17 Sept 2021)

  27. Adibhatla VA, Chih H-C, Hsu C-C, Cheng J, Abbod MF, Shieh J-S (2021) Applying deep learning to defect detection in printed circuit boards via a newest model of you-only-look-once. Math Biosci Eng 18(4):4411-4428

  28. Agarwal S, Du Terrail JO, Jurie F (2018) Recent advances in object detection in the age of deep convolutional neural networks. arXiv preprint arXiv: 1809.03193

  29. Yao J, Qi J, Zhang J, Shao H, Yang J, Li X (2021) A real-time detection algorithm for kiwifruit defects based on YOLOv5. Electronics 10(14):1711

  30. Dahua Technology. Available online: https://www.dahuasecurity.com/products/All-Products/Thermal-Cameras/Wizmind-Series/TPC-8-Series/TPC-PT8621C (Last Accessed on 04 Aug 2021)

  31. Zhang Y, Yongliang S, Jun Z (2019) An improved tiny-yolov3 pedestrian detection algorithm. Digital Signal Processing 183:17-23

  32. NguyenN-D, Do T, Ngo TD, Le D-D (2020) An evaluation of deep learning methods for small object detection. J Electr Comput Eng 2020(3189691):18

  33. Delleji T, Fekih H, Chtourou Z (2020) Deep learning-based approach for detection and classification of micro/mini drones. In 2020 4th International Conference on Advanced Systems and Emergent Technologies (IC_ASET), pp. 332–337

  34. Kisantal M, Wojna Z, Murawski J, Naruniec J, Cho K (2020) Augmentation for small object detection. arXiv preprint arXiv: 1902.07296

  35. Tong K, Wu Y, Zhou F (2020) Recent advances in small object detection based on deep learning: a review. Image and Vision Computing, Science Direct, ELSEVIER 97

  36. Pang J, Li C, Shi J, Xu Z, Feng H (2019) R2-CNN: fast tiny object detection in large-scale remote sensing images. IEEE Trans Geosci Remote Sens 57(8)

  37. Zhang Y, Bai Y, Ding M, Ghanem B (2020) Multi-task generative adversarial network for detecting small objects in the wild. Int J Comput Vis pp. 1810-1828

  38. Chen C, Liu M-Y, Tuzel O, Xiao J (2016) R-CNN for small object detection. Asian Conference on Computer Vision ACCV, pp.214-230

  39. Du Z, Yin J, Yang J (2019) Expanding receptive field YOLO for small object detection. Journal of Physics. Conference Series, 3rd International Conference on Electrical, Mechanical and Computer Engineering, Guizhou, China 1314

  40. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2018) Focal loss for dense object detection. IEEE Transactions on PatternAnalysis and Machine Intelligence 42(2):318–327

  41. Zheng Z, Wang P, Liu W, Ye R, Ren D (2020) Distance-IoU loss: faster and better learning for bounding box regression. AAAI Conference on Artificial Intelligence 34(07)

  42. Rezatofighi H, Tsoi N, Gwak JY, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: a metric and a loss for bounding box regression. arXiv preprint arXiv: 1902.09630

  43. Madasamy K, Shanmuganathan V, Kandasamy V, Lee MY, Thangadurai M (2021) OSDDY: embedded system-based object surveillance detection system with small drone using deep YOLO. EURASIP Journal on Image and Video Processing 2021:19

  44. Wang X, Song J (2021) ICIoU: improved loss based on complete intersection over union for bounding box regression. IEEE Access 9:105686–105695

    Article  Google Scholar 

  45. Zhou J, Tian Y, Yuan C, Yin K, Yang G, Wen M (2019) Improved UAV opium poppy detection using an updated YOLOv3 model. Sensors 19(22):4851

  46. Wicaksono AS, Supianto AA (2018) Hyper parameter optimization using genetic algorithm on machine learning methods for online news popularity prediction. International Journal of Advanced Computer Science and Applications(IJACSA) 9(12)

  47. Chawla S (2016) Application of genetic algorithm and backpropagation neural network for effective personalize web search-based on clustered query sessions. International Journal of Applied Evolutionary Computation (IJAEC) 7(1):33–49

    Article  Google Scholar 

  48. Kingma DP, Ba JL (2014) A method for stochastic optimization. arXiv preprint arXiv: 1412.6980

  49. Nicolai W, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. arXiv preprint arXiv: 1703.07402

  50. Jiang N, Peng X, Yu X, Wang Q, Xing J, Li G, Zhao J, Guo G, Han Z (2021) Anti-UAV: a large multi-modal benchmark for UAV tracking. arXiv preprint arXiv: 2101.08466

  51. Zhao J, Wang G, Li J, Jin L, Fan N, Wang M, Wang X, Yong T, Deng Y, Guo Y, Ge S, Guo G (2021) The 2nd Anti-UAV Workshop & Challenge: methods and results. arXiv preprint arXiv: 2108.09909

  52. website Roboflow. PyTorch Object Detection, YOLOv5 is Here, https://models.roboflow.com/object-detection/yolov5 (Last Accessed on 03 Dec 2021)

  53. Yuxin F, Liao B, Wang X, Fang J, Qi J, Wu R, Niu J, Liu W (2021) You only look at one sequence: rethinking transformer in vision through object detection. arXiv preprint arXiv: 2106.00666

  54. Adibhatla VA, Chih H-C, Hsu C-C, Cheng J, Abbod MF, Shieh J-S (2021) Applying deep learning to defect detection in printed circuit boards via a newest model of you-only-look-once. Math Biosci Eng 18

  55. Sahin O, Ozer S (2021) YOLODrone: improved YOLO architecture for object detection in drone images. In 2021 44th International Conference on Telecommunications and Signal Processing (TSP), pp. 361-365. IEEE

  56. ImageNet. ImageNet Large Scale Visual Recognition Challenge 2017 (ILSVRC2017). Available online: https://image-net.org/challenges/LSVRC/2017/ (Last Accessed on 22 Jul 2022)

Download references

Acknowledgements

This research is supported by the Tunisian Ministry of National Defense, Science and Technology for Defense Lab (STD), and Military Research Center through a research and development project.

Funding

This research was funded by the Military Research Center, Taeib Mhiri, Aouina, 2045, Tunis, Tunisia.

Author information

Author notes

  1. Feten Slimeni and Hedi Fekih are authors contributed equally to this work.

Authors and Affiliations

  1. Science and Technology for Defense Lab (STD), Military Research Center, Taeib Mhiri, Aouina, 2045, Tunis, Tunisia

    Tijeni Delleji, Feten Slimeni, Hedi Fekih, Achref Jarray, Wadi Boughanmi & Zied Chtourou

  2. SM@RTS Laboratory, Digital Research Centre of Sfax, Tunis road km 10, El-Ons City, 3021, Sfax, Tunisia

    Tijeni Delleji, Hedi Fekih & Abdelaziz Kallel

Authors
  1. Tijeni Delleji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feten Slimeni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hedi Fekih
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Achref Jarray
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wadi Boughanmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Abdelaziz Kallel
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zied Chtourou
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

T.D. (Tijeni Delleji) presented the ideas, carried out the experiments, and written the paper. F.S. (Feten Slimeni) contributed to programming, writing, and review. H.F (Hedi Fekih) contributed to review and helped in obtaining the real-time images of flying mini-UAVs. A.J. (Achref Jarry) and W.B. (Wadi Boughanmi) contributed to the original draft preparation. A.K (Abdelaziz Kallel) contributed to review and editing the final version of the manuscript. Z.C. (Zied Chtourou) took the responsibility of supervision. All authors have read and agreed the published version of the manuscript.

Corresponding author

Correspondence to Tijeni Delleji.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Delleji, T., Slimeni, F., Fekih, H. et al. An Upgraded-YOLO with Object Augmentation: Mini-UAV Detection Under Low-Visibility Conditions by Improving Deep Neural Networks. Oper. Res. Forum 3, 60 (2022). https://doi.org/10.1007/s43069-022-00163-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s43069-022-00163-7

Keywords

Search

Navigation