Abstract
Performance analysis of object detection combined with post-processing schemes are challenging especially that the spatial resolution of images is low in wide-area aerial imagery. In this paper, we present the quantitative results of ten object detection algorithms combined with several post-processing schemes including filtered dilation, heuristic filtering, sieving and closing, a three-stage scheme which involves thresholding with respect to area and compactness, and the proposed scheme of median filtering, opening and closing, followed by linear Gaussian filtering with nonmaximum suppression. We verified the sieving and closing as well as the three-stage scheme display better Fβ-score and PASCAL value via four vehicle detection algorithms. We evaluated combinations of ten object detection and segmentation methods with two post-processing schemes by adopting a set of recent evaluation metrics, i.e., Jaccard Index (JI), Fbw measure, the structure similarity measure (SSIM) and the enhanced alignment measure (EAM). Automatic detection outputs are compared with their ground truth in low-resolution aerial datasets. Classified detection results are established on ten algorithms each combined with the selected post-processing schemes. We take two widely used datasets (VIVID and VEDAI) for performance analysis, compare the detections and time cost of each algorithm either without or with the proposed scheme, and verified our approach via replacing either datasets or algorithms. Quantitative evaluation under a set of enhanced measures proves our test with validity, efficiency, and accuracy.
Similar content being viewed by others
Change history
12 January 2021
A Correction to this paper has been published: <ExternalRef><RefSource>https://doi.org/10.1007/s11042-020-10464-w</RefSource><RefTarget Address="10.1007/s11042-020-10464-w" TargetType="DOI"/></ExternalRef>
References
Achanta R, Süsstrunk S Saliency detection using maximum symmetric surround. In: Proc IEEE Int Conf Image Process (ICIP), Sep. 26–29, 2010, Hong Kong, China, pp 2653–2656
Achanta R, Hemami S, Estrada F, Süsstrunk S Frequency-tuned salient region detection. In: Proc IEEE Conf Comput Vis and Pattern Recognit (CVPR), Jun. 20–25, 2009, Miami, FL, pp 1597–1604
Ali FB, Powers DMW (2014) Fusion-based fastICA method: facial expression recognition. J Imag Graph 2(1):1–7
Bernard J, Shao S-C, Tunc C, Kheddouci H, Hariri S (2018) Quasi-cliques’ analysis for IRC channel thread detection. In: Int Conf Complex Networks Appl Springer, Cham, pp 578–589
Chen P, Yan W-Q Object detection based on saturation of visual perception. Multimedia Tools and Appl 79:1–20. https://doi.org/10.1007/s11042-020-08866-x
Chen K-Q, Fu K, Yan M-L, Gao X, Sun X, Wei X (2018) Semantic segmentation of aerial images with shuffling convolutional neural networks. IEEE Geosci Remote Sens Lett 15(2):173–177
Chen C, Zhong J-D, Tan Y (2019) Multiple oriented and small object detection with convolutional neural networks for aerial image. Remote Sens 11(18):2176. https://www.mdpi.com/2072-4292/11/18/2176. Accessed 18 Sept 2019
Chen X-Y, Li H-L, Wu Q-B, Ngan KN, Xu L-F (2020) High-quality R-CNN object detection using multi-path detection calibration network. IEEE Trans Cir Syst Video Technol 30:1–13. https://ieeexplore.ieee.org/document/9064498. Accessed 13 Apr 2020
Chen W-P, Qiao Y-T, Li Y-J (2020) Inception-SSD: an improved single shot detector for vehicle detection. J Ambient Intell Human Comput 11:1–7. https://doi.org/10.1007/s12652-020-02085-w
Fan D-P, Cheng M-M, Liu Y, Li T, Borji A Structure-measure: A new way to evaluate foreground maps. In: Proc IEEE Int Conf Comput Vis (ICCV), Oct. 22–29, 2017, Venice, Italy, pp 4548–4557
Fan D-P, Gong C, Cao Y, Ren B, Chen M-M, Borji A Enhanced-alignment measure for binary foreground map evaluation. In: Proc 27th Int Joint Conf Artificial Intelligence (IJCAI), July 13–19, 2018, Stockholm, Sweden, pp 698–704
Franchi G, Fehri A, Yao A (2020) Deep morphological networks. Pattern Recognit 102:107246
Fu Z-H, Chen Y-W, Yong H-W, Jiang R-X, Zhang L, Hua X-S (2019) Foreground gating and background refining network for surveillance object detection. IEEE Trans Image Process 28(12):6077–6090
Gao X (2016) Automatic detection, segmentation, and tracking of vehicles in wide-area aerial imagery, Thesis, Department of Electrical and Computer Engineering, The University of Arizona, Tucson, AZ, USA
Gao X (2018) Vehicle detection in wide-area aerial imagery: cross-association of detection schemes with post-processings. Int J Image Mining 3(2):106–116
Gao X (2018) A thresholding scheme of eliminating false detections on vehicles in wide-area aerial imagery. Int J Signal Image Syst Eng 11(4):217–224
Gao X, Ram S, Rodríguez JJ A performance comparison of automatic detection schemes in wide-area aerial imagery. In: 2016 IEEE Southwest Symp. Image Anal. Inter-pret. (SSIAI), March 6–8, 2016, Santa Fe, NM, pp 125–128
Gao X (2019) Diagnosing Abnormal Electrocardiogram (ECG) via Deep Learning. Practical Applications of Electrocardiogram. InTech Open Access, pp 3–17; [Online] Available: https://www.intechopen.com/books/practical-applications-of-electrocardiogram/diagnosing-abnormal-electrocardiogram-ecg-via-deep-learning. Accessed 03 Apr 2019
Gao X, Ram S, Rodríguez JJ (2020) A post-processing scheme for the performance improvement of vehicle detection in wide-area aerial imagery. Signal Image Video Process 14(3):625–633,635
Gao X, Ram S, Rodríguez JJ (2020) Exploiting bilinear interpolation and predictive particle swarm optimization for tilt correction of vehicle license plates. Int J Image Mining, in press
Gleason J, Nefian AV, Bouyssounousse X, Fong T, Bebis G Vehicle detection from aerial imagery. In: 2011 IEEE Int Conf Robotics Automat (ICRA’2011), May 9–13, 2011, Shanghai, China, pp 2065–2070
Han S-K, Yoo J-S, Kwon S-C (2019) Real-time vehicle detection method in bird-view unmanned aerial vehicle imagery. Sensors 19(18):3958
Hou X-D, Zhang L-Q Saliency detection: a spectral residual approach. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 18–23, 2007, Minneapolis, MN, pp 1–8
Huang Z-H, Leng J-S (2014) Texture extraction in natural scenes using region-based method. J Digital Inf Manag 12(4):246–254
Huang F, Qi J-Q, Lu H-C, Zhang L-H, Ruan X (2017) Salient object detection via multiple instance learning. IEEE Trans Image Process 26(4):1911–1922
Huang X-H, He P, Rangarajan A, Ranka S (2020) Intelligent intersection: two-stream convolutional networks for real-time near-accident detection in traffic video. ACM Trans Spatial Alg Syst 6(2):1–28
Jiao L-C, Zhang F, Liu F, Yang S-Y, Li L-L, Feng Z-X, Qu R (2019) A survey of deep learning-based object detection. IEEE Access 7:128837–128868
Karim S, Zhang Y, Yin S-L, Laghari AA, Brohi AA (2019) Impact of compressed and down-scaled training images on vehicle detection in remote sensing imagery. Multimedia Tools Appl 78(22):32565–32583
Kasturi R, Goldgof D, Soundararajan P, Manohar V, Boonstra M, Korzhova V (2006) Performance evaluation protocol for face, person and vehicle detection & tracking in video analysis and content extraction (VACE-II). Computer Science & Eng., Univ. South Florida, Tampa, FL, pp 17–18
Kasturi R, Goldgof D, Soundararajan P, Manohar V, Garofolo J, Bowers R, Zhang J (2009) Framework for performance evaluation of face, text, and vehicle detection and tracking in video: data, metrics, and protocol. IEEE Trans Pattern Anal Mach Intell 31(2):319–336
Kim J-W, Han D-Y, Tai Y-W, Kim J-M (2016) Salient region detection via high-dimensional color transform and local spatial support. IEEE Trans Image Process 25(1):9–23
Koga Y, Miyazaki H, Shibasaki R (2020) A method for vehicle detection in high-resolution satellite images that uses a region-based object detector and unsupervised domain adaptation. Remote Sens 12(3):575. [Online] Available: https://www.mdpi.com/2072-4292/12/3/575. Accessed 09 Feb 2020
Kurz F, Azimi S-M, Sheu C-Y, D’Angelo P (2019) Deep learning segmentation and 3D reconstruction of road markings using multi-view aerial imagery. ISPRS Int J Geo-Inf 8(47):1–16
Li S, Zhou G-Q, Zheng Z-Z, Liu Y-L, Li X-W, Zhang Y, Yue T The relation between accuracy and size of structure element for vehicle detection with high resolution highway aerial images. In: 2013 Int Geosci Remote Sens Symp (IGARSS' 2013), Jul. 21–26, 2013, Melbourne, Australia, pp 2645–2648
Li J, Dai Y-R, Li C-C, Shu J-Q, Li D-D, Yang T, Lu Z-Y (2019) Visual detail augmented mapping for small object detection. Remote Sens 11(1):14. [Online] Available: https://www.mdpi.com/2072-4292/11/1/14. Accessed 21 Dec 2018
Liu L-C, Chen CLP, You X-G, Tang Y-Y, Zhang Y-S, Li S-T (2017) Mixed noise removal via robust constrained sparse representation. IEEE Trans Circuits Syst Video Technol 28(9):2177–2189
Liu C-Y, Ding Y-L, Zhu M, Xiu J-H, Li M-Y, Li Q-H (2019) Vehicle detection in aerial images using a fast-oriented region search and the vector of locally aggregated descriptors. Sensors 19(15):3294
Lu H-C, Li X-H, Zhang L-H, Ruan X, Yang M-H (2016) Dense and sparse reconstruction error-based saliency descriptor. IEEE Trans Image Process 25(4):1592–1603
Liu Z-Y, Tang J-T, Xiang Q, Zhao P (2020) Salient object detection for RGB-D images by generative adversarial network. Multimedia Tools Appl 79(24):1–23. https://doi.org/10.1007/s11042-020-09188-8
Ma B-D, Liu Z-B, Jiang F-H, Yan Y-H, Yuan J-B, Hui B-S (2019) Vehicle detection in aerial images using rotation-invariant cascaded forest. IEEE Access 7:59613–59623
Mancas M, Gosselin B, Macq B, Unay D (2007) Computational attention for defect localization. In: Proc. ICVS Workshop Comput. Attent. Appl. (WCAA), Bielefeld, Germany, pp 1–10
Mandal M, Shah M, Meena P, Devi S, Vipparthi SK (Mar. 2020) AVDNet: a small-sized vehicle detection network for aerial visual data. IEEE Geosci Remote Sens Lett 17(3):494–498
Margolin R, Zelnik-Manor L, Tal A How to evaluate foreground maps? In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR), Jun. 23-28, 2014, Columbus, OH, pp 248–255
Murray N, Vanrell M, Otazu X, Parraga CA (2013) Low-level spatio-chromatic grouping for saliency estimation. IEEE Trans Pattern Anal Machine Intell 35(11):2810–2816
Nascimento JC, Marques JS (2006) Performance evaluation of object detection algorithms for video surveillance. IEEE Trans Multimedia 8(4):761–774
Pan Z-W, Hariri S, Pacheco J (2019) Context aware intrusion detection for building automation systems. Comput Security 85:181–201
Pflugfelder R, Weissenfeld A, Wagner J On learning vehicle detection in satellite video. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 13–19, 2020, Seattle, WA, pp 1–9; [Online] Available: https://arxiv.org/abs/2001.10900
Philip RC, Ram S, Gao X, Rodríguez JJ A comparison of tracking algorithm performance for objects in wide-area aerial imagery. In: 2014 IEEE Southwest Symp. Image Anal. Inter-pret. (SSIAI), April 6–8, 2014, San Diego, CA, pp 109–112
Porter R, Fraser AM, Hush D (2010) Wide-area motion imagery. IEEE Signal Process Mag 27(5):56–65
Prokaj J (2013) Exploitation of Wide-area Motion Imagery, Ph.D. Dissertation, University of Southern California, CA, pp 1–143
Prokaj J, Medioni G Persistent tracking for wide-area aerial surveillance. In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR), Jun. 23–28, 2014, Columbus, OH, pp 1186–1193
Qiu H-Q, Li H-L, Wu Q-B, Meng F-M, Ngan KN, Shi H-C (2019) A2 RMNet: adaptively aspect ratio multi-scale network for object detection in remote sensing images. Remote Sens. 11(13):1594. [Online] Available: https://www.mdpi.com/2072-4292/11/13/1594
Qiu H-Q, Li H-L, Wu Q-B, Meng F-M, Xu L-F, Ngan KN, Shi H-C Hierarchical context features embedding for object detection. IEEE Trans Multimedia. https://doi.org/10.1109/TMM.2020.2971175
Qiu H-Q, Li H-L, Wu Q-B, Shi H-C Offset bin classification network for accurate object detection. In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR), Jun. 14–19, 2020, Seattle, WA, pp 13188-13197; [Online] Available: https://openaccess.thecvf.com/content_CVPR_2020/papers/Qiu_Offset_Bin_Classification_Network_for_Accurate_Object_Detection_CVPR_2020_paper.pdf. Accessed 05 Aug 2020
Ram S (2017) Sparse representations and nonlinear image processing for inverse imaging solutions. Ph.D. Dissertation, Department of Electrical and Computer Engineering, The University of Arizona, Tucson, AZ, USA
Ram S, Rodríguez JJ Vehicle detection in wide area images using multi-scale structure enhancement and symmetry. IEEE Int Conf Image Process (ICIP'2016), Sept. 25–28, 2016, Phoenix, AZ, USA, pp 3817–3821
Ray KS, Chakraborty S (2019) Object detection by spatio-temporal analysis and tracking of the detected objects in a video with variable background. J Vis Commun Image Represent 58:662–674
Razaque A, Amsaad F, Hariri S, Almasri M, Rizvi SS, Frej MBH (2020) Enhanced grey risk assessment model for support of cloud service provider. IEEE Access 8: 80812-80826
Ren S-Q, He K-M, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Machine Intell 39(6):1137–1149
Ryu S, Ham B, Song K-H Contextual information based visual saliency model. In: Proc IEEE Int Conf Image Process (ICIP), Sep. 15–18, 2013, Melbourne, Australia, pp 201–205
Saha BN, Ray N (2009) Image thresholding by variational minimax optimization. Pattern Recognit 42(5):843–856
Salem MA, Ghamry N, Meffert B (2009) Daubechies versus biorthogonal wavelets for moving object detection in traffic monitoring systems. Informatik-Berichte 229:8–9
Samarabandu J, Liu X-Q (2007) An edge-based text region extraction algorithm for indoor mobile robot navigation. Int J Signal Process 3(4):273–280
Shaikh SH, Saeed K, Chaki N (2014) Moving object detection using background subtraction. Springer, pp 30–31
Shao S-C, Tunc C, Satam P, Hariri S (2017) Real-time IRC threat detection framework. In: IEEE 2nd Int Workshops Found Appl Self* Syst (FAS* W), pp 318–323
Shao S-C, Tunc C, Al-Shawi A, Hariri S (2018) Autonomic author identification in internet relay chat (IRC). In: IEEE/ACS 15th Int Conf Comput Syst Appl (AICCSA), pp 1–8
Shao S-C, Tunc C, Al-Shawi A, Hariri S (2019) One-class classification with deep autoencoder for author verification in internet relay chat. In: IEEE/ACS 16th Int Conf Comput Syst Appl (AICCSA), pp 1–8
Sharma B, Katiyar VK, Gupta AK, Singh A (2014) The automated vehicle detection of highway traffic images by differential morphological profile. J Transp Technol 4:150–156
Shen J-Q, Liu N-Z, Sun H, Zhou H-Y (2019) Vehicle detection in aerial images based on lightweight deep convolutional network and generative adversarial network. IEEE Access 7:148119–148130
Shi J-P, Xu L, Jia J-Y Just noticeable defocus blur detection and estimation. In: Proc. IEEE Conf Comput Vis Pattern Recognit (CVPR), Jun. 7–12, 2015, Boston, MA, pp 657–665
Sommer L, Schuchert T, Beyerer J (2019) Comprehensive analysis of deep learning-based vehicle detection using aerial images. IEEE Trans Cir Syst Video Technol 29(9):2733–2747
Song J-G, Park H-Y (2019) Object recognition in very low-resolution images using deep collaborative learning. IEEE Access 7:134071–134082
Stanković RS, Falkowski BJ (2003) The Haar wavelet transform: its status and achievements. Computer Elect Eng 29:25–44
Tayara H, Soo K-G, Chong K-T (2018) Vehicle detection and counting in high-resolution aerial images using convolutional regression neural network. IEEE Access 6:2220–2230
Tong K, Wu Y-Q, Zhou F (2020) Recent advances in small object detection based on deep learning: a review. Image Vis Comput 97:103910
Trujillo-Pino A, Krissian K, Alemán-Flores M, Santana-Cedrés D (2013) Accurate subpixel edge location based on partial area effect. Image Vis Comput 31(1):72–90
Türmer S (2014) Car detection in low-frame rate aerial imagery of dense urban areas, Ph.D. Dissertation, Technische Univ. ät München
Unser M, Chenouard N, Ville DVD (2011) Steerable pyramid and tight wavelet frames in L2(Rd). IEEE Trans Image Process 20(10):2705–2721
Vasu B-K (2018) Visualizing resiliency of deep convolutional network interpretations for aerial imagery. Master’s Thesis, Rochester Institute of Technology
Wang H-R, Celik T (2018) Sparse representation based hyper-spectral image classification. Signal Image Video Process 12(5):1009–1017
Wilson JN, Ritter GX (2000) Handbook of computer vision algorithms in image algebra. CRC Press, pp 114–115
Wu Y-Q, Hou W, Wu S-H (2011) Brain MRI segmentation using KFCM and Chan-Vese model. Trans Tianjin Univ 17(3):215–219
Xiang X-Z, Zhai M-L, Lv N, El Saddik A (2018) Vehicle counting based on vehicle detection and tracking from aerial videos. Sensors 18(8):2560
Xiao Y-Z, Tian Z-Q, Yu J-C, Zhang Y-S, Liu S, Du S-Y, Lan X-G (2020) A review of object detection based on deep learning. Multimedia Tools Appl 79(23):1–63. https://doi.org/10.1007/s11042-020-08976-6
Yang G-B, Du Q-S (2010) Application and Practical Examples of MATLAB Image / Video Processing. Publishing House of Electronics Library, pp 149–150
Yang M-Y, Liao W-T, Li X-B, Rosenhahn B Deep-learning for vehicle detection in aerial images. IEEE Int Conf Image Process (ICIP’2018), Oct. 7–10, 2018, Athens, Greece, pp 3080–3084
Yang J-X, Xie X-M, Yang W-Z (2019) Effective contexts for UAV vehicle detection. IEEE Access 7:85042–85054
Yang B, Zhang S, Tian Y, Li B-J (2019) Front-vehicle detection in video images based on temporal and spatial characteristics. Sensors 19(7):1728
Zhang X-X, Zhu X (2020) Moving vehicle detection in aerial infrared image sequences via fast image registration and improved YOLOv3 network. Int J Remote Sens 41(11):4312–4335
Zhang L, Gu Z-Y, Li H-Y SDSP: a novel saliency detection method by combining simple priors. In: Proc IEEE Int Conf Image Process (ICIP), Sep. 15–18, 2013, Melbourne, Australia, pp 171–175
Zhang T, Liu X-Y, Wang X-D, Walid A (2020) cuTensor-tubal: efficient primitives for tubal-rank tensor learning operations on GPUs. IEEE Trans Parallel Dist Syst 31(3):595–610
Zhang W, Liu C-S, Chang F-L, Song Y (2020) Multi-scale and occlusion aware network for vehicle detection and segmentation on UAV aerial images. Remote Sens 12(11):1760. [Online] Available: https://www.mdpi.com/2072-4292/12/11/1760
Zhao Z-Q, Zheng P, Xu S-T, Wu X (2019) Object detection with deep learning: a review. IEEE Trans Neural Network Learn Syst 30(11):3211–3231
Zheng Z-Z, Zhou G-Q, Wang Y, Liu Y-L, Li X-W, Wang X-T, Jiang L (2013) A novel vehicle detection method with high resolution highway aerial image. IEEE J Sel Top Appl Earth Observ Remote Sens 6(6):2338–2343
Zhou Y-F, Maskell S (2019) Detecting and tracking small moving objects in wide-area motion imagery using convolutional neural networks. Fusion, pp 1–8; [Online] Available: https://arxiv.org/abs/1911.01727
Zhou G-Y, Cui Y, Chen YL, Yang J, Rashvand HF (2010) SAR image edge detection using curvelet transform and Duda operator. Electron Lett 46(2):167–169
Zhou H, Wei L, Creighton D, Nahavandi S (2017) Orientation aware vehicle detection in aerial images. Electron Lett 53(21):1406–1408
Zhang X-F, Feng G-P, Gao X, Xu D-Z (2010) Blind multiuser detection for MC-CDMA with antenna array. Comput Elect Eng 36(1):160–168
Zhu, X-Z, Dai, J-F, Yuan L, Wei Y-C Towards high performance video object detection. In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR), Jun. 18–23, 2018, Salt Lake, UT, pp 7210–7218; [Online] Available: https://openaccess.thecvf.com/content_cvpr_2018/papers/Zhu_Towards_High_Performance_CVPR_2018_paper.pdf
Acknowledgments
The author declares no conflict of interests on this study. The author owes special thanks to anonymous reviewers for their suggestions on improving the quality of this manuscript. The author wishes to thank Dr. Jeno Szep, Dr. Pratik Satam, Dr. Sundaresh Ram, Prof. Jeffrey J. Rodríguez and Prof. Salim Hariri for their helpful guidance on constructing this set of research work.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gao, X. Performance evaluation of automatic object detection with post-processing schemes under enhanced measures in wide-area aerial imagery. Multimed Tools Appl 79, 30357–30386 (2020). https://doi.org/10.1007/s11042-020-09201-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09201-0