Abstract
Remote-sensing object detection is a challenging task due to the difficulties of separating the objects with arbitrary direction from complex backgrounds. Though substantial progress has been made, there still exist challenges for object detection under the scenario of small scale, large aspect ratio, and dense distribution. Besides, the current mainstream approach falls under anchor-based multi-stage method, which has a serious shortcoming of slower inference speed. To conquer the aforementioned issues, this paper used RoPoints (points in rotation objects), a new better representation of objects as a set of sample points to perform object localization and classification. Then, we propose an anchor-free refined rotation detector:ROPDet based on RoPoints for more accurate and faster object detection. In our method, there is no need to predefine a large number anchors with different shapes. We only need to learn RoPoints for each object followed by converting to the corresponding bounding box, which greatly accelerates the inference process. Extensive experiments on two public remote-sensing datasets DOTA and HRSC-2016 demonstrate the competitive ability in terms of accuracy and inference speed.
Similar content being viewed by others
References
Azimi, S.M., Vig, E., Bahmanyar, R., Körner, M., Reinartz, P.: Towards multi-class object detection in unconstrained remote sensing imagery. In: Asian Conference on Computer Vision. Springer, pp. 150–165 (2018)
Cai, Z., Vasconcelos, N.: Cascade r-cnn: delving into high quality object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6154–6162 (2018)
Chen, K., Pang, J., Wang, J., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Shi, J., Ouyang, W., et al.: Hybrid task cascade for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4974–4983 (2019)
Chen, W., Sun, T., Li, M., Jiang, H., Zhou, C.: A new image co-segmentation method using saliency detection for surveillance image of coal miners. Comput. Electr. Eng. 40(8), 227–235 (2014)
Chong, Y., Chen, W., Li, Z., Lam, W.H., Zheng, C., Li, Q.: Integrated real-time vision-based preceding vehicle detection in urban roads. Neurocomputing 116, 144–149 (2013)
Dai, J., Li, Y., He, K., Sun, J.: R-fcn: Object detection via region-based fully convolutional networks. In: Advances in Neural Information Processing Systems, pp. 379–387 (2016)
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y.: Deformable convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 764–773 (2017)
Ding, J., Xue, N., Long, Y., Xia, G.S., Lu, Q.: Learning roi transformer for detecting oriented objects in aerial images (2018). arXiv preprint arXiv:1812.00155
Ding, Q., Shang, J., Sun, Y., Wang, X., Liu, J.X.: Hc-hdsd: a method of hypergraph construction and high-density subgraph detection for inferring high-order epistatic interactions. Comput. Biol. Chem. 78, 440–447 (2019)
Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., Tian, Q.: Centernet: keypoint triplets for object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6569–6578 (2019)
Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)
Girshick, R.: Fast r-cnn. In: The IEEE International Conference on Computer Vision (ICCV) (2015)
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)
Howse, J.: OpenCV Computer Vision with Python. Packt Publishing Ltd, Birmingham (2013)
Huang, L., Yang, Y., Deng, Y., Yu, Y.: Densebox: Unifying landmark localization with end to end object detection (2015). arXiv preprint arXiv:1509.04874
Jiang, Y., Zhu, X., Wang, X., Yang, S., Li, W., Wang, H., Fu, P., Luo, Z.: R2cnn: rotational region cnn for orientation robust scene text detection (2017). arXiv preprint arXiv:1706.09579
Kong, T., Sun, F., Liu, H., Jiang, Y., Shi, J.: Foveabox: beyond anchor-based object detector (2019). arXiv preprint arXiv:1904.03797
Law, H., Deng, J.: Cornernet: detecting objects as paired keypoints. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 734–750 (2018)
Li, S., Shang, J., Chen, Q., Sun, Y., Liu, J.X.: A compressed sensing based multi-stage method for detecting epistatic interactions. Int. J. Data Mining Bioinform. 14(4), 354–372 (2016)
Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., Sun, J.: Light-head r-cnn: in defense of multi-stage object detector (2017). arXiv preprint arXiv:1711.07264
Liang, Y., Cai, Z., Yu, J., Han, Q., Li, Y.: Deep learning based inference of private information using embedded sensors in smart devices. IEEE Netw. 32(4), 8–14 (2018)
Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)
Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European Conference on Computer Vision. Springer, pp. 740–755 (2014)
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd: Single shot multibox detector. In: European Conference on Computer Vision. Springer, pp. 21–37 (2016)
Ma, J., Shao, W., Ye, H., Wang, L., Wang, H., Zheng, Y., Xue, X.: Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans. Multimedia 20(11), 3111–3122 (2018)
Ma, X., Zhang, F., Chen, X., Shen, J.: Privacy preserving multi-party computation delegation for deep learning in cloud computing. Inf. Sci. 459, 103–116 (2018)
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: integrated recognition, localization and detection using convolutional networks (2013). arXiv preprint arXiv:1312.6229
Shang, J., Sun, Y., Liu, J.X., Xia, J., Zhang, J., Zheng, C.H.: Cinoedv: a co-information based method for detecting and visualizing n-order epistatic interactions. BMC Bioinform. 17(1), 214 (2016)
Tian, Z., Shen, C., Chen, H., He, T.: Fcos: Fully convolutional one-stage object detection (2019). arXiv preprint arXiv:1904.01355
Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L.: Dota: a large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3974–3983 (2018)
Yang, X., Yang, J., Yan, J., Zhang, Y., Zhang, T., Guo, Z., Sun, X., Fu, K.: Scrdet: towards more robust detection for small, cluttered and rotated objects. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8232–8241 (2019)
Yang, Z., Liu, S., Hu, H., Wang, L., Lin, S.: Reppoints: Point set representation for object detection (2019). arXiv preprint arXiv:1904.11490
Zeng, Q., Martin, R.R., Wang, L., Quinn, J.A., Sun, Y., Tu, C.: Region-based bas-relief generation from a single image. Gr. Models 76(3), 140–151 (2014)
Zhang, S., Wen, L., Bian, X., Lei, Z., Li, S.Z.: Single-shot refinement neural network for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4203–4212 (2018)
Zhang, Z., Guo, W., Zhu, S., Yu, W.: Toward arbitrary-oriented ship detection with rotated region proposal and discrimination networks. IEEE Geosci. Remote Sens. Lett. 15(11), 1745–1749 (2018)
Zheng, Y., Xu, X., Qi, L.: Deep cnn-assisted personalized recommendation over big data for mobile wireless networks. Wirel. Commun. Mob. Comput. 2019 (2019)
Zhou, C., Liu, C.: Co-segmentation of multiple similar images using saliency detection and region merging. IET Comput. Vis. 8(3), 254–261 (2013)
Zhou, C., Liu, C.: An efficient segmentation method using saliency object detection. Multimedia Tools Appl. 74(15), 5623–5634 (2015)
Zhou, C., Wu, D., Qin, W., Liu, C.: An efficient multi-stage region merging method for interactive image segmentation. Comput. Electr. Eng. 54, 220–229 (2016)
Zhou, X., Wang, D., Krähenbühl, P.: Objects as points (2019). arXiv preprint arXiv:1904.07850
Zhou, X., Yao, C., Wen, H., Wang, Y., Zhou, S., He, W., Liang, J.: East: an efficient and accurate scene text detector. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 5551–5560 (2017)
Zhu, C., He, Y., Savvides, M.: Feature selective anchor-free module for single-shot object detection (2019). arXiv preprint arXiv:1903.00621
ZK, L., LB, W., YP, Y., et al.: A high resolution optical satellite image dataset for ship recognition and some new baselines (2017)
Acknowledgements
This work is supported in part by the National Natural Science Foundation of China under Grant nos. 61672254, 61802062, 61672246, 61572221, and 61300222, the Project of Department of Education of Guangdong Province of Grant number 2017KQNCX209, Key project of National Natural Science Foundation of China Grant no. U1536203, Natural Science Foundation of Hubei Province Grant no. 2015CFB687, the Fundamental Research Funds for the Central Universities, and HUST: 2016YXMS088 and 2016YXMS018. The authors appreciate the valuable suggestions from the anonymous reviewers and the Editors.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yang, Z., He, K., Zou, F. et al. ROPDet: real-time anchor-free detector based on point set representation for rotating object. J Real-Time Image Proc 17, 2127–2138 (2020). https://doi.org/10.1007/s11554-020-01013-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-020-01013-7