Abstract
Convolutional Neural Network (CNN) based object detection has achieved remarkable progress. However, most existing methods work on closed set assumption and can detect only objects of known classes. In real-world scenes, an image may contain unknown-class foreground objects that are unseen in training set but of potential interest, and open set object detection aims at detecting them as foreground, rather than rejecting them as background. A few methods have been proposed for this task, but they suffer from either low speed or unsatisfactory ability of unknown identification. In this paper, we propose a one-stage open set object detection method based on prototype learning. Benefiting from the compact distributions of known classes yielded by prototype learning, our method shows superior performance on identifying objects of both known and unknown classes from images in the open set scenario. It also inherits all advantages of YOLO v3 such as the high inference speed and the ability of multi-scale detection. To evaluate the performance of our method, we conduct experiments with both closed & open set settings, and especially assess the performance of unknown identification using recall and precision of the unknown class. The experimental results show that our method identifies unknown objects better while keeping the accuracy on known classes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dhamija, A.R., Gunther, M., Ventura, J., Boult, T.E.: The overlooked elephant of object detection: open set. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1021–1030 (2020)
Everingham, M., Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010). https://doi.org/10.1007/s11263-009-0275-4
Geng, C., Huang, S.J., Chen, S.: Recent advances in open set recognition: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3614–3631 (2021)
Geva, S., Sitte, J.: Adaptive nearest neighbor pattern classification. IEEE Trans. Neural Netw. 2(2), 318–322 (1991)
Girshick, R.: Fast r-cnn. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1440–1448 (2015)
Joseph, K.J., Khan, S., Khan, F.S., Balasubramanian, V.N.: Towards open world object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2017)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. 42(2), 318–327 (2020)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, C.L., Eim, I.J., Kim, J.: High accuracy handwritten Chinese character recognition by improved feature matching method. In: Proceedings of the Fourth International Conference on Document Analysis and Recognition, vol. 2, pp. 1033–1037 (1997)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Miller, D., Nicholson, L., Dayoub, F., Sunderhauf, N.: Dropout sampling for robust object detection in open-set conditions. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 3243–3249 (2018)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788 (2016)
Redmon, J., Farhadi, A.: Yolov3: An incremental improvement, Computer Vision and Pattern Recognition arXiv preprint arXiv:1804.02767 (2018)
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)
Snell, J., Swersky, K., Zemel, R.S.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, vol. 30, pp. 4077–4087 (2017)
Yang, H.M., Zhang, X.Y., Yin, F., Liu, C.L.: Robust classification with convolutional prototype learning. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3474–3482 (2018)
Yang, H.M., Zhang, X.Y., Yin, F., Yang, Q., Liu, C.L.: Convolutional prototype network for open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 1 (2020, early access)
Zou, Z., Shi, Z., Guo, Y., Ye, J.: Object detection in 20 years: A survey. arXiv preprint arXiv:1905.05055 (2019)
Acknowledgments
This work has been supported by the National Key Research and Development Program Grant No. 2018AAA0100400, the National Natural Science Foundation of China (NSFC) grant 61721004.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xiong, Y., Yang, P., Liu, CL. (2021). One-Stage Open Set Object Detection with Prototype Learning. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13108. Springer, Cham. https://doi.org/10.1007/978-3-030-92185-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-92185-9_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92184-2
Online ISBN: 978-3-030-92185-9
eBook Packages: Computer ScienceComputer Science (R0)