Skip to main content
SpringerLink
Log in

Object Detection: State of the Art and Beyond

  • Chapter
  • First Online:
Intelligent Scene Modeling and Human-Computer Interaction

Part of the book series: Human–Computer Interaction Series ((HCIS))

  • 760 Accesses

Abstract

As one of the fundamental problems of scene understanding and modeling, object detection has attracted extensive attention in the research communities of computer vision and artificial intelligence. Recently, inspired by the success of deep learning, various deep neural network-based models have been proposed and become the de facto solution for object detection. Therefore, in this chapter, we propose to present an overview of object detection techniques in the era of deep learning. We will first formulate the problem of object detection in the framework of deep learning, and then present two mainstream architectures, i.e., the one-stage model and the two-stage model, with the widely used detectors such as Fast R-CNN, YOLO, and their variants. Lastly, we will also discuss the potential and possible improvements on current methods and outline trends for further study.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://wordnet.princeton.edu/.

  2. 2.

    https://pytorch.org/.

  3. 3.

    https://www.tensorflow.org/.

  4. 4.

    https://github.com/facebookarchive/caffe2.

  5. 5.

    https://keras.io/.

  6. 6.

    https://mxnet.apache.org/.

  7. 7.

    https://github.com/facebookresearch/maskrcnn-benchmark/blob/master/MODEL_ZOO.md.

  8. 8.

    https://gluon-cv.mxnet.io/index.html.

References

  • Agarwal S, Terrail JOD, Jurie F (2018) Recent advances in object detection in the age of deep convolutional neural networks. arXiv:180903193

  • Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: optimal speed and accuracy of object detection. arXiv:200410934

  • Cai Z, Vasconcelos N (2018) Cascade r-cnn: delving into high quality object detection. In: Proceedings of the international conference on computer vision and pattern recognition, pp 6154–6162

    Google Scholar 

  • Chen K, Wang J, Pang J, Cao Y, Xiong Y, Li X, Sun S, Feng W, Liu Z, Xu J et al (2019a) Mmdetection: open mmlab detection toolbox and benchmark. arXiv:190607155

  • Chen Y, Han C, Li Y, Huang Z, Jiang Y, Wang N, Zhang Z (2019b) Simpledet: a simple and versatile distributed framework for object detection and instance recognition. arXiv:190305831

  • Chen Y, Yang T, Zhang X, Meng G, Pan C, Sun J (2019c) Detnas: neural architecture search on object detection. arXiv:190310979

  • Dai J, Li Y, He K, Sun J (2016) R-fcn: object detection via region-based fully convolutional networks. In: Advances in neural information processing systems, pp 379–387

    Google Scholar 

  • Dong Z, Li G, Liao Y, Wang F, Ren P, Qian C (2020) Centripetalnet: pursuing high-quality keypoint pairs for object detection. In: Proceedings of the international conference on computer vision and pattern recognition, pp 10519–10528

    Google Scholar 

  • Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q (2019) Centernet: keypoint triplets for object detection. arXiv:190408189

  • Duan K, Xie L, Qi H, Bai S, Huang Q, Tian Q (2020) Corner proposal network for anchor-free, two-stage object detection. In: Proceedings of the European conference on computer vision, pp 1–17

    Google Scholar 

  • Everingham M, Eslami SA, Van Gool L, Williams CK, Winn J, Zisserman A (2015) The pascal visual object classes challenge: a retrospective. Int J Comput Vis 111(1):98–136

    Article  Google Scholar 

  • Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2009) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645

    Article  Google Scholar 

  • Fu CY, Liu W, Ranga A, Tyagi A, Berg aC (2017) Dssd: deconvolutional single shot detector. arXiv:170106659

  • Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the kitti dataset. Int J Robot Res 32(11):1231–1237

    Article  Google Scholar 

  • Ghiasi G, Lin TY, Le QV (2019) Nas-fpn: Learning scalable feature pyramid architecture for object detection. In: Proceedings of the international conference on computer vision and pattern recognition, pp 7036–7045

    Google Scholar 

  • Girshick R (2015) Fast r-cnn. In: Proceedings of the international conference on computer vision, pp 1440–1448

    Google Scholar 

  • Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the international conference on computer vision and pattern recognition, pp 580–587

    Google Scholar 

  • Girshick R, Radosavovic I, Gkioxari G, Dollár P, He K (2018) Detectron. https://github.com/facebookresearch/detectron

  • Han J, Zhang D, Cheng G, Liu N, Xu D (2018) Advanced deep-learning techniques for salient and category-specific object detection: a survey. IEEE Signal Process Mag 35(1):84–100

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the international conference on computer vision and pattern recognition, pp 770–778

    Google Scholar 

  • He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

    Google Scholar 

  • Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:170404861

  • Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Fischer I, Wojna Z, Song Y, Guadarrama S, et al (2017) Speed/accuracy trade-offs for modern convolutional object detectors. In: Proceedings of the international conference on computer vision and pattern recognition, pp 7310–7311

    Google Scholar 

  • Huang L, Yang Y, Deng Y, Yu Y (2015) Densebox: unifying landmark localization with end to end object detection. arXiv:150904874

  • Huang Z, Huang L, Gong Y, Huang C, Wang X (2019) Mask scoring r-cnn. In: Proceedings of the international conference on computer vision and pattern recognition, pp 6409–6418

    Google Scholar 

  • Jeong J, Park H, Kwak N (2017) Enhancement of ssd by concatenating feature maps for object detection. arXiv:170509587

  • Jiao L, Zhang F, Liu F, Yang S, Li L, Feng Z, Qu R (2019) A survey of deep learning-based object detection. arXiv:190709408

  • Kirillov A, He K, Girshick R, Rother C, Dollár P (2019) Panoptic segmentation. In: Proceedings of the international conference on computer vision and pattern recognition, pp 9404–9413

    Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  • Kuznetsova A, Rom H, Alldrin N, Uijlings J, Krasin I, Pont-Tuset J, Kamali S, Popov S, Malloci M, Duerig T, et al (2018) The open images dataset v4: unified image classification, object detection, and visual relationship detection at scale. arXiv:181100982

  • Lan S, Ren Z, Wu Y, Davis LS, Hua G (2020) Saccadenet: a fast and accurate object detector. In: Proceedings of the international conference on computer vision and pattern recognition, pp 10397–10406

    Google Scholar 

  • Law H, Deng J (2018) Cornernet: detecting objects as paired keypoints. In: Proceedings of the European conference on computer vision, pp 734–750

    Google Scholar 

  • Law H, Teng Y, Russakovsky O, Deng J (2019) Cornernet-lite: efficient keypoint based object detection. arXiv:190408900

  • Li Y, Chen Y, Wang N, Zhang Z (2019) Scale-aware trident networks for object detection. arXiv:190101892

  • Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: Proceedings of the European conference on computer vision. Springer, pp 740–755

    Google Scholar 

  • Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the international conference on computer vision and pattern recognition, pp 2117–2125

    Google Scholar 

  • Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2018) Deep learning for generic object detection: a survey. arXiv:180902165

  • Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: single shot multibox detector. In: Proceedings of the European conference on computer vision. Springer, pp 21–37

    Google Scholar 

  • Ma N, Zhang X, Zheng HT, Sun J (2018) Shufflenet v2: practical guidelines for efficient cnn architecture design. In: Proceedings of the European conference on computer vision, pp 116–131

    Google Scholar 

  • Massa F, Girshick R (2018) Maskrcnn-benchmark: fast, modular reference implementation of instance segmentation and object detection algorithms in PyTorch. https://github.com/facebookresearch/maskrcnn-benchmark

  • Najibi M, Rastegari M, Davis LS (2016) G-cnn: an iterative grid based object detector. In: Proceedings of the international conference on computer vision and pattern recognition, pp 2369–2377

    Google Scholar 

  • Pang Y, Wang T, Anwer RM, Khan FS, Shao L (2019) Efficient featurized image pyramid network for single shot detector. In: Proceedings of the international conference on computer vision and pattern recognition, pp 7336–7344

    Google Scholar 

  • Pinheiro PO, Lin TY, Collobert R, Dollár P (2016) Learning to refine object segments. In: Proceedings of the European conference on computer vision. Springer, pp 75–91

    Google Scholar 

  • Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the international conference on computer vision and pattern recognition, pp 7263–7271

    Google Scholar 

  • Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv:180402767

  • Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the international conference on computer vision and pattern recognition, pp 779–788

    Google Scholar 

  • Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

    Google Scholar 

  • Rota Bulò S, Porzi L, Kontschieder P (2018) In-place activated batchnorm for memory-optimized training of dnns. In: Proceedings of the international conference on computer vision and pattern recognition, pp 5639–5647

    Google Scholar 

  • Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  • Singh B, Davis LS (2018) An analysis of scale invariance in object detection snip. In: Proceedings of the international conference on computer vision and pattern recognition, pp 3578–3587

    Google Scholar 

  • Singh B, Najibi M, Davis LS (2018) Sniper: efficient multi-scale training. In: Advances in neural information processing systems, pp 9310–9320

    Google Scholar 

  • Tian Z, Shen C, Chen H, He T (2019) Fcos: Fully convolutional one-stage object detection. arXiv:190401355

  • Tremblay J, To T, Birchfield S (2018) Falling things: a synthetic dataset for 3d object detection and pose estimation. In: International conference on computer vision and pattern recognition (workshop), pp 2038–2041

    Google Scholar 

  • Uijlings JR, Van De Sande KE, Gevers T, Smeulders AW (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171

    Article  Google Scholar 

  • Wang H, Wang Q, Gao M, Li P, Zuo W (2018) Multi-scale location-aware kernel representation for object detection. In: Proceedings of the international conference on computer vision and pattern recognition, pp 1248–1257

    Google Scholar 

  • Wang H, Kembhavi A, Farhadi A, Yuille AL, Rastegari M (2019a) Elastic: improving cnns with dynamic scaling policies. In: Proceedings of the international conference on computer vision and pattern recognition, pp 2258–2267

    Google Scholar 

  • Wang J, Chen K, Yang S, Loy CC, Lin D (2019b) Region proposal by guided anchoring. In: Proceedings of the international conference on computer vision and pattern recognition, pp 2965–2974

    Google Scholar 

  • Wu X, Sahoo D, Hoi SC (2019) Recent advances in deep learning for object detection. arXiv:190803673

  • Xia GS, Bai X, Ding J, Zhu Z, Belongie S, Luo J, Datcu M, Pelillo M, Zhang L (2018) Dota: a large-scale dataset for object detection in aerial images. In: Proceedings of the international conference on computer vision and pattern recognition, pp 3974–3983

    Google Scholar 

  • Yang S, Luo P, Loy CC, Tang X (2016) Wider face: a face detection benchmark. In: Proceedings of the international conference on computer vision and pattern recognition, pp 5525–5533

    Google Scholar 

  • Yang Z, Liu S, Hu H, Wang L, Lin S (2019) Reppoints: point set representation for object detection. arXiv:190411490

  • Yu F, Koltun V, Funkhouser T (2017) Dilated residual networks. In: Proceedings of the international conference on computer vision and pattern recognition, pp 472–480

    Google Scholar 

  • Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of the European conference on computer vision. Springer, pp 818–833

    Google Scholar 

  • Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the international conference on computer vision and pattern recognition, pp 6848–6856

    Google Scholar 

  • Zhao Q, Sheng T, Wang Y, Tang Z, Chen Y, Cai L, Ling H (2019a) M2det: a single-shot object detector based on multi-level feature pyramid network. Proc AAAI Conf Artif Intell 33:9259–9266

    Google Scholar 

  • Zhao ZQ, Zheng P, Xu St, Wu X (2019b) Object detection with deep learning: a review. IEEE Trans Neural Netw Learn Syst

    Google Scholar 

  • Zhou P, Ni B, Geng C, Hu J, Xu Y (2018) Scale-transferrable object detection. In: Proceedings of the international conference on computer vision and pattern recognition, pp 528–537

    Google Scholar 

  • Zhou X, Wang D, Krähenbühl P (2019a) Objects as points. arXiv:190407850

  • Zhou X, Zhuo J, Krahenbuhl P (2019b) Bottom-up object detection by grouping extreme and center points. In: Proceedings of the international conference on computer vision and pattern recognition, pp 850–859

    Google Scholar 

  • Zhu C, He Y, Savvides M (2019) Feature selective anchor-free module for single-shot object detection. arXiv:190300621

  • Zhu Y, Zhao C, Wang J, Zhao X, Wu Y, Lu H (2017) Couplenet: coupling global structure with local parts for object detection. In: Proceedings of the international conference on computer vision, pp 4126–4134

    Google Scholar 

  • Zou Z, Shi Z, Guo Y, Ye J (2019) Object detection in 20 years: a survey. arXiv:190505055

Download references

Author information

Authors and Affiliations

  1. Nanyang Technological University, Singapore, Singapore

    Hanhui Li, Xudong Jiang & Nadia Magnenat Thalmann

Authors
  1. Hanhui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xudong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nadia Magnenat Thalmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanhui Li .

Editor information

Editors and Affiliations

  1. Institute for Media Innovation, Nanyang Technological University, Singapore, Singapore

    Nadia Magnenat Thalmann

  2. Bournemouth University, Poole, UK

    Jian Jun Zhang

  3. Institute for Media Innovation, Nanyang Technological University, Singapore, Singapore

    Manoj Ramanathan

  4. School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausa, Lausanne, Switzerland

    Daniel Thalmann

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Li, H., Jiang, X., Magnenat Thalmann, N. (2021). Object Detection: State of the Art and Beyond. In: Thalmann, N.M., Zhang, J.J., Ramanathan, M., Thalmann, D. (eds) Intelligent Scene Modeling and Human-Computer Interaction. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-030-71002-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-71002-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-71001-9

  • Online ISBN: 978-3-030-71002-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation