Skip to main content
Book cover

International Conference on Pattern Recognition

ICPR 2021: Pattern Recognition. ICPR International Workshops and Challenges pp 361–374Cite as

Towards Corner Case Detection by Modeling the Uncertainty of Instance Segmentation Networks

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12664)

Abstract

State-of-the-art instance segmentation techniques currently provide a bounding box, class, mask, and scores for each instance. What they do not provide is an epistemic uncertainty estimate of these predictions. With our approach, we want to identify corner cases by considering the epistemic uncertainty. Corner cases are data/situations that are underrepresented or not covered in our data set. Our work is based on Mask R-CNN. We estimate the epistemic uncertainty by extending the architecture with Monte-Carlo dropout layers. By repeatedly executing the forward pass, we create a large number of predictions per instance. Afterward, we cluster the predictions of an instance based on the bounding box coordinates. It becomes possible to determine the epistemic position uncertainty for the bounding boxes and the classifier’s epistemic class uncertainty. For the epistemic uncertainty regarding the bounding box position and the class assignment, we provide a criterion for detecting corner cases utilizing the model’s epistemic uncertainty.

Keywords

  • Epistemic uncertainty estimation
  • Corner case detection
  • Object detection
  • Machine learning
  • Automated learning

F. Heidecker and A. Hannan—Contributed equally to this work.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-68799-1_26
  • Chapter length: 14 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-68799-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   149.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.

References

  1. Audi AG: Driving Dataset (2019). https://www.a2d2.audi/a2d2/en.html. Accessed 12 Apr 2020

  2. Blei, D., Jordan, M.: Variational inference for dirichlet process mixtures. J. Bayesian Anal. 1, 121–144 (2006)

    MathSciNet  CrossRef  Google Scholar 

  3. Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight Uncertainty in Neural Network. In: Proceedings of the 32nd ICML, vol. 37, pp. 1613–1622. PMLR, Lille, France (2015)

    Google Scholar 

  4. Choi, S., Lee, K., Lim, S., Oh, S.: Uncertainty-aware learning from demonstration using mixture density networks with sampling-free variance modeling. In: 2018 IEEE ICRA, pp. 6915–6922. IEEE, Brisbane, QLD, Australia (2018)

    Google Scholar 

  5. DeVries, T., Taylor, G.W.: Learning Confidence for Out-of-Distribution Detection in Neural Networks arxiv:1802.04865v1 (2018)

  6. Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45014-9_1

    CrossRef  Google Scholar 

  7. Gal, Y.: Uncertainty in Deep Learning. Ph.D. thesis, University of Cambridge (2016)

    Google Scholar 

  8. Gal, Y., Ghahramani, Z.: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In: Proceedings of The 33rd ICML, vol. 48, pp. 1050–1059. JMLR.org, New York (2016)

    Google Scholar 

  9. He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: 2017 IEEE ICCV, pp. 2980–2988. IEEE, Venice, Italy (2017)

    Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: 2016 IEEE CVPR, pp. 770–778. IEEE, Las Vegas, NV, USA (2016)

    Google Scholar 

  11. Hendrycks, D., Gimpel, K.: A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks arxiv:1610.02136v3 (2017)

  12. Hüllermeier, E., Waegeman, W.: Aleatoric and Epistemic Uncertainty in Machine Learning: A Tutorial Introduction (2019)

    Google Scholar 

  13. Ilg, E., Cicek, O., Galesso, S., Klein, A., Makansi, O., Hutter, F., Brox, T.: Uncertainty estimates and multi-hypotheses networks for optical flow. In: ECCV, pp. 652–667. Munich, Germany (2018)

    Google Scholar 

  14. Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles. In: Proceedings of the 31st NIPS, pp. 6405–6416. Curran Associates Inc, Red Hook, NY, USA (2017)

    Google Scholar 

  15. Liang, S., Li, Y., Srikant, R.: Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks arxiv:1706.02690v4 (2018)

  16. Liu, J.Z., Paisley, J., Kioumourtzoglou, M.A., Coull, B.: Accurate uncertainty estimation and decomposition in ensemble learning. In: Proceedings of the 33rd NIPS, pp. 8952–8963. Curran Associates Inc, Vancouver, Canada (2019)

    Google Scholar 

  17. Malinin, A., Gales, M.: Predictive Uncertainty Estimation via Prior Networks. In: Proceedings of the 32nd NIPS, pp. 7047–7058. Curran Associates Inc, Red Hook, NY, USA (2018)

    Google Scholar 

  18. Pinheiro, P.O., Collobert, R., Dollár, P.: Learning to Segment Object Candidates. In: Proceedings of the 28th NIPS, vol. 2, pp. 1990–1998. MIT Press, Cambridge, MA, USA (2015)

    Google Scholar 

  19. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You Only Look Once: Unified, Real-Time Object Detection. In: 2016 IEEE CVPR, pp. 779–788. IEEE, Las Vegas, NV, USA (2016)

    Google Scholar 

  20. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE TPAMI 39(6), 1137–1149 (2017)

    CrossRef  Google Scholar 

  21. Shalev, G., Adi, Y., Keshet, J.: Out-of-Distribution Detection using Multiple Semantic Label Representations. In: Proceedings of the 32nd NIPS, pp. 7375–7385. Curran Associates Inc. (2018)

    Google Scholar 

  22. Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE TPAMI 39(4), 640–651 (2017)

    CrossRef  Google Scholar 

  23. Sieman, R.: Strange Off-Road Dirt Bikes & Vehicles (2013), https://www.off-road.com/dirtbike/feature/strange-offroad-dirt-bikes-vehicles-53605.html?printable Accessed 12 Apr 2020

  24. Uijlings, J.R.R., van de Sande, K.E.A., Gevers, T., Smeulders, A.W.M.: Selective search for object recognition. IJCV 104(2), 154–171 (2013)

    CrossRef  Google Scholar 

  25. Yuan, Y., Chen, X., Wang, J.: Object-Contextual Representations for Semantic Segmentation arxiv:1909.11065v2 (2019)

Download references

Acknowledgment

This work results from the project KI Data Tooling (19A20001O) funded by BMWI (Deutsches Bundesministerium für Wirtschaft und Energie/German Federal Ministry for Economic Affairs and Energy) and the project DeCoInt\(^2\) supported by the German Research Foundation (DFG) within the priority program SPP 1835: “Kooperativ interagierende Automobile”, grant number SI 674/11-2.

Author information

Authors and Affiliations

  1. Universität Kassel, Wilhelmshöher Allee 73, 34121, Kassel, Germany

    Florian Heidecker, Abdul Hannan, Maarten Bieshaar & Bernhard Sick

Authors
  1. Florian Heidecker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdul Hannan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maarten Bieshaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bernhard Sick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Florian Heidecker .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell'Informazione, University of Firenze, Florence, Firenze, Italy

    Prof. Alberto Del Bimbo

  2. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Prof. Rita Cucchiara

  3. Department of Computer Science, Boston University, Boston, MA, USA

    Prof. Stan Sclaroff

  4. Dipartimento di Matematica e Informatica, University of Catania, Catania, Catania, Italy

    Dr. Giovanni Maria Farinella

  5. Cloud & AI, JD.COM, Beijing, China

    Tao Mei

  6. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Prof. Dr. Marco Bertini

  7. Computational Sciences Department, National Institute of Astrophysics, Optics and Electronics (INAOE), Tonantzintla, Puebla, Mexico

    Hugo Jair Escalante

  8. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Dr. Roberto Vezzani

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Heidecker, F., Hannan, A., Bieshaar, M., Sick, B. (2021). Towards Corner Case Detection by Modeling the Uncertainty of Instance Segmentation Networks. In: , et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12664. Springer, Cham. https://doi.org/10.1007/978-3-030-68799-1_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68799-1_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68798-4

  • Online ISBN: 978-3-030-68799-1

  • eBook Packages: Computer ScienceComputer Science (R0)