Abstract
In this paper, we propose a method for addressing the issue of unnoticed catastrophic deployment and domain shift in neural networks for semantic segmentation in autonomous driving. Our approach is based on the idea that deep learning-based perception for autonomous driving is uncertain and best represented as a probability distribution. As autonomous vehicles’ safety is paramount, it is crucial for perception systems to recognize when the vehicle is leaving its operational design domain, anticipate hazardous uncertainty, and reduce the performance of the perception system. To address this, we propose to encapsulate the neural network under deployment within an uncertainty estimation envelope that is based on the epistemic uncertainty estimation through the Monte Carlo Dropout approach. This approach does not require modification of the deployed neural network and guarantees expected model performance. Our defensive perception envelope has the capability to estimate a neural network’s performance, enabling monitoring and notification of entering domains of reduced neural network performance under deployment. Furthermore, our envelope is extended by novel methods to improve the application in deployment settings, including reducing compute expenses and confining estimation noise. Finally, we demonstrate the applicability of our method for multiple different potential deployment shifts relevant to autonomous driving, such as transitions into the night, rainy, or snowy domain. Overall, our approach shows great potential for application in deployment settings and enables operational design domain recognition via uncertainty, which allows for defensive perception, safe state triggers, warning notifications, and feedback for testing or development and adaptation of the perception stack.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Defensive Perception Repository (ours): https://osf.io/fjxw3/.
References
Banks, V.A., Plant, K.L., Stanton, N.A.: Driver error or designer error: using the perceptual cycle model to explore the circumstances surrounding the fatal tesla crash on 7th May 2016. Saf. Sci. 108, 278–285 (2018)
Behley, J., et al.: SemanticKITTI: a dataset for semantic scene understanding of LiDAR sequences. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2019)
Berman, B.: The key to autonomous vehicle safety is odd (2019). https://www.sae.org/news/2019/11/odds-for-av-testing
Blazek, P.J., Lin, M.M.: Explainable neural networks that simulate reasoning. Nat. Comput. Sci. 1(9), 607–618 (2021)
Chan, R., Rottmann, M., Gottschalk, H.: Entropy maximization and meta classification for out-of-distribution detection in semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5128–5137 (2021)
Charpentier, B., Zügner, D., Günnemann, S.: Posterior network: uncertainty estimation without OOD samples via density-based pseudo-counts. Adv. Neural. Inf. Process. Syst. 33, 1356–1367 (2020)
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. CoRR, abs/1802.02611 (2018). http://arxiv.org/abs/1802.02611
Du, X., Wang, Z., Cai, M., Li, Y.: VOS: learning what you don’t know by virtual outlier synthesis. arXiv preprint arXiv:2202.01197 (2022)
Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: appendix (2016)
Gawlikowski, J., et al.: A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342 (2021)
Hafner, F.M., Zeller, M., Schutera, M., Abhau, J., Kooij, J.F.P.: Backboneanalysis: structured insights into compute platforms from CNN inference latency. In: 2022 IEEE Intelligent Vehicles Symposium (IV), pp. 1801–1809 (2022). https://doi.org/10.1109/IV51971.2022.9827260
Harris, C.R., et al.: Array programming with NumPy. Nature 585(7825), 357–362 (2020). https://doi.org/10.1038/s41586-020-2649-2
Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580 (2012). http://arxiv.org/abs/1207.0580
Hüllermeier, E., Waegeman, W.: Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Mach. Learn. 110(3), 457–506 (2021)
Labach, A., Salehinejad, H., Valaee, S.: Survey of dropout methods for deep neural networks. arXiv e-prints, art. arXiv:1904.13310 (2019). https://doi.org/10.48550/arXiv.1904.13310
LeCun, Y., Cortes, C., Burges, C.J.C.: MNIST handwritten digit database. ATT Labs (2010). http://yann.lecun.com/exdb/mnist
National Highway Traffic Safety Administration. Summary report: Standing general order on crash reporting for automated driving systems (2022)
Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 427–436 (2015)
Rottmann, M., Schubert, M.: Uncertainty measures and prediction quality rating for the semantic segmentation of nested multi resolution street scene images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 1361–1369 (2019)
Rottmann, M., Colling, P., Hack, T.-P., Hüger, F., Schlicht, P., Gottschalk, H.: Prediction error meta classification in semantic segmentation: detection via aggregated dispersion measures of softmax probabilities. CoRR abs/1811.00648 (2018)
Schutera, M., Hafner, F.M., Vogt, H., et al.: Domain is of the essence: data deployment for city-scale multi-camera vehicle re-identification. In: IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 1–6 (2019). https://doi.org/10.1109/AVSS.2019.8909858. ISSN 2643-6205
Sensoy, M., Kandemir, M., Kaplan, L.M.: Evidential deep learning to quantify classification uncertainty. CoRR, abs/1806.01768 (2018). http://arxiv.org/abs/1806.01768
Spearman, C.: The proof and measurement of association between two things. Am. J. Psychol. 15(1), 72–101 (1904). http://www.jstor.org/stable/1412159. ISSN 00029556
Van Amersfoort, J., Smith, L., Teh, Y.W., Gal, Y.: Uncertainty estimation using a single deep deterministic neural network. In: Proceedings of Machine Learning Research, vol. 119, pp. 9690–9700 (2020). https://proceedings.mlr.press/v119/van-amersfoort20a.html
Xue, J.-R., Fang, J.-W., Zhang, P.: A survey of scene understanding by event reasoning in autonomous driving. Int. J. Autom. Comput. 15(3), 249–266 (2018)
Yu, F., et al.: BDD100K: a diverse driving dataset for heterogeneous multitask learning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Zhang, X., LeCun, Y.: Universum prescription: regularization using unlabeled data. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Acknowledgement
We want to thank our fellow researchers at Karlsruhe Institute of Technology and our colleagues at ZF Friedrichshafen AG - in particular, Dr. Jochen Abhau, and apl. Prof. Dr. Markus Reischl.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vogt, H., Buehler, S., Schutera, M. (2024). Defensive Perception: Estimation and Monitoring of Neural Network Performance Under Deployment. In: Cuzzolin, F., Sultana, M. (eds) Epistemic Uncertainty in Artificial Intelligence . Epi UAI 2023. Lecture Notes in Computer Science(), vol 14523. Springer, Cham. https://doi.org/10.1007/978-3-031-57963-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-57963-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-57962-2
Online ISBN: 978-3-031-57963-9
eBook Packages: Computer ScienceComputer Science (R0)