Abstract
Object detection using deep learning has recently gained significant attention due to its impressive results in a variety of applications, such as autonomous vehicles, surveillance, and image and video analysis. State-of-the-art models, such as YOLO, Faster-RCNN, and SSD, have achieved impressive performance on various benchmarks. However, it is crucial to ensure that the results produced by deep learning models are trustworthy, as they can have serious consequences, especially in an industrial context. In this paper, we introduce a novel confidence metric for object detection using neighborhood sampling. We evaluate our approach on MS-COCO and demonstrate that it significantly improves the trustworthiness of deep learning models for object detection. We also compare our approach against attribution-guided neighborhood sampling and show that such a heuristic does not yield better results.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
All data used in this study is publicly available. Refer to pertaining citations for access and download.
References
Chen, T., Navratil, J., Iyengar, V., Shanmugam, K.: Confidence scoring using whitebox meta-models with linear classifier probes. In: Chaudhuri, K., Sugiyama, M. (eds) Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, 89, 1467—1475 (2019)
Corbière, C., Thome, N., Bar-Hen, A., Cord, M., Pérez, P.: Addressing failure prediction by learning model confidence (2019). arXiv preprint arXiv:1910.04851
Corbière, C., Thome, N., Saporta, A., Vu, T.-H., Cord, M., Pérez, P.: Confidence estimation via auxiliary models (2020). arXiv preprint arXiv:2012.06508
Delseny, H., Gabreau, C., Gauffriau, A., Beaudouin, B., Ponsolle, L., Alecu, L., Bonnin, H., Beltran, B., Duchel, D., Ginestet, J.-B.: White paper machine learning in certified systems (2021). arXiv preprint arXiv:2103.10529
Denker, J.S., Lecun, Y.: Transforming neural-net output levels to probability distributions. Adv. Neural Inform. Process. Syst. 853—859 (1991)
Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In: International Conference on Machine Learning, 1050—1059 (2016)
Jha, S., Raj, S., Fernandes, S., Jha, S.K., Jha, S., Jalaian, B., Verma, G., Swami, A.: Attribution-based confidence metric for deep neural networks. In: NeurIPS Proceedings (2019)
Jiang, H., Kim, B., Guan, M., Gupta, M.: To trust or not to trust a classifier. Adv. Neural Inform. Process. Syst., 5541—5552 (2018)
Lam, D., Kuzma, R., McGee, K., Dooley, S., Laielli, M., Klaric, M., Bulatov, Y., McCord, B.: xview: Objects in context in overhead imagery (2018). arXiv preprint arXiv:1802.07856
Lee, K., Lee, H., Lee, K., Shin, J.: Training confidence-calibrated classifiers for detecting out-of-distribution samples. 96, 1–2 (2017). ArXiv preprint arXiv:1711.09325
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C.: Ssd: Single shot multibox detector. In European conference on computer vision, 21–37. Springer (2016)
Neumann, L., Zisserman, A., Vedaldi, A.: Relaxed softmax: efficient confidence auto-calibration for safe pedestrian detection (2018)
Papernot, N., McDaniel, P.: Deep k-nearest neighbors: towards confident, interpretable and robust deep learning (2018). arXiv preprint arXiv:1803.04765
Pérez, P., Gangnet, M.A.B.: Poisson image editing. ACM Trans. Graph. (SIGGRAPH’03) 22, 313–318 (2003)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 779–788 (2016)
Ren, K., Zheng, T., Qin, Z., Liud, X.: Adversarial attacks and defenses in deep learning. Engineering 6(3), 346–360 (2020)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In: Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28. Curran Associates Inc, New York (2015)
Rosenfeld, A., Thurston, M.: Edge and curve detection for visual scene analysis. IEEE Trans. Comput. 100(5), 562–569 (1971)
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks (2013). arXiv preprint arXiv:1312.6199
Wan, S., Wu, T.-Y., Wong, W. H., Lee, C.-Y.: Confnet: predict with confidence. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2921—2925 (2018)
Wang, X., Luo, Y., Crankshaw, D., Tumanov, A., Yu, F., Gonzalez, J. E.: Idk cascades: fast deep learning by learning not to overthink (2017). arXiv preprint arXiv:1706.00885
Acknowledgements
This work has been supported by the French government under the “France 2030” program, as part of the SystemX Technological Research Institute.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
Confidence score with (no asterisk) and without (asterisk) the use of attribution-based sampling. Transformations are used to force the model to produce erroneous predictions. The confidence score (CS) for both correct and incorrect prediction is calculated. We see that the mean CS is very similar across datasets, and for different transformations, whether we use attribution-based sampling or not
Neighborhood sampling has been successfully used to compute confidence in object classification in [7], but with the caveat that for images the neighborhood is high-dimensional, leading to a computational challenge that is solved by lowering the dimensions around high-attribution features.
We implemented the code for [7] and reproduced the good results for the following datasets: MNIST, FashionMNIST, and Cifar10. But we also tested these results when the high-attribution computation part of the code is removed.
The method is to perform predictions on the validation data of the aforementioned datasets, as well as on transformed data. The goal of the transformation is to force the model to produce invalid predictions. The transformations are (a) rotation (parameterized by the angle of rotation), and (b) alpha-blending with a random image in the dataset (parameterized with the percentage of blending).
The results in Fig. 6 show the average confidence score for all predictions in the validation dataset for the various transformations. We show the confidence scores computed with and without focusing on high-attribution features only. Most importantly, these results are computed using the exact same number of samples, which means that the computation without using attributions is actually faster, because it does not include the cost of computing the attributions for the input.
We conclude from this experiments that the use of attributions does not increase in practice the performance of confidence score computation using neighborhood sampling, and it is reasonably safe to remove this additional computation when performing neighborhood sampling.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gouguenheim, C., Berjaoui, A. Neighborhood sampling confidence metric for object detection. AI Ethics 4, 57–64 (2024). https://doi.org/10.1007/s43681-023-00395-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s43681-023-00395-1