Abstract
Being able to understand the logic behind predictions or recommendations on the instance level is at the heart of trustworthy machine learning models. Inherently interpretable models make this possible by allowing inspection and analysis of the model itself, thus exhibiting the logic behind each prediction, while providing an opportunity to gain insights about the underlying domain. Another important criterion for trustworthiness is the model’s ability to somehow communicate a measure of confidence in every specific prediction or recommendation. Indeed, the overall goal of this paper is to produce highly informative models that combine interpretability and algorithmic confidence. For this purpose, we introduce conformal predictive distribution trees, which is a novel form of regression trees where each leaf contains a conformal predictive distribution. Using this representation language, the proposed approach allows very versatile analyses of individual leaves in the regression trees. Specifically, depending on the chosen level of detail, the leaves, in addition to the normal point predictions, can provide either cumulative distributions or prediction intervals that are guaranteed to be well-calibrated. In the empirical evaluation, the suggested conformal predictive distribution trees are compared to the well-established conformal regressors, thus demonstrating the benefits of the enhanced representation.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
The European Commission Independent High-Level Expert Group on Artificial Intelligence: Ethics Guidelines for Trustworthy AI (2019)
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD. KDD’16, pp. 1135–1144. ACM (2016)
Freitas, A.A.: A survey of evolutionary algorithms for data mining and knowledge discovery. In: Advances in Evolutionary Computation, Springer (2002)
Johansson, U., Löfström, T., Boström, H.: Calibrating probability estimation trees using venn-abers predictors. In: SIAM International Conference on Data Mining, SDM Calgary, Canada, pp. 28–36 (2019)
Provost, F., Domingos, P.: Tree induction for probability-based ranking. Mach. Learn. 52(3), 199–215 (2003)
Vovk, V., Petej, I.: Venn-abers predictors. arXiv:1211.0025 (2012)
Vovk, V., Petej, I., Nouretdinov, I., Manokhin, V., Gammerman, A.: Computationally efficient versions of conformal predictive distributions. Neurocomputing. 397, 292–308 (2020)
Johansson, U., Linusson, H., Löfström, T., Boström, H.: Interpretable regression trees using conformal prediction. Exp. Syst. Appl. 97, 394–404 (2018)
Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer-Verlag, New York Inc (2005)
Papadopoulos, H., Haralambous, H.: Neural networks regression inductive conformal predictor and its application to total electron content prediction. In: ICANN. LNCS, vol. 6352, pp. 32–41. Springer (2010)
Boström, H., Linusson, H., Löfström, T., Johansson, U.: Accelerating difficulty estimation for conformal regression forests. Ann. Math. Artif. Intell. 81(1–2), 125–144 (2017)
Vovk, V., Shen, J., Manokhin, V., Xie, M.: Nonparametric predictive distributions based on conformal prediction. Mach. Learn. 108(3), 445–474 (2019)
Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R.J., Wasserman, L.: Distributionfree predictive inference for regression. J. Am. Stat. Assoc. 113(523), 1094–1111 (2018)
Kivaranovic, D., Johnson, K.D., Leeb, H.: Adaptive, distribution-free prediction intervals for deep networks. In: International Conference on Artificial Intelligence and Statistics, pp. 4346–4356. PMLR (2020)
Ndiaye, E., Takeuchi, I.: Root-finding approaches for computing conformal prediction set. arXiv:2104.06648 (2021)
Sesia, M., Romano, Y.: Conformal histogram regression. arXiv:2105.08747 (2021)
Gupta, C., Kuchibhotla, A.K., Ramdas, A.K.: Nested conformal prediction and quantile out-of-bag ensemble methods. arXiv:1910.10562 (2019)
Wisniewski, W., Lindsay, D., Lindsay, S.: Application of conformal prediction interval estimations to market makers’ net positions. In: Conformal and Probabilistic Prediction and Applications, pp. 285–301. PMLR (2020)
Kath, C., Ziel, F.: Conformal prediction interval estimation and applications to day-ahead and intraday power markets. Int. J. Forecast. 37(2), 777–799 (2021)
Vovk, V., Shen, J., Manokhin, V., Xie, M.: Nonparametric predictive distributions based on conformal prediction. In: Conformal and Probabilistic Prediction and Applications, COPA, Stockholm, Sweden. Proceedings of Machine Learning Research, vol. 60, pp. 82–102. PMLR (2017)
Vovk, V., Nouretdinov, I., Manokhin, V., Gammerman, A.: Cross-conformal predictive distributions. In: Conformal and Probabilistic Prediction and Applications, COPA 2018, 11-13 June 2018, Maastricht, The Netherlands. Proceedings of Machine Learning Research, vol. 91, pp. 37–51. PMLR (2018)
Vovk, V., Bendtsen, C.: Conformal predictive decision making. In: Conformal and Probabilistic Prediction and Applications, COPA. Proceedings of Machine Learning Research, vol. 91, pp. 52–62. PMLR (2018)
Johansson, U., Boström, H., Löfström, T., Linusson, H.: Regression conformal prediction with random forests. Mach. Learn. 97(1–2), 155–176 (2014)
Werner, H., Carlsson, L., Ahlberg, E., Boström, H.: Evaluating different approaches to calibrating conformal predictive systems. In: Conformal and Probabilistic Prediction and Applications, COPA. Proceedings of Machine Learning Research, vol. 128, pp. 134–150. PMLR (2020)
Löfström, T., Zhao, J., Linnusson, H., Jansson, K.: Predicting adverse drug events with confidence. In: Thirteenth Scandinavian Conference on Artificial Intelligence. IOS Press (2015)
Boström, H., Johansson, U.: Mondrian conformal regressors. In: Conformal and Probabilistic Prediction and Applications. Proceedings of Machine Learning Research, vol. 128, pp. 114–133. PMLR (2020)
Boström, H., Johansson, U., Löfström, T.: Mondrian conformal predictive distributions. In: Conformal and Probabilistic Prediction and Applications, COPA. Proceedings of Machine Learning Research, vol. 152, pp. 24–38. PMLR (2021)
Flake, G.W., Lawrence, S.: Efficient svm regression training with smo. Mach. Learn. 46(1–3), 271–290 (2002)
Bache, K., Lichman, M.: UCI Machine Learning Repository (2013)
Rasmussen, C.E., Neal, R.M., Hinton, G., van Camp, D., Revow, M., Ghahramani, Z., Kustra, R., Tibshirani, R.: Delve data for evaluating learning in valid experiments. www.cs.toronto.edu/delve (1996)
Alcalá-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S.: Keel datamining software tool: Data set repository, integration of algorithms and experimental analysis framework. Multiple-Valued Logic Soft Comput. 17(2–3), 255–287 (2011)
Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of American Statistical Association 32, 675–701 (1937)
Bergmann, B., Hommel, G.: Improvements of general multiple test procedures for redundant systems of hypotheses. In: Multiple Hypotheses Testing, pp. 100–115. Springer (1988)
Funding
Open access funding provided by Jönköping University. The authors acknowledge the Swedish Knowledge Foundation, Jönköping University, and the industrial partners for financially supporting the research through the AFAIR project with grant number 20200223, as part of the research and education environment SPARK at Jönköping University, Sweden.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Johansson, U., Löfström, T. & Boström, H. Conformal Predictive Distribution Trees. Ann Math Artif Intell (2023). https://doi.org/10.1007/s10472-023-09847-0
Accepted:
Published:
DOI: https://doi.org/10.1007/s10472-023-09847-0