A Metric to Improve the Robustness of Conformal Predictors in the Presence of Error Bars

Abstract

Conformal predictors, currently applied to many problems in various fields determine precise levels of confidence in new predictions on the basis only of the information present in the past data, without making recourse to any assumptions except that the examples are generated independently from the same probability distribution. In this paper, the robustness of their results is assessed for the cases in which the data are affected by error bars. This is the situation typical of the physical sciences, whose data are often the results of complex measurement procedures, unavoidably affected by noise. Assuming the noise presents a normal distribution, the Geodesic Distance on Gaussian Manifolds provides a statistical principled and quite effective method to handle the uncertainty in the data. A series of numerical tests prove that adopting this metric in conformal predictors improves significantly their performance, compared to the Euclidean distance, even for relatively low levels of noise.