Skip to main content
SpringerLink
Log in

Cross-conformal predictors

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

Inductive conformal predictors have been designed to overcome the computational inefficiency exhibited by conformal predictors for many underlying prediction algorithms. Whereas computationally efficient, inductive conformal predictors sacrifice different parts of the training set at different stages of prediction, which affects their informational efficiency. This paper introduces the method of cross-conformal prediction, which is a hybrid of the methods of inductive conformal prediction and cross-validation, and studies its validity and informational efficiency empirically. The computational efficiency of cross-conformal predictors is comparable to that of inductive conformal predictors, and they produce valid predictions in our empirical studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Balasubramanian, V.N.: Conformal predictions in multimedia pattern recognition. Ph.D. thesis, Arizona State University (2010)

  2. Balasubramanian, V.N., Chakraborty, S., Panchanathan, S.: Conformal predictions for information fusion: a comparative study of p-value combination methods. Ann. Math. Artif. Intell. (2013, this issue)

  3. Breiman, L., Spector, P.: Submodel selection and evaluation in regression: the X-random case. Int. Stat. Rev. 60, 291–319 (1992)

    Article  Google Scholar 

  4. Efron, B.: Bootstrap methods: another look at the jackknife. Ann. Stat. 7, 1–26 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  5. Efron, B.: Estimating the error rate of a prediction rule: some improvements on cross-validation. J. Am. Stat. Assoc. 78, 316–331 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  6. Fisher, R.A.: Combining independent tests of significance. Am. Stat. 2, 30 (1948)

    Google Scholar 

  7. Fraser, D.A.S.: Nonparametric Methods in Statistics. Wiley, New York (1957)

    MATH  Google Scholar 

  8. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  9. Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Data Anal. 29, 1189–1232 (2001)

    MATH  Google Scholar 

  10. Friedman, J.H.: Stochastic gradient boosting. Comput. Stat. Data Anal. 38, 367–378 (2002)

    Article  MATH  Google Scholar 

  11. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, New York (2009)

    Book  Google Scholar 

  12. Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 1137–1143. Morgan Kaufmann, San Mateo, CA (1995)

  13. Lei, J., Rinaldo, A., Wasserman, L.: A conformal prediction approach to explore functional data. Ann. Math. Artif. Intell. (2013). doi:10.1007/s10472-013-9366-6

    Google Scholar 

  14. Lei, J., Wasserman, L.: Distribution free prediction bands. Tech. Rep. arXiv: 1203.5422 [stat.ME]. e-Print archive (2012, to appear in JRRSB)

  15. Mosteller, F., Tukey, J.W.: Data analysis, including statistics. In: Lindzey, G., Aronson, E. (eds.) Handbook of Social Psychology, 2nd edn., vol. 2, pp. 80–203. Addison-Wesley, Reading, MA (1968)

    Google Scholar 

  16. Papadopoulos, H., Vovk, V., Gammerman, A.: Qualified predictions for large data sets in the case of pattern recognition. In: Proceedings of the First International Conference on Machine Learning and Applications (ICMLA), pp. 159–163. CSREA Press, Las Vegas, NV (2002)

    Google Scholar 

  17. Simard, P., LeCun, Y., Denker, J.: Efficient pattern recognition using a new transformation distance. In: Hanson, S., Cowan, J., Giles, C. (eds.) Advances in Neural Information Processing Systems, vol. 5, pp. 50–58. Morgan Kaufmann, San Mateo, CA (1993)

    Google Scholar 

  18. Stone, M.: Cross-validatory choice and assessment of statistical predictions (with discussion). J. R. Stat. Soc. Ser. B 36, 111–147 (1974)

    MATH  Google Scholar 

  19. Vanderlooy, S., van der Maaten, L., Sprinkhuizen-Kuyper, I.: Off-line learning with transductive confidence machines: an empirical evaluation. In: Perner, P. (ed.) Proceedings of the Fifth International Conference on Machine Learning and Data Mining in Pattern Recognition. Lecture Notes in Artificial Intelligence, vol. 4571, pp. 310–323. Springer, Berlin (2007)

    Chapter  Google Scholar 

  20. Vovk, V.: Conditional validity of inductive conformal predictors. In: Hoi, S.C.H., Buntine, W. (eds.) JMLR: Workshop and Conference Proceedings, vol. 25 (Asian Conference on Machine Learning), pp. 475–490 (2012). The journal version to appear in Machine Learning (ACML 2012 Special Issue)

  21. Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer, New York (2005)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Learning Research Centre, Department of Computer Science, Royal Holloway, University of London, Egham, Surrey, UK

    Vladimir Vovk

Authors
  1. Vladimir Vovk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Vovk.

Additional information

An extended abstract of an early version of this paper was published in the Proceedings of the Fifth Workshop on Information Theoretic Methods in Science and Engineering (WITMSE 2012, Amsterdam, August 2012, http://event.cwi.nl/witmse2012/proc.pdf). This work was partially supported by the Cyprus Research Promotion Foundation.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(ZIP 65.6 kb)

(ZIP 36.6 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vovk, V. Cross-conformal predictors. Ann Math Artif Intell 74, 9–28 (2015). https://doi.org/10.1007/s10472-013-9368-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-013-9368-4

Keywords

Mathematics Subject Classifications (2010)

Search

Navigation