Abstract
The recently introduced transductive confidence machines (TCMs) framework allows to extend classifiers such that they satisfy the calibration property. This means that the error rate can be set by the user prior to classification. An analytical proof of the calibration property was given for TCMs applied in the on-line learning setting. However, the nature of this learning setting restricts the applicability of TCMs. In this paper we provide strong empirical evidence that the calibration property also holds in the off-line learning setting. Our results extend the range of applications in which TCMs can be applied. We may conclude that TCMs are appropriate in virtually any application domain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bellotti, T.: Confidence Machines for Microarray Classification and Feature Selection. PhD thesis, Royal Holloway University of London, London, UK (February 2006)
Bellotti, T., Luo, Z., Gammerman, A., Van Delft, F., Saha, V.: Qualified predictions for microarray and proteomics pattern diagnostics with confidence machines. International Journal of Neural Systems 15(4), 247–258 (2005)
Blanzieri, E., Ricci, F.: Probability based metrics for nearest neighbor classification and case-based reasoning. In: Althoff, K.-D., Bergmann, R., Branting, L.K. (eds.) ICCBR 1999. LNCS (LNAI), vol. 1650, pp. 14–28. Springer, Heidelberg (1999)
Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13(1), 21–27 (1967)
Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29(2), 103–130 (1997)
Fisher, R.: The use of multiple measurements in taxonomics problems. Annals of Eugenics 7, 178–188 (1936)
Gammerman, A., Vovk, V.: Prediction algorithms and confidence measures based on algorithmic randomness theory. Theoretical Computer Science 287(1), 209–217 (2002)
Gammerman, A., Vovk, V., Vapnik, V.: Learning by transduction. In: Cooper, G., Moral, S. (eds.) UAI 1998. 14th Conference on Uncertainty in Artificial Intelligence, Madison, WI, July 24-26 1998, pp. 148–155. Morgan Kaufmann, San Francisco (1998)
Khardon, R., Roth, D., Servedio, R.: Efficiency versus convergence of boolean kernels for on-line learning algorithms. Journal of Artificial Intelligence Research 24, 341–356 (2005)
Melluish, T., Saunders, C., Nouretdinov, I., Vovk, V.: Comparing the Bayes and typicalness frameworks. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 360–371. Springer, Heidelberg (2001)
Newman, D., Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases (1998)
Proedrou, K., Nouretdinov, I., Vovk, V., Gammerman, A.: Transductive confidence machines for pattern recognition. Technical Report 01-02, Royal Holloway University of London, London, UK (2001)
Saunders, C., Gammerman, A., Vovk, V.: Transduction with confidence and credibility. In: Dean, T. (ed.) IJCAI 1999. 16th International Joint Conference on Artificial Intelligence, Stockholm, Sweden, July 31 - August 6, 1999, pp. 722–726. Morgan Kaufmann, San Francisco (1999)
Saunders, C., Gammerman, A., Vovk, V.: Computationally efficient transductive machines. In: Okamoto, T., Hartley, R., Kinshuk, Klus, J. (eds.) ICALT 2000. 11th International Conference on Algorithmic Learning Theory, Madison, WI, August 6-8, 2000, pp. 325–333. IEEE Computer Society Press, Los Alamitos (2000)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
Vanderlooy, S., van der Maaten, L., Sprinkhuizen-Kuyper, I.: Off-line learning with transductive confidence machines: an empirical evaluation. Technical Report MICC-IKAT 07-03, Universiteit Maastricht, Maastricht, The Netherlands (2007)
Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. LNCS. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Vanderlooy, S., van der Maaten, L., Sprinkhuizen-Kuyper, I. (2007). Off-Line Learning with Transductive Confidence Machines: An Empirical Evaluation. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2007. Lecture Notes in Computer Science(), vol 4571. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73499-4_24
Download citation
DOI: https://doi.org/10.1007/978-3-540-73499-4_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73498-7
Online ISBN: 978-3-540-73499-4
eBook Packages: Computer ScienceComputer Science (R0)