Abstract
An algorithm is proposed which adaptively and simultaneously estimates and combines classifiers originating from distinct classiffication frameworks for improved prediction. The methodology is developed and evaluated on simulations and real data. Analogies and similarities with generalised additive modeling, neural estimation and boosting are discussed. We contrast the approach with existing Bayesian model averaging methods. Areas for further research and development are indicated.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Breiman, L. (1992) Stacked regression. Technical report No. 367, August 1992, Department of Statistics, University of California, Berkeley, California 94720.
Breiman, L. (1994) Bagging predictors. Technical report No. 421, August 1994, Department of Statistics, University of California, Berkeley, California 94720.
Breiman, L. and Friedman, J. H. (1985) Estimating optimal transformations for multiple regression and correlation. J. Am. Statist. Ass., 80, 580–619.
Buja, A. (1990) Remarks on functional canonical variates, alternating least squares methods and ACE. The Annals of Statistics, 18, 1032–1069.
Buja, A., Hastie, T. and Tibshirani, R. (1989) Linear smoothers and additive models. The Annals of Statistics, 17, 453–555.
Buntine, W. (1992) Learning in Classification trees. Statistics and Computing, 2,63–73.
Clark, L. A. and Pregibon, D. (1992) Tree-based models. In: Statistical Models in S. (eds. Chambers, J. M. and Hastie, T. J.), Chapter 9. Pacific Grove, CA: Wadsworth and Brooks/Cole.
Devijver, P. A. and Kittler, J. V. (1982) Pattern Recognition. A Statistical Approach. Englewood Cliffs: Prentice-Hall.
Freund, Y. (1995) Boosting a weak learning algorithm by majority. Information and Computation, 121, 256–285.
Friedman, J. H. (1989) Regularized discriminant analysis. J. Am. Statist. Ass., 84, 165–175.
Friedman, J. H. (1991) Multivariate adaptive regression splines. The Annals of Statistics, 19, 1–141.
Friedman, J. H., Hastie, T. and Tibshirani, R. (1998) Additive Logistic Regression: A statistical view of boosting. Technical report. August 1998, Department of Statistics, Stanford University.
Hastie, T. and Pregibon, D. (1990) Shrinking trees. Technical report, AT&T Bell Laboratories, 600 Mountain Avenue, Murray Hill, NJ 07947.
Hastie, T. J. and Tibshirani, R. J. (1990) Generalized Additive Models. London: Chapman and Hall.
Key, J. T., Pericchi, L. R. and Smith, A. F. M. (1998) Bayesian model choice: what and why? Draft paper. Presented at the Sixth Conference on Bayesian Statistics in Valencia, 1998.
Madigan, D. and Raftery, A. E. (1994) Model selection and accounting for model uncertainty in graphical models using Occam’s window. J. Am. Statist. Ass., 89, 1535–1545. New York: Wiley.
Oliver, J. J. and Hand, D. J. (1996) Averaging over decision trees. Journal of Classification
Quinlan, J. R. (1992) Learning with continuous classes. In Proceedings of the Fifth Australian Joint Conference on Applied Artificial Intelligence, Eds. A. Adams and L. Sterling. Singapore: World Scientific, pp. 343–348.
Ripley, B. D. (1993) Statistical aspects of neural networks. In Networks and Chaos-Statistical and Probabilistic Aspects (eds. O. E. Barndorff-Nielsen, J. L. Jensen and W. S. Kendall), pp. 40–123. London: Chapman and Hall.
Ripley, B. D. (1994) Neural networks and related methods for Classification. J. R. Statist. Soc. B, 56, 409–456.
Schapire, R. E. (1990) The strength of weak learnability. Machine Learning, 5, 197–227.
Venables, W. N. and Ripley, B. D. (1994) Modern Applied Statistics with S-PLUS. New York: Springer-Verlag.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mertens, B.J.A., Hand, D.J. (1999). Adjusted Estimation for the Combination of Classifiers. In: Hand, D.J., Kok, J.N., Berthold, M.R. (eds) Advances in Intelligent Data Analysis. IDA 1999. Lecture Notes in Computer Science, vol 1642. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48412-4_27
Download citation
DOI: https://doi.org/10.1007/3-540-48412-4_27
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66332-4
Online ISBN: 978-3-540-48412-7
eBook Packages: Springer Book Archive