A Landmarker Selection Algorithm Based on Correlation and Efficiency Criteria

  • Daren Ler
  • Irena Koprinska
  • Sanjay Chawla
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3339)


Landmarking is a recent and promising meta-learning strategy, which defines meta-features that are themselves efficient learning algorithms. However, the choice of landmarkers is often made in an ad hoc manner. In this paper, we propose a new perspective and set of criteria for landmarkers. Based on the new criteria, we propose a landmarker generation algorithm, which generates a set of landmarkers that are each subsets of the algorithms being landmarked. Our experiments show that the landmarkers formed, when used with linear regression are able to estimate the accuracy of a set of candidate algorithms well, while only utilising a small fraction of the computational cost required to evaluate those candidate algorithms via ten-fold cross-validation.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bensusan, H.: God doesn’t always shave with Occam’s Razor: learning when and how to prune. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 119–124. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  2. 2.
    Bensusan, H., Giraud-Carrier, C.: Casa Batló is in Passeig de Gràcia or landmarking the expertise space. In: Proc. ECML, Workshop. on Meta-learning: Building automatic advice strategies for model selection and method combination, pp. 29–46 (2000)Google Scholar
  3. 3.
    Blake, C., Merz, C.: UCI repository of machine learning databases. Dept. of Information and Computer Sciences. University of California, Irvine (1998)Google Scholar
  4. 4.
    Brazdil, P., Gama, J., Henery, R.: Characterizing the applicability of classification algorithms using meta level learning. In: Bergadano, F., De Raedt, L. (eds.) ECML 1994. LNCS, vol. 784, pp. 84–102. Springer, Heidelberg (1994)Google Scholar
  5. 5.
    Brazdil, P., Soares, C., Costa, J.: Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results. Machine Learning 50(3), 251–277 (2003)MATHCrossRefGoogle Scholar
  6. 6.
    Fürnkranz, J., Petrak, J.: An evaluation of landmarking variants. In: Proc. ECML, Workshop. on integrating aspects of data mining, decision support and meta-learning, pp. 57–68 (2001)Google Scholar
  7. 7.
    Fürnkranz, J., Petrak, J., Brazdil, P., Soares, C.: On the use of fast subsampling estimates for algorithm recommendation. Technical Report, Austrian Research Institute for Artificial Intelligence (2002)Google Scholar
  8. 8.
    Giraud-Carrier, C., Vilalta, R., Brazdil, P.: Introduction to the special issue on meta-learning. Machine Learning 54(3), 187–193 (2004)CrossRefGoogle Scholar
  9. 9.
    Kalousis, A., Hilario, M.: Feature selection for meta-learning. In: Cheung, D., Williams, G.J., Li, Q. (eds.) PAKDD 2001. LNCS (LNAI), vol. 2035, pp. 222–233. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  10. 10.
    Kalousis, A., Hilario, M.: Model selection via meta-learning: a comparative study. Int. J. Artificial Intelligence Tools 10(4), 525–554 (2001)CrossRefGoogle Scholar
  11. 11.
    Köpf, C., Taylor, C., Keller, J.: Meta-analysis: from data characterisation for meta-learning to meta-regression. In: Proc. PKDD, Workshop. on Data Mining, Decision Support, Meta-learning and ILP (2000)Google Scholar
  12. 12.
    Michie, D., Spiegelhalter, D., Taylor, C.: Machine learning, neural and statistical classification, Ellis Horwood (1994)Google Scholar
  13. 13.
    Pfahringer, B., Bensusan, H., Giraud-Carrier, C.: Meta-learning by landmarking various learning algorithms. In: Proc. ICML, pp. 743–750 (2000)Google Scholar
  14. 14.
    Schaffer, C.: Technical note: selecting a classification method by cross-validation. Machine Learning 13(1), 135–143 (1993)Google Scholar
  15. 15.
    Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. J. Artificial Intelligence Review 18(2), 77–95 (2002)CrossRefGoogle Scholar
  16. 16.
    Wickens, T.: The geometry of multivariate statistics. LEA Publishers (1995)Google Scholar
  17. 17.
    Witten, I., Frank, E.: Data mining: practical machine learning tools with Java implementations. Morgan Kaufmann, San Francisco (2000)Google Scholar
  18. 18.
    Wolpert, D.: The supervised learning no-free-lunch theorems. In: Proc. Soft Computing in Industry - Recent Applications, pp. 25–42 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Daren Ler
    • 1
  • Irena Koprinska
    • 1
  • Sanjay Chawla
    • 1
  1. 1.School of Information TechnologiesUniversity of SydneyAustralia

Personalised recommendations