Meta-learning Framework for Prediction Strategy Evaluation

  • Rodica Potolea
  • Silviu Cacoveanu
  • Camelia Lemnaru
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 73)


The paper presents a framework which brings together the tools necessary to analyze new problems and make predictions related to the learning algorithms’ performance and automate the analyst’s work. We focus on minimizing the system dependence on user input while still providing the ability of a guided search for a suitable learning algorithm through performance metrics. Predictions are computed using different strategies for calculating the distance between datasets, selecting neighbors and combining existing results. The framework is available for free use on the internet.


Meta-learning framework Data set features Performance metrics Prediction strategies 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aha, D.W.: Generalizing from Case Studies: A Case Study. In: Proceedings of the Ninth International Conference on Machine Learning, pp. 1–10 (1992)Google Scholar
  2. 2.
    Bensusan, H., Giraud-Carrier, C., Kennedy, C.J.: A Higher-Order Approach to Meta-Learning. In: Proceedings of the ECML 2000 Workshop on Meta-Learning: Building Automatic Advice Strategies for Model Selection and Method Combination, pp. 33–42 (2000)Google Scholar
  3. 3.
    Brazdil, P.B., Soares, C., da Costa, J.P.: Ranking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results. Machine Learning 50, 251–277 (2003)CrossRefGoogle Scholar
  4. 4.
    Cacoveanu, S., Vidrighin, C., Potolea, R.: Evolutional Meta-Learning Framework for Automatic Classifier Selection. In: Proceedings of the 5th International Conference on Intelligent Computer Communication and Processing, Cluj-Napoca, pp. 27–30 (2009)Google Scholar
  5. 5.
    Caruana, R., Niculescu-Mizil, A.: Data Mining in Metric Space: An Empirical Analysis of Supervised Learning Performance Criteria. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 69–78 (2004)Google Scholar
  6. 6.
    Giraud-Carrier, C., Bensusan, H.: Discovering Task Neighbourhoods Through Landmark Learning Performances. In: Proceedings of the Fourth European Conference of Principles and Practice of Knowledge Discovery in Databases, pp. 325–330 (2000)Google Scholar
  7. 7.
    Japkowicz, N., Stephen, S.: The Class Imbalance Problem: A Systematic Study. Intelligent Data Analysis 6(5), 429–450 (2002)Google Scholar
  8. 8.
    Kalousis, A.: Algorithm Selection via Meta_Learning, PhD Thesis, Faculte des sciences de l’Universite de Geneve (2002)Google Scholar
  9. 9.
    Linder, C., Studer, R.: Support for Algorithm Selection with a CBR Approach. In: Proceedings of the 16th International Conference on Machine Learning, pp. 418–423 (1999)Google Scholar
  10. 10.
    Michie, D., Spiegelhalter, D.J., Taylor, C.C.: Machine Learning. Neural and Statistical Classification. Ellis Horwood Series in Artificial Intelligence (1994)Google Scholar
  11. 11.
    Niculescu-Mizil, A., et al.: Winning the KDD Cup Orange Challenge with Ensemble Selection. In: JMLR Workshop and Conference Proceedings 7 (2009)Google Scholar
  12. 12.
    Rendel, L., Seshu, R., Tcheng, D.: Layered concept learning and dynamically variable bias management. In: 10th Internatinal Joint Conf. on AI, pp. 308–314 (1987)Google Scholar
  13. 13.
    Schaffner, C.: Selecting a classification method by cross validation. Machine Learning 13, 135–143 (1993)Google Scholar
  14. 14.
    Sleeman, D., Rissakis, M., Craw, S., Graner, N., Sharma, S.: Consultant-2: Pre and post-processing of machine learning applications. International Journal of Human Computer Studies, 43–63 (1995)Google Scholar
  15. 15.
    UCI Machine Learning Data Repository, (last accessed January 2010)
  16. 16.
    Vilalta, R., Giraud-Carrier, C., Brazdil, P., Soares, C.: Using Meta-Learning to Support Data Mining. International Journal of Computer Science & Applications, 31–45 (2004)Google Scholar
  17. 17.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann Publishers, Elsevier Inc. (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Rodica Potolea
    • 1
  • Silviu Cacoveanu
    • 1
  • Camelia Lemnaru
    • 1
  1. 1.Technical University of Cluj-NapocaRomania

Personalised recommendations