Meta-Learning Architectures: Collecting, Organizing and Exploiting Meta-Knowledge

  • Joaquin Vanschoren
Part of the Studies in Computational Intelligence book series (SCI, volume 358)


While a valid intellectual challenge in its own right, meta-learning finds its real raison d’être in the practical support it offers Data Mining practitioners [20]. Indeed, the whole point of understanding how to learn in any given situation is to go out in the real world and learn as much as possible, from any source of data we encounter! However, almost any type of raw data will initially be very hard to learn from, and about 80% of the effort in discovering useful patterns lies in the clever preprocessing of data [47]. Thus, for machine learning to become a tool we can instantly apply in any given situation, or at least to get proper guidance when applying it, we need to build extended meta-learning systems that encompass the entire knowledge discovery process, from raw data to finished models, and that keep learning, keep accumulating meta-knowledge, every time they are presented with new problems.


Data Mining Learn Workshop Data Mining Ontology ECML PKDD Data Mining Experiment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aha, D.: Generalizing from case studies: A case study. In: Proceedings of the Ninth International Conference on Machine Learning, pp. 1–10 (1992)Google Scholar
  2. 2.
    Asuncion, A., Newman, D.: Uci machine learning repository. University of California, School of Information and Computer Science (2007)Google Scholar
  3. 3.
    Bensusan, H., Giraud-Carrier, C.: Discovering task neighbourhoods through landmark learning performances. In: Zighed, D.A., Komorowski, J., Żytkow, J.M. (eds.) PKDD 2000. LNCS (LNAI), vol. 1910, pp. 325–330. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  4. 4.
    Bernstein, A., Dänzer, M.: The nExT system: Towards true dynamic adaptations of semantic web service compositions. In: Franconi, E., Kifer, M., May, W. (eds.) ESWC 2007. LNCS, vol. 4519, pp. 739–748. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  5. 5.
    Bernstein, A., Provost, F., Hill, S.: Toward intelligent assistance for a data mining process: an ontology-based approach for cost-sensitive classification. IEEE Transactions on Knowledge and Data Engineering 17(4), 503–518 (2005)CrossRefGoogle Scholar
  6. 6.
    Blockeel, H., Vanschoren, J.: Experiment databases: Towards an improved experimental methodology in machine learning. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 6–17. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  7. 7.
    Brazdil, P., Giraud-Carrier, C., Soares, C., Vilalta, R.: Metalearning: Applications to data mining. Springer, Heidelberg (2009)zbMATHGoogle Scholar
  8. 8.
    Brazdil, P., Soares, C., Costa, J.P.D.: Ranking learning algorithms: Using ibl and meta-learning on accuracy and time results. Machine Learning 50, 251–277 (2003)zbMATHCrossRefGoogle Scholar
  9. 9.
    Chandrasekaran, B., Josephson, J.: What are ontologies, and why do we need them? IEEE Intelligent systems 14(1), 20–26 (1999)CrossRefGoogle Scholar
  10. 10.
    Chapman, P., Clinton, J., Kerber, R., Khabaza, T., Reinartz, T., Shearer, C., Wirth, R.: Crisp-dm 1.0. a step-by-step data mining guide (1999),
  11. 11.
    Charest, M., Delisle, S.: Ontology-guided intelligent data mining assistance: Combining declarative and …. In: Proceedings of the 10th IASTED International Conference on Artificial Intelligence and Soft Computing, pp. 9–14 (2006)Google Scholar
  12. 12.
    Charest, M., Delisle, S., Cervantes, O., Shen, Y.: Intelligent data mining assistance via cbr and ontologies. In: Proceedings of the 17th International Conference on Database and Expert Systems Applications (DEXA 2006) (2006)Google Scholar
  13. 13.
    Charest, M., Delisle, S., Cervantes, O., Shen, Y.: Bridging the gap between data mining and decision support: A case-based reasoning and …. Intelligent Data Analysis 12, 1–26 (2008)Google Scholar
  14. 14.
    Craw, S., Sleeman, D., Graner, N., Rissakis, M.: Consultant: Providing advice for the machine learning toolbox. In: Research and Development in Expert Systems IX: Proceedings of Expert Systems 1992, pp. 5–23 (1992)Google Scholar
  15. 15.
    Dean, M., Connolly, D., van Harmelen, F., Hendler, J., Horrocks, I., Orah, L., McGuinness, D., Patel-Schneider, P.F., Stein, L.A.: Web ontology language (owl) reference version 1.0. W3C Working Draft (2003),
  16. 16.
    Engels, R.: Planning tasks for knowledge discovery in databases; performing task-oriented user-guidance. In: Proceedings of the 2nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 1996), pp. 170–175 (1996)Google Scholar
  17. 17.
    Fikes, R., Nilsson, N.: Strips: A new approach to the application of theorem proving to problem solving. Artificial intelligence 2, 189–208 (1971)zbMATHCrossRefGoogle Scholar
  18. 18.
    Foster, I.: Service-oriented science. science 308(5723), 814 (2005)CrossRefGoogle Scholar
  19. 19.
    Giraud-Carrier, C.: The data mining advisor: meta-learning at the service of practitioners. In: Proceedings of the 4th International Conference on Machine Learning and Applications, pp. 113–119 (2005)Google Scholar
  20. 20.
    Giraud-Carrier, C.: Metalearning-a tutorial. Tutorial at the 2008 International Conference on Machine Learning and Applications, ICMLA 2008 (2008)Google Scholar
  21. 21.
    Grabczewski, K., Jankowski, N.: Versatile and efficient meta-learning architecture: Knowledge representation and …. In: IEEE Symposium on Computational Intelligence and Data Mining, pp. 51–58 (2007)Google Scholar
  22. 22.
    Grąbczewski, K., Jankowski, N.: Meta-learning with machine generators and complexity controlled exploration. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 545–555. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  23. 23.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.: The weka data mining software: An update. SIGKDD Explorations 11(1), 10–18 (2009)CrossRefGoogle Scholar
  24. 24.
    Hidalgo, M., Menasalvas, E., Eibe, S.: Definition of a metadata schema for describing data preparation tasks. In: Proceedings of the ECML/PKDD 2009 Workshop on 3rd generation Data Mining (SoKD 2009), pp. 64–75 (2009)Google Scholar
  25. 25.
    Hilario, M., Kalousis, A.: Building algorithm profiles for prior model selection in knowledge discovery systems. Engineering Intelligent Systems 8(2) (2000)Google Scholar
  26. 26.
    Hilario, M., Kalousis, A., Nguyen, P., Woznica, A.: A data mining ontology for algorithm selection and meta-mining. In: Proceedings of the ECML/PKDD 2009 Workshop on 3rd generation Data Mining (SoKD 2009), pp. 76–87 (2009)Google Scholar
  27. 27.
    Homann, J., Nebel, B.: The ff planning system: Fast plan generation through heuristic search. Journal of Artificial Intelligence Research 14, 253–302 (2001)Google Scholar
  28. 28.
    Horrocks, I., Patel-Schneider, P., Boley, H.: Swrl: A semantic web rule language combining owl and ruleml. W3C Member submission (2004),
  29. 29.
    Kalousis, A.: Algorithm selection via meta-learning. PhD Thesis. University of Geneve (2002)Google Scholar
  30. 30.
    Kalousis, A., Bernstein, A., Hilario, M.: Meta-learning with kernels and similarity functions for planning of data mining workflows. In: ICML/COLT/UAI 2008 Planning to Learn Workshop (PlanLearn), pp. 23–28 (2008)Google Scholar
  31. 31.
    Kalousis, A., Hilario, M.: Model selection via meta-learning: a comparative study. International Journal on Artificial Intelligence Tools 10(4), 525–554 (2001)CrossRefGoogle Scholar
  32. 32.
    Kalousis, A., Theoharis, T.: Noemon: Design, implementation and performance results of an intelligent assistant for classifier selection. Intelligent Data Analysis 3(4), 319–337 (1999)zbMATHCrossRefGoogle Scholar
  33. 33.
    Kaufman, K.: Inlen: a methodology and integrated system for knowledge discovery in databases. PhD Thesis, School of Information Technology and Engineering, George Mason University (1997)Google Scholar
  34. 34.
    Kaufman, K., Michalski, R.: Discovery planning: Multistrategy learning in data mining. In: Proceedings of the Fourth International Workshop on Multistrategy Learning, pp. 14–20 (1998)Google Scholar
  35. 35.
    Kietz, J., Serban, F., Bernstein, A., Fischer, S.: Towards cooperative planning of data mining workflows. In: Proceedings of the Third Generation Data Mining Workshop at the 2009 European Conference on Machine Learning (ECML 2009), pp. 1–12 (2009)Google Scholar
  36. 36.
    Klusch, M., Gerber, A., Schmidt, M.: Semantic web service composition planning with owls-xplan. In: Proceedings of the First International AAAI Fall Symposium on Agents and the Semantic Web (2005)Google Scholar
  37. 37.
    Kodratoff, Y., Sleeman, D., Uszynski, M., Causse, K., Craw, S.: Building a machine learning toolbox. In: Enhancing the Knowledge Engineering Process: Contributions from ESPRIT, pp. 81–108 (1992)Google Scholar
  38. 38.
    Le-Khac, N., Kechadi, M., Carthy, J.: Admire framework: Distributed data mining on data grid platforms. In: Proceedings of the 1st International Conference on Software and Data Technologies, vol. 2, pp. 67–72 (2006)Google Scholar
  39. 39.
    Levin, L.: Universal sequential search problems. Problemy Peredachi Informatsii 9(3), 115–116 (1973)zbMATHMathSciNetGoogle Scholar
  40. 40.
    Li, M., Vitányi, P.: An introduction to kolmogorov complexity and its applications. In: Text and Monographs in Computer Science. Springer, Heidelberg (1993)Google Scholar
  41. 41.
    Lindner, G., Studer, R.: Ast: Support for algorithm selection with a cbr approach. In: Żytkow, J.M., Rauch, J. (eds.) PKDD 1999. LNCS (LNAI), vol. 1704, pp. 418–423. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  42. 42.
    Liu, Z., Ranganathan, A., Riabov, A.: A planning approach for message-oriented semantic web service composition. In: Proceedings of the National Conference on AI, vol. 5(2), pp. 1389–1394 (2007)Google Scholar
  43. 43.
    METAL. Metal: A meta-learning assistant for providing user support in machine learning and data mining. ESPRIT Framework IV LRT Reactive Project Nr. 26.357 (2001)Google Scholar
  44. 44.
    Michalski, R., Kerschberg, L., Kaufman, K.: Mining for knowledge in databases: The inlen architecture, initial implementation and first results. Journal of Intelligent Information Systems 1(1), 85–113 (1992)CrossRefGoogle Scholar
  45. 45.
    Michie, D., Spiegelhalter, D., Taylor, C.: Machine learning. In: Neural and Statistical Classification. Ellis Horwood (1994)Google Scholar
  46. 46.
    MLT. Machine learning toolbox. Esprit Framework II Research Project Nr. 2154 (1993)Google Scholar
  47. 47.
    Morik, K., Scholz, M.: The miningmart approach to knowledge discovery in databases. Intelligent Technologies for Information Analysis, pp. 47–65 (2004)Google Scholar
  48. 48.
    Panov, P., Dzeroski, S., Soldatova, L.: Ontodm: An ontology of data mining. In: Proceedings of the 2008 IEEE International Conference on Data MIning Workshops, pp. 752–760 (2008)Google Scholar
  49. 49.
    Panov, P., Soldatova, L.N., Džeroski, S.: Towards an ontology of data mining investigations. In: Gama, J., Costa, V.S., Jorge, A.M., Brazdil, P.B. (eds.) DS 2009. LNCS, vol. 5808, pp. 257–271. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  50. 50.
    Pfahringer, B., Bensusan, H., Giraud-Carrier, C.: Meta-learning by landmarking various learning algorithms. In: Proceedings of the Seventeenth International Conference on Machine Learning, pp. 743–750 (2000)Google Scholar
  51. 51.
    Podpecan, V., Jursic, M., Zakova, M., Lavrac, N.: Towards a service-oriented knowledge discovery platform. In: Proceedings of the SoKD 2009 International Workshop on Third Generation Data Mining at ECML PKDD 2009, pp. 25–38 (2009)Google Scholar
  52. 52.
    Rendell, L., Seshu, R., Tcheng, D.: Layered concept learning and dynamically-variable bias management. In: Proceedings of the 10th International Joint Conference on Artificial Intelligence, pp. 308–314 (1987)Google Scholar
  53. 53.
    De Roure, D., Goble, C., Stevens, R.: The design and realisation of the myexperiment virtual research environment for social sharing of workflows. Future Generation Computer Systems 25, 561–567 (2009)CrossRefGoogle Scholar
  54. 54.
    Rowe, A., Kalaitzopoulos, D., Osmond, M.: The discovery net system for high throughput bioinformatics. Bioinformatics 19, 225–231 (2003)CrossRefGoogle Scholar
  55. 55.
    Sacerdoti, E.: Planning in a hierarchy of abstraction spaces. Artificial intelligence 5(2), 115–135 (1974)zbMATHCrossRefGoogle Scholar
  56. 56.
    Sirin, E., Parsia, B., Wu, D., Hendler, J., Nau, D.: Htn planning for web service composition using shop2. Journal of Web Semantics 1(4), 377–396 (2004)CrossRefGoogle Scholar
  57. 57.
    Sleeman, D., Rissakis, M., Craw, S., Graner, N., Sharma, S.: Consultant-2: Pre-and post-processing of machine learning applications. International Journal of Human-Computer Studies 43(1), 43–63 (1995)CrossRefGoogle Scholar
  58. 58.
    Soldatova, L., King, R.: An ontology of scientific experiments. Journal of the Royal Society Interface 3(11), 795–803 (2006)CrossRefGoogle Scholar
  59. 59.
    Talia, D., Trunfio, P., Verta, O.: Weka4ws: a wsrf-enabled weka toolkit for distributed data mining on grids. In: Jorge, A.M., Torgo, L., Brazdil, P.B., Camacho, R., Gama, J. (eds.) PKDD 2005. LNCS (LNAI), vol. 3721, pp. 309–320. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  60. 60.
    Taylor, I., Shields, M., Wang, I., Harrison, A.: The triana workflow environment: Architecture and applications. In: Workflows for e-Science, pp. 320–339. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  61. 61.
    Utgoff, P.: Shift of bias for inductive concept learning. In: Machine learning: An artificial intelligence approach, vol. II. Morgan Kaufmann, San Francisco (1986)Google Scholar
  62. 62.
    Van Someren, M.: Towards automating goal-driven learning. In: Proceedings of the Planning to Learn Workshop at the 18th European Conference of Machine Learning (ECML 2007), pp. 42–52 (2007)Google Scholar
  63. 63.
    Vanschoren, J., Assche, A.V., Vens, C., Blockeel, H.: Meta-learning from experiment databases: An illustration. In: Proceedings of the 16th Annual Machine Learning Conference of Belgium and The Netherlands (Benelearn 2007), pp. 120–127 (2007)Google Scholar
  64. 64.
    Vanschoren, J., Blockeel, H.: A community-based platform for machine learning experimentation. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) ECML PKDD 2009. Lecture Notes in Computer Science (LNAI), vol. 5782, pp. 750–754. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  65. 65.
    Vanschoren, J., Blockeel, H., Pfahringer, B.: Experiment databases: Creating a new platform for meta-learning research. In: Proceedings of the ICML/UAI/COLT Joint Planning to Learn Workshop (PlanLearn 2008), pp. 10–15 (2008)Google Scholar
  66. 66.
    Vanschoren, J., Blockeel, H., Pfahringer, B., Holmes, G.: Organizing the world’s machine learning information. Communications in Computer and Information Science 17, 693–708 (2008)CrossRefGoogle Scholar
  67. 67.
    Vanschoren, J., Pfahringer, B., Holmes, G.: Learning from the past with experiment databases. In: Ho, T.-B., Zhou, Z.-H. (eds.) PRICAI 2008. LNCS (LNAI), vol. 5351, pp. 485–496. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  68. 68.
    Wirth, R., Shearer, C., Grimmer, U., Reinartz, T., Schlosser, J., Breitner, C., Engels, R., Lindner, G.: Towards process-oriented tool support for knowledge discovery in databases. In: Komorowski, J., Żytkow, J.M. (eds.) PKDD 1997. LNCS, vol. 1263, pp. 243–253. Springer, Heidelberg (1997)Google Scholar
  69. 69.
    Zakova, M., Kremen, P., Zelezny, F., Lavrac, N.: Planning to learn with a knowledge discovery ontology. In: Second Planning to Learn Workshop at the Joint ICML/COLT/UAI Conference, pp. 29–34 (2008)Google Scholar
  70. 70.
    Záková, M., Podpecan, V., Zelezný, F., Lavrac, N.: Advancing data mining workflow construction: A framework and cases using the orange toolkit. In: Proceedings of the SoKD-2009 International Workshop on Third Generation Data Mining at ECML PKDD 2009, pp. 39–51 (2009)Google Scholar
  71. 71.
    Zhong, N., Liu, C., Ohsuga, S.: Dynamically organizing kdd processes. International Journal of Pattern Recognition and Artificial Intelligence 15(3), 451–473 (2001)CrossRefGoogle Scholar
  72. 72.
    Zhong, N., Matsui, Y., Okuno, T., Liu, C.: Framework of a multi-agent kdd system. In: Yin, H., Allinson, N.M., Freeman, R., Keane, J.A., Hubbard, S. (eds.) IDEAL 2002. LNCS, vol. 2412, pp. 337–346. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  73. 73.
    Zhong, N., Ohsuga, S.: The gls discovery system: its goal, architecture and current results. In: Raś, Z.W., Zemankova, M. (eds.) ISMIS 1994. LNCS, vol. 869, pp. 233–244. Springer, Heidelberg (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Joaquin Vanschoren
    • 1
  1. 1.Department of Computer ScienceK.U. LeuvenLeuvenBelgium

Personalised recommendations