Towards Integrative Machine Learning and Knowledge Extraction

  • Andreas HolzingerEmail author
  • Randy Goebel
  • Vasile Palade
  • Massimo Ferri
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10344)


This Volume is a result of workshop 15w2181 “Advances in interactive knowledge discovery and data mining in complex and big data sets” at the Banff International Research Station for Mathematical Innovation and Discovery. The workshop was dedicated to bring together experts with diverse backgrounds but with one common goal: to understand intelligence for the successful design, development and evaluation of algorithms that can learn from data, extract knowledge from experience, and to improve their learning behaviour over time – similarly as we humans do. Knowledge discovery, data mining, machine learning, artificial intelligence are more or less synonymously used with no strict definitions or boundaries. “Integrative” means to support not only the machine learning & knowledge extraction pipeline, ranging from dealing with data in arbitrarily high-dimensional spaces to the visualization of results into a lower dimension accessible to a human; it is taking into account seemingly disparate fields which can be very fruitful when brought together - for solving problems in complex application domains (e.g. health informatics). Here we want to emphasize that the most important findings in machine learning will be those we do not know yet. In this paper we provide: (1) a short motivation for the integrative approach; (2) brief summaries of the presentations given in Banff; and (3) some personally flavoured, subjective future research outlooks, e.g. in the combination of geometrical approaches with machine learning.


Integrative machine learning Knowledge discovery 



We are grateful to all participants of the Banff BIRS workshop 15w2181, specifically to our colleagues from the international HCI-KDD expert network and generally to all colleagues who constantly support our group in fostering the idea of an integrated machine learning approach and in bringing together diverse areas in an cross-disciplinary manner to stimulate fresh ideas and to encourage multi-disciplinary problem solving. The past has shown that many new discoveries are made in overlapping areas of seemingly disjunct fields and the interesting and most important discoveries are those which we have not yet found.


  1. 1.
    Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2, 1–127 (2009)CrossRefzbMATHGoogle Scholar
  2. 2.
    Gopnik, A., Glymour, C., Sobel, D.M., Schulz, L.E., Kushnir, T., Danks, D.: A theory of causal learning in children: causal maps and bayes nets. Psychol. Rev. 111, 3–32 (2004)CrossRefGoogle Scholar
  3. 3.
    Poole, D., Mackworth, A., Goebel, R.: Computational Intelligence: A Logical Approach. Oxford University Press, New York (1998)zbMATHGoogle Scholar
  4. 4.
    Michalski, R.S., Carbonell, J.G., Mitchell, T.M.: Machine Learning: An Artificial Intelligence Approach. Springer, Heidelberg (1983). doi: 10.1007/978-3-662-12405-5 CrossRefzbMATHGoogle Scholar
  5. 5.
    Ghahramani, Z.: Probabilistic machine learning and artificial intelligence. Nature 521, 452–459 (2015)CrossRefGoogle Scholar
  6. 6.
    Holzinger, A.: On knowledge discovery and interactive intelligent visualization of biomedical data - challenges in human computer interaction & biomedical informatics. In: Helfert, M., Fancalanci, C., Filipe, J. (eds.) DATA 2012, International Conference on Data Technologies and Applications, pp. 5–16 (2012)Google Scholar
  7. 7.
    Holzinger, A.: Human-computer interaction and knowledge discovery (HCI-KDD): what is the benefit of bringing those two fields to work together? In: Cuzzocrea, A., Kittl, C., Simos, D.E., Weippl, E., Xu, L. (eds.) CD-ARES 2013. LNCS, vol. 8127, pp. 319–328. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40511-2_22 CrossRefGoogle Scholar
  8. 8.
    Holzinger, A.: Trends in interactive knowledge discovery for personalized medicine: cognitive science meets machine learning. IEEE Intell. Inform. Bull. 15, 6–14 (2014)Google Scholar
  9. 9.
    Holzinger, A., Jurisica, I.: Knowledge discovery and data mining in biomedical informatics: the future is in integrative, interactive machine learning solutions. In: Holzinger, A., Jurisica, I. (eds.) Interactive Knowledge Discovery and Data Mining in Biomedical Informatics. LNCS, vol. 8401, pp. 1–18. Springer, Heidelberg (2014). doi: 10.1007/978-3-662-43968-5_1 CrossRefGoogle Scholar
  10. 10.
    Holzinger, A., Dehmer, M., Jurisica, I.: Knowledge discovery and interactive data mining in bioinformatics - state-of-the-art, future challenges and research directions. BMC Bioinf. 15, I1 (2014)CrossRefGoogle Scholar
  11. 11.
    Lee, S., Holzinger, A.: Knowledge discovery from complex high dimensional data. In: Michaelis, S., Piatkowski, N., Stolpe, M. (eds.) Solving Large Scale Learning Tasks. Challenges and Algorithms. LNAI, vol. 9580, pp. 148–167. Springer, Cham (2016). doi: 10.1007/978-3-319-41706-6_7 CrossRefGoogle Scholar
  12. 12.
    Holzinger, A.: Introduction to machine learning and knowledge extraction (make). Mach. Learn. Knowl. Extr. 1, 1–20 (2017)CrossRefGoogle Scholar
  13. 13.
    Shahriari, B., Swersky, K., Wang, Z., Adams, R.P., de Freitas, N.: Taking the human out of the loop: a review of bayesian optimization. Proc. IEEE 104, 148–175 (2016)CrossRefGoogle Scholar
  14. 14.
    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)CrossRefGoogle Scholar
  15. 15.
    Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., Hassabis, D.: Mastering the game of go with deep neural networks and tree search. Nature 529, 484–489 (2016)CrossRefGoogle Scholar
  16. 16.
    Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017)CrossRefGoogle Scholar
  17. 17.
    Holzinger, A., Plass, M., Holzinger, K., Crisan, G.C., Pintea, C.M., Palade, V.: A glass-box interactive machine learning approach for solving np-hard problems with the human-in-the-loop. arXiv:1708.01104 (2017)
  18. 18.
    Goebel, R.: Why visualization is an ai-complete problem (and why that matters). In: 20th International Conference on Information Visualisation (IV 2016), pp. 27–32. IEEE (2016)Google Scholar
  19. 19.
    Lopez, V., Fernandez, A., García, S., Palade, V., Herrera, F.: An insight into classification with imbalanced data: empirical results and current trends on using data intrinsic characteristics. Inf. Sci. 250, 113–141 (2013)CrossRefGoogle Scholar
  20. 20.
    Piatkowski, N., Lee, S., Morik, K.: Integer undirected graphical models for resource-constrained systems. Neurocomputing 173, 9–23 (2016)CrossRefGoogle Scholar
  21. 21.
    Hess, S., Morik, K., Piatkowski, N.: The primping routine-tiling through proximal alternating linearized minimization. Data Min. Knowl. Disc. 31, 1090–1131 (2017)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Holzinger, K., Palade, V., Rabadan, R., Holzinger, A.: Darwin or lamarck? Future challenges in evolutionary algorithms for knowledge discovery and data mining. In: Holzinger, A., Jurisica, I. (eds.) Interactive Knowledge Discovery and Data Mining in Biomedical Informatics: State-of-the-Art and Future Challenges. LNCS, vol. 8401, pp. 35–56. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  23. 23.
    Holzinger, A., Blanchard, D., Bloice, M., Holzinger, K., Palade, V., Rabadan, R.: Darwin, lamarck, or baldwin: applying evolutionary algorithms to machine learning techniques. In: Slezak, D., Dunin-Keplicz, B., Lewis, M., Terano, T. (eds.) IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), pp. 449–453. IEEE (2014)Google Scholar
  24. 24.
    Nagrecha, S., Thomas, P.B., Feldman, K., Chawla, N.V.: Predicting chronic heart failure using diagnoses graphs. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2017. LNCS, vol. 10410, pp. 295–312. Springer, Cham (2017). doi: 10.1007/978-3-319-66808-6_20 CrossRefGoogle Scholar
  25. 25.
    Sjöbergh, J., Kuwahara, M., Tanaka, Y.: Visualizing clinical trial data using pluggable components. In: 2012 16th International Conference on Information Visualisation (IV), pp. 291–296. IEEE (2012)Google Scholar
  26. 26.
    Dlotko, P., Ghrist, R., Juda, M., Mrozek, M.: Distributed computation of coverage in sensor networks by homological methods. Appl. Algebra Eng. Commun. Comput. 23(1/2), 1–30 (2012). doi: 10.1007/s00200-012-0167-7 zbMATHMathSciNetGoogle Scholar
  27. 27.
    Frosini, P.: Measuring shapes by size functions. In: Intelligent Robots and Computer Vision X: Algorithms and Techniques, International Society for Optics and Photonics, pp. 122–133 (1992)Google Scholar
  28. 28.
    Verri, A., Uras, C., Frosini, P., Ferri, M.: On the use of size functions for shape analysis. Biol. Cybern. 70, 99–107 (1993)CrossRefzbMATHGoogle Scholar
  29. 29.
    Edelsbrunner, H., Letscher, D., Zomorodian, A.: Topological persistence and simplification, pp. 454–463 cited By 72 (2000)Google Scholar
  30. 30.
    Carlsson, G., Zomorodian, A., Collins, A., Guibas, L.J.: Persistence barcodes for shapes. Int. J. Shape Model. 11, 149–187 (2005)CrossRefzbMATHGoogle Scholar
  31. 31.
    Edelsbrunner, H., Harer, J.: Persistent homology-a survey. Contemp. Math. 453, 257–282 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  32. 32.
    Frosini, P., Mulazzani, M.: Size homotopy groups for computation of natural size distances. Bull. Belg. Math. Soc. Simon Stevin 6, 455–464 (1999)zbMATHMathSciNetGoogle Scholar
  33. 33.
    Carlsson, G., Zomorodian, A.: The theory of multidimensional persistence. Discrete Comput. Geom. 42, 71–93 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  34. 34.
    Biasotti, S., Cerri, A., Frosini, P., Giorgi, D., Landi, C.: Multidimensional size functions for shape comparison. J. Math. Imaging Vis. 32, 161–179 (2008)CrossRefMathSciNetGoogle Scholar
  35. 35.
    Cerri, A., Di Fabio, B., Ferri, M., Frosini, P., Landi, C.: Betti numbers in multidimensional persistent homology are stable functions. Math. Methods Appl. Sci. 36, 1543–1557 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  36. 36.
    Cagliari, F., Di Fabio, B., Ferri, M.: One-dimensional reduction of multidimensional persistent homology. Proc. Am. Math. Soc. 138, 3003–3017 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  37. 37.
    Adcock, A., Rubin, D., Carlsson, G.: Classification of hepatic lesions using the matching metric. Comput. Vis. Image Underst. 121, 36–42 (2014)CrossRefGoogle Scholar
  38. 38.
    Di Fabio, B., Ferri, M.: Comparing persistence diagrams through complex vectors (2015)Google Scholar
  39. 39.
    Frosini, P.: G-invariant persistent homology. Math. Methods Appl. Sci. 38, 1190–1199 (2015)CrossRefzbMATHMathSciNetGoogle Scholar
  40. 40.
    Xie, Z., Sun, J., Palade, V., Wang, S., Liu, Y.: Evolutionary sampling: a novel way of machine learning within a probabilistic framework. Inf. Sci. 299, 262–282 (2015)CrossRefzbMATHMathSciNetGoogle Scholar
  41. 41.
    Jun, S., Palade, V., Xiao-Jun, W., Wei, F., Zhenyu, W.: Solving the power economic dispatch problem with generator constraints by random drift particle swarm optimization. IEEE Trans. Ind. Inform. 10, 222–232 (2014)CrossRefGoogle Scholar
  42. 42.
    Jun, S., Palade, V., Xiaojun, W., Wei, F.: Multiple sequence alignment with hiddenmarkov models learned by random driftparticle swarm optimization. IEEE/ACM Trans. Comput. Biol. Bioinform. 11, 243–257 (2014)CrossRefGoogle Scholar
  43. 43.
    Sun, J., Fang, W., Palade, V., Wu, X., Xu, W.: Quantum-behaved particle swarm optimization with gaussian distributed local attractor point. Appl. Math. Comput. 218, 3763–3775 (2011)zbMATHGoogle Scholar
  44. 44.
    Fogelberg, C., Palade, V.: Dense structural expectation maximisation with parallelisation for efficient large-network structural inference. Int. J. Artif. Intell. Tools 22, 1350011 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Andreas Holzinger
    • 1
    Email author
  • Randy Goebel
    • 2
  • Vasile Palade
    • 3
  • Massimo Ferri
    • 4
  1. 1.Holzinger Group, HCI-KDD, Institute for Medical Informatics/StatisticsMedical University GrazGrazAustria
  2. 2.Centre for Machine LearningUniversity of AlbertaEdmontonCanada
  3. 3.Cogent Computing Applied Research CentreCoventry UniversityCoventryUK
  4. 4.Vision Mathematics Group, Department of MathematicsUniversity of BolognaBolognaItaly

Personalised recommendations