Machine Learning and Knowledge Extraction in Digital Pathology Needs an Integrative Approach

  • Andreas HolzingerEmail author
  • Bernd Malle
  • Peter Kieseberg
  • Peter M. Roth
  • Heimo Müller
  • Robert Reihs
  • Kurt Zatloukal
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10344)


During the last decade pathology has benefited from the rapid progress of image digitizing technologies, which led to the development of scanners, capable to produce so-called Whole Slide images (WSI) which can be explored by a pathologist on a computer screen comparable to the conventional microscope and can be used for diagnostics, research, archiving and also education and training. Digital pathology is not just the transformation of the classical microscopic analysis of histological slides by pathologists to just a digital visualization. It is a disruptive innovation that will dramatically change medical work-flows in the coming years and help to foster personalized medicine. Really powerful gets a pathologist if she/he is augmented by machine learning, e.g. by support vector machines, random forests and deep learning. The ultimate benefit of digital pathology is to enable to learn, to extract knowledge and to make predictions from a combination of heterogenous data, i.e. the histological image, the patient history and the *omics data. These challenges call for integrated/integrative machine learning approach fostering transparency, trust, acceptance and the ability to explain step-by-step why a decision has been made.


Digital pathology Data integration Integrative machine learning Deep learning Transfer learning 



We are grateful for valuable comments from the international reviewers.


  1. 1.
    Madabhushi, A., Lee, G.: Image analysis and machine learning in digital pathology: Challenges and opportunities. Med. Image Anal. 33, 170–175 (2016)CrossRefGoogle Scholar
  2. 2.
    Holzinger, A.: Trends in interactive knowledge discovery for personalized medicine: Cognitive science meets machine learning. IEEE Intell. Inf. Bull. 15, 6–14 (2014)Google Scholar
  3. 3.
    Su, X., Kang, J., Fan, J., Levine, R.A., Yan, X.: Facilitating score and causal inference trees for large observational studies. J. Mach. Learn. Res. 13, 2955–2994 (2012)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Huppertz, B., Holzinger, A.: Biobanks – a source of large biological data sets: open problems and future challenges. In: Holzinger, A., Jurisica, I. (eds.) Interactive Knowledge Discovery and Data Mining in Biomedical Informatics. LNCS, vol. 8401, pp. 317–330. Springer, Heidelberg (2014). doi: 10.1007/978-3-662-43968-5_18 CrossRefGoogle Scholar
  5. 5.
    Mattmann, C.A.: Computing: A vision for data science. Nature 493, 473–475 (2013)CrossRefGoogle Scholar
  6. 6.
    Otasek, D., Pastrello, C., Holzinger, A., Jurisica, I.: Visual data mining: effective exploration of the biological universe. In: Holzinger, A., Jurisica, I. (eds.) Interactive Knowledge Discovery and Data Mining in Biomedical Informatics. LNCS, vol. 8401, pp. 19–33. Springer, Heidelberg (2014). doi: 10.1007/978-3-662-43968-5_2 CrossRefGoogle Scholar
  7. 7.
    Ghahramani, Z.: Probabilistic machine learning and artificial intelligence. Nature 521, 452–459 (2015)CrossRefGoogle Scholar
  8. 8.
    Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Hierarchical dirichlet processes. J. Am. Stat. Assoc. 101, 1566–1581 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Houlsby, N., Huszar, F., Ghahramani, Z., Hernndez-lobato, J.M.: Collaborative gaussian processes for preference learning. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems (NIPS 2012), pp. 2096–2104 (2012)Google Scholar
  10. 10.
    Holzinger, A.: Introduction to machine learning and knowledge extraction (make). Mach. Learn. Knowl. Extr. 1, 1–20 (2017)CrossRefGoogle Scholar
  11. 11.
    Shahriari, B., Swersky, K., Wang, Z., Adams, R.P., de Freitas, N.: Taking the human out of the loop: A review of bayesian optimization. Proc. IEEE 104, 148–175 (2016)CrossRefGoogle Scholar
  12. 12.
    Kim, W., Choi, B.J., Hong, E.K., Kim, S.K., Lee, D.: A taxonomy of dirty data. Data Min. Knowl. Disc. 7, 81–99 (2003)CrossRefMathSciNetGoogle Scholar
  13. 13.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)CrossRefGoogle Scholar
  14. 14.
    Lee, S., Holzinger, A.: Knowledge discovery from complex high dimensional data. In: Michaelis, S., Piatkowski, N., Stolpe, M. (eds.) Solving Large Scale Learning Tasks. Challenges and Algorithms. LNCS (LNAI), vol. 9580, pp. 148–167. Springer, Cham (2016). doi: 10.1007/978-3-319-41706-6_7 CrossRefGoogle Scholar
  15. 15.
    Holzinger, A., Plass, M., Holzinger, K., Crisan, G.C., Pintea, C.M., Palade, V.: A glass-box interactive machine learning approach for solving NP-hard problems with the human-in-the-loop (2017). arXiv:1708.01104
  16. 16.
    Holzinger, A.: Interactive machine learning for health informatics: When do we need the human-in-the-loop? Brain Informatics (BRIN) 3 (2016)Google Scholar
  17. 17.
    Holzinger, A., Plass, M., Holzinger, K., Crişan, G.C., Pintea, C.-M., Palade, V.: Towards interactive machine learning (iML): applying ant colony algorithms to solve the traveling salesman problem with the human-in-the-loop approach. In: Buccafurri, F., Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-ARES 2016. LNCS, vol. 9817, pp. 81–95. Springer, Cham (2016). doi: 10.1007/978-3-319-45507-5_6 CrossRefGoogle Scholar
  18. 18.
    Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: From data mining to knowledge discovery in databases. AI Mag. 17, 37–54 (1996)Google Scholar
  19. 19.
    Valiant, L.G.: A theory of the learnable. Commun. ACM 27, 1134–1142 (1984)CrossRefzbMATHGoogle Scholar
  20. 20.
    Holzinger, A.: On topological data mining. In: Holzinger, A., Jurisica, I. (eds.) Interactive Knowledge Discovery and Data Mining in Biomedical Informatics. LNCS, vol. 8401, pp. 331–356. Springer, Heidelberg (2014). doi: 10.1007/978-3-662-43968-5_19 CrossRefGoogle Scholar
  21. 21.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)CrossRefGoogle Scholar
  22. 22.
    Demichelis, F., Barbareschi, M., Dalla Palma, P., Forti, S.: The virtual case: a new method to completely digitize cytological and histological slides. Virchows Arch. 441, 159–161 (2002)CrossRefGoogle Scholar
  23. 23.
    Bloice, M., Simonic, K.M., Holzinger, A.: On the usage of health records for the design of virtual patients: a systematic review. BMC Med. Inform. Decis. Mak. 13, 103 (2013)CrossRefGoogle Scholar
  24. 24.
    Turkay, C., Jeanquartier, F., Holzinger, A., Hauser, H.: On computationally-enhanced visual analysis of heterogeneous data and its application in biomedical informatics. In: Holzinger, A., Jurisica, I. (eds.) Interactive Knowledge Discovery and Data Mining in Biomedical Informatics. LNCS, vol. 8401, pp. 117–140. Springer, Heidelberg (2014). doi: 10.1007/978-3-662-43968-5_7 CrossRefGoogle Scholar
  25. 25.
    Ferreira, R., Moon, B., Humphries, J., Sussman, A., Saltz, J., Miller, R., Demarzo, A.: The virtual microscope. In: Proceedings of the AMIA Annual Fall Symposium, pp. 449–453 (1997)Google Scholar
  26. 26.
    Barbareschi, M., Demichelis, F., Forti, S., Palma, P.D.: Digital pathology: Science fiction? Int. J. Surg. Pathol. 8, 261–263 (2000). PMID: 11494001CrossRefGoogle Scholar
  27. 27.
    Hamilton, P.W., Wang, Y., McCullough, S.J.: Virtual microscopy and digital pathology in training and education. Apmis 120, 305–315 (2012)CrossRefGoogle Scholar
  28. 28.
    Dandu, R.: Storage media for computers in radiology. Indian J. Radiol. Imag. 18, 287 (2008)CrossRefGoogle Scholar
  29. 29.
    Reeder, M.M., Felson, B.: Gamuts in Radiology: Comprehensive Lists of Roentgen Differential Diagnosis. Pergamon Press (1977)Google Scholar
  30. 30.
    Goolsby, A.W., Olsen, L., McGinnis, M., Grossmann, C.: Clincial data as the basic staple of health learning - Creating and Protecting a Public Good. National Institute of Health (2010)Google Scholar
  31. 31.
    McDermott, J.E., Wang, J., Mitchell, H., Webb-Robertson, B.J., Hafen, R., Ramey, J., Rodland, K.D.: Challenges in biomarker discovery: combining expert insights with statistical analysis of complex omics data. Expert Opinion Med. Diagn. 7, 37–51 (2013)CrossRefGoogle Scholar
  32. 32.
    Swan, A.L., Mobasheri, A., Allaway, D., Liddell, S., Bacardit, J.: Application of machine learning to proteomics data: Classification and biomarker identification in postgenomics biology. Omics-a J. Integr. Biol. 17, 595–610 (2013)CrossRefGoogle Scholar
  33. 33.
    Jeanquartier, F., Jean-Quartier, C., Schreck, T., Cemernek, D., Holzinger, A.: Integrating open data on cancer in support to tumor growth analysis. In: Renda, M.E., Bursa, M., Holzinger, A., Khuri, S. (eds.) ITBAM 2016. LNCS, vol. 9832, pp. 49–66. Springer, Cham (2016). doi: 10.1007/978-3-319-43949-5_4 CrossRefGoogle Scholar
  34. 34.
    Bleiholder, J., Naumann, F.: Data fusion. ACM Comput. Surv. (CSUR) 41, 1–41 (2008)CrossRefGoogle Scholar
  35. 35.
    Lafon, S., Keller, Y., Coifman, R.R.: Data fusion and multicue data matching by diffusion maps. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1784–1797 (2006)CrossRefGoogle Scholar
  36. 36.
    Blanchet, L., Smolinska, A.: Data fusion in metabolomics and proteomics for biomarker discovery. In: Jung, K. (ed.) Statistical Analysis in Proteomics. MMB, vol. 1362, pp. 209–223. Springer, New York (2016). doi: 10.1007/978-1-4939-3106-4_14 CrossRefGoogle Scholar
  37. 37.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2009)CrossRefzbMATHGoogle Scholar
  38. 38.
    Bishop, C.M.: Pattern Recognition and Machine Learning (2006)Google Scholar
  39. 39.
    Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc. 2, 121–167 (1998)CrossRefGoogle Scholar
  40. 40.
    Kotropoulos, C., Pitas, I.: Segmentation of ultrasonic images using support vector machines. Pattern Recogn. Lett. 24, 715–727 (2003)CrossRefzbMATHGoogle Scholar
  41. 41.
    Ricci, E., Perfetti, R.: Retinal blood vessel segmentation using line operators and support vector classificatio. IEEE Trans. Med. Imaging 26, 1357–1365 (2007)CrossRefGoogle Scholar
  42. 42.
    Orlando, J.I., Blaschko, M.: Learning fully-connected CRFs for blood vessel segmentation in retinal images. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8673, pp. 634–641. Springer, Cham (2014). doi: 10.1007/978-3-319-10404-1_79 Google Scholar
  43. 43.
    Bauer, S., Nolte, L.-P., Reyes, M.: Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6893, pp. 354–361. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23626-6_44 CrossRefGoogle Scholar
  44. 44.
    El-Naqa, I., Yang, Y., Wernick, M.N., Galatsanos, N.P., Nishikawa, R.M.: A support vector machine approach for detection of microcalcifications. IEEE Trans. Med. Imaging 21, 1552–1563 (2002)CrossRefGoogle Scholar
  45. 45.
    Han, J.W., Breckon, T.P., Randell, D.A., Landini, G.: The application of support vector machine classification to detect cell nuclei for automated microscopy. Mach. Vis. Appl. 23, 15–24 (2012)CrossRefGoogle Scholar
  46. 46.
    Breiman, L.: Random forests. Mach. Learn. 45, 4–32 (2001)zbMATHGoogle Scholar
  47. 47.
    Criminisi, A., Jamie, S. (eds.): Decision Forests for Computer Vision and Medical Image Analysis. Springer, London (2013)Google Scholar
  48. 48.
    Zikic, D., Glocker, B., Konukoglu, E., Criminisi, A., Demiralp, C., Shotton, J., Thomas, O.M., Das, T., Jena, R., Price, S.J.: Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7512, pp. 369–376. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33454-2_46 CrossRefGoogle Scholar
  49. 49.
    Glocker, B., Pauly, O., Konukoglu, E., Criminisi, A.: Joint classification-regression forests for spatially structured multi-object segmentation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 870–881. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33765-9_62 CrossRefGoogle Scholar
  50. 50.
    Richmond, D., Kainmueller, D., Glocker, B., Rother, C., Myers, G.: Uncertainty-driven forest predictors for vertebra localization and segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 653–660. Springer, Cham (2015). doi: 10.1007/978-3-319-24553-9_80 CrossRefGoogle Scholar
  51. 51.
    Criminisi, A.: Anatomy detection and localization in 3D medical images. In: Criminisi, A., Shotton, J. (eds.) Decision Forests for Computer Vision and Medical Image Analysis. Advances in Computer Vision and Pattern Recognition. Springer, London (2013)CrossRefGoogle Scholar
  52. 52.
    Štern, D., Ebner, T., Urschler, M.: From local to global random regression forests: exploring anatomical landmark localization. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 221–229. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_26 CrossRefGoogle Scholar
  53. 53.
    Štern, D., Ebner, T., Urschler, M.: Automatic localization of locally similar structures based on the scale-widening random regression forest. In: IEEE International Symposium on Biomedical Imaging (2017)Google Scholar
  54. 54.
    Hebb, D.: The Organization of Behavior. Wiley, New York (1949)Google Scholar
  55. 55.
    McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Mathe. Biophys. 5, 115–133 (1943)CrossRefzbMATHMathSciNetGoogle Scholar
  56. 56.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)CrossRefGoogle Scholar
  57. 57.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet Classification with Deep Convolutional Neural Networks. In: Advances in Neural Information Processing Systems (2012)Google Scholar
  58. 58.
    Singh, D., Merdivan, E., Psychoula, I., Kropf, J., Hanke, S., Geist, M., Holzinger, A.: Human activity recognition using recurrent neural networks. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2017. LNCS, vol. 10410, pp. 267–274. Springer, Cham (2017). doi: 10.1007/978-3-319-66808-6_18 CrossRefGoogle Scholar
  59. 59.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798–1828 (2013)CrossRefGoogle Scholar
  60. 60.
    Liu, Y., Gadepalli, K., Norouzi, M., Dahl, G.E., Kohlberger, T., Boyko, A., Venugopalan, S., Timofeev, A., Nelson, P.Q., Corrado, G.S., Hipp, J.D., Peng, L., Stumpe, M.C.: Detecting cancer metastases on gigapixel pathology images. arXiv: 1703.02442 (2017)
  61. 61.
    Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017)CrossRefGoogle Scholar
  62. 62.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Proceedings Medical Image Computing and Computer-Assisted Intervention (2015)Google Scholar
  63. 63.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)CrossRefGoogle Scholar
  64. 64.
    Cai, J., Lu, L., Xie, Y., Xing, F., Yang, L.: Pancreas segmentation in MRI using graph-based decision fusion on convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 674–682. Springer, Cham (2017). doi: 10.1007/978-3-319-66179-7_77 CrossRefGoogle Scholar
  65. 65.
    Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_27 CrossRefGoogle Scholar
  66. 66.
    Rozantsev, A., Lepetit, V., Fua, P.: On rendering synthetic images for training an object detector. Comput. Vis. Image Underst. 137, 24–37 (2015)CrossRefGoogle Scholar
  67. 67.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (2014)Google Scholar
  68. 68.
    Nie, D., Trullo, R., Petitjean, C., Ruan, S., Shen, D.: Medical image synthesis with context-aware generative adversarial networks. arXiv:1612.05362 (2016). Accepted MICCAI’17
  69. 69.
    Schmidhuber, J.: Deep learning in neural networks: An overview. Neural Netw. 61, 85–117 (2015)CrossRefGoogle Scholar
  70. 70.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016)Google Scholar
  71. 71.
    Malle, B., Kieseberg, P., Schrittwieser, S., Holzinger, A.: Privacy aware machine learning and the right to be forgotten. ERCIM News (Special Theme: Machine Learning) 107, 22–23 (2016)Google Scholar
  72. 72.
    Fosch Villaronga, E., Kieseberg, P., Li, T.: Humans forget, machines remember: Artificial intelligence and the right to be forgotten. Computer Security Law Review (2017)Google Scholar
  73. 73.
    Malle, B., Giuliani, N., Kieseberg, P., Holzinger, A.: The more the merrier - federated learning from local sphere recommendations. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2017. LNCS, vol. 10410, pp. 367–373. Springer, Cham (2017). doi: 10.1007/978-3-319-66808-6_24 CrossRefGoogle Scholar
  74. 74.
    Dwork, C.: Differential privacy: a survey of results. In: Agrawal, M., Du, D., Duan, Z., Li, A. (eds.) TAMC 2008. LNCS, vol. 4978, pp. 1–19. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-79228-4_1 CrossRefGoogle Scholar
  75. 75.
    Kieseberg, P., Hobel, H., Schrittwieser, S., Weippl, E., Holzinger, A.: Protecting anonymity in data-driven biomedical science. In: Holzinger, A., Jurisica, I. (eds.) Interactive Knowledge Discovery and Data Mining in Biomedical Informatics. LNCS, vol. 8401, pp. 301–316. Springer, Heidelberg (2014). doi: 10.1007/978-3-662-43968-5_17 CrossRefGoogle Scholar
  76. 76.
    Schrittwieser, S., Kieseberg, P., Echizen, I., Wohlgemuth, S., Sonehara, N., Weippl, E.: An algorithm for k-anonymity-based fingerprinting. In: Shi, Y.Q., Kim, H.-J., Perez-Gonzalez, F. (eds.) IWDW 2011. LNCS, vol. 7128, pp. 439–452. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-32205-1_35 CrossRefGoogle Scholar
  77. 77.
    Kieseberg, P., Schrittwieser, S., Mulazzani, M., Echizen, I., Weippl, E.: An algorithm for collusion-resistant anonymization and fingerprinting of sensitive microdata. Electron. Markets 24, 113–124 (2014)CrossRefGoogle Scholar
  78. 78.
    Haerder, T., Reuter, A.: Principles of transaction-oriented database recovery. ACM Comput. Surv. (CSUR) 15, 287–317 (1983)CrossRefMathSciNetGoogle Scholar
  79. 79.
    Bayer, R., McCreight, E.: Organization and maintenance of large ordered indexes. In: Broy, M., Denert, E. (eds.) Software Pioneers, pp. 245–262. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  80. 80.
    Fruhwirt, P., Kieseberg, P., Weippl, E.: Using internal MySQL/InnoDB B-tree index navigation for data hiding. In: Peterson, G., Shenoi, S. (eds.) DigitalForensics 2015. IAICT, vol. 462, pp. 179–194. Springer, Cham (2015). doi: 10.1007/978-3-319-24123-4_11 CrossRefGoogle Scholar
  81. 81.
    Kieseberg, P., Schrittwieser, S., Mulazzani, M., Huber, M., Weippl, E.: Trees cannot lie: Using data structures for forensics purposes. In: Intelligence and Security Informatics Conference (EISIC), 2011 European, pp. 282–285. IEEE (2011)Google Scholar
  82. 82.
    Pantazos, K., Lauesen, S., Lippert, S.: De-identifying an EHR database-Anonymity, correctness and readability of the medical record. Stud. Health Technol. Inf. 169, 862–866 (2011)Google Scholar
  83. 83.
    Neamatullah, I., Douglass, M.M., Lehman, L.W.H., Reisner, A., Villarroel, M., Long, W.J., Szolovits, P., Moody, G.B., Mark, R.G., Clifford, G.D.: Automated de-identification of free-text medical records. BMC Med. Inform. Decis. Mak. 8, 32 (2008)CrossRefGoogle Scholar
  84. 84.
    Al-hegami, A.S.: A biomedical named entity recognition using machine learning classifiers and rich feature set. Int. J. Comput. Sci. Netw. Secur. 17, 170–176 (2017)Google Scholar
  85. 85.
    Settles, B.: Biomedical named entity recognition using conditional random fields and rich feature sets. In: International Joint Workshop on Natural Language Processing in Biomedicine and its Applications, pp. 104–107 (2004)Google Scholar
  86. 86.
    Mavromatis, G.: Biomedical named entity recognition using neural networks 2015, 1–9 (2015)Google Scholar
  87. 87.
    Goldberg, Y., Levy, O.: word2vec explained: deriving Mikolov et al. negative-sampling word-embedding method. arXiv preprint arXiv:1402.3722 (2014)
  88. 88.
    Sweeney, L.: k-anonymity: A model for protecting privacy. Int. J. Uncertainty, Fuzziness and Knowl.-Based Syst. 10, 557–570 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  89. 89.
    Machanavajjhala, A., Gehrke, J., Kifer, D., Venkitasubramaniam, M.: l-diversity: Privacy beyond k-anonymity. In: Proceedings of the 22nd International Conference on Data Engineering, ICDE 2006, p. 24. IEEE (2006)Google Scholar
  90. 90.
    Li, N., Li, T., Venkatasubramanian, S.: t-closeness: Privacy beyond k-anonymity and l-diversity. In: IEEE 23rd International Conference on Data Engineering, ICDE 2007, pp. 106–115. IEEE (2007)Google Scholar
  91. 91.
    Nergiz, M.E., Atzori, M., Clifton, C.: Hiding the presence of individuals from shared databases. In: Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, pp. 665–676. ACM (2007)Google Scholar
  92. 92.
    Wong, R.C.W., Li, J., Fu, A.W.C., Wang, K.: (\(\alpha \), k)-anonymity: an enhanced k-anonymity model for privacy preserving data publishing. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 754–759. ACM (2006)Google Scholar
  93. 93.
    Campan, A., Truta, T.M.: Data and structural k-Anonymity in social networks. In: Bonchi, F., Ferrari, E., Jiang, W., Malin, B. (eds.) PInKDD 2008. LNCS, vol. 5456, pp. 33–54. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-01718-6_4 CrossRefGoogle Scholar
  94. 94.
    Malle, B., Kieseberg, P., Weippl, E., Holzinger, A.: The right to be forgotten: towards machine learning on perturbed knowledge bases. In: Buccafurri, F., Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-ARES 2016. LNCS, vol. 9817, pp. 251–266. Springer, Cham (2016). doi: 10.1007/978-3-319-45507-5_17 CrossRefGoogle Scholar
  95. 95.
    Malle, B., Kieseberg, P., Holzinger, A.: DO NOT DISTURB? classifier behavior on perturbed datasets. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2017. LNCS, vol. 10410, pp. 155–173. Springer, Cham (2017). doi: 10.1007/978-3-319-66808-6_11 CrossRefGoogle Scholar
  96. 96.
    Rafique, A., Azam, S., Jeon, M., Lee, S.: Face-deidentification in images using restricted boltzmann machines. In: ICITST, pp. 69–73 (2016)Google Scholar
  97. 97.
    Chi, H., Hu, Y.H.: Face de-identification using facial identity preserving features. In: 2015 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2015, pp. 586–590 (2016)Google Scholar
  98. 98.
    Yu, F., Fienberg, S.E., Slavković, A.B., Uhler, C.: Scalable privacy-preserving data sharing methodology for genome-wide association studies. J. Biomed. Inform. 50, 133–141 (2014)CrossRefGoogle Scholar
  99. 99.
    Simmons, S., Sahinalp, C., Berger, B.: Enabling privacy-preserving GWASs in heterogeneous human populations. Cell Syst. 3, 54–61 (2016)CrossRefGoogle Scholar
  100. 100.
    Im, H.K., Gamazon, E.R., Nicolae, D.L., Cox, N.J.: On sharing quantitative trait GWAS results in an era of multiple-omics data and the limits of genomic privacy. Am. J. Hum. Genet. 90, 591–598 (2012)CrossRefGoogle Scholar
  101. 101.
    Knoppers, B.M., Dove, E.S., Litton, J.E., Nietfeld, J.J.: Questioning the limits of genomic privacy. Am. J. Hum. Genet. 91, 577–578 (2012)CrossRefGoogle Scholar
  102. 102.
    Aggarwal, C.C., Li, Y., Philip, S.Y.: On the hardness of graph anonymization. In: 2011 IEEE 11th International Conference on Data Mining (ICDM), pp. 1002–1007. IEEE (2011)Google Scholar
  103. 103.
    Baxter, J.: A model of inductive bias learning. J. Artif. Intell. Res. 12, 149–198 (2000)zbMATHMathSciNetGoogle Scholar
  104. 104.
    Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 109–117. ACM (2004)Google Scholar
  105. 105.
    Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009)zbMATHGoogle Scholar
  106. 106.
    Parameswaran, S., Weinberger, K.Q.: Large margin multi-task metric learning. In: Lafferty, J., Williams, C., Shawe-Taylor, J., Zemel, R., Culotta, A. (eds.) Advances in Neural Information Processing Systems 23 (NIPS 2010), pp. 1867–1875 (2010)Google Scholar
  107. 107.
    McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: The sequential learning problem. In: Bower, G.H. (ed.) The Psychology of Learning and Motivation, vol. 24, pp. 109–164. Academic Press, San Diego (1989)Google Scholar
  108. 108.
    French, R.M.: Catastrophic forgetting in connectionist networks. Trends Cogn. Sci. 3, 128–135 (1999)CrossRefGoogle Scholar
  109. 109.
    Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgeting in gradient-based neural networks. arXiv:1312.6211v3 (2015)
  110. 110.
    Taylor, M.E., Stone, P.: Transfer learning for reinforcement learning domains: A survey. J. Mach. Learn. Res. 10, 1633–1685 (2009)zbMATHMathSciNetGoogle Scholar
  111. 111.
    Sycara, K.P.: Multiagent systems. AI Mag. 19, 79 (1998)Google Scholar
  112. 112.
    Lynch, N.A.: Distributed Algorithms. Morgan Kaufmann, San Francisco (1996)zbMATHGoogle Scholar
  113. 113.
    DeGroot, M.H.: Reaching a consensus. J. Am. Stat. Assoc. 69, 118–121 (1974)CrossRefzbMATHGoogle Scholar
  114. 114.
    Benediktsson, J.A., Swain, P.H.: Consensus theoretic classification methods. IEEE Trans. Syst. Man Cybern. 22, 688–704 (1992)CrossRefzbMATHGoogle Scholar
  115. 115.
    Weller, S.C., Mann, N.C.: Assessing rater performance without a gold standard using consensus theory. Med. Decis. Making 17, 71–79 (1997)CrossRefGoogle Scholar
  116. 116.
    Olfati-Saber, R., Fax, J.A., Murray, R.M.: Consensus and cooperation in networked multi-agent systems. Proc. IEEE 95, 215–233 (2007)CrossRefGoogle Scholar
  117. 117.
    Roche, B., Guegan, J.F., Bousquet, F.: Multi-agent systems in epidemiology: a first step for computational biology in the study of vector-borne disease transmission. BMC Bioinf. 9 (2008)Google Scholar
  118. 118.
    Kok, J.R., Vlassis, N.: Collaborative multiagent reinforcement learning by payoff propagation. J. Mach. Learn. Res. 7, 1789–1828 (2006)zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Andreas Holzinger
    • 1
    Email author
  • Bernd Malle
    • 1
    • 2
  • Peter Kieseberg
    • 1
    • 2
  • Peter M. Roth
    • 3
  • Heimo Müller
    • 1
    • 4
  • Robert Reihs
    • 1
    • 4
  • Kurt Zatloukal
    • 4
  1. 1.Holzinger Group, HCI-KDD, Institute for Medical Informatics/StatisticsMedical University GrazGrazAustria
  2. 2.SBA ResearchViennaAustria
  3. 3.Institute of Computer Graphics and VisionGraz University of TechnologyGrazAustria
  4. 4.Institute of PathologyMedical University GrazGrazAustria

Personalised recommendations