Skip to main content
SpringerLink
Log in

Introduction

  • Chapter
Kernel-based Data Fusion for Machine Learning

Part of the book series: Studies in Computational Intelligence ((SCI,volume 345))

Abstract

The history of learning has been accompanied by the pace of evolution and the progress of civilization. Some modern ideas of learning (e.g., pattern analysis and machine intelligence) can be traced back thousands of years in the analects of oriental philosophers [16] and Greek mythologies (e.g., The Antikythera Mechanism [83]). Machine learning, a contemporary topic rooted in computer science and engineering, has always being inspired and enriched by the unremitting efforts of biologists and psychologists in their investigation and understanding of the nature. The Baldwin effect [4], proposed by James Mark Baldwin 110 years ago, concerns the the costs and benefits of learning in the context of evolution, which has greatly influenced the development of evolutionary computation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aizerman, M., Braverman, E., Rozonoer, L.: Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control 25, 821–837 (1964)

    MathSciNet  Google Scholar 

  2. Bach, F.R., Jordan, M.I.: A Probabilistic Interpretation of Canonical Correlation Analysis. Internal Report 688, Department of Statistics. Department of Statistics, University of California, Berkeley (2005)

    Google Scholar 

  3. Bach, F.R., Jordan, M.I.: Kernel independent component analysis. Journal of Machine Learning Research 3, 1–48 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  4. Baldwin, M.J.: A New Factor in Evolution. The American Naturalist 30, 441–451 (1896)

    Article  Google Scholar 

  5. Barry, D.J.: Design Of and Studies With a Novel One Meter Multi-Element Spectroscopic Telescope. Ph.D dissertation, University of Cornell (1995)

    Google Scholar 

  6. Becker, S.: Mutual Information Maximization: models of cortical self-organization. Network: Computation in Neural System 7, 7–31 (1996)

    Article  MATH  Google Scholar 

  7. Bishop, C.M.: Pattern recognition and machine learning. Springer, New York (2006)

    MATH  Google Scholar 

  8. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the 5th Annual ACM Workshop on COLT, pp. 144–152. ACM Press, New York (1992)

    Chapter  Google Scholar 

  9. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    MATH  Google Scholar 

  10. Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)

    Article  MATH  Google Scholar 

  11. Brieman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)

    Google Scholar 

  12. Breiman, L.: Statistical Modeling: The Two Cultures. Statistical Science 16, 199–231 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  13. Cao, Y.: Efficient K-Means Clustering using JIT. MATLAB Central file exchange (2008), http://www.mathworks.com/matlabcentral/fileexchange/19344-efficient-k-means-clustering-using-jit

  14. Cauwenberghs, G., Poggio, T.: Incremental and decremental support vector machine learning. Advances in Neural Information Processing Systems 13, 409–415 (2001)

    Google Scholar 

  15. Chu, C.T., Kim, S.K., Lin, Y.A., Yu, Y.Y., Bradski, G., Ng, A.Y., Olukotun, K.: Map-Reduce for Machine Learning on Multicore. Advances in Neural Information Processing Systems 20, 281–288 (2008)

    Google Scholar 

  16. Confucius: The Analects. 500 B.C

    Google Scholar 

  17. Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1999)

    Google Scholar 

  18. Cover, T.M., Hart, P.E.: Nearest neighbor pattern classification. IEEE Trans. Information Theory. 13, 21–27 (1967)

    Article  MATH  Google Scholar 

  19. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)

    Google Scholar 

  20. Csiszar, I., Tusnady, G.: Information geometry and alternating minimization procedures. Statistics and Decisions suppl. 1, 205–237 (1984)

    MathSciNet  Google Scholar 

  21. Dash, D., Druzdzel, M.J.: Robust independence testing for constraint-based learning of causal structure. In: Proceedings of the 19th Conference on Uncertainty in Artificial Intelligence, pp. 167–174 (2003)

    Google Scholar 

  22. De Raedt, L., Dehaspe, L.: Clausal discovery. Machine Learning 26, 99–146 (1997)

    Article  MATH  Google Scholar 

  23. De Raedt, L., Van Laer, W.: Inductive constraint logic. In: Zeugmann, T., Shinohara, T., Jantke, K.P. (eds.) ALT 1995. LNCS, vol. 997, pp. 80–94. Springer, Heidelberg (1995)

    Google Scholar 

  24. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Communications of the ACM - 50th Anniversary issue: 1958 - 2008 51, 107–113 (2008)

    Google Scholar 

  25. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39, 1–38 (1977)

    MATH  MathSciNet  Google Scholar 

  26. Drucker, H., Schapire, R., Simard, P.: Improving performance in neural networks using a boosting algorithm. Advances in Neural Information Processing Systems 5, 42–49 (1993)

    Google Scholar 

  27. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons Inc., New York (2001)

    MATH  Google Scholar 

  28. Evans, J., Rzhetsky, A.: Machine Science. Science 329, 399–400 (2010)

    Article  Google Scholar 

  29. Freund, Y., Schapire, R.: A decision-theoretic generalization of online learning and an application to boosting. Journal of Computer and System Sciences 55, 119–139 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  30. Friedman, N.: Learning belief networks in the presence of missing values and hidden variables. In: Proceedings of the 14th ICML, pp. 125–133 (1997)

    Google Scholar 

  31. Friedman, C., Borlawsky, T., Shagina, L., Xing, H.R., Lussier, Y.A.: Bio-Ontology and text: bridging the modeling gap. Bioinformatics 22, 2421–2429 (2006)

    Article  Google Scholar 

  32. Fromont, E., Quiniou, R., Cordier, M.-O.: Learning Rules from Multisource Data for Cardiac Monitoring. In: Miksch, S., Hunter, J., Keravnou, E.T. (eds.) AIME 2005. LNCS (LNAI), vol. 3581, pp. 484–493. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  33. Galperin, M.Y.: The Molecular Biology Database Collection: 2008 Update. Nucleic acids research 4, D2–D4 (2008)

    Google Scholar 

  34. Gevaert, O.: A Bayesian network integration framework for modeling biomedical data. Ph.D dissertation, Katholieke Universiteit Leuven (2008)

    Google Scholar 

  35. Hardoon, D.R., Shawe-Taylor, J.: Canonical Correlation Analysis: An Overview with Application to Learning Methods. Neural Computation 16, 2639–2664 (2004)

    Article  MATH  Google Scholar 

  36. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, Heidelberg (2009)

    MATH  Google Scholar 

  37. Hettich, R., Kortanek, K.O.: Semi-infinite programming: theory, methods, and applications. SIAM Review 35, 380–429 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  38. Hotelling, H.: Relations between two sets of variates. Biometrika 28, 321–377 (1936)

    MATH  Google Scholar 

  39. Hubert, L., Arabie, P.: Comparing partitions. Journal of Classification 2, 193–218 (1985)

    Article  Google Scholar 

  40. Hucka, M., Finney, A., Sauro, H.M., et al.: The systems biology markup language (SBML): a medium for representation and exchange of biochemical network models. Bioinformatics 19, 524–531 (2003)

    Article  Google Scholar 

  41. Jaccard, P.: Distribution de la flore alpine dans le bassin des Dranses et dans quelques régions voisines. Bulletin de la Société Vaudoise des Sciences Naturelles 37, 241–272 (1901)

    Google Scholar 

  42. Kaliski, J., Haglin, D., Roos, C., Terlaky, T.: Logarithmic barrier decomposition methods for semi-infinite programming. International Transactions in Operations Research 4, 285–303 (1997)

    Article  MATH  Google Scholar 

  43. Klami, A., Kaski, S.: Generative models that discover dependencies between two data sets. In: Proc. of IEEE Machine Learning for Signal Processing XVI, pp. 123–128 (2006)

    Google Scholar 

  44. Kloft, M., Brefeld, U., Laskov, P., Sonnenburg, S.: Non-sparse Multiple Kernel Learning. In: NIPS 2008 Workshop: Kernel Learning - Automatic Selection of Optimal Kernels (2008)

    Google Scholar 

  45. Krogh, A., Vedelsby, J.: Neural network ensembles, cross-validation and active learning. Advances in Neural Information Processing Systems 7, 231–238 (1995)

    Google Scholar 

  46. Lai, P.L., Fyfe, C.: Kernel and Nonlinear Canonical Correlation Analysis. International Journal of Neural Systems 10, 365–377 (2000)

    Google Scholar 

  47. Lanckriet, G.R.G., Cristianini, N., Jordan, M.I., Noble, W.S.: Kernel Methods in Computational Biology. MIT Press, Cambridge (2004)

    Google Scholar 

  48. Lanckriet, G.R.G., De Bie, T., Cristianini, N., Jordan, M.I., Noble, W.S.: A statistical framework for genomic data fusion. Bioinformatics 20, 2626–2635 (2004)

    Article  Google Scholar 

  49. Looy, S.V., Verplancke, T., Benoit, D., Hoste, E., Van Maele, G., De Turck, F., Decruyenaere, J.: A novel approach for prediction of tacrolimus blood concentration in liver transplantation patients in the intensive care unit through support vector regression. Critical Care 11, R83 (2007)

    Article  Google Scholar 

  50. Lloyd, J.: Foundations of Logic Programming. Springer, New York (1987)

    MATH  Google Scholar 

  51. Mika, S., Rätsch, G., Weston, J., Schölkopf, B.: Fisher discriminant analysis with kernels. In: IEEE Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop, pp. 41–48 (1999)

    Google Scholar 

  52. Mika, S., Weston, J., Schölkopf, B., Smola, A., Müller, K.-R.: Constructing Descriptive and Discriminative Nonlinear Features: Rayleigh Coefficients in Kernel Feature Spaces. IEEE Trans. on PAMI 25, 623–628 (2003)

    Google Scholar 

  53. Muggleton, S., De Raedt, L.: Inductive Logic Programming: Theory and methods. The Journal of Logic Programming 19/20, 629–680 (1994)

    Article  Google Scholar 

  54. Myers, J.W.: Learning bayesian network from incomplete data with stochastic search algorithms. In: Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, pp. 476–485. Morgan Kaufmann Publishers, San Francisco (1999)

    Google Scholar 

  55. Needham, C.J., Bradford, J.R., Bulpitt, A.J., Westhead, D.R.: A Primer on Learning in Bayesian Networks for Computational Biology. PLOS Computational Biology 3, 1409–1416 (2007)

    Article  Google Scholar 

  56. Nesterov, Y., Nemirovskij, A.: Interior-point polynomial algorithms in convex programming. SIAM Press, Philadelphia (1994)

    MATH  Google Scholar 

  57. Newman, M.E.J.: Modularity and community structure in networks. Proc. Natl. Acad. Sci. USA. 103, 8577–8582 (2006)

    Article  Google Scholar 

  58. Parzen, E.: On Estimation of a Probability Density Function and Mode. Annals of Mathematical Statistics 33, 1065–1076 (1962)

    Article  MATH  MathSciNet  Google Scholar 

  59. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, San Francisco (1988)

    Google Scholar 

  60. Pekalska, E., Haasdonk, B.: Kernel Discriminant Analysis for Positive Definite and Indefinite Kernels. IEEE Trans. TPAMI 31, 1017–1031 (2009)

    Google Scholar 

  61. Rakotomamonjy, A., Bach, F.R., Canu, S., Grandvalet, Y.: Simple MKL. Journal of Machine Learning Research 9, 2491–2521 (2008)

    MathSciNet  Google Scholar 

  62. Ramoni, M., Sebastiani, P.: Robust learning with missing data. Machine Learning 45, 147–170 (2000)

    Article  Google Scholar 

  63. Reemtsen, R.: Some other approximation methods for semi-infinite optimization problems. Jounral of Computational and Applied Mathematics 53, 87–108 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  64. Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics 20, 53–65 (1987)

    Article  MATH  Google Scholar 

  65. Santosh, K.C., Lamiroy, B., Ropers, J.-P.: Inductive Logic Programming for Symbol Recognition. In: Proc. of the 10th International Conference on Document Analysis and Recognition, pp. 1330–1334 (2009)

    Google Scholar 

  66. Schölkopf, B., Smola, A.: Learning with Kernels. MIT Press, Cambridge (2002)

    Google Scholar 

  67. Schölkopf, B., Smola, A., Müller, K.-R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)

    Article  Google Scholar 

  68. Scheinberg, K.: An Efficient Implementation of an Active Set Method for SVMs. Journal of Machine Learning Research 7, 2237–2257 (2006)

    MathSciNet  Google Scholar 

  69. Shapior, B.E., Hucka, M., Finney, A., Doyle, J.: MathSBML: a package for manipulating SBML-based biological models. Bioinformatics 20, 2829–2831 (2004)

    Article  Google Scholar 

  70. Sonnenburg, S., Rätsch, G., Schäfer, C., Schölkopf, B.: Large Scale Multiple Kernel Learning. Jounral of Machine Learning Research 7, 1531–1565 (2006)

    Google Scholar 

  71. Sonnenburg, S., Räetsch, G., Henschel, S., Widmer, C., Behr, J., Zien, A., de Bona, F., Binder, A., Gehl, C., Franc, V.: The SHOGUN Machine Learning Toolbox. Journal of Machine Learning Research 11, 1799–1802 (2010)

    Google Scholar 

  72. Stephen, W.: Primal-Dual Interior-Point Methods. SIAM Press, Philadelphia (1997)

    MATH  Google Scholar 

  73. Strehl, A., Ghosh, J.: Cluster Ensembles: A Knowledge Reuse Framework for Combining Multiple Partitions. Journal of Machine Learning Research 3, 583–617 (2002)

    Article  MathSciNet  Google Scholar 

  74. Sutton, C.D.: Classification and Regression Trees, Bagging, and Boosting. Handbook of Statistics 24, 303–329 (2005)

    Article  MathSciNet  Google Scholar 

  75. Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific Press, Singapore (2002)

    Book  MATH  Google Scholar 

  76. Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Processing Letters 9, 293–300 (1999)

    Article  MathSciNet  Google Scholar 

  77. Taton, R.: La premire note mathmatique de Gaspard Monge (juin 1769). Rev. Histoire Sci. Appl. 19, 143–149 (1966)

    Google Scholar 

  78. Taylor, C.F., Paton, N.W., Garwood, K.L., et al.: A systematic approach to modeling, capturing, and disseminating proteomics experimental data. Nature Biotechnology 21, 247–254 (2003)

    Article  Google Scholar 

  79. Tretyakov, K.: Methods of Genomic Data Fusion: An Overview. Technical Report, Institute of Computer Science, University of Tartu (2006)

    Google Scholar 

  80. Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (1999)

    Google Scholar 

  81. Vapnik, V.: Statistical Learning Theory. Wiley Interscience, New York (1998)

    MATH  Google Scholar 

  82. Vapnik, V., Chervonenkis, A.: On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications 16, 264–280 (1971)

    Article  MATH  Google Scholar 

  83. Wikipedia: Antikythera mechanism, http://en.wikipedia.org/wiki/Antikythera_mechanism

  84. Ye, J.P., Ji, S.W., Chen, J.H.: Multi-class Discriminant Kernel Learning via Convex Programming. Jounral of Machine Learning Research 9, 719–758 (2008)

    MathSciNet  Google Scholar 

  85. Yu, K., Ji, L., Zhang, X.G.: Kernel Nearest-Neighbor Algorithm. Neural Processing Letters 15, 147–156 (2002)

    Article  MATH  Google Scholar 

  86. Yu, S., De Moor, B., Moreau, Y.: Learning with heterogeneous data sets by Weighted Multiple Kernel Canonical Correlation Analysis. In: Proc. of the Machine Learning for Signal Processing XVII, pp. 81–86. IEEE, Los Alamitos (2007)

    Google Scholar 

  87. Yu, S., Falck, T., Tranchevent, L.-C., Daemen, A., Suykens, J.A.K., De Moor, B., Moreau, Y.: L2-norm multiple kernel learning and its application to biomedical data fusion. BMC Bioinformatics 11, 1–53 (2010)

    Article  MATH  Google Scholar 

  88. Yu, S., Liu, X.H., Glänzel, W., De Moor, B., Moreau, Y.: Optimized data fusion for K-means Laplacian Clustering. Bioinformatics 26, 1–9 (2010)

    Article  Google Scholar 

  89. Yu, S., Tranchevent, L.-C., De Moor, B., Moreau, Y.: Gene prioritization and clustering by multi-view text mining. BMC Bioinformatics 11, 1–48 (2010)

    Article  MATH  Google Scholar 

  90. Yu, S., Tranchevent, L.-C., Leach, S., De Moor, B., Moreau, Y.: Cross-species gene prioritization by genomic data fusion. Internal Report (2010) (submitted for publication)

    Google Scholar 

  91. Yu, S., Tranchevent, L.-C., Liu, X., Glänzel, W., Suykens, J.A.K., De Moor, B., Moreau, Y.: Optimized data fusion for kernel K-means clustering. Internal Report 08-200, ESAT-SISTA, K.U.Leuven, Lirias number: 242275 (2008) (submitted for publication)

    Google Scholar 

  92. Zheng, W.J.: Engineering Approaches Toward Biological Information Integration at the Systems Level. Current Bioinformatics 1, 85–93 (2006)

    Article  Google Scholar 

Download references

Authors
  1. Shi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Léon-Charles Tranchevent
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bart De Moor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yves Moreau
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Yu, S., Tranchevent, LC., De Moor, B., Moreau, Y. (2011). Introduction. In: Kernel-based Data Fusion for Machine Learning. Studies in Computational Intelligence, vol 345. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19406-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-19406-1_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-19405-4

  • Online ISBN: 978-3-642-19406-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation