Abstract
Prior knowledge, in the form of simple advice rules, can greatly speed up convergence in learning algorithms. Online learning methods predict the label of the current point and then receive the correct label (and learn from that information). The goal of this work is to update the hypothesis taking into account not just the label feedback, but also the prior knowledge, in the form of soft polyhedral advice, so as to make increasingly accurate predictions on subsequent examples. Advice helps speed up and bias learning so that generalization can be obtained with less data. Our passive-aggressive approach updates the hypothesis using a hybrid loss that takes into account the margins of both the hypothesis and the advice on the current point. Encouraging computational results and loss bounds are provided.
Chapter PDF
Similar content being viewed by others
Keywords
- Loss Function
- Mycobacterium Tuberculosis Complex
- Mycobacterial Intersperse Repetitive Unit
- Loss Bound
- Advice Vector
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Aminian, M., Shabbeer, A., Bennett, K.P.: A conformal Bayesian network for classification of Mycobacterium tuberculosis complex lineages. BMC Bioinformatics, 11(suppl. 3), S4 (2010)
Asuncion, A., Newman, D.J.: UCI machine learning repository (2007)
Brudey, K., Driscoll, J.R., Rigouts, L., Prodinger, W.M., Gori, A., Al-Hajoj, S.A., Allix, C., Aristimuño, L., Arora, J., Baumanis, V., et al.: Mycobacterium tuberculosis complex genetic diversity: Mining the fourth international spoligotyping database (spoldb 4) for classification, population genetics and epidemiology. BMC Microbiology 6(1), 23 (2006)
Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., Singer, Y.: Online passive-aggressive algorithms. J. of Mach. Learn. Res. 7, 551–585 (2006)
Freund, Y., Schapire, R.E.: Large margin classification using the perceptron algorithm. Mach. Learn. 37(3), 277–296 (1999)
Fung, G., Mangasarian, O.L., Shavlik, J.W.: Knowledge-based support vector classifiers. In: Becker, S., Thrun, S., Obermayer, K. (eds.) NIPS, vol. 15, pp. 521–528 (2003)
Gagneux, S., Small, P.M.: Global phylogeography of Mycobacterium tuberculosis and implications for tuberculosis product development. The Lancet Infectious Diseases 7(5), 328–337 (2007)
Langford, J., Li, L., Zhang, T.: Sparse online learning via truncated gradient. J. Mach. Learn. Res. 10, 777–801 (2009)
Lauer, F., Bloch, G.: Incorporating prior knowledge in support vector machines for classification: A review. Neurocomp. 71(7-9), 1578–1594 (2008)
Li, Y., Long, P.M.: The relaxed online maximum margin algorithm. Mach. Learn. 46(1/3), 361–387 (2002)
Mangasarian, O.L.: Nonlinear Programming. McGraw-Hill, New York (1969)
Schölkopf, B., Simard, P., Smola, A., Vapnik, V.: Prior knowledge in support vector kernels. In: NIPS, vol. 10, pp. 640–646 (1998)
Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization Optimization and Beyond. MIT Press, Cambridge (2001)
Shabbeer, A., Cowan, L., Driscoll, J.R., Ozcaglar, C., Vandenberg, S.L., Yener, B., Bennett, K.P.: TB-Lineage: An online tool for classification and analysis of strains of Mycobacterium tuberculosis Complex (2010) (unpublished manuscript)
Towell, G.G., Shavlik, J.W.: Knowledge-based artificial neural networks. AIJ 70(1-2), 119–165 (1994)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer-Verlag (2000)
Zinkevich, M.: Online convex programming and generalized infinitesimal gradient ascent. In: Proc. 20th Int. Conf. on Mach. Learn, ICML 2003 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kunapuli, G., Bennett, K.P., Shabbeer, A., Maclin, R., Shavlik, J. (2010). Online Knowledge-Based Support Vector Machines. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2010. Lecture Notes in Computer Science(), vol 6322. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15883-4_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-15883-4_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15882-7
Online ISBN: 978-3-642-15883-4
eBook Packages: Computer ScienceComputer Science (R0)