Skip to main content
SpringerLink
Log in

Effective Kernelized Online Learning in Language Processing Tasks

  • Conference paper
Advances in Information Retrieval (ECIR 2014)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 8416))

Included in the following conference series:

Abstract

Kernel-based methods for NLP tasks have been shown to enable robust and effective learning, although their inherent complexity is manifest also in Online Learning (OL) scenarios, where time and memory usage grows along with the arrival of new examples. A state-of-the-art budgeted OL algorithm is here extended to efficiently integrate complex kernels by constraining the overall complexity. Principles of Fairness and Weight Adjustment are applied to mitigate imbalance in data and improve the model stability. Results in Sentiment Analysis in Twitter and Question Classification show that performances very close to the state-of-the-art achieved by batch algorithms can be obtained.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agarwal, A., Xie, B., Vovsha, I., Rambow, O., Passonneau, R.: Sentiment analysis of twitter data. In: Proceedings of LASM, pp. 30–38 (2011)

    Google Scholar 

  2. Baroni, M., Bernardini, S., Ferraresi, A., Zanchetta, E.: The wacky wide web: a collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation 43(3), 209–226 (2009)

    Article  Google Scholar 

  3. Basili, R., Zanzotto, F.M.: Parsing engineering and empirical robustness. Nat. Lang. Eng. 8(3), 97–120 (2002)

    Google Scholar 

  4. Cesa-Bianchi, N., Gentile, C.: Tracking the best hyperplane with a simple budget perceptron. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 483–498. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  5. Collins, M., Duffy, N.: Convolution kernels for natural language. In: Proceedings of Neural Information Processing Systems (NIPS 2001), pp. 625–632 (2001)

    Google Scholar 

  6. Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., Singer, Y.: Online passive-aggressive algorithms. Journal of Machine Learning Research 7, 551–585 (2006)

    MathSciNet  MATH  Google Scholar 

  7. Cristianini, N., Shawe-Taylor, J., Lodhi, H.: Latent semantic kernels. J. Intell. Inf. Syst. 18(2-3), 127–152 (2002)

    Article  Google Scholar 

  8. Croce, D., Moschitti, A., Basili, R.: Structured lexical similarity via convolution kernels on dependency trees. In: Proceedings of EMNLP, Scotland, UK (2011)

    Google Scholar 

  9. Davidov, D., Tsur, O., Rappoport, A.: Enhanced sentiment learning using twitter hashtags and smileys. In: COLING, pp. 241–249 (2010)

    Google Scholar 

  10. Dekel, O., Shalev-Shwartz, S., Singer, Y.: The forgetron: A kernel-based perceptron on a budget. SIAM J. Comput. 37(5), 1342–1372 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  11. Foster, J., Çetinoglu, Ö., Wagner, J., Roux, J.L., Hogan, S., Nivre, J., Hogan, D., van Genabith, J.: #hardtoparse: Pos tagging and parsing the twitterverse. In: Analyzing Microtext (2011)

    Google Scholar 

  12. Gönen, M., Alpaydin, E.: Multiple kernel learning algorithms. Journal of Machine Learning Research 12, 2211–2268 (2011)

    MATH  Google Scholar 

  13. Jaakkola, T., Meila, M., Jebara, T.: Maximum entropy discrimination. In: Solla, S.A., Leen, T.K., Müller, K.R. (eds.) NIPS, pp. 470–476. The MIT Press (1999)

    Google Scholar 

  14. Jansen, B.J., Zhang, M., Sobel, K., Chowdury, A.: Twitter power: Tweets as electronic word of mouth. J. Am. Soc. Inf. Sci. Technol. 60(11), 2169–2188 (2009)

    Article  Google Scholar 

  15. Joachims, T.: Learning to Classify Text Using Support Vector Machines. Kluwer Academic Publishers (2002)

    Google Scholar 

  16. Kouloumpis, E., Wilson, T., Moore, J.: Twitter sentiment analysis: The good the bad and the omg? In: ICWSM (2011)

    Google Scholar 

  17. Kwok, C.C., Etzioni, O., Weld, D.S.: Scaling question answering to the web. In: World Wide Web, pp. 150–161 (2001)

    Google Scholar 

  18. Landauer, T., Dumais, S.: A solution to plato’s problem: The latent semantic analysis theory of acquisition, induction and representation of knowledge. Psychological Review 104 (1997)

    Google Scholar 

  19. Li, X., Roth, D.: Learning question classifiers: the role of semantic information. Natural Language Engineering 12(3), 229–249 (2006)

    Article  Google Scholar 

  20. Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. In: Machine Learning, pp. 285–318 (1988)

    Google Scholar 

  21. Morik, K., Brockhausen, P., Joachims, T.: Combining statistical learning with a knowledge-based approach - a case study in intensive care monitoring. In: ICML, pp. 268–277. Morgan Kaufmann Publishers Inc., San Francisco (1999)

    Google Scholar 

  22. Moschitti, A., Pighin, D., Basili, R.: Tree kernels for semantic role labeling. Computational Linguistics 34 (2008)

    Google Scholar 

  23. Moschitti, A., Quarteroni, S., Basili, R., Manandhar, S.: Exploiting syntactic and shallow semantic kernels for question/answer classification. In: Proceedings of ACL 2007 (2007)

    Google Scholar 

  24. Orabona, F., Keshet, J., Caputo, B.: The projectron: a bounded kernel-based perceptron. In: Proceedings of ICML 2008, pp. 720–727. ACM, USA (2008)

    Google Scholar 

  25. Pak, A., Paroubek, P.: Twitter as a corpus for sentiment analysis and opinion mining. In: LREC (2010)

    Google Scholar 

  26. Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2(1-2), 1–135 (2008)

    Article  Google Scholar 

  27. Rosenblatt, F.: The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review 65(6), 386–408 (1958)

    Article  MathSciNet  Google Scholar 

  28. Sahlgren, M.: The Word-Space Model. Ph.D. thesis, Stockholm University (2006)

    Google Scholar 

  29. Salton, G., Wong, A., Yang, C.: A vector space model for automatic indexing. Communications of the ACM 18 (1975)

    Google Scholar 

  30. Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, New York (2004)

    Book  Google Scholar 

  31. Van Hulse, J., Khoshgoftaar, T.M., Napolitano, A.: Experimental perspectives on learning from imbalanced data. In: Proceedings of the ICML. ACM, USA (2007)

    Google Scholar 

  32. Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience (1998)

    Google Scholar 

  33. Wang, Z., Vucetic, S.: Online passive-aggressive algorithms on a budget. Journal of Machine Learning Research - Proceedings Track 9, 908–915 (2010)

    Google Scholar 

  34. Wilson, T., Kozareva, Z., Nakov, P., Ritter, A., Rosenthal, S., Stoyonov, V.: Semeval-2013 task 2: Sentiment analysis in twitter. In: Proceedings of the 7th International Workshop on Semantic Evaluation (2013)

    Google Scholar 

  35. Zanzotto, F.M., Pennacchiotti, M., Moschitti, A.: A machine learning approach to textual entailment recognition. Natural Language Engineering 15-04 (2009)

    Google Scholar 

  36. Zhang, D., Lee, W.S.: Question classification using support vector machines. In: Proceedings of SIGIR 2003, pp. 26–32. ACM, New York (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DICII, University of Roma, Tor Vergata, 00133, Roma, Italy

    Simone Filice

  2. DIE, University of Roma, Tor Vergata, 00133, Roma, Italy

    Giuseppe Castellucci

  3. DII, University of Roma, Tor Vergata, 00133, Roma, Italy

    Danilo Croce & Roberto Basili

Authors
  1. Simone Filice
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giuseppe Castellucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Danilo Croce
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roberto Basili
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    Maarten de Rijke  & Tom Kenter  & 

  2. Centrum Wiskunde en Informatica, Amsterdam, The Netherlands and Delft University of Technology, Delft, The Netherlands

    Arjen P. de Vries

  3. University of Illinois at Urbana-Champaign, Urbana, IL, USA

    ChengXiang Zhai

  4. University of Twente, Twente, The Netheralnds and Erasmus University Rotterdam, Rotterdam, The Netherlands

    Franciska de Jong

  5. SalesPredict, Haifa, Israel

    Kira Radinsky

  6. Microsoft Research, Cambridge, UK

    Katja Hofmann

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Filice, S., Castellucci, G., Croce, D., Basili, R. (2014). Effective Kernelized Online Learning in Language Processing Tasks. In: de Rijke, M., et al. Advances in Information Retrieval. ECIR 2014. Lecture Notes in Computer Science, vol 8416. Springer, Cham. https://doi.org/10.1007/978-3-319-06028-6_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-06028-6_29

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-06027-9

  • Online ISBN: 978-3-319-06028-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation