Advertisement

Language Resources and Evaluation

, Volume 47, Issue 4, pp 1049–1088 | Cite as

Bootstrapping polarity classifiers with rule-based classification

  • Michael WiegandEmail author
  • Manfred Klenner
  • Dietrich Klakow
Original Paper

Abstract

In this article, we examine the effectiveness of bootstrapping supervised machine-learning polarity classifiers with the help of a domain-independent rule-based classifier that relies on a lexical resource, i.e., a polarity lexicon and a set of linguistic rules. The benefit of this method is that though no labeled training data are required, it allows a classifier to capture in-domain knowledge by training a supervised classifier with in-domain features, such as bag of words, on instances labeled by a rule-based classifier. Thus, this approach can be considered as a simple and effective method for domain adaptation. Among the list of components of this approach, we investigate how important the quality of the rule-based classifier is and what features are useful for the supervised classifier. In particular, the former addresses the issue in how far linguistic modeling is relevant for this task. We not only examine how this method performs under more difficult settings in which classes are not balanced and mixed reviews are included in the data set but also compare how this linguistically-driven method relates to state-of-the-art statistical domain adaptation.

Keywords

Polarity classification Sentiment analysis Bootstrapping methods Feature engineering Text classification 

Notes

Acknowledgments

This article is an extension of (Wiegand and Klakow 2010). We refer the reader to the section on related work for the full list of extensions. This work was funded by the German Federal Ministry of Education and Research (Software-Cluster) under grant no. ”01IC10S01“ and the Cluster of Excellence for Multimodal Computing and Interaction. The authors would like to thank Ivan Titov for running his statistical domain adaptation method on the data used in this article and Sabrina Wilske for insightful comments.

References

  1. Akkaya, C., Wiebe, J., & Mihalcea, R. (2009). Subjectivity word sense disambiguation. In Proceedings of the conference on empirical methods in natural language processing (EMNLP) (pp. 190–199). Singapore.Google Scholar
  2. Akkaya, C., Wiebe, J., Conrad, A., & Mihalcea, R. (2011). Improving the impact of subjectivity word sense disambiguation on contextual opinion analysis. In Proceedings of the conference on computational natural language learning (pp. 87–96). Portland, OR, USA.Google Scholar
  3. Andreevskaia, A., & Bergler, S. (2008). When specialists and generalists work together: Overcoming domain dependence in sentiment tagging. In Proceedings of the annual meeting of the association for computational linguistics: Human language technologies (ACL/HLT) (pp. 290–298), Columbus, OH, USA.Google Scholar
  4. Aue, A., & Gamon, M. (2005). Customizing sentiment classifiers to new domains: A case study. In Proceedings of recent advances in natural language processing (RANLP). Borovets, Bulgaria.Google Scholar
  5. Baccianella, S., Esuli, A., & Sebastiani, F. (2010). SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In Proceedings of the conference on language resources and evaluation (LREC) (pp. 2200–2204). Valletta.Google Scholar
  6. Balamurali, A., Joshi, A., & Bhattacharyya, P. (2011). Harnessing wordnet senses for supervised sentiment classification. In Proceedings of the conference on empirical methods in natural language processing (EMNLP) (pp. 1081–1091). Edinburgh, Scotland, UK.Google Scholar
  7. Becker, I., & Aharonson, V. (2010). Last but definitely not least: On the role of the last sentence in automatic polarity-classification. In Proceedings of the annual meeting of the association for computational linguistics (ACL) (pp. 331–335). Uppsala, Sweden.Google Scholar
  8. Beineke, P., Hastie, T., & Vaithyanathan, S. (2004). The sentimental factor: Improving review classification via human-provided information. In Proceedings of the annual meeting of the association for computational linguistics (ACL) (pp. 263–270). Barcelona, Spain.Google Scholar
  9. Blitzer, J., McDonald, R., & Pereira, F. (2006). Domain adaptation with structural correspondence learning. In Proceedings of the conference on empirical methods in natural language processing (EMNLP) (pp. 120–128). Sydney, Australia.Google Scholar
  10. Blitzer, J., Dredze, M., & Pereira, F. (2007). Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the annual meeting of the association for computational linguistics (ACL) (pp. 440–447). Prague, Czech Republic.Google Scholar
  11. Choi, Y., & Cardie, C. (2008). Learning with compositional semantics as structural inference for subsentential sentiment analysis. In Proceedings of the conference on empirical methods in natural language processing (EMNLP) (pp. 793–801). Waikiki, HI, USA.Google Scholar
  12. Council, I., McDonald, R., & Velikovich, L. (2010). What’s great and what’s not: Learning to classify the scope of negation for improved sentiment analysis. In Proceedings of the workshop on negation and speculation in natural language processing (pp. 51–59). Uppsala, Sweden.Google Scholar
  13. Dasgupta, S., & Ng, V. (2009). Mine the easy, classify the hard: A semi-supervised approach to automatic sentiment classification. In Proceedings of the joint conference of the annual meeting of the association for computational linguistics and the international joint conference on natural language processing of the Asian federation of natural language processing (ACL/IJCNLP) (pp. 701–709). Suntec, Singapore.Google Scholar
  14. Esuli, A., & Sebastiani, F. (2006). Derminining term subjectivity and term orientation for opinion mining. In Proceedings of the conference on European chapter of the association for computational linguistics (EACL) (pp. 193–200). Trento, Italy.Google Scholar
  15. Esuli, A., & Sebastiani, F. (2007). PageRanking wordnet synsets: An application to opinion mining. In Proceedings of the annual meeting of the association for computational linguistics (ACL) (pp. 424–431). Prague, Czech Republic.Google Scholar
  16. Hatzivassiloglou, V., & McKeown, K. R. (1997). Predicting the semantic orientation of adjectives. In Proceedings of the conference on european chapter of the association for computational linguistics (EACL) (pp. 174–181). Madrid, Spain.Google Scholar
  17. Hatzivassiloglou, V., & Wiebe, J. (2000). Effects of adjective orientation and gradability on sentence subjectivity. In Proceedings of the international conference on computational linguistics (COLING), Saarbrücken (pp. 299–305). Germany.Google Scholar
  18. Hu, M., & Liu, B. (2004). Mining and summarizing customer reviews. In Proceedings of the ACM SIGKDD international conference on knowledge discovery & data mining (KDD) (pp. 168–177). Seattle, WA, USA.Google Scholar
  19. Huang, Y., & Lowe, H. J. (2007). A novel hybrid approach to automated negation detection in clinical radiology reports. Journal of the American Medical Infomatics Association (JAMIA), 14, 304–311.CrossRefGoogle Scholar
  20. Jason, G. (1988). Hedging as a fallacy of language. Informal Logic, 10(3), 169–175.Google Scholar
  21. Jia, L., Yu, C., & Meng, W. (2009). The effect of negation on sentiment analysis and retrieval effectiveness. In Proceedings of the conference on information and knowledge management (CIKM) (pp. 1827–1830). Singapore.Google Scholar
  22. Joachims, T. (1999). Making large-scale SVM learning practical. In: Schölkopf, B., Burges, C., Smola, A. (Eds.) Advances in Kernel methods—Support vector learning. Cambridge MA: MIT Press.Google Scholar
  23. Kennedy, A., & Inkpen, D. (2006). Sentiment classification of movie reviews using contextual valence shifters. Computational Intelligence, 22(2), 110–125.CrossRefGoogle Scholar
  24. Klenner, M. (2009). Süsse Beklommenheit und schmerzvolle Ekstase. Automatische Sentimentanalyse in den Werken von Eduard von Keyserling. In Proceedings of the biennial GSCL-conference (pp. 91–97). Potsdam, Germany.Google Scholar
  25. Klenner, M., Fahrni, A., & Petrakis, S. (2009a). PolArt: A robust tool for sentiment analysis. In Proceedings of the nordic conference on computational linguistics (NoDaLiDa) (pp. 235–238). Odense, Denmark.Google Scholar
  26. Klenner, M., Petrakis, S., & Fahrni, A. (2009b). Robust Compositional Polarity Classification. In Proceedings of Recent Advances in Natural Language Processing (RANLP) (pp. 180–184). BorovetsGoogle Scholar
  27. Melville, P., Gryc, W., & Lawrence, R. D. (2009). Text analysis of blogs by combining lexical knowledge with text classification. In Proceedings of the ACM SIGKDD conference on knowledge discovery and data mining (KDD) (pp. 1275–1283). Paris, France.Google Scholar
  28. Min, H. J., & Park, J. C. (2011). Detecting and blocking false sentiment propagation. In Proceedings of the international joint conference on natural language processing (IJCNLP) (pp. 354–362). Chian Mai, Thailand.Google Scholar
  29. Moilanen, K., & Pulman, S. (2007). Sentiment construction. In Proceedings of recent advances in natural language processing (RANLP). Borovets, Bulgaria.Google Scholar
  30. Morante, R. (2010). Descriptive analysis of negation cues in biomedical texts. In Proceedings of the conference on language resources and evaluation (LREC) (pp. 1429–1436). Valletta, Malta.Google Scholar
  31. Neviarouskaya, A., Prendinger, H., & Ishizuka, M. (2009). Semantically distinct verb classes involved in sentiment analysis. In Proceedings of the IADIS international conference on applied computing (pp. 27–34). Rome, Italy.Google Scholar
  32. Ng, V., Dasgupta, S., & Arifin, S. M. N. (2006). Examining the role of linguistic knowledge sources in the automatic identification and classification of reviews. In Proceedings of the international conference on computational linguistics and annual meeting of the association for computational linguistics (COLING/ACL) (pp. 611–618). Sydney, Australia.Google Scholar
  33. Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the conference on empirical methods in natural language processing (EMNLP) (pp. 79–86). Philadelphia, PA, USA.Google Scholar
  34. Polanyi, L., & Zaenen, A. (2006). Contextual valence shifters. In Computing attitude and affect in text: Theory and applications (pp. 1–10). Springer.Google Scholar
  35. Porter, M. F. (1980). An algorithm for suffix stripping. Program, 14(3), 130–137.CrossRefGoogle Scholar
  36. Prabowo, R., & Thelwall, M. (2009). Sentiment analysis: A combined approach. Journal of Informetrics, 3(1), 143–157.CrossRefGoogle Scholar
  37. Qiu, L., Zhang, W., Hu, C., & Zhao, K. (2009). SELC: A self-supervised model for sentiment classification. In Proceedings of the conference on information and knowledge management (CIKM) (pp. 929–936). Hong Kong, China.Google Scholar
  38. Rao, D., & Ravichandran, D. (2009). Semi-supervised polarity lexicon induction. In Proceedings of the conference on European chapter of the association for computational linguistics (EACL) (pp. 675–682). Athens, Greece.Google Scholar
  39. Salvetti, F., Reichenbach, C., & Lewis, S. (2006). Opinion Polarity Identification of Movie Reviews. In Shanahan, J. G., Qu, Y., & Wiebe, J. (Eds.) Computing Attitude and Affect in Text: Theory and Applications (pp. 303–316). Berlin: Springer.CrossRefGoogle Scholar
  40. Schmid, H. (1994). Probabilistic part-of-speech tagging using decision trees. In Proceedings of the international conference on new methods in language processing (pp. 44–49). Manchester, UK.Google Scholar
  41. Shaikh, M. A. M., Prendinger, H., & Ishizuka, M. (2007). Assessing sentiment of text by semantic dependency and contextual valence analysis. In Proceedings of the international conference of affective computing and intelligent interface (ACII)(pp. 191–202). Lisbon, Portugal: Springer.Google Scholar
  42. Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011). Lexicon-based methods for sentiment analysis. Computational Linguistics, 37(2), 267–307.CrossRefGoogle Scholar
  43. Tan, S., Wang, & Y., Cheng, X. (2008). Combining learn-based and lexicon-based techniques for sentiment detection without using labeled examples. In Proceedings of the ACM special interest group on information retrieval (SIGIR) (pp. 744–745). Singapore.Google Scholar
  44. Tan, S., Cheng, X., Wang, Y., & Xu, H. (2009). Adapting naive bayes to domain adaptation for sentiment analysis. In Proceedings of the European conference in information retrieval (ECIR) (pp. 337–349). Toulouse, France.Google Scholar
  45. Titov, I. (2011). Domain adaptation by constraining inter-domain variability of latent feature representation. In Proceedings of the annual meeting of the association for computational linguistics (ACL) (pp. 62–71). Portland, OR, USA.Google Scholar
  46. Turney, P., & Littman, M. (2003). Measuring praise and criticism: Inference of semantic orientation from association. In Proceedings of ACM transactions on information systems (TOIS) (pp. 315–346).Google Scholar
  47. Velikovich, L., Blair-Goldensohn, S., Hannan, K., & McDonald, R. (2010). The viability of web-derived polarity lexicons. In Proceedings of the human language technology conference of the North American chapter of the ACL (HLT/NAACL) (pp. 777–785). Los Angeles, CA, USA.Google Scholar
  48. Wiebe, J., & Mihalcea, R. (2006). Word sense and subjectivity. In Proceedings of the international conference on computational linguistics and annual meeting of the association for computational linguistics (COLING/ACL) (pp. 1065–1072). Syndney, Australia.Google Scholar
  49. Wiebe, J., & Riloff, E. (2005). Creating subjective and objective sentence classifiers from unannotated texts. In Proceedings of the international conference on intelligent text processing and computational linguistics (CICLing) (pp. 486–497). Mexico City, Mexico.Google Scholar
  50. Wiegand, M., & Klakow, D. (2009a). Predictive features in semi-supervised learning for polarity classification and the role of adjectives. In Proceedings of the Nordic conference on computational linguistics (NoDaLiDa) (pp. 198–205). Odense, Denmark.Google Scholar
  51. Wiegand, M., & Klakow, D. (2009b). The role of knowledge-based features in polarity classification at sentence level. In Proceedings of the international FLAIRS conference (FLAIRS) (pp. 296–301). Sanibel Island, FL, USA.Google Scholar
  52. Wiegand, M., & Klakow, D. (2010). Bootstrapping supervised machine-learning polarity classifiers with rule-based classification. In Proceedings of the workshop on computational approaches to subjectivity and sentiment analysis (WASSA) (pp. 59–66). Lisbon, Portugal.Google Scholar
  53. Wiegand, M., Balahur, A., Roth, B., Klakow, D., & Montoyo, A. (2010). A survey on the role of negation in sentiment analysis. In Proceedings of the workshop on negation and speculation in natural language processing (pp. 60–68). Uppsala, Sweden.Google Scholar
  54. Wilson, T., Wiebe, J., & Hoffmann, P. (2009). Recognizing contextual polarity: An exploration for phrase-level analysis. Computational Linguistics, 35, 399–433.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Michael Wiegand
    • 1
    Email author
  • Manfred Klenner
    • 2
  • Dietrich Klakow
    • 1
  1. 1.Spoken Language SystemsSaarland UniversitySaarbrückenGermany
  2. 2.Institute of Computational LinguisticsZürich UniversityZürichSwitzerland

Personalised recommendations