TISA: Topic Independence Scoring Algorithm
Textual analysis using machine learning is in high demand for a wide range of applications including recommender systems, business intelligence tools, and electronic personal assistants. Some of these applications need to operate over a wide and unpredictable array of topic areas, but current in-domain, domain adaptation, and multi-domain approaches cannot adequately support this need, due to their low accuracy on topic areas that they are not trained for, slow adaptation speed, or high implementation and maintenance costs.
To create a true domain-independent solution, we introduce the Topic Independence Scoring Algorithm (TISA) and demonstrate how to build a domain-independent bag-of-words model for sentiment analysis. This model is the best preforming sentiment model published on the popular 25 category Amazon product reviews dataset. The model is on average 89.6% accurate as measured on 20 held-out test topic areas. This compares very favorably with the 82.28% average accuracy of the 20 baseline in-domain models. Moreover, the TISA model is highly uniformly accurate, with a variance of 5 percentage points, which provides strong assurance that the model will be just as accurate on new topic areas. Consequently, TISAs models are truly domain independent. In other words, they require no changes or human intervention to accurately classify documents in never before seen topic areas.
KeywordsTopic Area Domain Adaptation Sentiment Analysis Label Data Weighted Vote
Unable to display preview. Download preview PDF.
- 1.Baccianella, S., Esuli, A., Sebastiani, F.: Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In: Proceedings of the Seventh Conference on International Language Resources and Evaluation (LREC 2010), Valletta, Malta. European Language Resources Association (ELRA) (2010)Google Scholar
- 2.Blitzer, J., Dredze, M., Pereira, F.: Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In: Annual Meeting of Association For Computational Linguistics, vol. 45, pp. 440–447 (2007)Google Scholar
- 3.Blitzer, J., Kakade, S., Foster, D.P.: Domain adaptation with coupled subspaces. Journal of Machine Learning Research - Proceedings Track 15, 173–181 (2011)Google Scholar
- 4.Chen, M., Weinberger, K.Q., Blitzer, J.: Co-training for domain adaptation. In: NIPS 2011, pp. 2456–2464 (2011)Google Scholar
- 5.Fellbaum, C.: Wordnet. In: Theory and Applications of Ontology: Computer Applications, pp. 231–243 (2010)Google Scholar
- 7.Martineau, J.: Identifying and Isolating Text Classification Signals from Domain and Genre Noise for Sentiment Analysis. PhD thesis, University of Maryland, Baltimore County, Computer Science and Electrical Engineering (December 2011)Google Scholar
- 8.Martineau, J., Finin, T., Joshi, A., Patel, S.: Improving binary classification on text problems using differential word features. In: Proceeding of the 18th ACM Conference on Information and Knowledge Management, pp. 2019–2024. ACM (2009)Google Scholar
- 9.Paltoglou, G., Thelwall, M.: A study of Information Retrieval weighting schemes for sentiment analysis. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 1386–1395. Association for Computational Linguistics (2010)Google Scholar
- 10.Pan, S., Ni, X., Sun, J., Yang, Q., Chen, Z.: Cross-domain sentiment classification via spectral feature alignment. In: Proceedings of the 19th International Conference on World Wide Web, pp. 751–760. ACM (2010)Google Scholar
- 11.Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up?: sentiment classi cation using machine learning techniques. In: Proceedings of the ACL 2002 Conference on Empirical Methods in Natural Language Processing, vol. 10, pp. 79–86. Association for Computational Linguistics, Morristown (2002)Google Scholar
- 12.Strapparava, C., Valitutti, A.: Wordnet-affect: an affective extension of wordnet. In: Proceedings of LREC, vol. 4, pp. 1083–1086 (2004)Google Scholar