An Empirical Study of Massively Parallel Bayesian Networks Learning for Sentiment Extraction from Unstructured Text
- Cite this paper as:
- Chen W., Zong L., Huang W., Ou G., Wang Y., Yang D. (2011) An Empirical Study of Massively Parallel Bayesian Networks Learning for Sentiment Extraction from Unstructured Text. In: Du X., Fan W., Wang J., Peng Z., Sharaf M.A. (eds) Web Technologies and Applications. APWeb 2011. Lecture Notes in Computer Science, vol 6612. Springer, Berlin, Heidelberg
Extracting sentiments from unstructured text has emerged as an important problem in many disciplines, for example, to mine on-line opinions from the Internet. Many algorithms have been applied to solve this problem. Most of them fail to handle the large scale web data. In this paper, we present a parallel algorithm for BN(Bayesian Networks) structure leaning from large-scale dateset by using a MapReduce cluster. Then, we apply this parallel BN learning algorithm to capture the dependencies among words, and, at the same time, finds a vocabulary that is efficient for the purpose of extracting sentiments. The benefits of using MapReduce for BN structure learning are discussed. The performance of using BN to extract sentiments is demonstrated by applying it to real web blog data. Experimental results on the web data set show that our algorithm is able to select a parsimonious feature set with substantially fewer predictor variables than in the full data set and leads to better predictions about sentiment orientations than several usually used methods.
KeywordsSentiment Analysis Bayesian Networks MapReduce Cloud Computing Opinion Mining
Unable to display preview. Download preview PDF.