An Empirical Study of Massively Parallel Bayesian Networks Learning for Sentiment Extraction from Unstructured Text

  • Wei Chen
  • Lang Zong
  • Weijing Huang
  • Gaoyan Ou
  • Yue Wang
  • Dongqing Yang
Conference paper

DOI: 10.1007/978-3-642-20291-9_47

Part of the Lecture Notes in Computer Science book series (LNCS, volume 6612)
Cite this paper as:
Chen W., Zong L., Huang W., Ou G., Wang Y., Yang D. (2011) An Empirical Study of Massively Parallel Bayesian Networks Learning for Sentiment Extraction from Unstructured Text. In: Du X., Fan W., Wang J., Peng Z., Sharaf M.A. (eds) Web Technologies and Applications. APWeb 2011. Lecture Notes in Computer Science, vol 6612. Springer, Berlin, Heidelberg

Abstract

Extracting sentiments from unstructured text has emerged as an important problem in many disciplines, for example, to mine on-line opinions from the Internet. Many algorithms have been applied to solve this problem. Most of them fail to handle the large scale web data. In this paper, we present a parallel algorithm for BN(Bayesian Networks) structure leaning from large-scale dateset by using a MapReduce cluster. Then, we apply this parallel BN learning algorithm to capture the dependencies among words, and, at the same time, finds a vocabulary that is efficient for the purpose of extracting sentiments. The benefits of using MapReduce for BN structure learning are discussed. The performance of using BN to extract sentiments is demonstrated by applying it to real web blog data. Experimental results on the web data set show that our algorithm is able to select a parsimonious feature set with substantially fewer predictor variables than in the full data set and leads to better predictions about sentiment orientations than several usually used methods.

Keywords

Sentiment Analysis Bayesian Networks MapReduce Cloud Computing Opinion Mining 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Wei Chen
    • 1
  • Lang Zong
    • 1
  • Weijing Huang
    • 1
  • Gaoyan Ou
    • 1
  • Yue Wang
    • 1
  • Dongqing Yang
    • 1
  1. 1.Key Laboratory of High Confidence Software Technologies, (Ministry of Education), School of EECSPeking UniversityBeijingChina

Personalised recommendations