Scalable Aspect-Based Summarization in the Hadoop Environment

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 654)


In the present-day scenario, selecting a good product is a cumbersome process. The reviews from the shopping sites may confuse the user while purchasing the product. It becomes hard for the customers to go through all the reviews, even when they read they may get into a baffling state. Some consumers may like to buy the best product based on its features and its extra comfort. Meanwhile, the size of the datasets for analysis process is huge which cannot be handled by traditional systems. In order to handle the large datasets, we are proposing a parallel approach using Hadoop cluster for extracting the feature and opinion. Then by using online sentiment dictionary and interaction information method, predict the sentiments followed by summarization using clustering. After classifying each opinion words, our summarization system generates an easily readable summary for that particular product based on aspects.


Sentiment summarization Aspect Hadoop Mapreduce 


  1. 1.
    Baccianella, S., Esuli, A., Sebastiani, F.: SentiWordNet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. InLREC 10, 2200–2204 (2010)Google Scholar
  2. 2.
    Blair-Goldensohn, S., Hannan, K., McDonald, R., Neylon, T., Reis, G.A., Reynar, J.: Building a sentiment summarizer for local service reviews. In: WWW Workshop on NLP in the Information Explosion Era, vol. 14 (2008)Google Scholar
  3. 3.
    Briscoe, T., Carroll, J., Watson, R.: The second release of the RASP system. In: Proceedings of the COLING/ACL on Interactive presentation sessions, pp. 77–80. Association for Computational Linguistics (2006)Google Scholar
  4. 4.
    Carenini, G., Cheung, J.C.K.: Extractive versus NLG-based abstractive summarization of evaluative text: the effect of corpus controversiality. In: Proceedings of the 5th International Natural Language Generation Conference, pp. 33–41. Association for Computational Linguistics (2008)Google Scholar
  5. 5.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  6. 6.
    Ganesan, K., Zhai, C., Han, J.: Opinosis: a graph-based approach to abstractive summarization of highly redundant opinions. In: Proceedings of the 23rd international conference on computational linguistics, pp. 340–348. Association for Computational Linguistics (2010)Google Scholar
  7. 7.
    Ganesan, K., Zhai, C., Viegas, E.: Micropinion generation: an unsupervised approach to generating ultra-concise summaries of opinions. In: Proceedings of the 21st International Conference on World Wide Web, pp. 869–878. ACM (2012)Google Scholar
  8. 8.
    Gerani, S., Mehdad, Y., Carenini, G., Ng, R.T., Nejat, B.: Abstractive summarization of product reviews using discourse structure. In: Proceedings of EMNLP (2014)Google Scholar
  9. 9.
    Gindl, S., Weichselbraun, A., Scharl, A.: Cross-domain contextualisation of sentiment lexicons (2010)Google Scholar
  10. 10.
    Hamid, F., Tarau, P.: Anti-Summaries: enhancing graph-based techniques for summary extraction with sentiment polarity. In: Computational Linguistics and Intelligent Text Processing, pp. 375–389. Springer International Publishing (2015)Google Scholar
  11. 11.
    Hu, M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and data mining, pp. 168–177. ACM (2004)Google Scholar
  12. 12.
    Kansal, H., Toshniwal, D.: Aspect based summarization of context-based opinion words. Proc. Comp. Sci. 35, 166–175 (2014)CrossRefGoogle Scholar
  13. 13.
    Kim, H.D., Ganesan, K., Sondhi, P., Zhai, C.: Comprehensive review of opinion summarization (2011)Google Scholar
  14. 14.
    Liu, B., Zhang, L.: A survey of opinion mining and sentiment analysis. In: Mining text data, pp. 415–463. Springer US (2012)Google Scholar
  15. 15.
    Liu, B., Blasch, E., Chen, Y., Shen, D., Chen, G.: Scalable sentiment classification for big data analysis using naive bayes classifier. In: Big Data, 2013 IEEE International Conference on IEEE, pp. 99–104 (2013)Google Scholar
  16. 16.
    Liu, J., Cao, Y., Lin, C.Y., Huang, Y., Zhou, M. Low-Quality product review detection in opinion summarization. In: EMNLP-CoNLL, pp. 334–342 (2007)Google Scholar
  17. 17.
    Lu, Y., Zhai, C., Sundaresan, N.: Rated aspect summarization of short comments. In: Proceedings of the 18th International Conference on World Wide Web, pp. 131–140. ACM (2009)Google Scholar
  18. 18.
    Marrese-Taylor, E., Velásquez, J.D., Bravo-Marquez, F.: A novel deterministic approach for aspect-based opinion mining in tourism products reviews. Expert Syst. Appl. 41(17), 7764–7775 (2014)CrossRefGoogle Scholar
  19. 19.
    McAuley, J., Leskovec, J.: Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM Conference on Recommender Systems, pp. 165–172. ACM (2013)Google Scholar
  20. 20.
    Moghaddam, S.: Beyond sentiment analysis: mining defects and improvements from customer feedback. In: Advances in Information Retrieval, pp. 400–410. Springer International Publishing (2015)Google Scholar
  21. 21.
    Mukherjee, A., Liu, B.: Aspect extraction through semi-supervised modeling. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers, vol. 1, pp. 339–348. Association for Computational Linguistics (2012)Google Scholar
  22. 22.
    Pan, S.J., Ni, X., Sun, J.T., Yang, Q., Chen, Z.: Cross-domain sentiment classification via spectral feature alignment. In: Proceedings of the 19th International Conference on World Wide Web, pp. 751–760. ACM (2010)Google Scholar
  23. 23.
    Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? Sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, vol. 10, pp. 79–86. Association for Computational Linguistics (2002)Google Scholar
  24. 24.
    Shimada, K., Tadano, R., Endo, T.: Multi-aspects review summarization with objective information. Proc. Soc. Behav. Sci. 27, 140–149 (2011)CrossRefGoogle Scholar
  25. 25.
    Turney, P.D.: Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 417–424. Association for Computational Linguistics (2002)Google Scholar
  26. 26.
    Van de Cruys, T.: Two multivariate generalizations of pointwise mutual information. In: Proceedings of the Workshop on Distributional Semantics and Compositionality, pp. 16–20. Association for Computational Linguistics (2011)Google Scholar
  27. 27.
    Wang, D., Liu, Y.: Opinion summarization on spontaneous conversations. Comput. Speech Lang. 34(1), 61–82 (2015)CrossRefGoogle Scholar
  28. 28.
    Yu, J., Zha, Z.J., Wang, M., Chua, T.S.: Aspect ranking: identifying important product aspects from online consumer reviews. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 1496–1505. Association for Computational Linguistics (2011)Google Scholar
  29. 29.
    Yuan, X., Sa, N., Begany, G., Yang, H.: What users prefer and why: a user study on effective presentation styles of opinion summarization. In: Human-Computer Interaction–INTERACT 2015, pp. 249–264. Springer International Publishing (2015)Google Scholar
  30. 30.
    Zhu, L., Gao, S., Pan, S.J., Li, H., Deng, D., Shahabi, C.: The Pareto principle is everywhere: finding informative sentences for opinion summarization through leader detection. In: Recommendation and Search in Social Networks, pp. 165–187. Springer International Publishing (2015)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringA.V.C. College of EngineeringMayiladuthuraiIndia
  2. 2.Department of Computer Science and EngineeringNational Institute of TechnologyTiruchirapalliIndia

Personalised recommendations