Improving Topical Social Media Sentiment Analysis by Correcting Unknown Words Automatically

  • Rayner AlfredEmail author
  • Rui Wen Teoh
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 937)


In the digital world, social media has become one of the most popular communication mediums that allow users to share their views on various topics in their social network. For example, Twitter users are allowed to share their thoughts on various topic by sending tweets with a maximum length of 140 characters. Hence, social media driven information contains opinions and sentiments on various topics of interest which are extremely useful for companies to design marketing strategies. Sentiment Analysis is widely used to assist people to understand the massive amount of data available online and identify the polarity of the topical based social media opinions. However, social media platforms’ users come from all over the world and have variation in terms of informal language and short notation used on social media platforms. Therefore, the identification on the polarity of topical social media has become more challenging and the accuracy on the polarity of topical social media opinions might be influenced. This paper investigates the effectiveness of applying different spelling correction algorithms, such as Levenshtein distance and Peter Norvig’s algorithm for spelling correction of unknown words found in social media such as Twitter, before carrying out sentiment analysis. The evaluation of spelling correction algorithms on sentiment analysis is carried out by comparing the polarities of manually annotated tweets with the polarities obtained from the sentiment analysis algorithm. Based on the results obtained, there are slight improvements in term of percentage of matched polarity, where 1.6% improvement by using the Levenshtein distance-based algorithm and 2.0% improvement by using the Peter Norvig’s algorithm.


Sentiment analysis Informal language Spelling correction Polarity 


  1. 1.
    Fazal, M.K., Aurangzeb, K., Muhammad, Z.A., Shakeel, A.: Context-aware spelling corrector for sentiment analysis. MAGNT Res. Rep. 2(5), 1–10 (2014). ISSN 1444-8939Google Scholar
  2. 2.
    Ayushi, D., Thish, G., Vasudeva, V.: Manwitter sentiment analysis: the good, the bad, and the neutral. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Denver, Colorado, 4–5 June 2015, pp. 520–526. Association for Computational Linguistics (2015)Google Scholar
  3. 3.
    Read, J.: Using emoticons to reduce dependency in machine learning techniques for sentiment classification. In: Proceedings of the ACL Student Research Workshop (2005), pp 43–48 (2005)Google Scholar
  4. 4.
    Baccianella, S.A., Esuli, F.S.: SENTIWORDNET 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In: Proceedings of LREC (2010)Google Scholar
  5. 5.
    Liu, B., Li, S., Lee, W.S., Yu, P.S.: Text classification by labeling words. In: Proceedings of the National Conference on Artificial Intelligence, pp 425–430. AAAI Press/MIT Press, Menlo Park/Cambridge/London (2004)Google Scholar
  6. 6.
    Zubair, A.M., Aurangzeb, K., Shakeel, A., Fazal, M.K.: A review of feature extraction in sentiment analysis. J. Basic Appl. Sci. Res. 4(3), 181–186 (2014)Google Scholar
  7. 7.
    Jaccard, P.: Distribution de la flore alpine dans le bassin des Dranses et dans quelques régions voisines. Bulletin de la Société Vaudoise des Sciences Naturelles 37, 241–272 (1901)Google Scholar
  8. 8.
    Nerbonne, J., Heeringa, W., Kleiweg, P.: Edit distance and dialect proximity. In: Time Warps, String Edits and Macromolecules: The Theory and Practice of Sequence Comparison, 2nd edn, p. 15 (1999)Google Scholar
  9. 9.
    Gilleland, M.: Levenshtein distance in three flavors (2009). Accessed 10 Aug 2018
  10. 10.
    Yannakoudakis, E.J., Fawthrop, D.: The rules of spelling errors. Inf. Process. Manage. 19(2), 87–99 (1983)CrossRefGoogle Scholar
  11. 11.
    Bilal, A.: Lexical normalisation of Twitter data.–1038. Accessed 10 Aug 2018
  12. 12.
    Abinash, T., Ankit, A., Santanu, K.R.: Classification of sentiment reviews using n-gram machine learning approach. Expert Syst. Appl. 57, 117–126 (2016)CrossRefGoogle Scholar
  13. 13.
    Bo, P., Lillian, L.: Opinion Mining and Sentiment Analysis. Found. Trends Inf. Retrieval 2(1–2), 1–135 (2008)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Knowledge Technology Research UnitUniversiti Malaysia SabahKota KinabaluMalaysia

Personalised recommendations