Predicting the Stock Market Behavior Using Historic Data Analysis and News Sentiment Analysis in R

  • A. C. JishagEmail author
  • A. P. Athira
  • Muchintala Shailaja
  • S. Thara
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1045)


Predicting the stock market has always been an attractive topic, mainly due to its vitality in the economic and financial sectors. Yet, predictions of the stock market pose a challenging exercise, even to the brightest and sharpest minds in the business. Prediction of stock market is never an easy task, due to the complexity and dynamic characteristics of the data it deals with. Bulk amount of the data output generated by the stock market is considered to be a treasure house of knowledge for investors; several studies have been conducted in an attempt to predict the stock market trends. Hence, it is imminent to uncover the behavior of the stock market data in order to avoid future investment risks for the investors. Here we tried a different approach for solving this problem by combining two different components: sentiment analysis on stock-related news reports and historic data analysis. The primary aim of this study was to construct an efficient model to predict trends in the stock market, with minimum error ratio and with maximum accuracy possible for the prediction. This model achieved notably better accuracy as compared to the models created in the previous studies. Two datasets were used in this study. A historical dataset containing the stock values of over ten 11, xxxx companies in the previous years, and a sentiment dataset containing the stock market news reports from social media and other online sources. The first step was to analyze the stock reports and classify them either as a positive or a negative sentiment. Lexicon method of text sentiment classification was used for this purpose. Predictions at this stage achieved an accuracy of 67.14%. The second step of this study used ts and ARIMA functions to predict stock trend, using the historical dataset. In the final step, results from both the components were combined together, to predict stock prices in future. This improved the prediction accuracy up to 89.80%.


Historic data analysis Lexicon method Sentiment analysis Stock market prediction 


  1. 1.
    Bing, L.I., Ou, C.: Public sentiment analysis in Twitter data for prediction of a company s stock price movements. In: IEEE 11th International Public, Conference E-bus. Eng (2014)Google Scholar
  2. 2.
    Yahya Eru Cara, B.D.T.: Stock price prediction using linear regression based on sentiment analysis. In: International Conference Advance Computer Science Information System, pp. 147154 (2015)Google Scholar
  3. 3.
    Hana Alostad, H.D.: Directional prediction of stock prices using breaking news on Twitter. In: IEEE/WIC/ACM International Conference on Web Intelligence Intelligent Agent Technology, pp. 07 (2015)Google Scholar
  4. 4.
    Patrick Uhr, M.F., Zenkert, J.: Sentiment analysis in financial market. In: IEEE International Conference System Man, Cybernetics, pp. 912917 (2014)Google Scholar
  5. 5.
    Shynkevichl, Y., Mcginnityl, T.M., Colemanl, S., Belatrechel, A.: Stock Price Prediction based on StockSpecific and Sub-Industry-Specific News Articles (2015)Google Scholar
  6. 6.
    Abdullah, S.S., Rahaman, M.S., Rahman, M.S.: Analysis of stock market using text mining and natural language processing. In: 2013 International Conference Informatics, Electronics Vis, pp. 16 (2013)Google Scholar
  7. 7.
    Price, S.M., Shriwas, J., Farzana, S.: Using text mining and rule based technique for prediction. Int. J. Emerg. Technol. Adv. Eng. 4(1) (2014)Google Scholar
  8. 8.
    Desai, R.: Stock market prediction using data mining 1, 2(2), 27802784 (2014)Google Scholar
  9. 9.
    Journal, I., Social, O.F., Studies, H.: TIME SERIES ANALYSIS ON STOCK MARKET FOR TEXT MINING 6(1), 69–91 (2014)Google Scholar
  10. 10.
    Kim, Y., Jeong, S.R., Ghani, I.: Text Opinion Mining to Analyze News for Stock Market Prediction Int. J. Adv. Soft Comput. Its Appl. 6(1), 113 (2014)Google Scholar
  11. 11.
    Thanh, H.T.P., Meesad, P.: Stock market trend prediction based on text mining of corporate web and time series data. J. Adv. Comput. Intell. Intell. Informatics 18(1) (2014)Google Scholar
  12. 12.
    Thara, S., Athul Krishna, N.S.: Aspect sentiment identification using random fourier features. Int. J. Intell. Syst. Appl. 10(9), 32–39Google Scholar
  13. 13.
    Thara, S. Sidharth, S.: Aspect based sentiment classication: SVD features. In: 2017 International Conference on Advances in Computing, Communications and Informatics, ICACCI 2017, pp. 2370–2374 (2017)Google Scholar
  14. 14.
    Rafeek, R., Remya, R., Detecting contextual word polarity using aspect-based sentiment analysis and logistic regression. In: ICSTM 2017-Proceedings, : IEEE International Conference on Smart Technologies and Management for Computing. Communication, Controls, Energy and Materials (2017)Google Scholar
  15. 15.
    Vijayan, V.K, Bindu, K.R., Parameswaran, L.A.: Comprehensive study of text classication algorithms. In: 2017 International Conference on Computing and Network Communications, CoCoNet 2015 (2015)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  • A. C. Jishag
    • 1
    Email author
  • A. P. Athira
    • 1
  • Muchintala Shailaja
    • 1
  • S. Thara
    • 1
  1. 1.Amrita Vishwa VidyapeethamAmritapuriIndia

Personalised recommendations