Skip to main content
SpringerLink
Log in
Book cover

Advances in Combining Intelligent Methods pp 1–23Cite as

Real-Time Investors’ Sentiment Analysis from Newspaper Articles

  • Chapter
  • First Online:

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 116 ))

Abstract

Recently, investor sentiment measures have become one of the more widely examined areas in behavioral finance. They are capable of both explaining and forecasting stock returns. The purpose of this paper is to present a method, based on a combination of a Naïve Bayes classifier and the n-gram probabilistic language model, which can create a sentiment index for specific stocks and indices of the New York Stock Exchange. An economic useful proxy for investor sentiment is constructed from U.S. news articles mainly provided by The New York Times. Initially, a large amount of articles for ten big companies and indices is collected and processed, in order to be able to extract a sentiment score from each one of them. Then, the classifier is trained from the positive, negative and neutral articles, so that it is possible afterwards to examine the sentiment of any unseen newspaper article, for any company or index. Subsequently, the classification task is tested and validated for its accuracy and efficiency. The widely used Baker and Wurgler sentiment index [2] is used as a comparison measure for predicting stock returns. In a sample of S&P 500 index from 2004 to 2010 on monthly basis, it is shown that the new sentiment index created has, on average, twice the predictive ability of Baker and Wurgler’s index, for the existing time frame.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://code.google.com/p/word2vec .

  2. 2.

    http://developer.nytimes.com/docs/read/article_search_api_v2 .

  3. 3.

    https://lucene.apache.org/ .

  4. 4.

    http://json.org/ .

  5. 5.

    https://www.mongodb.org/.

  6. 6.

    https://research.stlouisfed.org/fred2/series/SP500/downloaddata .

References

  1. Antweiler, W., Frank, M.: Is all that talk just noise? The information content of Internet stock message boards. J. Finance 59(3), 1259–1294 (2004)

    Article  Google Scholar 

  2. Baker, M., Wurgler, J.: Investor sentiment and the cross-section of stock returns. J. Finance 61(4), 1645–1680 (2006)

    Article  Google Scholar 

  3. Barber, B.M., Odean, T., Zhu, N.: Do retail trades move markets? Rev. Finan. Stud. 22, 151–186 (2009)

    Article  Google Scholar 

  4. Bollen, J., Mao, H., Zeng, X.: Twitter mood predicts the stock market. J. Comput. Sci. 2(1), 1–8 (2010)

    Article  Google Scholar 

  5. Bram, J., Ludvigson, S.C.: Does consumer confidence forecast household expenditure? A sentiment index horse race. Econ. Policy Rev. 4(2) (1998)

    Google Scholar 

  6. Das, S., Chen, M.: Yahoo! for Amazon: sentiment extraction from small talk on the Web. Manag. Sci. 53(9), 1375–1388 (2007)

    Article  Google Scholar 

  7. Dergiades, T.: Do investors’ sentiment dynamics affect stock returns? Evidence from the US economy. Econ. Lett. 116(3), 404–407 (2012). ISSN 0165-1765, http://dx.doi.org/10.1016/j.econlet.2012.04.018

  8. Dickinson, B., Hu, W.: Sentiment analysis of investor opinions on twitter. Soc. Network 4, 62–71 (2015)

    Article  Google Scholar 

  9. Fang, H., Tao, T., Zhai, Ch. X.: A formal study of information retrieval heuristics. In: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ‘04). ACM, New York, NY, USA, pp. 49–56 (2004). doi:http://dx.doi.org/10.1145/1008992.1009004

  10. Garcia, D.: Sentiment during recessions. J. Finance 68(3), 1267–1299 (2013)

    Article  Google Scholar 

  11. Iyengar, S.: Is Anyone Responsible? How Television Frames Political Issues. University of Chicago Press, Chicago (1991)

    Book  Google Scholar 

  12. Klemola, A., Nikkinen, J., Peltomäki, J.: Investor Sentiment in the Stock Market Inferred from Google Search Volumes (2010)

    Google Scholar 

  13. McDonald, B.: Bill McDonald’s Word Lists Page, University of Notre Dame (2013). http://www3.nd.edu/~mcdonald/Word_Lists.html

  14. Pang, B., Lee, L.: Opinion Mining and Sentiment Analysis (2008)

    Google Scholar 

  15. Plous S (1993) The Psychology of Judgment and Decision Making. McGraw-Hill. ISBN 978-0-07-050477-6

    Google Scholar 

  16. Price, V., Tewksbury, D.: News Values and Public Opinion: A Theoretical Account of Media Priming and Framing. In: Barett, G.A., Boster, F.J. (eds.) Progress in Communication Sciences: Advances in Persuasion, vol. 13, pp. 173–212. Ablex, Greenwich, CT (1997)

    Google Scholar 

  17. Price, V., Tewksbury, D., Powers, E.: Switching trains of thought: the impact of news frames on readers’ cognitive responses. Commun. Res. 24(5), 481–506 (1997)

    Article  Google Scholar 

  18. Qiu, L., Welch, I.: Investor Sentiment Measures. Brown University and NBER (2006)

    Google Scholar 

  19. Schmeling, M.: Institutional and individual sentiment: smart money and noise trader risk? 23, 127–145 (2007)

    Google Scholar 

  20. Sehgal, V., Song, C.: SOPS: Stock Prediction using Web Sentiment. In: Seventh IEEE International Conference on Data Mining—Workshops, pp. 21–26 (2009)

    Google Scholar 

  21. Shiller, R.J.: Irrational Exuberance, 2nd edn. Princeton University Press, Princeton, New Jersey (2005)

    Google Scholar 

  22. Shleifer, A., Summers, L.H.: The noise trader approach to finance. J. Econ. Perspect. 4, 19–33 (1990)

    Article  Google Scholar 

  23. Spärck Jones, K.: A statistical interpretation of term specificity and its application in retrieval. J. Documentation 28, 11–21 (1972). doi:10.1108/eb026526

    Article  Google Scholar 

  24. Statman, M.: Normal investors, then and now. CFA Institute, Finan. Anal. J. 61(2), 31–37 (2005)

    Google Scholar 

  25. Tumarkin, R., Whitelaw, R.F.: News or Noise? Internet Message Board Activity and Stock Prices (2000)

    Google Scholar 

  26. Tversky, A., Kahneman, D.: The Framing of decisions and the psychology of choice. Science 211(4481), 453–458 (1981). doi:10.1126/science.7455683

    Article  MathSciNet  MATH  Google Scholar 

  27. Vliegenthart, R., Schuck, A.R.T., Boomgaarden, H.G., De Vreese, C.H.: News coverage and support for european integration, 1990–2006. Int. J. Public Opin. Res. 20(4), 415–436 (2008)

    Article  Google Scholar 

  28. Wolpert, D.H.: Stacked generalization. Neural Networks 5, 241 (1992)

    Article  Google Scholar 

Download references

Acknowledgments

We would like to give special thanks to Dr. Theologos Dergiades, Academic Associate of the School of Science & Technology at the International Hellenic University, Thessaloniki, Greece, for his advice and assistance whenever it was needed.

Author information

Authors and Affiliations

  1. Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece

    Konstantinos Arvanitis & Nick Bassiliades

Authors
  1. Konstantinos Arvanitis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nick Bassiliades
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nick Bassiliades .

Editor information

Editors and Affiliations

  1. Sch of Engg, Dept of Comp Engg & Informa, University of Patras Sch of Engg, Dept of Comp Engg & Informa, Patras, Greece

    Ioannis Hatzilygeroudis

  2. Fac of Engg & Compu, dept of Compu, Coventry University, Reader in Perv, Fac of Engg & Compu, dept of Compu, Coventry, United Kingdom

    Vasile Palade

  3. Dept of Edu Sci Early child, Lab of info, Democritus Univer Thrace,Sch of Edu Sci Dept of Edu Sci Early child, Lab of info, Alexandroupolis, Greece

    Jim Prentzas

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Arvanitis, K., Bassiliades, N. (2017). Real-Time Investors’ Sentiment Analysis from Newspaper Articles. In: Hatzilygeroudis, I., Palade, V., Prentzas, J. (eds) Advances in Combining Intelligent Methods. Intelligent Systems Reference Library, vol 116 . Springer, Cham. https://doi.org/10.1007/978-3-319-46200-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46200-4_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46199-1

  • Online ISBN: 978-3-319-46200-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation