Advertisement

Predicting Flu Epidemics Using Twitter and Historical Data

  • Giovanni Stilo
  • Paola Velardi
  • Alberto E. Tozzi
  • Francesco Gesualdo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8609)

Abstract

Recently there has been a growing attention on the use of web and social data to improve traditional prediction models in politics, finance, marketing and health, but even though a correlation between observed phenomena and related social data has been demonstrated in many cases, yet the effectiveness of the latter for long-term or even mid-term predictions has not been shown. In epidemiological surveillance, the problem is compounded by the fact that infectious diseases models (such as susceptible-infected-recovered-susceptible, SIRS) are very sensitive to current conditions, such that small changes can produce remarkable differences in future outcomes. Unfortunately, current or nearly-current conditions keep changing as data are collected and updated by the epidemiological surveillance organizations. In this paper we show that the time series of Twitter messages reporting a combination of symptoms that match the influenza-like-illness (ILI) case definition represent a more stable and reliable information on “current conditions”, to the point that they can replace, rather than simply integrate, official epidemiological data. We estimate the effectiveness of these data at predicting current and past flu seasons (17 seasons overall), in combination with official historical data on past seasons, obtaining an average correlation of 0.85 over a period of 17 weeks covering the flu season.

Keywords

Twitter mining epidemiological surveillance predictability of health-related phenomena 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Yu, S., Kak, S.: A Survey of Prediction using Social Media (2012), http://arxiv.org/ftp/arxiv/papers/1203/1203.1647.pdf (retrieved)
  2. 2.
    Asur, S., Huberman, B.H.: Predicting the future with social media, http://www.hpl.hp.com/research/scl/papers/socialmedia/socialmedia.pdf (retrieved)
  3. 3.
    Radinsky, K., Horvitz, E.: Mining the web to predict future events. In: WSDM 2013, pp. 255–264 (2013)Google Scholar
  4. 4.
    Ruiz, E.J., Hristidis, V., Castillo, C., Gionis, A., Jaimes, A.: Correlating financial time series with micro-blogging activity. In: Proceedings of the Fifth ACM International Conference on Web Search and Data Mining, WSDM 2012, pp. 513–522. ACM, New York (2012)Google Scholar
  5. 5.
    Carrifere-Swallow, Y., Labbfe, F.: Nowcasting with Google Trends in an Emerging Market. Central Bank of Chile, Working Papers n. 588 (2010)Google Scholar
  6. 6.
    Ginsberg, J., Mohebbi, M.H., Patel, R.S., Brammer, L., Smolinski, M.S., et al.: Detecting influenza epidemics using search engine query data. Nature 457, 1012–1014 (2009)CrossRefGoogle Scholar
  7. 7.
    Althouse, B.M., Ng, Y.Y., Cummings, D.A.T.: Prediction of Dengue Incidence Using Search Query Surveillance. PLoS Negl. Trop. Dis. 5(8) (2011)Google Scholar
  8. 8.
    Xu, D., Liu, Y., Zhang, M., Ma, S., Ciu, A., Ru, L.: Predicting Epidemic Tendency through Search Behaviour Analysis. In: Proc. of 22nd IJCAI (2011)Google Scholar
  9. 9.
    Lamb, A., Paul, M.J., Dredze, M.: Separating Fact from Fear: Tracking Flu Infections on Twitter. In: NAACL (2013)Google Scholar
  10. 10.
    Achrekar, H., Gandhe, A., Lazarus, R., Yu, S., Liu, B.: Predicting Flu Trends using Twitter Data. In: CPNS 2011 (2011)Google Scholar
  11. 11.
    Shaman, J., Karspeck, A., Yang, W., Tamerius, J., Lipsitch, M.: Real-time influenza forecasts during the 2012-2013 season. Nature Communications 4, 2837 (2013), doi:10.1038/ncomms3837CrossRefGoogle Scholar
  12. 12.
    Gesualdo, F., Stilo, G., Agricola, E., Gonfiantini, M.V., Pandolfi, E., Velardi, P., Tozzi, A.E.: Influenza-like illness surveillance on Twitter through automated learning of naïve language. PloS One Public Library of Science One, Journal (2013)Google Scholar
  13. 13.
    Velardi, P., Stilo, G., Tozzi, A.E., Gesualdo, F.: Twitter mining for fine-grained syndromic surveillance. Artificial Intelligence in Medicine, Special Issue on Text Mining and Information Analysis (in press, 2014)Google Scholar
  14. 14.
    Stilo, G., De Vincenzi, M., Tozzi, A.E., Velardi, P.: Automated Learning of Everyday Patients’ Language for Medical Blog Analytics. In: Proceedings of the Recent Advances in Natural Language Processing (RANLP 2013) Hissar, Hissar (Bulgaria), September 9-11 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Giovanni Stilo
    • 1
  • Paola Velardi
    • 1
  • Alberto E. Tozzi
    • 2
  • Francesco Gesualdo
    • 2
  1. 1.Dipartimento di Informatica Sapienza Università di RomaItaly
  2. 2.Bambino Gesù Children HospitalRomaItaly

Personalised recommendations