Stance Classification in Out-of-Domain Rumours: A Case Study Around Mental Health Disorders

  • Ahmet Aker
  • Arkaitz Zubiaga
  • Kalina Bontcheva
  • Anna Kolliakou
  • Rob Procter
  • Maria Liakata
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10540)


Social media being a prolific source of rumours, stance classification of individual posts towards rumours has gained attention in the past few years. Classification of stance in individual posts can then be useful to determine the veracity of a rumour. Research in this direction has looked at rumours in different domains, such as politics, natural disasters or terrorist attacks. However, work has been limited to in-domain experiments, i.e. training and testing data belong to the same domain. This presents the caveat that when one wants to deal with rumours in domains that are more obscure, training data tends to be scarce. This is the case of mental health disorders, which we explore here. Having annotated collections of tweets around rumours emerged in the context of breaking news, we study the performance stability when switching to the new domain of mental health disorders. Our study confirms that performance drops when we apply our trained model on a new domain, emphasising the differences in rumours across domains. We overcome this issue by using a little portion of the target domain data for training, which leads to a substantial boost in performance. We also release the new dataset with mental health rumours annotated for stance.


Social media Stance classification Veracity Rumours Mental health 



This work was partially supported by the European Union under grant agreement No. 654024 SoBigData, PHEME project under the grant agreement No. 611223 and by the Deutsche Forschungsgemeinschaft (DFG) under grant No. GRK 2167, Research Training Group “User-Centred Social Media”. Rob Procter and Maria Liakata were supported by the Alan Turing Institute.


  1. 1.
    Baroni, M., Dinu, G., Kruszewski, G.: Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In: Proceedings of ACL, pp. 238–247 (2014)Google Scholar
  2. 2.
    Buhmann, M.D.: Radial Basis Functions: Theory and Implementations. Cambridge Monographs on Applied and Computational Mathematics, vol. 12, pp. 147–165 (2003)Google Scholar
  3. 3.
    Celli, F., Ghosh, A., Alam, F., Riccardi, G.: In the mood for sharing contents: emotions, personality and interaction styles in the diffusion of news. Inf. Process. Manag. 52(1), 93–98 (2016)CrossRefGoogle Scholar
  4. 4.
    Chen, D., Manning, C.D.: A fast and accurate dependency parser using neural networks. In: Proceedings of EMNLP, pp. 740–750 (2014)Google Scholar
  5. 5.
    Guo, W., Diab, M.: Modeling sentences in the latent space. In: Proceedings of ACL, pp. 864–872. Association for Computational Linguistics (2012)Google Scholar
  6. 6.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. ACM SIGKDD Explor. Newsl. 11(1), 10–18 (2009)CrossRefGoogle Scholar
  7. 7.
    Hamidian, S., Diab, M.: Rumor detection and classification for twitter data. In: Proceedings of SOTICS, pp. 71–77 (2015)Google Scholar
  8. 8.
    Hamidian, S., Diab, M.T.: Rumor identification and belief investigation on twitter. In: Proceedings of NAACL-HLT, pp. 3–8 (2016)Google Scholar
  9. 9.
    Hinduja, S., Patchin, J.W.: Bullying, cyberbullying, and suicide. Arch. Suicide Res. 14(3), 206–221 (2010)CrossRefGoogle Scholar
  10. 10.
    Kolliakou, A., Ball, M., Derczynski, L., Chandran, D., Gkotsis, G., Deluca, P., Jackson, R., Shetty, H., Stewart, R.: Novel psychoactive substances: An investigation of temporal trends in social media and electronic health records. Eur. Psychiatry 38, 15–21 (2016)CrossRefGoogle Scholar
  11. 11.
    Liang, P.: Semi-supervised learning for natural language. Ph.D. thesis, Massachusetts Institute of Technology (2005)Google Scholar
  12. 12.
    Liu, X., Nourbakhsh, A., Li, Q., Fang, R., Shah, S.: Real-time rumor debunking on twitter. In: Proceedings of CIKM, pp. 1867–1870. ACM (2015)Google Scholar
  13. 13.
    Lukasik, M., Cohn, T., Bontcheva, K.: Classifying tweet level judgements of rumours in social media. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, pp. 2590–2595 (2015)Google Scholar
  14. 14.
    Lukasik, M., Srijith, P.K., Vu, D., Bontcheva, K., Zubiaga, A., Cohn, T.: Hawkes processes for continuous time sequence classification: an application to rumour stance classification in twitter. In: Proceedings of the 54th Meeting of the Association for Computational Linguistics, pp. 393–398. Association for Computer Linguistics (2016)Google Scholar
  15. 15.
    Mendoza, M., Poblete, B., Castillo, C.: Twitter under crisis: can we trust what we rt? In: Proceedings of the Workshop on Social Media Analytics, pp. 71–79. ACM (2010)Google Scholar
  16. 16.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)Google Scholar
  17. 17.
    Qazvinian, V., Rosengren, E., Radev, D.R., Mei, Q.: Rumor has it: Identifying misinformation in microblogs. In: Proceedings of EMNLP, pp. 1589–1599 (2011)Google Scholar
  18. 18.
    Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., Potts, C.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of EMNLP, vol. 1631, p. 1642. Citeseer (2013)Google Scholar
  19. 19.
    Tausczik, Y.R., Pennebaker, J.W.: The psychological meaning of words: liwc and computerized text analysis methods. J. Lang. Soc. Psychol. 29(1), 24–54 (2010)CrossRefGoogle Scholar
  20. 20.
    Zeng, L., Starbird, K., Spiro, E.S.: # unconfirmed: classifying rumor stance in crisis-related social media messages. In: Proceedings of ICWSM (2016)Google Scholar
  21. 21.
    Zubiaga, A., Aker, A., Bontcheva, K., Liakata, M., Procter, R.: Detection and resolution of rumours in social media: a survey (2017). arXiv preprint: arXiv:1704.00656
  22. 22.
    Zubiaga, A., Kochkina, E., Liakata, M., Procter, R., Lukasik, M.: Stance classification in rumours as a sequential task exploiting the tree structure of social media conversations. In: Proceedings of COLING (2016)Google Scholar
  23. 23.
    Zubiaga, A., Liakata, M., Procter, R., Wong Sak Hoi, G., Tolmie, P.: Analysing how people orient to and spread rumours in social media by looking at conversational threads. PLoS ONE 11(3), 1–29 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Ahmet Aker
    • 1
    • 2
  • Arkaitz Zubiaga
    • 3
  • Kalina Bontcheva
    • 1
  • Anna Kolliakou
    • 4
  • Rob Procter
    • 3
    • 5
  • Maria Liakata
    • 3
    • 5
  1. 1.University of SheffieldSheffieldUK
  2. 2.University of Duisburg-EssenDuisburgGermany
  3. 3.University of WarwickCoventryUK
  4. 4.King’s College LondonLondonUK
  5. 5.Alan Turing InstituteLondonUK

Personalised recommendations