An Operational Deep Learning Pipeline for Classifying Life Events from Individual Tweets

  • Xinsong Du
  • Jiang Bian
  • Mattia ProsperiEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 898)


We here present an operational deep learning pipeline for classifying life events from individual tweets, using job loss as a use case and Twitter data collected between 2010 and 2013 (historic sample from the public stream). The pipeline includes identification of keywords through snowball sampling, multiple rater manual annotation, supervised deep learning, text processing (word embedding, bag of words) and architecture selection (convolutional, shallow-and-wide convolutional, and long-short-term memory) with parameter optimization, external validation and feedback learning. After model optimization, a shallow-and-wide network with a pre-trained 200-dimensional word2vec achieved a precision of 78% (over an average single keyword precision of 50%) and an area under receiver operating characteristic of 86%. Precision and recall also increased by 5% using bag of words. When tested on tweets with ambiguous annotations (i.e. tweets that were hard for human annotators to classify), the network achieved 65% precision. Finally, on a random set of tweets that did not contain any of the snowballed keywords, 30% were classified as job loss events; this putatively false positive set can be used to reinforce the learner’s training. In conclusion, the pipeline streamlines both the manual and automated process, providing feedback reinforcement (snowballing and external tweets), and shows good performance on classifying individual tweets on the use case, potentially saving human resources needed to collate such data for research studies.


Deep learning Job loss Twitter Classification 



MP, JB, and XD are in part supported by US NSF grant SES 1734134.


  1. 1.
    Reece, A.G., Reagan, A.J., Lix, K.L.M., Dodds, P.S., Danforth, C.M., Langer, E.J.: Forecasting the onset and course of mental illness with Twitter data. Sci. Rep. 7(1), 13006 (2017). Scholar
  2. 2.
    Tausczik, Y.R., Pennebaker, J.W.: The psychological meaning of words: LIWC and computerized text analysis methods. J. Lang. Soc. Psychol. 29(1), 24–54 (2010). Scholar
  3. 3.
    Alsaedi, N., Burnap, P., Rana, O.: Can we predict a riot? Disruptive event detection using Twitter. ACM Trans. Internet Technol. 17(2), 18:1–18:26 (2017). Scholar
  4. 4.
    Sumner, C., Byers, A., Boochever, R., Park, G.J.: Predicting dark triad personality traits from Twitter usage and a linguistic analysis of tweets. In: 2012 11th International Conference on Machine Learning and Applications, vol. 2, pp. 386–393 (2012).
  5. 5.
    Makazhanov, A., Rafiei, D.: Predicting political preference of Twitter users. In: 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013), pp. 298–305 (2013).
  6. 6.
    Conover, M., Gonçalves, B., Ratkiewicz, J., Flammini, A., Menczer, F.: Predicting the political alignment of Twitter users. In: 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, pp. 192–199 (2011).
  7. 7.
    Ramos, J.P.: Using TF-IDF to determine word relevance in document queries. Presented at the First International Conference on Machine Learning, New Brunswick, NJ, USA (2003)Google Scholar
  8. 8.
    Hofmann, T.: Unsupervised learning by probabilistic latent semantic analysis. Mach. Learn. 42(1–2), 177–196 (2001). Scholar
  9. 9.
    Won, D., Steinert-Threlkeld, Z.C., Joo, J.: Protest activity detection and perceived violence estimation from social media images. ArXiv:1709.06204 [Cs] (2017).
  10. 10.
    Zhang, Z., He, Q., Gao, J., Ni, M.: A deep learning approach for detecting traffic accidents from social media data. Transp. Res. Part C: Emerg. Technol. 86, 580–596 (2018). Scholar
  11. 11.
    Founta, A.-M., Chatzakou, D., Kourtellis, N., Blackburn, J., Vakali, A., Leontiadis, I.: A unified deep learning architecture for abuse detection. ArXiv:1802.00385 [Cs] (2018).
  12. 12.
    Badjatiya, P., Gupta, S., Gupta, M., Varma, V.: Deep learning for hate speech detection in tweets. In: Proceedings of the 26th International Conference on World Wide Web Companion, pp. 759–760. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland (2017).
  13. 13.
    Pitsilis, G.K., Ramampiaro, H., Langseth, H.: Detecting offensive language in tweets using deep learning. ArXiv:1801.04433 [Cs] (2018).
  14. 14.
    Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation, pp. 1532–1543. Association for Computational Linguistics (2014).
  15. 15.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality, p. 9 (n.d.)Google Scholar
  16. 16.
    Bian, J., et al.: Mining Twitter to assess the public perception of the “internet of things”. PLoS One 11(7), e0158450 (2016). Scholar
  17. 17.
    McHugh, M.L.: Interrater reliability: the kappa statistic. Biochem. Med. 22(3), 276–282 (2012)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Goodman, L.A.: Snowball sampling. Ann. Math. Stat. 32(1), 148–170 (1961). Scholar
  19. 19.
    Nadeau, C., Bengio, Y.: Inference for the generalization error. In: Proceedings of the 12th International Conference on Neural Information Processing Systems, pp. 307–313. MIT Press, Cambridge (1999).
  20. 20.
    LeCun, Y., Bengio, Y.: Convolutional networks for images, speech, and time-series, p. 15 (n.d.)Google Scholar
  21. 21.
    Le, H.T., Cerisara, C., Denis, A.: Do convolutional networks need to be deep for text classification? ArXiv:1707.04108 [Cs] (2017).
  22. 22.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). Scholar
  23. 23.
    Das, S., Wong, W.K., Dietterich, T., Fern, A., Emmott, A.: Incorporating expert feedback into active anomaly discovery. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 853–858 (2016).
  24. 24.
    Yin, W., Kann, K., Yu, M., Schütze, H.: Comparative study of CNN and RNN for natural language processing (2017).
  25. 25.
    Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. ArXiv:1509.01626 [Cs] (2015).
  26. 26.
    The Xapian Project (n.d.). Assessed 28 June 2018
  27. 27.
    Welcome to (n.d.). Assessed 28 June 2018
  28. 28.
    TensorFlow (n.d.). Assessed 28 June 2018
  29. 29.
    Palinkas, L.A., Horwitz, S.M., Green, C.A., Wisdom, J.P., Duan, N., Hoagwood, K.: Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm. Policy Ment. Health 42(5), 533–544 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Health Outcomes and Biomedical InformaticsUniversity of FloridaGainesvilleUSA
  2. 2.Department of EpidemiologyUniversity of FloridaGainesvilleUSA

Personalised recommendations