Identifying Conversational Message Threads by Integrating Classification and Data Clustering

  • Giacomo DomeniconiEmail author
  • Konstantinos Semertzidis
  • Gianluca Moro
  • Vanessa Lopez
  • Spyros Kotoulas
  • Elizabeth M. Daly
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 737)


Conversational message thread identification regards a wide spectrum of applications, ranging from social network marketing to virus propagation, digital forensics, etc. Many different approaches have been proposed in literature for the identification of conversational threads focusing on features that are strongly dependent on the dataset. In this paper, we introduce a novel method to identify threads from any type of conversational texts overcoming the limitation of previously determining specific features for each dataset. Given a pool of messages, our method extracts and maps in a three dimensional representation the semantic content, the social interactions and the timestamp; then it clusters each message into conversational threads. We extend our previous work by introducing a deep learning approach and by performing new extensive experiments and comparisons with classical learning algorithms.


Conversational message Thread identification Data clustering Classification 


  1. 1.
    Jurczyk, P., Agichtein, E.: Discovering authorities in question answer communities by using link analysis. In: CIKM, Lisbon, Portugal, 6–10 November 2007, pp. 919–922 (2007)Google Scholar
  2. 2.
    Coussement, K., den Poel, D.V.: Improving customer complaint management by automatic email classification using linguistic style features as predictors. Decis. Support Syst. 44, 870–882 (2008)CrossRefGoogle Scholar
  3. 3.
    Glass, K., Colbaugh, R.: Toward emerging topic detection for business intelligence: Predictive analysis of meme’ dynamics. CoRR abs/1012.5994 (2010)Google Scholar
  4. 4.
    Khan, F.M., Fisher, T.A., Shuler, L., Wu, T., Pottenger, W.M.: Mining chatroom conversations for social and semantic interactions. In: Technical report LU-CSE-02-011, Lehigh University (2002)Google Scholar
  5. 5.
    Hofmann, T.: Probabilistic latent semantic indexing. In: ACM SIGIR, pp. 50–57. ACM (1999)Google Scholar
  6. 6.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)zbMATHGoogle Scholar
  7. 7.
    Shen, D., Yang, Q., Sun, J., Chen, Z.: Thread detection in dynamic text message streams. In: SIGIR, Washington, USA, 6–11 August 2006, pp. 35–42 (2006)Google Scholar
  8. 8.
    Huang, J., Zhou, B., Wu, Q., Wang, X., Jia, Y.: Contextual correlation based thread detection in short text message streams. J. Intell. Inf. Syst. 38, 449–464 (2012)CrossRefGoogle Scholar
  9. 9.
    Adams, P.H., Martell, C.H.: Topic detection and extraction in chat. In: ICSC 2008, pp. 581–588 (2008)Google Scholar
  10. 10.
    Yeh, J.: Email thread reassembly using similarity matching. In: CEAS, 27–28 July 2006, Mountain View, California, USA (2006)Google Scholar
  11. 11.
    Domeniconi, G., Semertzidis, K., Lopez, V., Daly, E.M., Kotoulas, S., Moro, G.: A novel method for unsupervised and supervised conversational message thread detection. In: Proceedings of the 5th International Conference on Data Management Technologies and Applications, vol. 1, DATA, pp. 43–54 (2016)Google Scholar
  12. 12.
    Zhao, Q., Mitra, P.: Event detection and visualization for social text streams. In: ICWSM, Boulder, Colorado, USA, 26–28 March 2007Google Scholar
  13. 13.
    Lena, P., Domeniconi, G., Margara, L., Moro, G.: Gota: go term annotation of biomedical literature. BMC Bioinform. 16, 346 (2015)CrossRefGoogle Scholar
  14. 14.
    Ester, M., Kriegel, H., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD 1996, Portland, Oregon, USA, pp. 226–231 (1996)Google Scholar
  15. 15.
    Bouguettaya, A., Yu, Q., Liu, X., Zhou, X., Song, A.: Efficient agglomerative hierarchical clustering. Expert Syst. Appl. 42, 2785–2797 (2015)CrossRefGoogle Scholar
  16. 16.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)CrossRefGoogle Scholar
  17. 17.
    Najafabadi, M.M., Villanustre, F., Khoshgoftaar, T.M., Seliya, N., Wald, R., Muharemagic, E.: Deep learning applications and challenges in big data analytics. J. Big Data 2, 1 (2015)CrossRefGoogle Scholar
  18. 18.
    Zhao, Q., Mitra, P., Chen, B.: Temporal and information flow based event detection from social text streams. In: AAAI, 22–26 July 2007, Vancouver, British Columbia, Canada, pp. 1501–1506 (2007)Google Scholar
  19. 19.
    Porter, M.F.: An algorithm for suffix stripping. Program 14, 130–137 (1980)CrossRefGoogle Scholar
  20. 20.
    Domeniconi, G., Moro, G., Pasolini, R., Sartori, C.: A comparison of term weighting schemes for text classification and sentiment analysis with a supervised variant of tf.idf. In: Data Management Technologies and Applications (DATA 2015), Revised Selected Papers, pp. 39–58, vol. 553. Springer, Heidelberg (2016)Google Scholar
  21. 21.
    Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Inf. Process. Manage. 24, 513–523 (1988)CrossRefGoogle Scholar
  22. 22.
    Singhal, A.: Modern information retrieval: a brief overview. IEEE Data Eng. Bull. 24, 35–43 (2001)Google Scholar
  23. 23.
    Manning, C.D., Raghavan, P., Schütze, H., et al.: Introduction to Information Retrieval, vol. 1. Cambridge University Press, Cambridge (2008)CrossRefzbMATHGoogle Scholar
  24. 24.
    Aumayr, E., Chan, J., Hayes, C.: Reconstruction of threaded conversations in online discussion forums. In: Weblogs and Social Media (2011)Google Scholar
  25. 25.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep learning. Book in preparation for MIT Press (2016)Google Scholar
  26. 26.
    Sugomori, Y.: Java Deep Learning Essentials. Packt Publishing Ltd., Birmingham (2016)Google Scholar
  27. 27.
    Ulrich, J., Murray, G., Carenini, G.: A publicly available annotated corpus for supervised email summarization. In: AAAI08 EMAIL Workshop (2008)Google Scholar
  28. 28.
    Soboroff, I., de Vries, A.P., Craswell, N.: Overview of the TREC 2006 enterprise track. In: TREC, Gaithersburg, Maryland, USA, 14–17 November 2006 (2006)Google Scholar
  29. 29.
    Dehghani, M., Shakery, A., Asadpour, M., Koushkestani, A.: A learning approach for email conversation thread reconstruction. J. Inf. Sci. 39, 846–863 (2013)CrossRefGoogle Scholar
  30. 30.
    Erera, S., Carmel, D.: Conversation detection in email systems. In: Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., White, R.W. (eds.) ECIR 2008. LNCS, vol. 4956, pp. 498–505. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-78646-7_48 CrossRefGoogle Scholar
  31. 31.
    Wu, Y., Oard, D.W.: Indexing emails and email threads for retrieval. In: SIGIR, pp. 665–666 (2005)Google Scholar
  32. 32.
    Hall, M.A., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. 11, 10–18 (2009)CrossRefGoogle Scholar
  33. 33.
    Raschka, S.: Python Machine Learning. Packt Publishing, Birmingham (2015)Google Scholar
  34. 34.
    Wang, X., Xu, M., Zheng, N., Chen, M.: Email conversations reconstruction based on messages threading for multi-person. In: ETTANDGRS 2008, vol. 1, pp. 676–680 (2008)Google Scholar
  35. 35.
    Joshi, S., Contractor, D., Ng, K., Deshpande, P.M., Hampp, T.: Auto-grouping emails for faster e-discovery. PVLDB 4, 1284–1294 (2011)Google Scholar
  36. 36.
    Wang, H., Wang, C., Zhai, C., Han, J.: Learning online discussion structures by conditional random fields. In: SIGIR 2011, Beijing, China, 25–29 July 2011, pp. 435–444 (2011)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Giacomo Domeniconi
    • 1
    Email author
  • Konstantinos Semertzidis
    • 2
  • Gianluca Moro
    • 1
  • Vanessa Lopez
    • 3
  • Spyros Kotoulas
    • 3
  • Elizabeth M. Daly
    • 3
  1. 1.Department of Computer Science and Engineering (DISI)University of Bologna at CesenaCesenaItaly
  2. 2.Department of Computer Science and EngineeringUniversity of IoanninaIoanninaGreece
  3. 3.IBM Research - Damastown Industrial Estate MulhuddartDublin 15Ireland

Personalised recommendations