BursT: A Dynamic Term Weighting Scheme for Mining Microblogging Messages

  • Chung-Hong Lee
  • Chih-Hong Wu
  • Tzan-Feng Chien
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6677)


One of the basic human needs is to exchange information and socialize with each other. Online microblogging services such as Twitter allow users to post very short messages related to everything ranging from mundane daily life routines to breaking news events. A key challenging issue of mining such social messages is how to analyze the real-time distributed messages and extract significant features of them in a dynamic environment. In this work, we propose a novel term weighting method, called BursT, using sliding window techniques for weighting message streams. The experimental results show that our weighting technique has an outstanding performance to reflect the shifts of concept drift. The result of this work can be extended to perform a periodic feature extraction, and also be able to integrate other sophisticated clustering methods to enhance the efficiency for real-time event mining in social networks.


information retrieval text mining term weighting scheme social networks social mining 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tsymbal, A.: The Problem of Concept Drift: Definitions and Related Work. Trinity College, Dublin (2004)Google Scholar
  2. 2.
    Yang, Y., Pierce, T., Carbonell, J.: A Study of Retrospective and On-line Event Detection. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 28–36. ACM, Melbourne (1998)Google Scholar
  3. 3.
    Lee, C.H., Chien, T.F., Yang, H.C.: DBHTE: A Novel Algorithm for Extracting Real-time Microblogging Topics. In: The Conference Proceedings at the ISCA 23rd International Conference on Computer Applications in Industry and Engineering (CAINE 2010), Las Vegas, USA (2010)Google Scholar
  4. 4.
    Brants, T., Chen, F., Farahat, A.: A System for New Event Detection. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, pp. 330–337. ACM, Toronto (2003)CrossRefGoogle Scholar
  5. 5.
    Bun, K.K., Ishizuka, M.: Topic Extraction from News Archive Using TFPDF Algorithm. In: Proceedings of the 3rd International Conference on Web Information Systems Engineering, pp. 73–82. IEEE Computer Society, Los Alamitos (2002)Google Scholar
  6. 6.
    Kleinberg, J.: Bursty and Hierarchical Structure in Streams. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 91–101. ACM, Edmonton (2002)CrossRefGoogle Scholar
  7. 7.
    Kleinberg, J.: Temporal Dynamics of On-Line Information Streams. In: Conference Temporal Dynamics of On-Line Information Streams (2005) Google Scholar
  8. 8.
    Cataldi, M., Caro, L.D., Schifanella, C.: Emerging Topic Detection on Twitter Based on Temporal and Social Terms Evaluation. In: Proceedings of the Tenth International Workshop on Multimedia Data Mining, pp. 1–10. ACM, Washington, D.C (2010)CrossRefGoogle Scholar
  9. 9.
    Cheong, M., Lee, V.: Integrating Web-based Intelligence Retrieval and Decision-making from the Twitter Trends Knowledge Base. In: Proceeding of the 2nd ACM Workshop on Social Web Search and Mining, pp. 1–8. ACM, Hong Kong (2009)CrossRefGoogle Scholar
  10. 10.
    Chen, J., Nairn, R., Nelson, L., Bernstein, M., Chi, E.: Short and Tweet: Experiments on Recommending Content from Information Streams. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 1185–1194. ACM, Atlanta (2010)Google Scholar
  11. 11.
    Bifet, A.: Adaptive Stream Mining: Pattern Learning and Mining from Evolving Data Streams. IOS Press, Amsterdam (2010)zbMATHGoogle Scholar
  12. 12.
    Finch, T.: Incremental Calculation of Weighted Mean and Variance. University of Cambridge (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Chung-Hong Lee
    • 1
  • Chih-Hong Wu
    • 1
  • Tzan-Feng Chien
    • 1
  1. 1.Dept of Electrical EngineeringNational Kaohsiung University of Applied SciencesKaohsiungTaiwan

Personalised recommendations