BursT: A Dynamic Term Weighting Scheme for Mining Microblogging Messages
One of the basic human needs is to exchange information and socialize with each other. Online microblogging services such as Twitter allow users to post very short messages related to everything ranging from mundane daily life routines to breaking news events. A key challenging issue of mining such social messages is how to analyze the real-time distributed messages and extract significant features of them in a dynamic environment. In this work, we propose a novel term weighting method, called BursT, using sliding window techniques for weighting message streams. The experimental results show that our weighting technique has an outstanding performance to reflect the shifts of concept drift. The result of this work can be extended to perform a periodic feature extraction, and also be able to integrate other sophisticated clustering methods to enhance the efficiency for real-time event mining in social networks.
Keywordsinformation retrieval text mining term weighting scheme social networks social mining
Unable to display preview. Download preview PDF.
- 1.Tsymbal, A.: The Problem of Concept Drift: Definitions and Related Work. Trinity College, Dublin (2004)Google Scholar
- 2.Yang, Y., Pierce, T., Carbonell, J.: A Study of Retrospective and On-line Event Detection. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 28–36. ACM, Melbourne (1998)Google Scholar
- 3.Lee, C.H., Chien, T.F., Yang, H.C.: DBHTE: A Novel Algorithm for Extracting Real-time Microblogging Topics. In: The Conference Proceedings at the ISCA 23rd International Conference on Computer Applications in Industry and Engineering (CAINE 2010), Las Vegas, USA (2010)Google Scholar
- 5.Bun, K.K., Ishizuka, M.: Topic Extraction from News Archive Using TFPDF Algorithm. In: Proceedings of the 3rd International Conference on Web Information Systems Engineering, pp. 73–82. IEEE Computer Society, Los Alamitos (2002)Google Scholar
- 7.Kleinberg, J.: Temporal Dynamics of On-Line Information Streams. In: Conference Temporal Dynamics of On-Line Information Streams (2005) Google Scholar
- 10.Chen, J., Nairn, R., Nelson, L., Bernstein, M., Chi, E.: Short and Tweet: Experiments on Recommending Content from Information Streams. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 1185–1194. ACM, Atlanta (2010)Google Scholar
- 12.Finch, T.: Incremental Calculation of Weighted Mean and Variance. University of Cambridge (2009)Google Scholar