Fuzzy Bag-of-Topics Model for Short Text Representation
Text representation is the keystone in many NLP tasks. For short text representation learning, the traditional Bag-of-Words model (BoW) is often criticized for sparseness and neglecting semantic information. Fuzzy Bag-of-Words (FBoW) and Fuzzy Bag-of-Words Cluster (FBoWC) model are the improved model of BoW, which can learn dense and meaningful document vectors. However, word clusters in FBoWC model are obtained by K-means cluster algorithm, which is unstable and may result in incoherent word clusters if not initialized properly. In this paper, we propose the Fuzzy Bag-of-Topics model (FBoT) to learn short text vector. In FBoT model, word communities, which are more coherent than word clusters in FBoWC, are used as basis terms in text vector. Experimental results of short text classification on two datasets show that FBoT achieves the highest classification accuracies.
KeywordsShort text Representation learning Word communities
This work was supported in part by Shanghai Innovation Action Plan Project under the grant No. 16511101200.
- 3.Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient Estimation of Word Representations in Vector Space. http://arxiv.org/abs/1301.3781
- 4.Banerjee, S., Ramanathan, K., Gupta, A.: Clustering short texts using Wikipedia. In: 30th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 787–788. ACM, Amsterdam (2007)Google Scholar
- 5.Hu, X., Sun, N., Zhang, C., Chua, T.S., et al.: Exploring internal and external semantics for the clustering of short texts using world knowledge. In: 18th ACM Conference on Information and Knowledge Management, pp. 919–928. ACM, Hong Kong (2009)Google Scholar
- 7.Sahami, M., Heilman, T.D.: A web-based kernel function for measuring the similarity of short text snippets. In: 15th International Conference on World Wide Web, pp. 377–386. ACM, Edinburgh (2006)Google Scholar
- 9.Dumain S.: Latent Semantic Indexing (LSI): TREC-3 Report. In: Harman, M. (ed.) The Third Text REtrieval Conference, vol. 500, no. 226, pp. 219–230. NIST Special Publication, Gaithersburg (1995)Google Scholar
- 11.Le, Q., Mikolov, T.: Distributed Representations of Sentences and Documents. http://arxiv.org/abs/1405.4053
- 12.Phan, X.H., Nguyen, C.T., Le, D.T., Nguyen, L.M., Horiguchi, S., Ha, Q.T.: A hidden topic-based framework towards building applications with short web documents. Trans. KDE 23, 961–976 (2011)Google Scholar
- 13.Phan, X.H., Nguyen, L.M., Horiguchi, S.: Learning to classify short and sparse text and web with hidden topics from large-scale data collections. In: 17th International World Wide Web Conference, pp. 91–100. ACM, Beijing (2008)Google Scholar
- 14.Daniel, L.S., Yang, Q., Li, L.: Lifelong machine learning systems: beyond learning algorithms. In: Proceedings of the AAAI Spring Symposium on Lifelong Machine Learning, pp. 49–55. AAAI, Palo Alto (2013)Google Scholar