Fuzzy Bag-of-Topics Model for Short Text Representation

  • Hao Jia
  • Qing LiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11305)


Text representation is the keystone in many NLP tasks. For short text representation learning, the traditional Bag-of-Words model (BoW) is often criticized for sparseness and neglecting semantic information. Fuzzy Bag-of-Words (FBoW) and Fuzzy Bag-of-Words Cluster (FBoWC) model are the improved model of BoW, which can learn dense and meaningful document vectors. However, word clusters in FBoWC model are obtained by K-means cluster algorithm, which is unstable and may result in incoherent word clusters if not initialized properly. In this paper, we propose the Fuzzy Bag-of-Topics model (FBoT) to learn short text vector. In FBoT model, word communities, which are more coherent than word clusters in FBoWC, are used as basis terms in text vector. Experimental results of short text classification on two datasets show that FBoT achieves the highest classification accuracies.


Short text Representation learning Word communities 



This work was supported in part by Shanghai Innovation Action Plan Project under the grant No. 16511101200.


  1. 1.
    Spärck Jones, K.: A statistical interpretation of term specificity and its application to retrieval. J. Doc. 28, 11–21 (1972)CrossRefGoogle Scholar
  2. 2.
    Zhao, R., Mao, K.: Fuzzy bag-of-words model for document representation. IEEE Trans. Fuzzy Syst. 26, 794–804 (2018)CrossRefGoogle Scholar
  3. 3.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient Estimation of Word Representations in Vector Space.
  4. 4.
    Banerjee, S., Ramanathan, K., Gupta, A.: Clustering short texts using Wikipedia. In: 30th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 787–788. ACM, Amsterdam (2007)Google Scholar
  5. 5.
    Hu, X., Sun, N., Zhang, C., Chua, T.S., et al.: Exploring internal and external semantics for the clustering of short texts using world knowledge. In: 18th ACM Conference on Information and Knowledge Management, pp. 919–928. ACM, Hong Kong (2009)Google Scholar
  6. 6.
    Wang, L., Jia, Y., Han, W.: Instant message clustering based on extended vector space model. In: Kang, L., Liu, Y., Zeng, S. (eds.) ISICA 2007. LNCS, vol. 4683, pp. 435–443. Springer, Heidelberg (2007). Scholar
  7. 7.
    Sahami, M., Heilman, T.D.: A web-based kernel function for measuring the similarity of short text snippets. In: 15th International Conference on World Wide Web, pp. 377–386. ACM, Edinburgh (2006)Google Scholar
  8. 8.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)zbMATHGoogle Scholar
  9. 9.
    Dumain S.: Latent Semantic Indexing (LSI): TREC-3 Report. In: Harman, M. (ed.) The Third Text REtrieval Conference, vol. 500, no. 226, pp. 219–230. NIST Special Publication, Gaithersburg (1995)Google Scholar
  10. 10.
    Hofmann, T.: Unsupervised learning by probabilistic latent semantic analysis. Mach. Learn. 42, 177–196 (2001)CrossRefGoogle Scholar
  11. 11.
    Le, Q., Mikolov, T.: Distributed Representations of Sentences and Documents.
  12. 12.
    Phan, X.H., Nguyen, C.T., Le, D.T., Nguyen, L.M., Horiguchi, S., Ha, Q.T.: A hidden topic-based framework towards building applications with short web documents. Trans. KDE 23, 961–976 (2011)Google Scholar
  13. 13.
    Phan, X.H., Nguyen, L.M., Horiguchi, S.: Learning to classify short and sparse text and web with hidden topics from large-scale data collections. In: 17th International World Wide Web Conference, pp. 91–100. ACM, Beijing (2008)Google Scholar
  14. 14.
    Daniel, L.S., Yang, Q., Li, L.: Lifelong machine learning systems: beyond learning algorithms. In: Proceedings of the AAAI Spring Symposium on Lifelong Machine Learning, pp. 49–55. AAAI, Palo Alto (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer Engineering and ScienceShanghai UniversityShanghaiChina

Personalised recommendations