Skip to main content
SpringerLink
Log in
Book cover

International Conference on Neural Information Processing

ICONIP 2018: Neural Information Processing pp 473–482Cite as

Fuzzy Bag-of-Topics Model for Short Text Representation

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11305))

Abstract

Text representation is the keystone in many NLP tasks. For short text representation learning, the traditional Bag-of-Words model (BoW) is often criticized for sparseness and neglecting semantic information. Fuzzy Bag-of-Words (FBoW) and Fuzzy Bag-of-Words Cluster (FBoWC) model are the improved model of BoW, which can learn dense and meaningful document vectors. However, word clusters in FBoWC model are obtained by K-means cluster algorithm, which is unstable and may result in incoherent word clusters if not initialized properly. In this paper, we propose the Fuzzy Bag-of-Topics model (FBoT) to learn short text vector. In FBoT model, word communities, which are more coherent than word clusters in FBoWC, are used as basis terms in text vector. Experimental results of short text classification on two datasets show that FBoT achieves the highest classification accuracies.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://gist.github.com/kunalj101.

  2. 2.

    http://scikit-learn.org.

  3. 3.

    https://radimrehurek.com/gensim.

References

  1. Spärck Jones, K.: A statistical interpretation of term specificity and its application to retrieval. J. Doc. 28, 11–21 (1972)

    Article  Google Scholar 

  2. Zhao, R., Mao, K.: Fuzzy bag-of-words model for document representation. IEEE Trans. Fuzzy Syst. 26, 794–804 (2018)

    Article  Google Scholar 

  3. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient Estimation of Word Representations in Vector Space. http://arxiv.org/abs/1301.3781

  4. Banerjee, S., Ramanathan, K., Gupta, A.: Clustering short texts using Wikipedia. In: 30th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 787–788. ACM, Amsterdam (2007)

    Google Scholar 

  5. Hu, X., Sun, N., Zhang, C., Chua, T.S., et al.: Exploring internal and external semantics for the clustering of short texts using world knowledge. In: 18th ACM Conference on Information and Knowledge Management, pp. 919–928. ACM, Hong Kong (2009)

    Google Scholar 

  6. Wang, L., Jia, Y., Han, W.: Instant message clustering based on extended vector space model. In: Kang, L., Liu, Y., Zeng, S. (eds.) ISICA 2007. LNCS, vol. 4683, pp. 435–443. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74581-5_48

    Chapter  Google Scholar 

  7. Sahami, M., Heilman, T.D.: A web-based kernel function for measuring the similarity of short text snippets. In: 15th International Conference on World Wide Web, pp. 377–386. ACM, Edinburgh (2006)

    Google Scholar 

  8. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  9. Dumain S.: Latent Semantic Indexing (LSI): TREC-3 Report. In: Harman, M. (ed.) The Third Text REtrieval Conference, vol. 500, no. 226, pp. 219–230. NIST Special Publication, Gaithersburg (1995)

    Google Scholar 

  10. Hofmann, T.: Unsupervised learning by probabilistic latent semantic analysis. Mach. Learn. 42, 177–196 (2001)

    Article  Google Scholar 

  11. Le, Q., Mikolov, T.: Distributed Representations of Sentences and Documents. http://arxiv.org/abs/1405.4053

  12. Phan, X.H., Nguyen, C.T., Le, D.T., Nguyen, L.M., Horiguchi, S., Ha, Q.T.: A hidden topic-based framework towards building applications with short web documents. Trans. KDE 23, 961–976 (2011)

    Google Scholar 

  13. Phan, X.H., Nguyen, L.M., Horiguchi, S.: Learning to classify short and sparse text and web with hidden topics from large-scale data collections. In: 17th International World Wide Web Conference, pp. 91–100. ACM, Beijing (2008)

    Google Scholar 

  14. Daniel, L.S., Yang, Q., Li, L.: Lifelong machine learning systems: beyond learning algorithms. In: Proceedings of the AAAI Spring Symposium on Lifelong Machine Learning, pp. 49–55. AAAI, Palo Alto (2013)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by Shanghai Innovation Action Plan Project under the grant No. 16511101200.

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, China

    Hao Jia & Qing Li

Authors
  1. Hao Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qing Li .

Editor information

Editors and Affiliations

  1. The Chinese Academy of Sciences, Beijing, China

    Long Cheng

  2. City University of Hong Kong, Kowloon, Hong Kong

    Andrew Chi Sing Leung

  3. Kobe University, Kobe, Japan

    Seiichi Ozawa

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jia, H., Li, Q. (2018). Fuzzy Bag-of-Topics Model for Short Text Representation. In: Cheng, L., Leung, A., Ozawa, S. (eds) Neural Information Processing. ICONIP 2018. Lecture Notes in Computer Science(), vol 11305. Springer, Cham. https://doi.org/10.1007/978-3-030-04221-9_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-04221-9_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-04220-2

  • Online ISBN: 978-3-030-04221-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation