Extracting Keyphrases from Research Papers Using Word Embeddings

  • Wei Fan
  • Huan Liu
  • Suge Wang
  • Yuxiang ZhangEmail author
  • Yaocheng Chang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11441)


Unsupervised random-walk keyphrase extraction models mainly rely on global structural information of the word graph, with nodes representing candidate words and edges capturing the co-occurrence information between candidate words. However, integrating different types of useful information into the representation learning process to help better extract keyphrases is relatively unexplored. In this paper, we propose a random-walk method to extract keyphrases using word embeddings. Specifically, we first design a new word embedding learning model to integrate local context information of the word graph (i.e., the local word collocation patterns) with some crucial features of candidate words and edges. Then, a novel random-walk ranking model is designed to extract keyphrases by leveraging such word embeddings. Experimental results show that our approach outperforms 8 state-of-the-art unsupervised methods on two real datasets consistently for keyphrase extraction.


Keyphrase extraction Word embeddings Ranking model 



This work was partially supported by grants from the National Natural Science Foundation of China (No. 61632011, 61573231, U1633110, U1533104, U1333109) and Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province (No. CICIP2018004).


  1. 1.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. PAMI 35(8), 1798–1828 (2013)CrossRefGoogle Scholar
  2. 2.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3(1), 993–1022 (2003)zbMATHGoogle Scholar
  3. 3.
    Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. ACL 5(1), 135–146 (2017)Google Scholar
  4. 4.
    Caragea, C., Bulgarov, F., Godea, A., Gollapalli, S.D.: Citation-enhanced keyphrase extraction from research papers: a supervised approach. In: Proceedings of EMNLP, pp. 1435–1446 (2014)Google Scholar
  5. 5.
    Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)CrossRefGoogle Scholar
  6. 6.
    Florescu, C., Caragea, C.: Positionrank: an unsupervised approach to keyphrase extraction from scholarly documents. In: Proceedings of ACL, pp. 1105–1115 (2017)Google Scholar
  7. 7.
    Gollapalli, S.D., Caragea, C.: Extracting keyphrases from research papers using citation networks. In: Proceedings of AAAI, pp. 1629–1635 (2014)Google Scholar
  8. 8.
    Hasan, K.S., Ng, V.: Automatic keyphrase extraction: a survey of the state of the art. In: Proceedings of ACL, pp. 1262–1273 (2014)Google Scholar
  9. 9.
    Liu, Y., Liu, Z., Chua, T.S., Sun, M.: Topical word embeddings. In: Proceedings of AAAI, pp. 2418–2424 (2015)Google Scholar
  10. 10.
    Liu, Z., Huang, W., Zheng, Y., Sun, M.: Automatic keyphrase extraction via topic decomposition. In: Proceedings of EMNLP, pp. 366–376 (2010)Google Scholar
  11. 11.
    Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008)CrossRefGoogle Scholar
  12. 12.
    Mihalcea, R., Tarau, P.: Textrank: bringing order into text. In: Proceedings of EMNLP, pp. 404–411 (2004)Google Scholar
  13. 13.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of NIPS, pp. 3111–3119 (2013)Google Scholar
  14. 14.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: bringing order to the web. Technical report, Stanford InfoLab (1999)Google Scholar
  15. 15.
    Sterckx, L., Demeester, T., Deleu, J., Develder, C.: Topical word importance for fast keyphrase extraction. In: Proceedings of WWW, pp. 121–122 (2015)Google Scholar
  16. 16.
    Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: large-scale information network embedding. In: Proceedings of WWW, pp. 1067–1077 (2015)Google Scholar
  17. 17.
    Teneva, N., Cheng, W.: Salience rank: efficient keyphrase extraction with topic modeling. In: Proceedings of ACL, pp. 530–535 (2017)Google Scholar
  18. 18.
    Wan, X., Xiao, J.: Single document keyphrase extraction using neighborhood knowledge. In: Proceedings of AAAI, pp. 855–860 (2008)Google Scholar
  19. 19.
    Wang, R., Liu, W., McDonald, C.: Corpus-independent generic keyphrase extraction using word embedding vectors. In: Proceedings of DL-WSDM, pp. 39–46 (2015)Google Scholar
  20. 20.
    Wang, Y., Jin, Y., Zhu, X., Goutte, C.: Extracting discriminative keyphrases with learned semantic hierarchies. In: Proceedings of COLING, pp. 932–942 (2016)Google Scholar
  21. 21.
    Zhang, W., Feng, W., Wang, J.: Integrating semantic relatedness and words’ intrinsic features for keyword extraction. In: Proceedings of IJCAI, pp. 139–160 (2013)Google Scholar
  22. 22.
    Zhang, Y., Chang, Y., Liu, X., Gollapalli, S.D., Li, X., Xiao, C.: Mike: keyphrase extraction by integrating multidimensional information. In: Proceedings of CIKM, pp. 1349–1358 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Wei Fan
    • 1
  • Huan Liu
    • 1
  • Suge Wang
    • 2
  • Yuxiang Zhang
    • 1
    Email author
  • Yaocheng Chang
    • 1
  1. 1.School of Computer Science and TechnologyCivil Aviation University of ChinaTianjinChina
  2. 2.School of Computer and Information TechnologyShanxi UniversityTaiyuanChina

Personalised recommendations