, Volume 115, Issue 3, pp 1293–1313 | Cite as

A novel classification method for paper-reviewer recommendation

  • Shu Zhao
  • Dong Zhang
  • Zhen Duan
  • Jie Chen
  • Yan-ping Zhang
  • Jie Tang


Reviewer recommendation problem in the research field usually refers to invite experts to comment on the quality of papers, proposals, etc. How to effectively and accurately recommend reviewers for the submitted papers and proposals is a meaningful and still tough task. At present, many unsupervised recommendation methods have been researched to solve this task. In this paper, a novel classification method named Word Mover’s Distance–Constructive Covering Algorithm (WMD–CCA, for short) is proposed to solve the reviewer recommendation problem as a classification issue. A submission or a reviewer is described by some tags, such as keywords, research interests, and so on. First, the submission or the reviewer is represented as some vectors by a word embedding method. That is to say, each tag describing a submission or a reviewer is represented as a vector. Second, the Word Mover’s Distance (WMD, for short) method is used to measure the minimum distances between submissions and reviewers. Actually, the papers usually have research field information, and utilizing them well might improve the reviewer recommendation accuracy. So finally, the reviewer recommendation task is transformed into a classification problem which is solved by a supervised learning method- Constructive Covering Algorithm (CCA, for short). Comparative experiments are conducted with 4 public datasets and a synthetic dataset from Baidu Scholar, which show that the proposed method WMD–CCA effectively solves the reviewer recommendation task as a classification issue and improves the recommendation accuracy.


Reviewer recommendation Classification Word embedding Word Mover’s Distance Constructive Covering Algorithm 

Mathematics Subject Classification


JEL Classification

C63 C89 



This work was partially supported by National Natural Science Foundation of China (Grants #61402006, #61602003 and #61673020), National High Technology Research and Development Program (863 Plan)(Grant #2015AA124102), Innovation Zone Project Program for Science and Technology of China’s National Defense (Grant No. 2017-0001-863015-0009), the National Key Research and Development Program of China (2017YFB1401903), the Provincial Natural Science Foundation of Anhui Province (Grants #1508085MF113 and #1708085QF156), Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry (Forty-ninth batch) and the Recruitment Project of Anhui University for Academic and Technology Leader.


  1. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Scholar
  2. Bo, P., & Lee, L. Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. In Meeting on association for computational linguistics, University of Michigan, USA, 2005 (pp. 115–124).Google Scholar
  3. Charlin, L., Zemel, R. S., & Boutilier, C. (2012). A framework for optimizing paper matching. Paper presented at the Proceedings of the twenty-seventh conference on uncertainty in artificial intelligence, Barcelona, Spain.Google Scholar
  4. Conry, D., Koren, Y., & Ramakrishnan, N. Recommender systems for the conference paper assignment problem. In ACM conference on recommender systems, New York, 2009 (pp. 357–360).Google Scholar
  5. Deep, K., & Das, K. N. (2008). Quadratic approximation based hybrid genetic algorithm for function optimization. Applied Mathematics and Computation, 203(1), 86–98.CrossRefzbMATHGoogle Scholar
  6. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391.CrossRefGoogle Scholar
  7. Diaz, F., Mitra, B., & Craswell, N. (2016). Query expansion with locally-trained word embeddings. Paper presented at the proceedings of the 54th annual meeting of the association for computational linguistics, ACL 2016, Berlin.Google Scholar
  8. Fan, Z. P., Chen, Y., Ma, J., & Zhu, Y. (2009). Decision support for proposal grouping: A hybrid approach using knowledge rule and genetic algorithm. Expert Systems with Applications, 36(2), 1004–1013.CrossRefGoogle Scholar
  9. Fang, H., & Zhai, C. X. Probabilistic models for expert finding. In European conference on Ir research, Rome, 2007 (pp. 418–430).Google Scholar
  10. Jin, J., Geng, Q., Zhao, Q., & Zhang, L. Integrating the trend of research interest for reviewer assignment. In International conference on world wide web companion, Perth, 2017 (pp. 1233–1241).Google Scholar
  11. Karimzadehgan, M., & Zhai, C. X. Constrained multi-aspect expertise matching for committee review assignment. In ACM conference on information and knowledge management, Hong Kong, 2009 (pp. 1697–1700).Google Scholar
  12. Karimzadehgan, M., Zhai, C. X., & Belford, G. Multi-aspect expertise matching for review assignment. In ACM conference on information and knowledge management, Napa Valley, 2008 (pp. 1113–1122).Google Scholar
  13. Kolasa, T., & Krol, D. (2011). A survey of algorithms for paper-reviewer assignment problem. IETE Technical Review, 28(2), 123–134.CrossRefGoogle Scholar
  14. Kolasa, T., & Król, D. (2010). ACO-GA approach to paper-reviewer assignment problem in CMS. Berlin: Springer.CrossRefGoogle Scholar
  15. Kusner, M., Sun, Y., Kolkin, N., & Weinberger, K. From word embeddings to document distances. In International conference on machine learning, Lille, 2015 (pp. 957–966).Google Scholar
  16. Li, K., Cao, Z., & Qu, D. Fair reviewer assignment considering academic social network. In Asia-Pacific Web, 2017 (pp. 362–376).Google Scholar
  17. Liu, O., Wang, J., Ma, J., & Sun, Y. (2016). An intelligent decision support approach for reviewer assignment in R&D project selection. Amsterdam: Elsevier.Google Scholar
  18. Liu, X., Suel, T., & Memon, N. (2014) A robust model for paper reviewer assignment. In Eighth ACM conference on recommender systems, RecSys ‘14, Foster City, Silicon Valley, CA, USA (pp. 25–32).Google Scholar
  19. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013a). Efficient estimation of word representations in vector space. Computer Science. arXiv:1301.3781.
  20. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013b). Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 26, 3111–3119.Google Scholar
  21. Mimno, D., & Mccallum, A. Expertise modeling for matching papers with reviewers. In ACM SIGKDD International conference on knowledge discovery and data mining, San Jose, California, USA, August, 2007 (pp. 500–509).Google Scholar
  22. Peng, H., Hu, H., Wang, K., & Wang, X. (2017). Time-aware and topic-based reviewer assignment. Paper presented at the database systems for advanced applications—22nd international conference, Suzhou, China.Google Scholar
  23. Protasiewicz, J., Pedrycz, W., Kozłowski, M., Dadas, S., Stanisławek, T., Kopacz, A., et al. (2016). A recommender system of reviewers and experts in reviewing problems. Knowledge-Based Systems, 106(C), 164–178.CrossRefGoogle Scholar
  24. Ren, Y., Zhang, Y., Zhang, M., & Ji, D. Improving Twitter sentiment classification using topic-enriched multi-prototype word embeddings. In Thirtieth AAAI conference on artificial intelligence, 2016 (pp. 3038–3044).Google Scholar
  25. Rubner, Y., Tomasi, C., & Guibas, L. J. A metric for distributions with applications to image databases. In International conference on computer vision, Bombay, 1998 (pp. 59–66).Google Scholar
  26. Salton. (1971). The SMART retrieval system—experiments in automatic document processing. Upper Saddle River: Prentice-hall Inc.Google Scholar
  27. Sanders, N. J. (2011). Sanders-twitter sentiment corpus.Google Scholar
  28. Shen, Y., Rong, W., Jiang, N., Peng, B., Tang, J., & Xiong, Z. (2017). Word embedding based correlation model for question/answer matching. Paper presented at the proceedings of the thirty-first AAAI conference on artificial intelligence, San Francisco, California.Google Scholar
  29. Shon, H. S., Han, S. H., Kim, K. A., Cha, E. J., & Ryu, K. H. (2017). Proposal reviewer recommendation system based on big data for a national research management institute. Journal of Information Science, 43(2), 147–158.CrossRefGoogle Scholar
  30. Sun, Y. H., Ma, J., Fan, Z. P., & Wang, J. (2008). A hybrid knowledge and model approach for reviewer assignment. Expert Systems with Applications, 34(2), 817–824.CrossRefGoogle Scholar
  31. Tang, J., Fong, A. C. M., Wang, B., & Zhang, J. (2012). A unified probabilistic framework for name disambiguation in digital library. IEEE Transactions on Knowledge and Data Engineering, 24(6), 975–987.CrossRefGoogle Scholar
  32. Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. ArnetMiner:extraction and mining of academic social networks. In ACM SIGKDD international conference on knowledge discovery and data mining, Las Vegas, Nevada, USA, 2008 (pp. 990–998).Google Scholar
  33. Tayal, D. K., Saxena, P. C., Sharma, A., Khanna, G., & Gupta, S. (2014). New method for solving reviewer assignment problem using type-2 fuzzy sets and fuzzy functions. Applied Intelligence, 40(1), 54–73.CrossRefGoogle Scholar
  34. Wang, D. (2008). Fast constructive-covering algorithm for neural networks and its implement in classification. Applied Soft Computing Journal, 8(1), 166–173.CrossRefGoogle Scholar
  35. Xu, Y., Ma, J., Sun, Y., Hao, G., Xu, W., & Zhao, D. (2010). A decision support approach for assigning reviewers to proposals. Expert Systems with Applications, 37(10), 6948–6956.CrossRefGoogle Scholar
  36. Yang, K. H., Kuo, T. L., Lee, H. M., & Ho, J. M. A Reviewer recommendation system based on collaborative intelligence. In Ieee/wic/acm international joint conference on web intelligence and intelligent agent technology, Milan, 2009 (pp. 564–567).Google Scholar
  37. Zhang, Y., Xing, H., Zou, H., Zhao, S., & Wang, X. A three-way decisions model based on constructive covering algorithm. In International conference on rough sets and knowledge technology, Halifax, 2013 (pp. 346–353).Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2018

Authors and Affiliations

  • Shu Zhao
    • 1
  • Dong Zhang
    • 1
  • Zhen Duan
    • 1
  • Jie Chen
    • 1
  • Yan-ping Zhang
    • 1
  • Jie Tang
    • 2
  1. 1.School of Computer Science and TechnologyAnhui UniversityHefeiChina
  2. 2.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations