Research into Reviewer Assignment Problem (RAP) is still in its early stage but there is great world-wide interest, as the foregoing process of peer-review which is the brickwork of science authentication. The RAP approach can be divided into three phases: identifying assignment procedure, computing the matching degree between manuscripts and reviewers, and optimizing the assignment so as to achieve the given objectives. Methodologies for addressing the above three phases have been developed from a variety of research disciplines, including information retrieval, artificial intelligent, operations research, etc. This survey is not only to cover variations of RAP that have appeared in the literature, but also to identify the practical challenge and current progress for developing intelligent RAP systems.


reviewer assignment information retrieval conference system 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Basu, C., Hirsh, H., Cohen, W.: Recommendation as Classification: Using Social and Content-based Information in Recommendation. In: 15th national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, pp. 714–720. AAAI, USA (1998)Google Scholar
  2. 2.
    Basu, C., Hirsh, H., Cohen, W., Nevill-Manning, C.: Technical paper recommendation: A study in combining multiple information sources. Journal of Artificial Intelligence Research 14, 231–252 (2001)MATHGoogle Scholar
  3. 3.
    Bausch, D.O., Brown, G.G., Hundley, D.R., Rapp, S.H., Rosenthal, R.E.: Mobilizing Marine Corps Officers. Interfaces 21(4), 26–38 (1991)Google Scholar
  4. 4.
    Benferhat, S., Lang, J.: Conference Paper Assignment. International Journal of Intelligent Systems 16, 1183–1192 (2001)MATHCrossRefGoogle Scholar
  5. 5.
    Biswas, H.K., Hasan, M.M.: Using Publications and Domain Knowledge to Build Research Profiles: an Application in Automatic Reviewer Assignment. In: 2007 International Conference on Information and Communication Technology, pp. 82–86 (2007)Google Scholar
  6. 6.
    Cameron, D., Aleman-Meza, B., Arpinar, B.: Collecting Expertise of Researchers for Finding Relevant Experts in a Peer-Review Setting. In: First International ExpertFinder Workshop (2007)Google Scholar
  7. 7.
    Caron, G., Hansen, P., Jaumard, B.: The assignment problem with seniority and job priority constraints. Operations Research 47(3), 449–454 (1999)MATHMathSciNetCrossRefGoogle Scholar
  8. 8.
    Carter, M.W., Tovey, C.A.: When is the classroom assignment problem hard? Operations Research 40(1), S28–S39 (1992)Google Scholar
  9. 9.
    Casati, F., Giunchiglia, F., Marchese, M.: Publish and perish: Why the Current Publication and Review Model is Killing Research and Wasting Your Money, http://www.acm.org/ubiquity/views/v8i03_fabio.html
  10. 10.
    Cohen, P.R., Kjeldsen, R.: Information retrieval by constrained spreading activation in semantic network. Information Processing & Management 23(4), 255–268 (1987)CrossRefGoogle Scholar
  11. 11.
    Cohen, W., Fan, W.: Web-collaborative Filtering: Recommending Music by Crawling the Web. Computer Networks 33(1-6), 685–698 (2000)CrossRefGoogle Scholar
  12. 12.
    Cook, W.D., Golany, B., Kress, M., Penn, M., Raviv, T.: Optimal Allocation of Proposals to Reviewers to Facilitate Effective Ranking. Management Science 51(4), 655–661 (2005)CrossRefGoogle Scholar
  13. 13.
    Dell’Amico, M., Martello, S.: The k-cardinality assignment problem. Discrete Applied Mathematics 76(1-3), 103–121 (1997)MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Dumais, S., Nielsen, J.: Automating the assignment of submitted manuscripts to reviewers. Research and Development in information Retrieval, 233–244 (1992)Google Scholar
  15. 15.
    Geller, J.: How IJCAI 1999 can prove the value of AI by using AI. In: 15th International Joint Conference on Artificial Intelligence, pp. 55–58 (1997)Google Scholar
  16. 16.
    Geller, J., Scherl, R.: Challenge: Technology for Automated Reviewer Selection (1997), http://njit.edu/~geller/finalchall.ps1997
  17. 17.
    Goldsmith, J., Solan, R.H.: The AI Conference Paper Assignment Problem, http://www.cs.uic.edu/~sloan/my-papers/GodlsmithSloanPaperAssignment.pdf
  18. 18.
    Gupta, D., Digiovanni, M., Narita, H., Goldberg, K.: Jester 2.0: A new Lineartime Collaborative Filtering Algorithm Applied to Jokes. In: Workshop on Recommender Systems at ACM SIGIR 1999 (1999)Google Scholar
  19. 19.
    Hansen, P., Wendell, R.E.: A note on airline commuting. Interfaces 11(12), 85–87 (1982)Google Scholar
  20. 20.
    Hartvigsen, D., Wei, J.C., Czuchlewski, R.: The Conference Paper-Reviewer Assignment Problem. Decision Sciences 30(3), 865–876 (1999)CrossRefGoogle Scholar
  21. 21.
    Hettich, S., Pazzani, M.J.: Mining for Proposal Reviewers: Lessons Learned at the National Science Foundation. In: 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 862–871. ACM Press, New York (2006)CrossRefGoogle Scholar
  22. 22.
    Hill, W., Stead, L., Rosenstein, M., Furnas, G.: Recommending and Evaluating Choices in a Virtual Community of Use. In: SIGCHI conference on Human factors in computing systems, pp. 195–201. ACM Press/Addison-Wesley Publishing Co., New York (1995)Google Scholar
  23. 23.
    Hofmann, T.: Probabilistic Latent Semantic Analysis. In: 15th Conference on Uncertainty in Artificial Intelligence, pp. 289–296 (1999)Google Scholar
  24. 24.
    Janak, S.L., Taylor, M.S., Floudas, C.A., Burka, M., Mountzizris, T.J.: Novel and Effective Integer Optimization Approach for the NSF Panel-Assignment Problem: A Multiresource and Preference-Constrained Generalized Assignment Problem. Ind. Eng. Chem. Res. 45(1), 258–265 (2006)CrossRefGoogle Scholar
  25. 25.
    Klingman, D., Phillips, N.: Topological and computational aspects of preemptive multicriteria military personnel assignment problems. Manage. Sci. 30(1), 1362–1375 (1984)MathSciNetGoogle Scholar
  26. 26.
    Konstan, J., Miller, B., Maltz, D., Herlocker, L., Gordon, L., Riedl, J.: Grouplens: Applying Collaborative Filtering to Usenet News. Communications of the ACM 40(3), 77–87 (1997)CrossRefGoogle Scholar
  27. 27.
    LeBlanc, L.J., Randels, D., Swann, T.K.: Heery International’s Spreadsheet Optimization Model for Assigning Managers to Construction Projects. INTERFACE 30(6), 95–106 (2000)CrossRefGoogle Scholar
  28. 28.
    Merelo-Guervós, J.J., Castillo-Valdivieso, P.: Conference Paper Assignment Using a Combined Greedy/ Evolutionary Algorithm. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 602–611. Springer, Heidelberg (2004)Google Scholar
  29. 29.
    Merelo-Guervós, J.J., García-Castellano, F.J., Castillo, P.A., Arenas, M.G.: How Evolutionary Computation and Perl saved my conference. In: Sánchez, L. (ed.) Segundo Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados, pp. 93–99 (2003)Google Scholar
  30. 30.
    Popescul, A., Ungar, L.H., Pennock, D.M., Lawrence, S.: Probabilistic Models for Unified Collaborative and Content-base Recommendation in Sparse-Data Environments. In: 17th Conference on Uncertainty in Artificial Intelligence, pp. 437–444. Morgan Kaufmann Publishers Inc., San Francisco (2001)Google Scholar
  31. 31.
    Rodriguez, M.A., Bollen, J., Van de Sompel, H.: Mapping the Bid Behavior of Conference Referees. Journal of Informetrics 1(1), 62–82 (2007)CrossRefGoogle Scholar
  32. 32.
    Rodriguez, M.A., Bollen, J.: An Algorithm to Determine Peer-Reviewers. Arxiv preprint cs.DL/0605112 (2006)Google Scholar
  33. 33.
    Schirrer, A., Doerner, K.F., Hartl, R.F.: Reviewer Assignment for Scientific Articles using Memetic Algorithms. OR/CS Interfaces Series 39, 113–134 (2007)Google Scholar
  34. 34.
    Scott, A.: Peer review and the relevance of science. Futures 39, 827–845 (2007)CrossRefGoogle Scholar
  35. 35.
    Shardanand, U., Maes, P.: Social Information Filtering: Algorithms for Automating “Word of Mouth”. In: SIGCHI conference on Human factors in computing systems, pp. 210–217. ACM Press/Addison-Wesley Publishing Co., New York (1995)Google Scholar
  36. 36.
    Sun, Y.H., Ma, J., Fan, Z.P., Wang, J.: A Hybrid Knowledge and Model Approach for Reviewer Assignment. In: 40th Annual Hawaii International Conference on System Sciences, p. 47. IEEE Computer Society, Washington (2007)CrossRefGoogle Scholar
  37. 37.
    Tian, Q., Ma, J., Liu, O.: A Hybrid Knowledge and Model System for R&D Project Selection. Expert Systems with Applications 23(3), 265–271 (2002)CrossRefGoogle Scholar
  38. 38.
    Veronika, A., Riantini, L.S., Trigunarsyah, B.: Corrective Action Recommendation For Project Cost Variance in Construction Material Management. In: Kanok-Nukulchai, W., Munasinghe, S., Anwar, N. (eds.) 10th East Asia-Pacific Conference on Structural Engineering and Construction 2005, pp. 23–28 (2006)Google Scholar
  39. 39.
    Watanabe, S., Ito, T., Ozono, T., Shintani, T.: A Paper Recommendation Mechanism for the Research Support System Papits. In: International Workshop on Data Engineering Issues in E-Commerce, pp. 71–80 (2005)Google Scholar
  40. 40.
    Weber, R.: The Journal Review Process: a Manifesto for Change. Communications of the Association for Information Systems 2(2-3) (1999)Google Scholar
  41. 41.
    Yarowsky, D., Florian, R.: Taking the load off the conference chairs: towards a digital paper-routing assistant. In: 1999 Joint SIGDAT Conference on Empirical Methods in NLP and Very-Large Corpora (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Fan Wang
    • 1
  • Ben Chen
    • 1
  • Zhaowei Miao
    • 2
  1. 1.School of BusinessSun Yat-Sen UniversityGuangzhouP.R. China
  2. 2.Management SchoolXiamen UniversityXiamenP.R. China

Personalised recommendations