Assigning Papers to Referees

Abstract

Refereed conferences require every submission to be reviewed by members of a program committee (PC) in charge of selecting the conference program. There are many software packages available to manage the review process. Typically, in a bidding phase PC members express their personal preferences by ranking the submissions. This information is used by the system to compute an assignment of the papers to referees (PC members).

We study the problem of assigning papers to referees. We propose to optimize a number of criteria that aim at achieving fairness among referees/papers. Some of these variants can be solved optimally in polynomial time, while others are NP-hard, in which case we design approximation algorithms. Experimental results strongly suggest that the assignments computed by our algorithms are considerably better than those computed by popular conference management software.

References

  1. 1.

    Linklings: http://www.linklings.com/

  2. 2.

    Softconf: http://www.softconf.com/

  3. 3.

    Alkan, A., Demange, G., Gale, D.: Fair allocation of indivisible goods and criteria of justice. Econometrica 59(4), 1023–1039 (1991)

    MATH  Article  MathSciNet  Google Scholar 

  4. 4.

    Apers, P.: Acceptance rates major database conferences. http://wwwhome.cs.utwente.nl/~apers/rates.html

  5. 5.

    Bezáková, I., Dani, V.: Allocating indivisible goods. SIGecom Exch. 5(3), 11–18 (2005)

    Article  Google Scholar 

  6. 6.

    Bogomolnaia, A., Moulin, H.: A new solution to the random assignment problem. J. Econ. Theory 100(2), 295–328 (2001)

    MATH  Article  MathSciNet  Google Scholar 

  7. 7.

    Bogomolnaia, A., Moulin, H.: A simple random assignment problem with a unique solution. Econ. Theory 75(3), 257–279 (2004)

    Google Scholar 

  8. 8.

    Chaudhuri, S.: Microsoft’s academic conference management service. http://cmt.research.microsoft.com/cmt/

  9. 9.

    Cook, W., Golany, B., Kress, M., Penn, M., Raviv, T.: Optimal allocation of proposals to reviewers to facilitate effective ranking. Manag. Sci. 51(4), 655–661 (2005)

    Article  Google Scholar 

  10. 10.

    Ghouilla-Houri, A.: Charactérisations des matrices totalment unimodulaires. C. R. Acad. Sci. Paris 254, 1192–1193 (1962)

    MathSciNet  Google Scholar 

  11. 11.

    Goldsmith, J., Sloan, R.H.: The AI conference assignment problem. In: Proceedings of the Workshop on Preference Handling for Artificial Intelligence (2007)

  12. 12.

    Halevi, S.: Websubrev, http://people.csail.mit.edu/shaih/websubrev/

  13. 13.

    Hartvigsen, D., Wei, J.C., Czuchlewski, R.: The conference paper-reviewer assignment problem. Decis. Sci. 30(3), 865–876 (1999)

    Article  Google Scholar 

  14. 14.

    Irving, R.W., Kavitha, T., Mehlhorn, K., Michail, D., Paluch, K.: Rank-maximal matchings. ACM Trans. Algorithms 2(4), 602–610 (2006)

    Article  MathSciNet  Google Scholar 

  15. 15.

    Janak, S.L., Taylor, M.S., Floudas, C.A., Burka, M., Mountzizris, T.J.: Novel and effective integer optimization approach for the NSF panel-assignment problem. Ind. Eng. Chem. Res. 45(1), 258–265 (2005)

    Article  Google Scholar 

  16. 16.

    Karp, R.M.: Reducibility among combinatorial problems. In: Complexity of Computer Computations, pp. 85–103. Plenum, New York (1972)

    Google Scholar 

  17. 17.

    Katta, A.-K., Sethuraman, J.: A solution to the random assignment problem on the full preference domain. J. Econ. Theory 131(1), 231–250 (2005)

    Article  MathSciNet  Google Scholar 

  18. 18.

    Kohler, E.: HotCRP. http://www.cs.ucla.edu/~kohler/hotcrp/

  19. 19.

    Lenstra, J.K., Shmoys, D.B., Tardos, É.: Approximation algorithms for scheduling unrelated parallel machines. Math. Program. 46, 259–271 (1990)

    MATH  Article  MathSciNet  Google Scholar 

  20. 20.

    Mehlhorn, K., Michail, D.: Network problems with non-polynomial weights and applications. Manuscript

  21. 21.

    Merelo-Gurvós, J.J., Castillo-Valdivieso, P.: Conference paper assignment using a combined greedy/evolutionary algorithm. In: Proceedings of Parallel Problem Solving from Nature, pp. 602–611 (2004)

  22. 22.

    Moulin, H.: Fair Division and Collective Welfare. MIT Press, Cambridge (2003)

    Google Scholar 

  23. 23.

    O’Dell, R., Wattenhofer, M., Wattenhofer, R.: The paper assignment problem. Technical Report 491, ETH Zürich (2005)

  24. 24.

    Shiloach, Y.: Another look at the degree constrained subgraph problem. Inf. Process. Lett. 12(2), 89–92 (1981)

    MATH  Article  MathSciNet  Google Scholar 

  25. 25.

    Shmoys, D.B., Tardos, É.: An approximation algorithm for the generalized assignment problem. Math. Program. 62, 461–474 (1993)

    Article  MathSciNet  Google Scholar 

  26. 26.

    Svensson, L.-G.: Strategy-proof allocation of indivisible goods. Social Welf. Choice 16(4), 557–567 (1999)

    MATH  Article  MathSciNet  Google Scholar 

  27. 27.

    Tungodden, B.: Egalitarianism: Is leximin the only option? Econ. Philos. 16, 229–245 (2000)

    Article  Google Scholar 

  28. 28.

    Voronkov, A.: Personal communication

  29. 29.

    Voronkov, A.: Easychair. http://www.easychair.org

  30. 30.

    Voronkov, A.: Easychair—users. http://www.easychair.org/users.cgi

  31. 31.

    Wang, F., Chen, B., Miao, Z.: A survey on reviewer assignment problem. In: Proceedings of the 21st International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 718–727 (2008)

  32. 32.

    Zhou, L.: On a conjecture by Gale about one-sided matching problems. J. Econ. Theory 52(1), 123–135 (1990)

    MATH  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Julián Mestre.

Additional information

J. Mestre’s research is supported by an Alexander von Humboldt Fellowship.

Rights and permissions

Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Reprints and Permissions

About this article

Cite this article

Garg, N., Kavitha, T., Kumar, A. et al. Assigning Papers to Referees. Algorithmica 58, 119–136 (2010). https://doi.org/10.1007/s00453-009-9386-0

Download citation

Keywords

  • Fair assignment
  • Rank-maximal matchings
  • Leximin principle