How to Keep Bad Papers Out of Conferences (with Minimum Reviewer Effort)
Reviewing conference submissions is both labour-intensive and diffuse. A lack of focus leads to reviewers spending much of their scarce time on papers which will not be accepted, which can prevent them from identifying several classes of problems with papers that will be. We identify opportunities for automation in the review process and propose protocols which allow human reviewers to better focus their limited time and attention, making it easier to select only the best “genetic” material to incorporate into their conference’s “DNA.” Some of the protocols that we propose are difficult to “game” without uneconomic investment on the part of the attacker, and successfully attacking others requires attackers to provide a positive social benefit to the wider research community.
KeywordsProgram Committee Threat Model Signalling Protocol Bibliographic Coupling Mechanical Assistance
Unable to display preview. Download preview PDF.
- 4.Simkin, M.V., Roychowdhury, V.P.: A Mathematical Theory of Citing. Journal of the American Society for Information Science and Technology 58(11) (2007)Google Scholar
- 5.Stamp, M.: Risks of monoculture. Communications of the ACM 47(3) ( March 2004)Google Scholar
- 6.Tang, J., Zhang, D., Yao, L.: Social Network Extraction of Academic Researchers. In: Seventh IEEE International Conference on Data Mining (ICDM), pp. 292–301 (2007)Google Scholar
- 7.Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., Su, Z.: ArnetMiner: extraction and mining of academic social networks. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM (August 2008)Google Scholar