Reviewers’ scores do not predict impact: bibliometric analysis of the proceedings of the human–robot interaction conference
- 635 Downloads
The peer review process is an essential component for the progress of science. The ACM/IEEE International Conference on Human–Robot Interaction is the prime publication channel for the field and this study evaluates its peer review process. The results show that the number of accepted papers are unevenly distributed across countries, organizations and authors. The contributions from the US outweigh all others contributions. A Binary Logistic Regression analysis showed that only for 85.5% of the papers the reviewers’ scores accurately predict its acceptance or rejection. Moreover, there was no significant correlation between the reviewers’ scores and the citations the papers later attract. 73% of the authors only ever submitted one paper and the proportion of newcomers at the conferences ranges from 63–77%.
KeywordsHuman–robot interaction HRI Conference Proceedings Acceptance rate
The author would like to thank Utku Yalcin and Subha Krishna for the data processing.
- Bartneck, C. (2010). The all-in publication policy. Fourth International Conference on Digital Society (ICDS 2010), IEEE (pp. 37–40). doi: 10.1109/ICDS.2010.14.
- Langford, J. (2015). The NIPS experiment. http://cacm.acm.org/blogs/blog-cacm/181996-the-nips-experiment/fulltext.
- Matthews, D. (2016). High rejection rates by journals pointless. https://www.timeshighereducation.com/news/high-rejection-rates-by-journals-pointless.
- Meho, L. I., & Rogers, Y. (2008). Citation counting, citation ranking, and h-index of human–computer interaction researchers: A comparison of scopus and web of science. Journal of the American Society for Information Science and Technology, 59(11), 1711–1726. doi: 10.1002/asi.20874.CrossRefGoogle Scholar
- Price, E. (2014). The NIPS experiment. http://blog.mrtz.org/2014/12/15/the-nips-experiment.html.
- Safi, M. (2015). Journal accepts bogus paper requesting removal from mailing list. http://www.theguardian.com/australia-news/2014/nov/25/journal-accepts-paper-requesting-removal-from-mailing-list.
- Rocha da Silva, P. (2015). Selecting for impact: new data debunks old beliefs. http://blog.frontiersin.org/2015/12/21/4782/.