Assessing non-standard article impact using F1000 labels
Faculty of 1000 (F1000) is a post-publishing peer review web site where experts evaluate and rate biomedical publications. F1000 reviewers also assign labels to each paper from a standard list or article types. This research examines the relationship between article types, citation counts and F1000 article factors (FFa). For this purpose, a random sample of F1000 medical articles from the years 2007 and 2008 were studied. In seven out of the nine cases, there were no significant differences between the article types in terms of citation counts and FFa scores. Nevertheless, citation counts and FFa scores were significantly different for two article types: “New finding” and “Changes clinical practice”: FFa scores value the appropriateness of medical research for clinical practice and “New finding” articles are more highly cited. It seems that highlighting key features of medical articles alongside ratings by Faculty members of F1000 could help to reveal the hidden value of some medical papers.
KeywordsFaculty of F1000 Altmetrics Beyond impact Research assessment Post-publishing peer review
- Bornmann, L., & Leydesdorff, L. (2012). The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000. Digital Libraries; Applications. http://arxiv.org/abs/1211.1154.
- Cronin, B. (1984). The citation process. The role and significance of citations in scientific communication. London: Taylor Graham.Google Scholar
- F1000. (2012a). About F1000. http://f1000.com/prime/about/whatis.
- F1000. (2012b). F1000 Faculty. http://f1000.com/prime/thefaculty.
- Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008). Comparison of SCImago journal rank indicator with journal impact factor. FASEB journal: official publication of the Federation of American Societies for Experimental Biology, 22(8), 2623–2628. doi: 10.1096/fj.08-107938.CrossRefGoogle Scholar
- Field, A. (2009). Discovering statistics using SPSS. Thousand Oaks: SAGE.Google Scholar
- Fienberg, S. E., & Martin, M. E. (1985). Sharing research data. Washington: Natl Academy.Google Scholar
- Huggett, S. (2012). F1000 Journal Rankings: An alternative way to evaluate the scientific impact of scholarly communications. Research Trends, 26, 7–11.Google Scholar
- Lewison, G. (2005). Citations to papers from other documents. Handbook of Quantitative Science and Technology. http://www.springerlink.com/index/T2H0245570526217.pdf.
- Li, & Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. 17th International Conference on Science and Technology Indicators (Vol. 3, pp. 1–11).Google Scholar
- Mahdi, S., D’Este, P., & Neely, A. D. (2008). Citation counts: Are they good predictors of RAE scores?: A bibliometric analysis of RAE 2001. London: AIM Research.Google Scholar
- Moed, H. F. (2005). Citation analysis in research evaluation (Vol. 9). Norwell: Kluwer Academic.Google Scholar
- Oppenheim, C., & Summers, M. A. C. (2008). Citation counts and the Research Assessment Exercise, part VI: Unit of assessment 67 (music). Information Research, 13(2), 3.Google Scholar
- Priem, & Hemminger, B. M. H. (2010). Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday, 15(7), http://frodo.lib.uic.edu/ojsjournals/index.php/fm/. Retrieved from http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2874.
- Priem, Taraborelli, D., Groth, P., & Neylon, C. (2011). Altmetrics: A manifesto. http://altmetrics.org/manifesto.
- Sarli, C. C., & Holmes, K. L. (2012). The becker medical library model for assessment of research impact. St Louis: Bernard Becker Medical Library, Washington University School of Medicine.Google Scholar
- Small, H. (2004). On the shoulders of Robert Merton: Towards a normative theory of citation. Scientometrics, 60(1), 71–79. http://www.springerlink.com/index/X6VTVM1209131570.pdf.Google Scholar
- Smith, A. T., & Eysenck, M. (2002). The correlation between RAE ratings and citation counts in psychology. London.Google Scholar
- Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.Google Scholar
- Wardle, D. A. (2010). Do’Faculty of 1000′(F1000) ratings of ecological publications serve as reasonable predictors of their future impact? Ideas in Ecology and Evolution, 3, 11–15.Google Scholar
- Zaman, M. uz, & Britain, G. (2004). Review of the academic evidence on the relationship between teaching and research in higher education. https://www.education.gov.uk/publications/eOrderingDownload/RR506.pdf.