Open access scientometrics and the UK Research Assessment Exercise
First Online: 13 November 2008 Received: 05 December 2007 DOI:
Cite this article as: Harnad, S. Scientometrics (2009) 79: 147. doi:10.1007/s11192-009-0409-z Abstract
Scientometric predictors of research performance need to be validated by showing that they have a high correlation with the external criterion they are trying to predict. The UK Research Assessment Exercise (RAE) — together with the growing movement toward making the full-texts of research articles freely available on the web — offer a unique opportunity to test and validate a wealth of old and new scientometric predictors, through multiple regression analysis: Publications, journal impact factors, citations, co-citations, citation chronometrics (age, growth, latency to peak, decay rate), hub/authority scores, h-index, prior funding, student counts, co-authorship scores, endogamy/exogamy, textual proximity, download/co-downloads and their chronometrics, etc. can all be tested and validated jointly, discipline by discipline, against their RAE panel rankings in the forthcoming parallel panel-based and metric RAE in 2008. The weights of each predictor can be calibrated to maximize the joint correlation with the rankings. Open Access Scientometrics will provide powerful new means of navigating, evaluating, predicting and analyzing the growing Open Access database, as well as powerful incentives for making it grow faster.
References Brody, T.
(2003), Citebase search: Autonomous citation database for e-print archives.
ECS Technical Report
http://eprints.ecs.soton.ac.uk/10677/ Brody, T., Harnad, S., Carr, L.
(2006), Earlier web usage statistics as predictors of later citation impact.
Journal of the American Association for Information Science and Technology (JASIST)
, 57(8): 1060–1072.
http://eprints.ecs.soton.ac.uk/10713/ CrossRef Garfield, E.
, (1955) Citation indexes for science: A new dimension in documentation through association of ideas.
, 122: 108–111,
http://www.garfield.library.upenn.edu/papers/science_v122(3159)p108y1955.html CrossRef Giles, C. L., Bollacker, K., Lawrence, S.
(1998), CiteSeer: An automatic citation indexing system.
Digital Libraries 98: Third ACM Conf. on Digital Libraries
, ACM Press, New York, 1998, pp. 89–98.
http://www.neci.nj.nec.com/homepages/lawrence/citeseer.html CrossRef Guédon, J.-C.
In Oldenburg’s Long Shadow: Librarians, Research Scientists, Publishers, and the Control of Scientific Publishing
. Washington, DC: The Association of Research Libraries.
http://www.arl.org/resources/pubs/mmproceedings/138guedon.shtml Hajjem, C., Harnad, S., Gingras, Y.
(2005), Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact.
IEEE Data Engineering Bulletin
, 28(4) 39-47.
http://eprints.ecs.soton.ac.uk/12906/ Harnad, S.
(2006), Online, continuous, metrics-based research assessment. Technical Report, ECS, University of Southampton.
http://eprints.ecs.soton.ac.uk/12130/ Harnad, S., Brody, T.
(2004), Comparing the impact of Open Access (OA) vs. non-OA articles in the same journals,
, 10(6) June.
http://www.dlib.org/dlib/june04/harnad/06harnad.html Harnad, S., Carr, L., Brody, T., Oppenheim, C.
(2003), Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier.
, 35 (April 2003).
http://www.ariadne.ac.uk/issue35/harnad/ Hirsch, J. E.
(2005), An index to quantify an individual’s scientific research output.
Proceedings of the National Academy of Sciences
, 102(46): 16569–16572,
http://www.pnas.org/cgi/content/abstract/102/46/16569 CrossRef Kleinberg, J. M.
(1999), Hubs, authorities, and communities.
ACM Computing Surveys
http://www.cs.brown.edu/memex/ACM_HypertextTestbed/papers/10.html Kline, P. (2000), The New Psychometrics: Science, Psychology and Measurement. Routledge. Lawrence, S.
(2001), Online or invisible?
, 411(6837): 521,
http://www.neci.nec.com/:_lawrence/papers/online-nature01/ CrossRef Landauer, T. K., Foltz, P. W., Laham, D.
(1998), Introduction to latent semantic analysis.
, 25: 259–284.
CrossRef Moed, H. F. (2005), Citation Analysis in Research Evaluation. NY Springer. Oppenheim, Charles
(1996), Do citations count? Citation indexing and the research assessment exercise,
, 9: 155–61.
http://uksg.metapress.com/index/5YCDB0M2K3XGAYA6.pdf CrossRef Page, L., Brin, S., Motwani, R., Winograd, T.
The PageRank Citation Ranking: Bringing Order to the Web
http://dbpubs.stanford.edu:8090/pub/1999-66 Shadbolt, N., Brody, T., Carr, L., Harnad, S.
(2006), The open research web: a preview of the optimal and the inevitable. In:
Open Access: Key Strategic, Technical and Economic Aspects
, chapter 21. Chandos.
http://eprints.ecs.soton.ac.uk/12453/ Copyright information
© Springer Science+Business Media B.V. 2008