Scientometrics

, Volume 79, Issue 1, pp 147–156 | Cite as

Open access scientometrics and the UK Research Assessment Exercise

Article

Abstract

Scientometric predictors of research performance need to be validated by showing that they have a high correlation with the external criterion they are trying to predict. The UK Research Assessment Exercise (RAE) — together with the growing movement toward making the full-texts of research articles freely available on the web — offer a unique opportunity to test and validate a wealth of old and new scientometric predictors, through multiple regression analysis: Publications, journal impact factors, citations, co-citations, citation chronometrics (age, growth, latency to peak, decay rate), hub/authority scores, h-index, prior funding, student counts, co-authorship scores, endogamy/exogamy, textual proximity, download/co-downloads and their chronometrics, etc. can all be tested and validated jointly, discipline by discipline, against their RAE panel rankings in the forthcoming parallel panel-based and metric RAE in 2008. The weights of each predictor can be calibrated to maximize the joint correlation with the rankings. Open Access Scientometrics will provide powerful new means of navigating, evaluating, predicting and analyzing the growing Open Access database, as well as powerful incentives for making it grow faster.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Brody, T. (2003), Citebase search: Autonomous citation database for e-print archives. ECS Technical Report. http://eprints.ecs.soton.ac.uk/10677/
  2. Brody, T., Harnad, S., Carr, L. (2006), Earlier web usage statistics as predictors of later citation impact. Journal of the American Association for Information Science and Technology (JASIST), 57(8): 1060–1072. http://eprints.ecs.soton.ac.uk/10713/CrossRefGoogle Scholar
  3. Garfield, E., (1955) Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122: 108–111, http://www.garfield.library.upenn.edu/papers/science_v122(3159)p108y1955.htmlCrossRefGoogle Scholar
  4. Giles, C. L., Bollacker, K., Lawrence, S. (1998), CiteSeer: An automatic citation indexing system. Digital Libraries 98: Third ACM Conf. on Digital Libraries, ACM Press, New York, 1998, pp. 89–98. http://www.neci.nj.nec.com/homepages/lawrence/citeseer.htmlCrossRefGoogle Scholar
  5. Guédon, J.-C. (2002), In Oldenburg’s Long Shadow: Librarians, Research Scientists, Publishers, and the Control of Scientific Publishing. Washington, DC: The Association of Research Libraries. http://www.arl.org/resources/pubs/mmproceedings/138guedon.shtmlGoogle Scholar
  6. Hajjem, C., Harnad, S., Gingras, Y. (2005), Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. IEEE Data Engineering Bulletin, 28(4) 39-47. http://eprints.ecs.soton.ac.uk/12906/Google Scholar
  7. Harnad, S. (2006), Online, continuous, metrics-based research assessment. Technical Report, ECS, University of Southampton. http://eprints.ecs.soton.ac.uk/12130/
  8. Harnad, S., Brody, T. (2004), Comparing the impact of Open Access (OA) vs. non-OA articles in the same journals, D-Lib Magazine, 10(6) June. http://www.dlib.org/dlib/june04/harnad/06harnad.html
  9. Harnad, S., Carr, L., Brody, T., Oppenheim, C. (2003), Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier. Ariadne, 35 (April 2003). http://www.ariadne.ac.uk/issue35/harnad/
  10. Hirsch, J. E. (2005), An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46): 16569–16572, http://www.pnas.org/cgi/content/abstract/102/46/16569CrossRefGoogle Scholar
  11. Kleinberg, J. M. (1999), Hubs, authorities, and communities. ACM Computing Surveys, 31(4), http://www.cs.brown.edu/memex/ACM_HypertextTestbed/papers/10.html
  12. Kline, P. (2000), The New Psychometrics: Science, Psychology and Measurement. Routledge.Google Scholar
  13. Lawrence, S. (2001), Online or invisible? Nature, 411(6837): 521, http://www.neci.nec.com/:_lawrence/papers/online-nature01/CrossRefGoogle Scholar
  14. Landauer, T. K., Foltz, P. W., Laham, D. (1998), Introduction to latent semantic analysis. Discourse Processes, 25: 259–284.CrossRefGoogle Scholar
  15. Moed, H. F. (2005), Citation Analysis in Research Evaluation. NY Springer.Google Scholar
  16. Oppenheim, Charles (1996), Do citations count? Citation indexing and the research assessment exercise, Serials, 9: 155–61. http://uksg.metapress.com/index/5YCDB0M2K3XGAYA6.pdfCrossRefGoogle Scholar
  17. Page, L., Brin, S., Motwani, R., Winograd, T. (1999), The PageRank Citation Ranking: Bringing Order to the Web. http://dbpubs.stanford.edu:8090/pub/1999-66
  18. Shadbolt, N., Brody, T., Carr, L., Harnad, S. (2006), The open research web: a preview of the optimal and the inevitable. In: Jacobs, N. (Ed.), Open Access: Key Strategic, Technical and Economic Aspects, chapter 21. Chandos. http://eprints.ecs.soton.ac.uk/12453/

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  1. 1.Chaire de recherche du Canada, Institut des sciences cognitivesUniversité du Québec à MontréalMontréalCanada

Personalised recommendations