Scientometrics

, Volume 78, Issue 3, pp 467–479 | Cite as

Testing bibliometric indicators by their prediction of scientists promotions

  • Pablo Jensen
  • Jean-Baptiste Rouquier
  • Yves Croissant
Article

Abstract

We have developed a method to obtain robust quantitative bibliometric indicators for several thousand scientists. This allows us to study the dependence of bibliometric indicators (such as number of publications, number of citations, Hirsch index...) on the age, position, etc. of CNRS scientists. Our data suggests that the normalized h-index (h divided by the career length) is not constant for scientists with the same productivity but different ages.

We also compare the predictions of several bibliometric indicators on the promotions of about 600 CNRS researchers. Contrary to previous publications, our study encompasses most disciplines, and shows that no single indicator is the best predictor for all disciplines. Overall, however, the Hirsch index h provides the least bad correlations, followed by the number of papers published. It is important to realize however that even h is able to recover only half of the actual promotions. The number of citations or the mean number of citations per paper are definitely not good predictors of promotion.

Due to space constraints, this paper is a short version of a more detailed article. [JENSEN & AL., 2008B]

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bornmann, L., Daniel, H-D. (2005). Does the h-index for ranking of scientists really work? Scientometrics, 65 (3): 391–392.CrossRefGoogle Scholar
  2. Bornmann, L., Daniel, H-D. (2007), Convergent validation of peer review decisions using the h-index. Extent of and reasons for type I and type II errors. Journal of Informetrics, 1: 204–213.CrossRefGoogle Scholar
  3. Brooks, T. A. (1996), Evidence of complex citer motivations. Journal of the American Society for Information Science, 37:34–36.MathSciNetGoogle Scholar
  4. Cronin, B., Meho, L. (2006), Using the h-index to rank influential information scientists. Journal of the American Society for Information Science and Technology, 57: 1275–1278.CrossRefGoogle Scholar
  5. Davidson, R., Mackinnon, J. (1981), Several tests for model specification in the presence of alternative hypotheses. Econometrica, 49: 781–793.MATHCrossRefMathSciNetGoogle Scholar
  6. Hirsch, J. E. (2005), An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sciences, 102: 16569–16572.CrossRefGoogle Scholar
  7. Iglesias, J. E., Pecharromán, C. (2007), Scaling the h-index for different scientific ISI fields. Scientometrics, 73: 303–320.CrossRefGoogle Scholar
  8. Jensen, P., Rouquier, J-B., Croissant, Y. (2008A). Scientists connected with society are more active academically. Science and Public Policy, 35 (7) (August 2008).Google Scholar
  9. Jensen, P., Rouquier, J-B., Croissant, Y. (2008B), Testing bibliometric indicators by their prediction of scientists promotions: full version of the paper published in Scientometrics. http://perso.enslyon.fr/pablo.jensen/full_scientometrics.pdf.
  10. Kostoff, R. N. (1998), The use and misuse of citation analysis in research evaluation. Scientometrics, 43 (1): 27–43.CrossRefGoogle Scholar
  11. Larédo, P., Callon, M., Mustar, P., Birac, A. M., Fourest, B. (1992), Defining the strategic profile of research labs: the research compass card method. In: A. Van Raan, R. De Bruin, H. Moed, A. Nederhof, R. Tijssen (Eds), Science and Technology in a Policy Context, 184–199, Leiden, DSWO Press. 10.Google Scholar
  12. Leibnitz, H. M. (1989), The measurement of quality and reputation in the world of learning. Minerva, 27: 483.CrossRefGoogle Scholar
  13. Leydesdorff, L. (1998), Theories of citation? Scientometrics, 43: 5–25.CrossRefGoogle Scholar
  14. Liu, M. (1993), The complexities of citation practice: a review of citation studies. Journal of Documentation, 49: 370–408.CrossRefGoogle Scholar
  15. McCain, K. W., Turner, K. (1989), Citation context analysis and aging patterns of journal articles in molecular genetics. Scientometrics, 17: 127–163.CrossRefGoogle Scholar
  16. Nowotny, H., Scott, P., Gibbons, M. (2003), Mode 2 revisited: the new production of knowledge introduction. Minerva, 41 (3): 179–194.CrossRefGoogle Scholar
  17. Porter, TH. M. (1995), Trust in Numbers. Princeton University Press.Google Scholar
  18. Saad, G. (2006), Exploring the h-index at the author and journal levels using bibliometric data of productive consumer scholars and business-related journals respectively. Scientometrics, 69: 117–120.CrossRefGoogle Scholar
  19. Van Raan, A. F. J. (2006), Comparison of the hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research. Scientometrics, 67: 491–502.Google Scholar
  20. Whitehead, A. N. (2007), Introduction to Mathematics. Rough Draft Printing.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  • Pablo Jensen
    • 1
    • 2
    • 3
    • 4
  • Jean-Baptiste Rouquier
    • 1
    • 2
    • 5
  • Yves Croissant
    • 1
    • 4
  1. 1.Université de LyonLyonFrance
  2. 2.Institut des Systèmes Complexes Rhône-Alpes (IXXI)LyonFrance
  3. 3.Laboratoire de PhysiqueÉcole Normale Supérieure de Lyon and CNRSLyonFrance
  4. 4.Laboratoire d’Économie des TransportsUniversité Lyon 2 and CNRSLyonFrance
  5. 5.Laboratoire d’Informatique du ParallèlismeÉcole Normale Supérieure de Lyon and CNRSLyonFrance

Personalised recommendations