Scientometrics

, Volume 90, Issue 3, pp 843–853 | Cite as

Future publication success in science is better predicted by traditional measures than by the h index

Article

Abstract

Although the use of bibliometric indicators for evaluations in science is becoming more and more ubiquitous, little is known about how future publication success can be predicted from past publication success. Here, we investigated how the post-2000 publication success of 85 researchers in oncology could be predicted from their previous publication record. Our main findings are: (i) Rates of past achievement were better predictors than measures of cumulative achievement. (ii) A combination of authors’ past productivity and the past citation rate of their average paper was most successful in predicting future publication success (R2 ≈ 0.60). (iii) This combination of traditional bibliographic indicators clearly outperformed predictions based on the rate of the h index (R2 between 0.37 and 0.52). We discuss implications of our findings for views on creativity and for science evaluation.

Keywords

Science Evaluation Creativity Bibliometry h index 

References

  1. Alonso, S., Cabrerizo, F. J., Herrera-Viedema, E., & Herrera, F. (2009). H index: A review focused in its variants, computation and standardisation for different scientific fields. Journal of Informetrics, 3, 273–289.CrossRefGoogle Scholar
  2. Bornmann, L. (2011). Mimicry in science? Scientometrics, 86, 173–177.CrossRefGoogle Scholar
  3. Bornmann, L., & Daniel, H.-D. (2005). Does the h index for ranking of scientists really work? Scientometrics, 65, 391–392.CrossRefGoogle Scholar
  4. Bornmann, L., & Daniel, H.-D. (2009). The state of h index research. EMBO Reports, 10, 2–6.CrossRefGoogle Scholar
  5. Bornmann, L., Mutz, R., & Daniel, H.-D. (2008a). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59, 830–837.CrossRefGoogle Scholar
  6. Bornmann, L., Wallon, G., & Ledin, A. (2008b). Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data. Research Evaluation, 17, 149–156.CrossRefGoogle Scholar
  7. Bornmann, L., Mutz, R., & Daniel, H.-D. (2009). Do we need the h index and its variants in addition to standard bibliometric measures? Journal of the American Society for Information Science and Technology, 60, 1286–1289.CrossRefGoogle Scholar
  8. Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571–582.CrossRefGoogle Scholar
  9. Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgement. Science, 243, 1668–1674.CrossRefGoogle Scholar
  10. Egghe, L. (2005). Power laws in the information production process: Lotkaian informetrics. Kidlington: Elsevier Academic Press.Google Scholar
  11. Feist, G. J. (1993). A structural model of scientific eminence. Psychological Science, 4, 366–371.CrossRefGoogle Scholar
  12. Henzinger, M., Suñol, J., & Weber, I. (2010). The stability of the h index. Scientometrics, 84, 465–479.CrossRefGoogle Scholar
  13. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Science, 102, 16569–16572.Google Scholar
  14. Hirsch, J. E. (2007). Does the h index have predictive power? Proceedings of the National Academy of Science, 104, 19193–19198.Google Scholar
  15. Hönekopp, J., & Kleber, J. (2008). Sometimes the impact factor outshines the h index. Retrovirology, 5 (88).Google Scholar
  16. Huber, J. C. (2001). A new method for analysing scientific productivity. Journal of the American Society for Information Science and Technology, 52, 1089–1099.CrossRefGoogle Scholar
  17. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h index. Chinese Science Bulletin, 52, 855–863.CrossRefGoogle Scholar
  18. Kelly, C. D., & Jennions, M. D. (2006). The h index and carrier assessment by numbers. Trends in Ecology and Evolution, 21, 167–170.CrossRefGoogle Scholar
  19. Lawrence, P. A. (2007). The mismeasurement of science. Current Biology, 17, R583.CrossRefGoogle Scholar
  20. Lehmann, S., Jackson, A. D., & Lautrup, B. E. (2008). A quantitative analysis of indicators of scientific performance. Scientometrics, 76, 369–390.CrossRefGoogle Scholar
  21. Lovegrove, B. G., & Johnson, S. D. (2008). Assessment of research performance in biology: How well do peer review and bibliometry correlate. BioScience, 58, 160–164.CrossRefGoogle Scholar
  22. Macilwain, C. (2010). Wild goose chase. Nature, 463, 291.CrossRefGoogle Scholar
  23. Poynard, T., Thabut, D., Munteanu, M., Ratziu, V., Benhamou, Y., & Deckmyn, O. (2010). Hirsch index and truth survival in clinical research. PLoS ONE, 5, e12044.CrossRefGoogle Scholar
  24. Redner, S. (1998). How popular is your paper? An empirical study of the citation distribution. The European Physical Journal B, 4, 131–134.CrossRefGoogle Scholar
  25. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.CrossRefGoogle Scholar
  26. Schreiber, M. (2008). To share the fame in a fair way, h m modifies h for multi-authored manuscripts. New Journal of Physics, 10, 040201.CrossRefGoogle Scholar
  27. Simonton, D. K. (1997). Creative productivity: A predictive and explanatory model of career trajectories and landmarks. Psychological Review, 104, 66–89.CrossRefGoogle Scholar
  28. Simonton, D. K. (2003). Scientific creativity as constrained stochastic behaviour: The integration of product, person and process perspectives. Psychological Bulletin, 129, 475–494.CrossRefGoogle Scholar
  29. van Raan, A. F. J. (2004). Measuring science. Capita selecta of current main issues. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research, pp. 19–50. Dordrecht: Kluwer Academic Publishers.Google Scholar
  30. van Raan, A. F. J. (2006). Comparison of the Hirsch index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67, 491–502.Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2011

Authors and Affiliations

  1. 1.Department of PsychologyNorthumbria UniversityNewcastle upon TyneUK

Personalised recommendations