Scientometrics

, Volume 98, Issue 1, pp 369–385

Impact of bibliometric studies on the publication behaviour of authors

Article

Abstract

It has been widely discussed how individuals change the way they act and react in studies just because they are under observation. In this paper, we try to analyse how this so-called Hawthorne effect applies to researchers that are the subject of bibliometric investigations. This encompasses individual assessments as well as international performance comparisons. We test various bibliometric indicators for notable changes in the last decade from a world-wide perspective and deduce explanations for changes from the observations. We then concentrate on the behaviour of German authors in particular, to show national trends. The German publication behaviour is evaluated in regard to citation rates and collaborations in publications and size, publisher country and impact of the journals chosen for publication. We can conclude that authors adapt their publication behaviour to aim for journals that are more internationally known and have a US publisher. Also, a trend from more specialized journals to journals with a broader scope can be observed that raises the question whether the implicit penalization of specialized fields in the bibliometrics leads to undesired shifts in conducted research.

Keywords

Bibliometrics National studies Research evaluation Author behaviour Hawthorne effect 

References

  1. Adams, J. (2005). Early citation counts correlate with accumulated impact. Scientometrics, 63, 567–581.CrossRefGoogle Scholar
  2. Aksnes, D. W. (2003). A macro study of self-citation. Scientometrics, 56, 235–246.CrossRefGoogle Scholar
  3. Aksnes, D. W., & Sivertsen, G. (2004). The effect of highly cited papers on national citation indicators. Scientometrics, 59, 213–224.CrossRefGoogle Scholar
  4. Bornmann, L. (2011). Mimicry in science? Scientometrics, 86, 173–177.CrossRefGoogle Scholar
  5. Butler, L. (2003). Modifying publication practices in response to funding formulas. Research Evaluation, 17(1), 39–46.CrossRefGoogle Scholar
  6. Glänzel, W., Thijs, B., & Schlemmer, B. (2004). A bibliometric approach to the role of author self-citations in scientific communication. Scientometrics, 59, 63–77.CrossRefGoogle Scholar
  7. Gondal, N. (2011). The local and global structure of knowledge production in an emergent research field: An exponential random graph analysis. Social Networks, 33(1), 20–30.CrossRefMathSciNetGoogle Scholar
  8. Grupp, H., Schmoch, U., & Hinze, S. (2001). International alignment and scientific regard as macro-indicators for international comparisons of publications. Scientometrics, 51(2), 359–380.CrossRefGoogle Scholar
  9. Jimenez-Contreras, E., Delgado Lopez-Cozar, E., Ruiz-Perez, R., & Fernandez, V. M. (2002). Impact-factor rewards affect Spanish research. Nature, 417(6892), 898.CrossRefGoogle Scholar
  10. Jones, S. R. G. (1992). Was there a Hawthorne effect? American Journal of Sociology, 98(3), 451–468.CrossRefGoogle Scholar
  11. Ltd, Evidence. (2007). The use of bibliometrics to measure research quality in UK higher education institutions. London: Universities UK.Google Scholar
  12. Michels, C., & Schmoch, U. (2011), Growth of science and database coverage (Full paper) paper presented at the 13th conference of the international society for scientometrics and informetrics (ISSI), 4–7 July 2011, Durban, South Africa.Google Scholar
  13. Michels, C., & Schmoch, U. (2012). The Growth of science and database coverage. Scientometrics, 93(3), 831–846.CrossRefGoogle Scholar
  14. Moed, H. (2005). Citation analysis in research evaluation. Dordrecht: Springer. (Information Science and Knowledge Management, 9).Google Scholar
  15. Mueller, P. S., Murali, N. S., Cha, S. S., Erwin, P. J., & Ghosh, A. K. (2006). The association between impact factors and language of general internal medicine journals. Swiss Medical Weekly, 136, 441–443.Google Scholar
  16. Schmoch, U., Michels, C., Neuhäusler, P., & Schulze, N. (2012). Performance and Structure of the German Science System 2011. Germany in an international comparison, China’s profile, behaviour of German authors, comparison of the Web of Science and Scopus. Berlin: Commission of Expert on Research and Innovation (www.e-fi.de).
  17. Schneider, J. W. (2009). An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway. European Political Science, 8(3), 364–378.CrossRefGoogle Scholar
  18. Van Leeuwen, T. N., Moed, H. F., Tussen, R. J. W., Visser, M. S., & Van Raan, A. F. J. (2001). Language biases in the coverage of the science citation index and its consequences for international comparisons of national research performance. Scientometrics, 51(1), 335–346.CrossRefGoogle Scholar
  19. van Raan, A. J. F. (2004). Measuring science. Capita selecta of current main issues. In W. Glänzel, H. Moed, & U. Schmoch (Eds.), Handbook of quantitative science and technology research. The use of publication and patent statistics in studies on R&D systems (pp. 19–50). Dodrecht: Kluwer Academic Publishers.Google Scholar
  20. Waltman, L, & Van Eck, N.J. (2013): Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison. Scientometrics in press.Google Scholar
  21. Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 63(1), 117–131.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2013

Authors and Affiliations

  1. 1.Fraunhofer Institute for Systems and Innovation ResearchKarlsruheGermany
  2. 2.German University of Administrative SciencesSpeyerGermany

Personalised recommendations