Abstract
This paper describes various problems which may occur in quantitative research evaluation. It is shown that problems already arise when trying to define such seemingly simple scientometric elements as “personnel” or “budget”. This has major consequences on the construction of indicators. Furthermore, it is demonstrated that different data sources as well as different data and indicators result in different, sometimes even contradicting outcomes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
ANONYMUS, [Oesterreichische] Evaluierungsverordnung (EvalVO) (Verordnung des Bundesministeriums fuer Wissenschaft und Verkehr ueber Grundsaetze fuer die Durchfuehrung von Evaluierungen in Forschung und Lehre der Universitaeten. BGBlNr. II 224/1997), Bundesministerium fuer Wissenschaft und Verkehr, Vienna, 1997, http://www.bmwf.gv.at/3uniwes/03unirecht/evalvo/evalvo.htm (state: November 2001).
DANIEL, H.-D., Methodische Probleme institutsvergleichender Analysen der Forschungsproduktivitaet, In: H.-D. DANIEL, R. FISCH (Eds), Evaluation von Forschung: Methoden, Ergebnisse, Stellungnahmen, Universitaetsverlag Konstanz, Konstanz, 1988, 215-241.
KING, J., A review of bibliometric and other science indicators and their role in research evaluation, Journal of Information Science, 13 (1987) 261-276.
KORWITZ, U., Welchen “Rang” hat ein Wissenschaftler? Nachrichten fuer Dokumentation, 46 (1995) 267-272.
LEWISON, G., New bibliometric techniques for the evaluation of medical schools, Scientometrics, 41 (1998) 5-16.
MOED, H. F., TH. N. VAN LEEUWEN, Improving the accuracy of institute for Scientific Information's journal impact factors, Journal of the American Society for Information Science, 46 (1995) 461-467.
VINKLER, P., Relations of relative scientometric impact indicators. The Relative Publication Strategy Index, Scientometrics, 40 (1997) 163-169.
VINKLER, P., General performance indexes calculated for research institutes of the Hungarian Academy of Sciences based on scientometric indicators, Scientometrics, 41 (1998) 185-200.
VINKLER, P., An attempt for defining some basic categories of scientometrics and classifying the indicators of evaluative scientometrics (Opinion Paper), Scientometrics, 50 (2001) 539-544.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Schloegl, C., Gorraiz, J., Bart, C. et al. Evaluating two Austrian university departments: Lessons learned. Scientometrics 56, 289–299 (2003). https://doi.org/10.1023/A:1022318618200
Issue Date:
DOI: https://doi.org/10.1023/A:1022318618200