Scientometric impact assessment of a research policy instrument: the case of rating researchers on scientific outputs in South Africa
- First Online:
- 288 Downloads
The influence of the National Research Foundation’s (NRF) rating system on the productivity of the South African social science researchers is investigated scientometrically for the period from 1981 to 2006. Their output performance is mainly indicated by their research publications. Following international best practice in scientometrics as well as the behavioural reinforcement theory, we employed the “before/after control impact (BACI) method”, as well as the well known econometric breakpoint test as proposed by Chow. We use as control group the publications in the field of clinical medicine. The field is not supported by NRF and hence clinical medicine researchers are not affected by the evaluation and rating system. The findings show a positive impact of the NRF programme on the research outputs of social sciences researchers and the implementation of the programme has increased the relevant population of research articles by an average of 24.5% (during the first 5 years) over the expected number of publication without the programme. The results confirm the scientometric findings of other studies (e.g. that of Nederhof) that ratings promulgate research productivity.
KeywordsScientometrics Incentives Research policy Quasi experimental design South Africa
- Bertrand, M., Djankov, S., Hanna, R., & Mullainathan, S. (2006). Does corruption produce unsafe drivers? National Bureau of Economic Research NBER Working Paper #12274, Cambridge, MA.Google Scholar
- Bouabid, H., Dalimi, M., & Elmajid, Z. (2010). Impact evaluation of the voluntary early retirement policy on research and technology outputs of the faculties of science in Morocco. Scientometrics, 85, 125–132.Google Scholar
- Duflo, E., Glennerster, R., & Kremer, M. (2006). Using randomization in development economics research: A toolkit. National Bureau of Economic Research (NBER), Technical Working Paper 333, Cambridge, MA.Google Scholar
- Gujarati, D. N. (2003). Basic econometrics. New York: McGraw-Hill.Google Scholar
- Habib, A., & Morrow, S. (2007). Research, research productivity and the state in South Africa. Journal of Higher Education in Africa, 5, 113–130.Google Scholar
- National Research Foundation. (n.d.). NRF profile, [online]. Available from http://www.nrf.ac.za/about_overview.php. Last accessed 23 June 2011.
- Pouris, A. (1991). Effects of funding policies on research publications in South Africa. South African Journal of Science, 87, 78–81.Google Scholar
- Pouris, A. (2007). The national research foundation rating system: Why scientists let their ratings lapse. South African Journal of Science, 103, 439–442.Google Scholar
- Shadish, R. W., Cook, D. T., & Campbell, T. D. (2003). Experimental and quasi experimental designs for generalised causal inference. Boston: Houghton Mifflin Company.Google Scholar
- Skinner, B. F. (1969). Contingencies of reinforcement: A theoretical analysis. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
- Tassey, G. (2003). Methods for assessing the economic impacts of government R&D. National Institute of Standards and Technology, [Online]. Available at http://www.nist.gov/director/planning/upload/report03-1.pdf.
- Tertiary Education Commission (TEC) (2007). PBRF evaluating research excellence. The 2006 assessment. [Online]. Available at http://www.tec.govt.nz/Documents/Publications/pbrf-full-report-2006.pdf. Last accessed 23 June 2011.