Scientometrics

, Volume 101, Issue 1, pp 85–107

A propensity score approach in the impact evaluation on scientific production in Brazilian biodiversity research: the BIOTA Program

  • Fernando A. B. Colugnati
  • Sergio Firpo
  • Paula F. Drummond de Castro
  • Juan E. Sepulveda
  • Sergio L. M. Salles-Filho
Article

DOI: 10.1007/s11192-014-1397-1

Cite this article as:
Colugnati, F.A.B., Firpo, S., de Castro, P.F.D. et al. Scientometrics (2014) 101: 85. doi:10.1007/s11192-014-1397-1

Abstract

Evaluation has become a regular practice in the management of science, technology and innovation (ST&I) programs. Several methods have been developed to identify the results and impacts of programs of this kind. Most evaluations that adopt such an approach conclude that the interventions concerned, in this case ST&I programs, had a positive impact compared with the baseline, but do not control for any effects that might have improved the indicators even in the absence of intervention, such as improvements in the socio-economic context. The quasi-experimental approach therefore arises as an appropriate way to identify the real contributions of a given intervention. This paper describes and discusses the utilization of propensity score (PS) in quasi-experiments as a methodology to evaluate the impact on scientific production of research programs, presenting a case study of the BIOTA Program run by FAPESP, the State of São Paulo Research Foundation (Brazil). Fundamentals of quasi-experiments and causal inference are presented, stressing the need to control for biases due to lack of randomization, also a brief introduction to the PS estimation and weighting technique used to correct for observed bias. The application of the PS methodology is compared to the traditional multivariate analysis usually employed.

Keywords

Quasi-experiment Propensity score Impact evaluation Biota program Bibliometrics 

Mathematics Subject Classification

62P25 

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2014

Authors and Affiliations

  • Fernando A. B. Colugnati
    • 1
    • 4
  • Sergio Firpo
    • 2
  • Paula F. Drummond de Castro
    • 1
  • Juan E. Sepulveda
    • 3
  • Sergio L. M. Salles-Filho
    • 1
  1. 1.Laboratory of Studies on the Organization of Research and Innovation – GEOPI, Department of Science and Technology Policy, Institute of GeosciencesUniversity of CampinasCampinasBrazil
  2. 2.São Paulo School of Economics/FGV‎São PauloBrazil
  3. 3.Institute of Economics/UNICAMPCampinasBrazil
  4. 4.Department of Clinical MedicineFederal University of Juiz de ForaJuiz de ForaBrazil

Personalised recommendations