Abstract
Research policies in the more developed nations are ever more oriented towards the introduction of productivity incentives and competition mechanisms intended to increase efficiency in research institutions. Assessments of the effects of these policy interventions on public research activity often neglect the normal, inherent variation in the performance of research institutions over time. In this work, we propose a cross-time bibliometric analysis of research performance by all Italian universities in two consecutive periods (2001–2003 and 2004–2008) not affected by national policy interventions. Findings show that productivity and impact increased at the level of individual scientists. At the level of university, significant variation in the rank was observed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In Italy there are 95 universities. Of these, only 63 (which are the units of analysis for this study) have at least 6 research staff in at least one of the UDA considered.
Articles, reviews and conference proceedings.
The research staff considered are assistant, associate and professors.
This data was obtained as a weighted average of the average productivity of the 205 SDSs considered, with the weightings based on the portion of the national research staff belonging to each SDS.
Here, the impact of a publication is standardized with respect to a WoS international benchmark: the average value of the citations of all publications indexed in the WoS in the same subject category and year.
The choice to standardize citations with respect to median value (rather than to the average, as is frequently observed in the literature) is justified by the fact that the distribution of citations is highly skewed in almost all subject categories (Lundberg 2007). In the aggregate analysis presented in the previous section it was not possible to utilize the average, because the international average was not available, nor was it opportune to use the national average, since aggregate value would clearly result as invariant.
Observed as of 30/06/2009.
References
Abramo, G., D’Angelo, C. A., & Pugini, F. (2008a). The measurement of Italian universities’ research productivity by a non parametric-bibliometric methodology. Scientometrics, 76(2), 225–244.
Abramo, G., D’Angelo, C. A., & Di Costa, F. (2008b). Assessment of sectoral aggregation distortion in research productivity measurements. Research Evaluation, 17(2), 111–121.
Abramo, G., D’Angelo, C. A., & Di Costa, F. (2009). Research collaboration and productivity: Is there correlation? Higher Education, 57(2), 155–171.
Auranen, O., & Nieminen, M. (2010). University research funding and publication performance—An international comparison. Research Policy, 39(6), 822–834.
Bonaccorsi, A., Daraio, C., & Simar, L. (2006). Advanced indicators of productivity of universities. An application of robust nonparametric methods to Italian data. Scientometrics, 66(2), 389–410.
Butler, L. (2003). Explaining Australia’s increased share of ISI publications. The effects of a funding formula based on publication counts. Research Policy, 32(1), 143–155.
Franceschet, M., & Costantini, A. (2010). The effect of scholar collaboration on impact and quality of academic papers. Journal of Informetrics, 4(4), 540–553.
Giuffrida, C., D’Angelo, C. A., Abramo, G. (2010). A heuristic approach to author name disambiguation in large-scale bibliometric databases. Journal of the American Society for Information Science and Technology (forthcoming).
Gómez, I., Bordons, M., Fernández, M. T., & Morillo, F. (2009). Structure and research performance of Spanish universities. Scientometrics, 79(1), 131–146.
He, Z. L., Geng, X. S., & Campbell-Hunt, C. (2009). Research collaboration and research output: A longitudinal study of 65 biomedical scientists in a New Zealand university. Research Policy, 38(2), 306–317.
Hung, W.-C., Lee, L.-C., & Tsai, M.-H. (2009). An international comparison of relative contributions to academic productivity. Scientometrics, 81(3), 703–718.
Lundberg, J. (2007). Lifting the crown-citation z-score. Journal of Informetrics, 1(2), 145–154.
Moed, H. F. (2008). UK Research Assessment Exercises: Informed judgments on research quality or quantity? Scientometrics, 74(1), 153–161.
van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Abramo, G., D’Angelo, C.A. & Di Costa, F. A national-scale cross-time analysis of university research performance. Scientometrics 87, 399–413 (2011). https://doi.org/10.1007/s11192-010-0319-0
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-010-0319-0