Abstract
As indicator weights obtaining is often difficult in all types of evaluation, this paper describes an approach to improve the indicator weights of scientific and technological competitiveness evaluation of Chinese universities. As a public institution funded by Chinese government, the research center for Chinese science evaluation of Wuhan University has completed five annual evaluations for the scientific and technological competitiveness of Chinese universities since 2005, whose abundant and reliable data motivated us to try to improve the weights obtained by the AHP (analytical hierarchy process). Based on these data, we calculated the objective weights of the indicator using the representative mathematical methods of the least square and the variation coefficient. As the weights of AHP can be influenced by the knowledge, experience and preference of experts and the calculated objective weights neglect the subjective judgement information, we integrated the subjective and objective weights by respectively using the additive and multiplicative model to reflect both the subjective considerations of experts and the objective information, and obtained three kinds of integrative weights. Finally, we selected the integrative weights of multiplicative model as the best weights by comparing and analyzing the evaluation results in 2005 and 2009 of each kind of weights. The results show that the evaluation effect of the weights of multiplicative model is indeed the best for all types of Chinese universities among these kinds of weights, and the experts and university principals enquired also basically reached a consensus on the university rankings of the integrative weights of multiplicative model.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bonnevie-Nebelong, E. (2006). Methods for journal evaluation: Journal citation identity, journal citation image and internationalization. Scientometrics, 66(2), 411–424.
Butler, L. (2002). Identifying ‘highly-rated’ journals—an Australian case study. Scientometrics, 53(2), 207–227.
Diakoulaki, D., Mavrotas, G., & Papayannakis, L. (1992). A multicriteria approach for evaluating the performance of industrial firms. Omega, 20, 467–474.
Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178, 471–479.
Guo, Y. J. (2007). The theory, method and application of comprehensive evaluation. Beijing: Science Press.
Higgins, J. C. (1989). Performance measurement in universities. European Journal of Operational Research, 38, 358–368.
Huang, M. H., Chang, H. W., & Chen, D. Z. (2006). Research evaluation of research-oriented universities in Taiwan from 1993 to 2003. Scientometrics, 67(3), 419–435.
Hwang, C. L., & Lin, M. J. (1987). Group decision making under multiple criteria: Methods and applications. Berlin: Springer.
Hwang, C. L., & Yoon, K. (1981). Multiple attribute decision making: Methods and applications. Berlin: Springer.
Kao, C., & Pao, H. L. (2009). An evaluation of research performance in management of 168 Taiwan universities. Scientometrics, 78(2), 261–277.
Ma, J., Fan, Z. P., & Huang, L. H. (1999). A subjective and objective integrated approach to determine attribute weights. European Journal of Operational Research, 112, 397–404.
Madu, C. N. (1994). A quality confidence procedure for GDSS application in multicriteria decision making. IIE Transactions, 26(3), 31–39.
Mao, D. X. (2002). A combinational evaluation method resulting in consistency between subjective and objective evaluation in the least squares sense. Journal of Chinese Management Science, 10(5), 95–97.
Mcgrath, W. E. (1987). Ratings and rankings: Multiple comparisons of mean ratings. College and Research Libraries, 48, 169–172.
Morrissey, A. J., & Browne, J. (2004). Waste management models and their application to sustainable waste management. Waste Management, 24, 297–308.
Qiu, J. P. (2005). The evaluation report of Chinese universities and specialties of 2005–2006. Beijing: Science Press.
Qiu, J. P. (2006). The evaluation report of Chinese universities and specialties of 2006–2007. Beijing: Science Press.
Qiu, J. P. (2007). The evaluation report of Chinese universities and specialties of 2007–2008. Beijing: Science Press.
Qiu, J. P. (2008). The evaluation report of Chinese universities and specialties of 2008–2009. Beijing: Science Press.
Qiu, J. P. (2009). The evaluation report of Chinese universities and specialties of 2009–2010. Beijing: Science Press.
Raan, A. (1999). Advanced bibliometric methods for the evaluation of universities. Scientometrics, 45(3), 418–420.
Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234–281.
Saaty, T. L. (1980). The analytic hierarchy process. New York: McGraw-Hill.
Saaty, T. L. (1990). Multicriteria decision making: The analytic hierarchy process. Pittsburgh: RWS Publications.
Thomson Scientific (2010). ESI v2.0 Reference Card. Retrieved January 24, 2010 from http://scientific.thomson.com/media/scpdf/esi-0805-q.pdf.
Zeleny, M. (1982). Multiple criteria decision making. NY: McGraw-Hill.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ding, J., Qiu, J. An approach to improve the indicator weights of scientific and technological competitiveness evaluation of Chinese universities. Scientometrics 86, 285–297 (2011). https://doi.org/10.1007/s11192-010-0268-7
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-010-0268-7