Advertisement

Scientometrics

, Volume 86, Issue 2, pp 285–297 | Cite as

An approach to improve the indicator weights of scientific and technological competitiveness evaluation of Chinese universities

  • Jingda Ding
  • Junping Qiu
Article

Abstract

As indicator weights obtaining is often difficult in all types of evaluation, this paper describes an approach to improve the indicator weights of scientific and technological competitiveness evaluation of Chinese universities. As a public institution funded by Chinese government, the research center for Chinese science evaluation of Wuhan University has completed five annual evaluations for the scientific and technological competitiveness of Chinese universities since 2005, whose abundant and reliable data motivated us to try to improve the weights obtained by the AHP (analytical hierarchy process). Based on these data, we calculated the objective weights of the indicator using the representative mathematical methods of the least square and the variation coefficient. As the weights of AHP can be influenced by the knowledge, experience and preference of experts and the calculated objective weights neglect the subjective judgement information, we integrated the subjective and objective weights by respectively using the additive and multiplicative model to reflect both the subjective considerations of experts and the objective information, and obtained three kinds of integrative weights. Finally, we selected the integrative weights of multiplicative model as the best weights by comparing and analyzing the evaluation results in 2005 and 2009 of each kind of weights. The results show that the evaluation effect of the weights of multiplicative model is indeed the best for all types of Chinese universities among these kinds of weights, and the experts and university principals enquired also basically reached a consensus on the university rankings of the integrative weights of multiplicative model.

Keywords

Indicator weights Improvement Approach Evaluation 

References

  1. Bonnevie-Nebelong, E. (2006). Methods for journal evaluation: Journal citation identity, journal citation image and internationalization. Scientometrics, 66(2), 411–424.CrossRefGoogle Scholar
  2. Butler, L. (2002). Identifying ‘highly-rated’ journals—an Australian case study. Scientometrics, 53(2), 207–227.CrossRefGoogle Scholar
  3. Diakoulaki, D., Mavrotas, G., & Papayannakis, L. (1992). A multicriteria approach for evaluating the performance of industrial firms. Omega, 20, 467–474.CrossRefGoogle Scholar
  4. Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178, 471–479.CrossRefGoogle Scholar
  5. Guo, Y. J. (2007). The theory, method and application of comprehensive evaluation. Beijing: Science Press.Google Scholar
  6. Higgins, J. C. (1989). Performance measurement in universities. European Journal of Operational Research, 38, 358–368.CrossRefGoogle Scholar
  7. Huang, M. H., Chang, H. W., & Chen, D. Z. (2006). Research evaluation of research-oriented universities in Taiwan from 1993 to 2003. Scientometrics, 67(3), 419–435.Google Scholar
  8. Hwang, C. L., & Lin, M. J. (1987). Group decision making under multiple criteria: Methods and applications. Berlin: Springer.zbMATHGoogle Scholar
  9. Hwang, C. L., & Yoon, K. (1981). Multiple attribute decision making: Methods and applications. Berlin: Springer.zbMATHGoogle Scholar
  10. Kao, C., & Pao, H. L. (2009). An evaluation of research performance in management of 168 Taiwan universities. Scientometrics, 78(2), 261–277.CrossRefGoogle Scholar
  11. Ma, J., Fan, Z. P., & Huang, L. H. (1999). A subjective and objective integrated approach to determine attribute weights. European Journal of Operational Research, 112, 397–404.zbMATHCrossRefGoogle Scholar
  12. Madu, C. N. (1994). A quality confidence procedure for GDSS application in multicriteria decision making. IIE Transactions, 26(3), 31–39.CrossRefGoogle Scholar
  13. Mao, D. X. (2002). A combinational evaluation method resulting in consistency between subjective and objective evaluation in the least squares sense. Journal of Chinese Management Science, 10(5), 95–97.Google Scholar
  14. Mcgrath, W. E. (1987). Ratings and rankings: Multiple comparisons of mean ratings. College and Research Libraries, 48, 169–172.Google Scholar
  15. Morrissey, A. J., & Browne, J. (2004). Waste management models and their application to sustainable waste management. Waste Management, 24, 297–308.CrossRefGoogle Scholar
  16. Qiu, J. P. (2005). The evaluation report of Chinese universities and specialties of 2005–2006. Beijing: Science Press.Google Scholar
  17. Qiu, J. P. (2006). The evaluation report of Chinese universities and specialties of 2006–2007. Beijing: Science Press.Google Scholar
  18. Qiu, J. P. (2007). The evaluation report of Chinese universities and specialties of 2007–2008. Beijing: Science Press.Google Scholar
  19. Qiu, J. P. (2008). The evaluation report of Chinese universities and specialties of 2008–2009. Beijing: Science Press.Google Scholar
  20. Qiu, J. P. (2009). The evaluation report of Chinese universities and specialties of 2009–2010. Beijing: Science Press.Google Scholar
  21. Raan, A. (1999). Advanced bibliometric methods for the evaluation of universities. Scientometrics, 45(3), 418–420.CrossRefGoogle Scholar
  22. Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234–281.zbMATHCrossRefMathSciNetGoogle Scholar
  23. Saaty, T. L. (1980). The analytic hierarchy process. New York: McGraw-Hill.zbMATHGoogle Scholar
  24. Saaty, T. L. (1990). Multicriteria decision making: The analytic hierarchy process. Pittsburgh: RWS Publications.Google Scholar
  25. Thomson Scientific (2010). ESI v2.0 Reference Card. Retrieved January 24, 2010 from http://scientific.thomson.com/media/scpdf/esi-0805-q.pdf.
  26. Zeleny, M. (1982). Multiple criteria decision making. NY: McGraw-Hill.zbMATHGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2010

Authors and Affiliations

  1. 1.Research Center for Chinese Science EvaluationWuhan UniversityWuhanChina

Personalised recommendations