Research fund evaluation based on academic publication output analysis: the case of Chinese research fund evaluation
- 94 Downloads
Government funding is a key scientific research resource, and it has made a concrete contribution to the world’s science and technology development. But these funds come from common taxpayers, so we need to evaluate the effectiveness of these funds. Generally speaking, policymakers adopt the method of peer review to make assessments. Compared to kinds of shortcomings of peer review, the paper here proposed the benchmarking evaluation method based on the academic publication outputs of supporting funds, mainly guiding indicators from scientometrics. At first, with the academic publication output extracted from the concluding report project manager submitted to the government after the fund finished, we designed the analysis framework to search and define the research field the fund belonged to. And then from the following three perspectives, including quantity, quality and relative influence, we compared the research fund output to the field output. Later, we took one fund program from national program on key basic research project of China (973 Program) in the field of quantum physics as an example to make an empirical analysis to demonstrate its effectiveness. At last, we found that the funded program performance was superior to the field, and even about 11.65% of the research achievement reaches the top 1/1000 of the world, but the research was lacking in remarkable papers, so it needs further improvement.
KeywordsResearch fund evaluation Academic publication output analysis Benchmarking bibliometrics Research achievement Scientometrics
This work is supported by National Social Science Foundation of China (No. 17CTQ029). Also we would like to thank anonymous reviewers for their helpful comments.
- Bo, Y., & Zeng, J. (2011). Metrological analysis of national social sciences funding annual projects (2000–2009). Science and Technology Management Research, 31, 45–48.Google Scholar
- Chen, H., Roco, M. C., Son, J., Jiang, S., Larson, C. A., & Gao, Q. (2013). Global nanotechnology development from 1991 to 2012: Patents, scientific publications, and effect of NSF funding. Journal of Nanoparticle Research, 15, 1–21.Google Scholar
- Costas, R., & Leeuwen, T. N. (2012). Approaching the “reward triangle”: General analysis of the presence of funding acknowledgments and “peer interactive communication” in scientific publications. Journal of the American Society for Information Science and Technology, 63, 1647–1661.CrossRefGoogle Scholar
- Feng, M., & Yishan, W. (2009). A survey study on motivations for citation: A case study on periodicals research and library and information science community in China. Journal of Data and Information, 2, 28–43.Google Scholar
- Guo, H., Pan, Y., Ma, Z., Su, C., Yu, Z., & Xu, B. (2011). Quantitative analysit of papers sponsored by Natural Science Foundation of China. Science & Technology Review, 29, 61–66.Google Scholar
- Hu, Z., & Wu, Y. (2017). A probe into causes of non-citation based on survey data. Social Science Information, 56, 1–13.Google Scholar
- Huang, Y., Zhang, Y., Youtie, J., Porter, A. L., & Wang, X. (2016). How does national scientific funding support emerging interdisciplinary research: A comparison study of big data research in the US and China. PLoS ONE, 11, e154509.Google Scholar
- LI, G. (2004) Improving the efficiency of scientific research depends on the ouput impact. Invention and Innovation, 1–5.Google Scholar
- Siyao, L., & Yongxin, Z. (2016). Basic research is the supply side of innovation—an interview with Yang Wei, director of the National Natural Science Foundation of China. Beijing: People’s Daily.Google Scholar
- Wang, J., & Shapira, P. (2015). Is there a relationship between research sponsorship and publication impact? An analysis of funding acknowledgments in nanotechnology papers. PLoS ONE, 10, 555–582.Google Scholar
- Wikipedia (2018) Benchmarking. https://en.wikipedia.org/wiki/Benchmarking. Accessed 24 November 2018.
- Yu, Z., Ma, Z., Guo, H., Wang, N., & Jia, J. (2013). A quantitative analysis of papers sponsored by National Science and Technology programs. Journal of Intelligence, 32, 1–5.Google Scholar
- Zhang, A., Gao, P., & Liu, S. (2010). Analysis on the output of social science paper subsidized by science foundations of countries. Information Science, 28, 705–708.Google Scholar
- Zou, G., & Tang, Y. (2011). Survey and analysis on input and output of major Projects of Natural Science Fund in China. Science and Technology Management Research, 31, 32–35.Google Scholar