Skip to main content
SpringerLink
Log in

Integrated figure of merit of public sector research evaluation

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

An approach for evaluation of research is described that integrates output indicators of four stages downstream the innovation process: immediate, intermediate, pre- ultimate and ultimate outputs. Indexes of leading output indicators are constructed. The indexes are integrated cumulatively to form an overall index of key output indicators, which is the integrated figure of merit (IFM). Data for the indicators are obtained from records and key informants, and the indicators are grouped by normalized weights. The paper also discusses the limitations and the methodological, conceptual and political/organizational issues of such an approach to research evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R. N. Kostoff, Research impact assessment: Problems, progress, promise,Proceedings of the Fourth International Conference on Management of Technology, Miami, Florida, February 27–March 4, 1994.

  2. E. Geisler, Key output indicators in performance evaluation of research and development organizations,Technological Forecasting and Social Change, 47 (1994) 189–204.

    Article  Google Scholar 

  3. A. H. Rubenstein, E. Geisler, Evaluating the outputs and impacts of R&D/innovation,International Journal of Technology Management, Special publication on the Role of Technology in Corporate Policy (1992) 181–204.

  4. J. A. D. Holbrook, Basic indicators of scientific and technological performance,Science and Public Policy, 19 (1992) 267–273.

    Google Scholar 

  5. E. Mansfield, Academic research and industrial innovation,Research Policy, 20 (1991) 1–12.

    Article  Google Scholar 

  6. E. Mansfield, Academic research and industrial innovation: A further note,Research Policy, 21 (1992) 295–296.

    Article  Google Scholar 

  7. National Science Board,Science & Engineering Indicators — 1993, Washington, D.C., U.S. Government Printing Office (1993). The numbers cited are in constant 1987 dollars computed by using GDP implicit price deflators.

    Google Scholar 

  8. J. David Roessner, Evaluating government innovation programs: Lessons from the U.S. experience,Research Policy, 18 (1989) 343–360.

    Article  Google Scholar 

  9. L. Branscomb, National laboratories: The search for new missions and new structures, In:L. Branscomb (Ed.)Empowering Technology. Implementing a US Strategy, MIT Press, Cambridge, MA, 1993.

    Google Scholar 

  10. B. Bozeman, Evaluating government technology transfer: Early impacts of the cooperative technology paradigm,Policy Studies Journal, 22 (1994) 322–340.

    Article  Google Scholar 

  11. S. Kassicieh, H. R. Radosevich (Eds),From Lab to Market: Commercialization of Public Sector Technology, Plenum Press, New York, 1994.

    Google Scholar 

  12. K. Hughes, The interpretation and measurement of R&D intensity — A note,Research Policy, 17 (1988) 301–307.

    Article  Google Scholar 

  13. D. Evered, S. Hannett (Eds),The Evaluation of Scientific Research, John Wiley & Sons, Chichester, UK, 1989.

    Google Scholar 

  14. B. Bozeman, M. Crow, The environments of US R&D laboratories: Political and market influences,Policy Sciences, 23 (1990) 25–56.

    Article  Google Scholar 

  15. D. Frey, E. Geisler, Commercialization of energy related technology to industry: The case of the US national energy laboratories,International Journal of Global Energy Issues, forthcoming (1996).

  16. R. Radosevich, S. Kassicieh, Strategic challenges and proposed responses to competitiveness through public sector technology,California Management Review, 35 (1993) 33–50.

    Google Scholar 

  17. Z. Griliches (Ed.),R&D, Patents, and Productivity, The University of Chicago Press, Chicago, 1984.

    Google Scholar 

  18. B. Bozeman, J. Melkers,Evaluating R&D Impacts: Methods and Practices, Kluwer Academic Publishers, Boston, MA, 1993.

    Google Scholar 

  19. R. Kostoff, Co-word analysis in:B. Bozeman, J. Melkers, ——op. cit., pp. 63–78.

    Google Scholar 

  20. R. Kostoff, Database tomography: Origins and applications,Competitive Intelligence Review, Special Issue on Technology, 5 (1994).

  21. A. Link, Methods for evaluating the return on R&D investments, In:B. Bozeman, J. Melkers, ——op. cit., pp. 1–16.

    Google Scholar 

  22. K. Azumi, F. Hull, Inventive payoff from R&D in Japanese industry: Convergence with the West,IEEE Transactions on Engineering Management, 37 (1990) 3–9.

    Article  Google Scholar 

  23. R. Szakonyi, Measuring R&D effectiveness — I,Research · Technology Management, (March–April, 1994) 27–32.

  24. Z. Griliches, Research costs and social returns: Hybrid corn and related innovations,Journal of Political Economy, 66 (1958).

  25. B. Martin, J. Irvine, Assessing basic research: Some partial indicators of scientific progress in radioastronomy,Research Policy, 12 (1983) 62–90.

    Article  Google Scholar 

  26. S. Green, M. Gavin, L. Aiman-Smith, Assessing a multidimensional measure of radical technological innovation,IEEE Transactions on Engineering Management, 42 (1995) 203–214.

    Article  Google Scholar 

  27. E. Geisler, Evaluation of R&D: Approaches, Methods, Techniques, Paper presented at TIMS/ORSA National Meeting, Orlando, Florida, April 24–29, 1992.

  28. E. Geisler, An integrated cost-performance model of research and development evaluation,Omega, 23 (1995) 281–294.

    Article  Google Scholar 

  29. A. H. Rubenstein, E. Geisler, The use of indicators and measures of the R&D process in evaluating science and technology programs, In:J. D. Roessner (Ed.),Government Innovation Policy, St. Martin Press, 1988, pp. 185–204.

  30. H. Schmied, About the quantification of the economic impact of public investment into scientific research,International Journal of Technology Management 2 (1987) 711–730.

    Google Scholar 

  31. W. Conrad Fernelius, W. Waldo, Role of basic research in industrial innovation,Research Management, (July 1980) 36–40.

  32. D. Hall, A. Nauda, An integrative approach for selecting IR&D projects,IEEE Transactions on Engineering Management, 37 (1990) 126–133.

    Article  Google Scholar 

  33. Y. Kuwahara, Y. Takeda, A managerial approach to research and development cost-effectiveness evaluation,IEEE Transactions on Engineering Management, 37 (1990) 134–138.

    Article  Google Scholar 

  34. E. Geisler, Measuring the unquantifiable: Issues in the use of indicators in unstructured phenomena,International Journal of Operations and Quantitative Management, 1 (1995) 145–161.

    Google Scholar 

  35. T. R. Zenger, Explaining organizational diseconomies of scale in R&D: Agency problems and the allocation of engineering talent, ideas, and effort by firm size,”Management Science, 40 (1994) 708–729.

    Google Scholar 

  36. D. L. Goodhue, Understanding user evaluations of information systems,Management Science, 14 (1995) 1827–1844.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management, College of Business & Economics, University of Wisconsin-Whitewater, 53190, Whitewater, WI, (USA)

    E. Geisler

Authors
  1. E. Geisler
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Geisler, E. Integrated figure of merit of public sector research evaluation. Scientometrics 36, 379–395 (1996). https://doi.org/10.1007/BF02129601

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02129601

Keywords

Search

Navigation