, Volume 59, Issue 2, pp 253–276 | Cite as

Using a bibliometric approach to support research policy making: The case of the Flemish BOF-key

  • Koenraad Debackere
  • Wolfgang Glänzel


In this paper, we describe the development of a methodology and an instrument to support a major research funding allocation decision by the Flemish government. Over the last decade, and in parallel with the decentralization and the devolution of the Belgian federal policy authority towards the various regions and communities in the country, science and technology policy have become a major component of regional policy making. In the Flemish region, there has been an increasing focus on basing the funding allocation decisions that originate from this policy decentralization on “objective, quantifiable and repeatable” decision parameters. One of the data sources and indicator bases that have received ample attention in this evolution is the use of bibliometric data and indicators. This has now led to the creation of a dedicated research and policy support staff, called “Steunpunt O&O Statistieken,” and the first time application of bibliometric data and methods to support a major inter-university funding allocation decision. In this paper, we analyze this evolution. We show how bibliometric data have for the first time been used to allocate 93 million Euro of public research money between 6 Flemish universities for the fiscal year 2003, based on Web-of-Science SCI data provided to “Steunpunt O&O Statistieken” via a license agreement with Thomson-ISI. We also discuss the limitations of the current approach that was based on inter-university publication and citation counts. We provide insights into future adaptations that might make it more representative of the total research activity at the universities involved (e.g., by including data for the humanities) and of its visibility (e.g., by including impact measures). Finally, based on our current experience and interactions with the universities involved, we speculate on the future of the specific bibliometric approach that has now been adopted. More specifically, we hypothesize that the allocation method now developed and under further improvement will become more criticized if it turns out that it (1) also starts influencing intra-university research allocation decisions and, as a consequence (2) introduces adverse publication and citation behaviors at the universities involved.


Citation Count Allocation Rule Bibliometric Indicator Gross Regional Product Funding Allocation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Amram, M. (2002), Value Sweep: Mapping Corporate Growth Opportunities, Harvard Business School Press, Boston.Google Scholar
  2. Audretsch, D. B., Bozeman, B., Combs, K. L., Feldman, M., Link, A. N., Siegel, D. S., Stephan, P., Tassey, G., Wessner, C. (2002), The economics of science and technology, The Journal of Technology Transfer, 27(2): 155-203.CrossRefGoogle Scholar
  3. Boer, F. P. (2002), Financial management of R&D 2002, Research Technology Management, July–August Issue: 23–35.Google Scholar
  4. Brockhoff, K. (1994), Forschung und Entwicklung: Planung und Kontrolle (4. Auflage), R. Oldenbourg Verlag, München.Google Scholar
  5. CORDIS Focus (2002), European Innovation Scoreboard 2002, E.C. Luxembourg.Google Scholar
  6. Debackere, K., GlÄnzel, W., Meyer, M. (2002), A Reflection on the Opportunities and Limitations in Using Publication and Patent Based Indicators, Working Paper, Steunpunt O&O Statistieken, K.U. Leuven.Google Scholar
  7. EIRMA (1970), Methods for the Evaluation of R&D Projects, Vol. I, Paris.Google Scholar
  8. FWO-Vlaanderen (2002), Jaarverslag.Google Scholar
  9. Gibbons, M., Georghiou, L. (1986), Research Evaluation, OECD, Paris.Google Scholar
  10. Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., Trow, M. (1994), The New Production of Knowledge, Sage Publications, London.Google Scholar
  11. GlÄnzel, W. (1996), The need for standards in bibliometric research and technology, Scientometrics, 35(1): 167-176.CrossRefGoogle Scholar
  12. GlÄnzel, W., Schubert, A. (2002), A new classification scheme of science fields and subfields designed for scientometric evaluation purposes, Scientometrics, 56(3): 357-367.CrossRefGoogle Scholar
  13. Griliches, Z. (1984), R&D, Patents and Productivity, Chicago University Press, Chicago.Google Scholar
  14. HBPWB (2003), Horizontaal Begrotingsprogramma Wetenschapsbeleid, AWI Publication, Ministery of the Flemish Community, Brussels.Google Scholar
  15. IWT-Vlaanderen (2002), Jaarverslag.Google Scholar
  16. Katz, J. S., Hicks, D. M. (1996), A Systemic view of British science, Scientometrics, 35(1): 133-154.CrossRefGoogle Scholar
  17. Luwel, M. (1999), Is the Science Citation Index US-biased, Scientometrics, 46(3): 549-562.CrossRefGoogle Scholar
  18. Luwel, M., Noyons, E. C. M., Moed, H. F. (1999), Bibliometric assessment of research performance in Flanders: Policy background and implications, R & D Management, 29(2): 133-141.CrossRefGoogle Scholar
  19. Moed, H. F. M., De Bruin, R. E., Nederhof, A. J., van Raan, A. F. J., R. J. W. Tijssen (1992), State of Art Bibliometric Macro-Indicators. An Overview of Demand and Supply, Office for Official Publications of the European Community, Luxembourg.Google Scholar
  20. NSF (1961), Investing in Scientific Progress, National Science Foundation Report 61-27, Washington.Google Scholar
  21. NSF (1967), Patterns of National R&D Resources, 1953–1968, Reviews of Data on Science Resources, National Science Foundation Report 67-7, Washington.Google Scholar
  22. NSF (1976), Indicators of International Trends in Technological Innovation, National Science Foundation, Washington.Google Scholar
  23. OECD (2000), Knowledge, Technology and Economic Growth: Recent Evidence from OECD Countries.Google Scholar
  24. OECD (2001), Benchmarking Industry-Science Relationships, Science, Technology and Industry Outlook 2000.Google Scholar
  25. Paxson, D. A. (2001), Introduction to real R&D options, (Special Issue), R&D Management Journal, 31(2): 109-248.CrossRefGoogle Scholar
  26. Roberts, E. B. (1964), The Dynamics of Research and Development, Harper & Row Publishers, New York.Google Scholar
  27. Roman, D. D. (1968), Research and Development Management: The Economics and Administration of Technology, Prentice-Hall, Englewood Cliffs, New Jersey.Google Scholar
  28. Roussel, P. A., Saad, K. N., Erickson, T. J. (1991), Third Generation R&D, Harvard Business School Press, Boston.Google Scholar
  29. Seiler, R. E. (1965), Improving the Effectiveness of Research and Development, McGraw-Hill Book Company, New York.Google Scholar
  30. Tidd, J., Bessant, J., Pavitt, K. (2001), Managing Innovation, John Wiley & Sons, Chichester.Google Scholar
  31. Tijssen, R. J. W., Visser, M. S., van Leeuwen, T. N. (2001), Searching for scientific excellence: Scientometric measurements and citation analyses of national research systems, Proceedings of the 8th International Conference on Scientometrics & Informetrics, pp. 675-689.Google Scholar
  32. Van Den Berghe, H., Houben, J. A., De Bruin, R. E., Moed H. F., Kint, A., Luwel, M., Spruyt, E. H. J. (1998), Bibliometric indicators of university research performance in Flanders, Journal of the American Society for Information Science, 49(1): 59-67.Google Scholar
  33. Villers, R. (1964), Research and Development: Planning and Control, Financial Executives Institute, New York.Google Scholar
  34. Vinck, D. (Ed.) (1991), Gestion de la Recherche, De Boeck Publishing, Brussels.Google Scholar
  35. VRWB (2002), Het ‘grote’ begrotingsadvies wetenschaps-en technologisch innovatiebeleid 2002, Brussel.Google Scholar

Copyright information

© Kluwer Academic Publishers/Akadémiai Kiadó 2004

Authors and Affiliations

  • Koenraad Debackere
    • 1
  • Wolfgang Glänzel
    • 1
  1. 1.Katholieke Universiteit Leuven, Steunpunt O&O StatistiekenLeuvenBelgium

Personalised recommendations