Research Portfolio Analysis in Science Policy: Moving from Financial Returns to Societal Benefits
- 1.7k Downloads
Funding agencies and large public scientific institutions are increasingly using the term “research portfolio” as a means of characterizing their research. While portfolios have long been used as a heuristic for managing corporate R&D (i.e. R&D aimed at gaining tangible economic benefits), they remain ill-defined in a science policy context where research is aimed at achieving societal outcomes. In this article we analyze the discursive uses of the term “research portfolio” and propose some general considerations for their application in science policy. We explore the use of the term in private R&D and related scholarly literature in existing science policy practices, and seek insight in relevant literature in science policy scholarship. While the financial analogy can in some instances be instructive, a simple transposition from the world of finance or of corporate R&D to public research is problematic. However, we do identify potentially fruitful uses of portfolio analysis in science policy. In particular, our review suggests that the concept of research portfolio can indeed be a useful analytical instrument for tackling complex societal challenges. Specifically, the strands of scholarship identified suggest that the use of research portfolio should: i) recognize the diversity of research lines relevant for a given societal challenge, given the uncertainty and ambiguity of research outcomes; ii) examine the relationships between research options of a portfolio and the expected societal outcomes; and iii) adopt a systemic perspective to research portfolios – i.e. examine a portfolio as a functional whole, rather than as the sum of its parts. We argue that with these considerations, portfolio-driven approaches may foster social inclusion in science policy decisions, help deliberation between “alternative” portfolios to tackle complex societal challenges, as well as promote cost-effectiveness and transparency.
KeywordsResearch portfolio Prioritisation Research landscape Societal challenges
We thank Tommaso Ciarli, Jochen Gläser, Jordi Molas-Gallart, Richard Wooley, and two anonymous referees for their insightful comments and suggestions. We acknowledge support from the UK Economic and Social Research Council (Grant RES-360-25-0076, Mapping the Development of Emergent Technologies) and the FP7 EU Marie Curie Integration Grant to IR (MapRePort).
- Agarwal, Pankaj, and David B Searls. 2009. Can literature analysis identify innovation drivers in drug discovery? Nature Reviews. Drug Discovery 8. Nature Publishing Group: 865–78. doi: 10.1038/nrd2973.
- Bazilian, Morgan, and Fabien Roque (eds.). 2008. No Analytical Methods for Energy Diversity and Security: Portfolio Optimization in the Energy Sector: A Tribute to the Work of Dr. Shimon Awerbuch. Amsterdam: Elsevier.Google Scholar
- Bernard Cohen, I. 1993. Analogy, Homology, and Metaphor in the Interactions between the Natural Sciences and the Social Sciences, Especially Economics. In Non-natural Social Science: Reflecting on the Enterprise of More Heat than Light, ed. Neil de Marchi, 7–44. Durham: Duke University Press.Google Scholar
- Boaz, Annette, Siobhan Fitzpatrick, and Ben Shaw. 2008. Assessing the impact of research on policy: A review of the literature for a project on bridging research and policy through outcome evaluation Final report with references and appendices, February 2008.Google Scholar
- Brooks, Harvey. 1978. The Problem of Research Priorities. Daedelus 107: 171–190.Google Scholar
- Buxton, Martin, Leonie Sundmacher, Jorge Mestre-Ferrandiz, Liz Allen, Nick Black, David Cox, Helen Munn, Briony Rayfield, Eddy Nason, and Jon Sussex. 2008. Medical Research: What's it worth? Estimating the economic benefits from medical research in the UK. London: Health Economics Research Group, Office of Health Economics, RAND Europe.Google Scholar
- Chalmers, Iain, M.B. Bracken, and Ben Djulbegovic. 2014. How to increase value and reduce waste when research priorities are set. The Lancet: 7–16.Google Scholar
- Cozzens, Susan, and Michelle Snoeck. 2010. Knowledge to Policy Contributing to the Measurement of Social, Health, and Environmental Benefits. Paper prepared for the Workshop on the Science of Science Measurement: 1–39.Google Scholar
- Dolby, Kevin, Jimmy Whitworth, Marta Tufet, Suzi Morris, Jessica Burnett, Lily Ickowitz-Seidler, Annie Sanderson, Dave Carr, and Jo Scott. 2012. Malaria 1990–2009. London: Wellcome Trust.Google Scholar
- European Commission. 2005. Impact assessment and ex ante evaluation. Brussels.Google Scholar
- Ghiselin, Michael T. 1978. The Economy of the Body. The American Economic Review 68: 233–237.Google Scholar
- Gläser, Jochen. 2012. Jochen Gläser on the possibility of a sociological middle-range theory linking science. TUTS-WP-1-2012. Technical University Technology Studies Working Papers. Berlin.Google Scholar
- Guthrie, Susan, Benoit Guerin, Helen Wu, Sharif Ismail, and Steven Wooding. 2013. Alternatives to Peer Review in Research Project Funding.Google Scholar
- Haak, Laurel L., Will Ferriss, Kevin Wright, Michael E. Pollard, Kirk Barden, Matt A. Probus, Michael Tartakovsky, and Charles J. Hackett. 2012. The electronic Scientific Portfolio Assistant: Integrating scientific knowledge databases to support program impact assessment. Science and Public Policy 39: 464–475. doi: 10.1093/scipol/scs030.CrossRefGoogle Scholar
- Hanney, Stephen R. 2003. The utilisation of health research in policy-making: Concepts, examples and methods of assessment. Health Research 28: 1–28.Google Scholar
- Hausmann, Ricardo, César A. Hidalgo, Sebastián Bustos, Michele Coscia, Alexander Simoes, and Muhammed A. Yildirim. 2013. The atlas of economic complexity: Mapping paths to prosperity. Cambridge: Massachusetts Institute of Technology and Centre for International Development, Harvard University.Google Scholar
- Hicks, Diana. 2014. “What are grand challenges?” The selected works of Diana Hicks. http://works.bepress.com/diana_hicks/38. (unpublished).
- Holbrook, J. Britt, and Robert Frodeman. 2011. Peer review and the ex ante assessment of societal impacts. Research Evaluation 20: 239–246. doi: 10.3152/095820211X12941371876788.
- Ismail, Sharif, Jan Tiessen, and Steven Wooding. 2010. Strengthening Research Portfolio Evaluation at the Medical Research Council.Google Scholar
- Kuhn, Thomas S. 1979. Metaphor in science. In Metaphor and Thought, ed. A. Ortony. Cambridge: Cambridge University Press.Google Scholar
- Lakoff, George, and Mark Johnson. 1980. Metaphors we live by. Chicago: University of Chicago Press.Google Scholar
- Liggins, Charlene, Lisa Pryor, and Marie A. Bernard. 2010. Challenges and Opportunities in Advancing Models of Care for Older Adults: An Assessment of the National Institute on Aging Research Portfolio. Journal of the American Geriatrics Society 58: 2345–2349. doi: 10.1111/j.1532-5415.2010.03157.x.CrossRefGoogle Scholar
- Marburger, John. 2005. Presentation to the Annual Meeting of the Consortium of Social Science Associations (Washington, D.C.).Google Scholar
- Markowitz, Harry. 1952. Portfolio Selection. Journal of Finance 7: 77–91.Google Scholar
- Marres, Noortje, and Esther Weltevrede. 2013. SCRAPING THE SOCIAL? Journal of Cultural Economy 6:313–335. doi: 10.1080/17530350.2013.772070.
- Mirowski, Philip. 1991. More Heat than Light: Economics as Social Physics, Physics as Nature’s Economics. Cambridge: Cambridge University Press.Google Scholar
- National Research Council. 2005. A prospective evaluation of applied energy research and development at DOE (Phase One). Washington.Google Scholar
- National Research Council. 2012. A Review of NASA Human Research Program’s Scientific Merit Processes: Letter Report. Washington: The National Academies Press.Google Scholar
- National Science Board. 2001. Federal Research Resources: A Process for Setting Priorities. National Science Foundation.Google Scholar
- Nicholson, Joshua M., and John P. Ioannidis. 2012. Research grants: Conform and be funded. Nature 492: 34–36.Google Scholar
- Porter, Theodore M. 1995. Trust in numbers: The pursuit of objectivity in science and public life. Princeton: Princeton University Press.Google Scholar
- Portfolio Review Group. 2014. Report of the Portfolio Review Group: 2012-2013 University of California Systemwide Research Portfolio: Cycle 1 Programs Findings and Recommendations. University of California.Google Scholar
- Rafols, Ismael, Loet Leydesdorff, Alice O’Hare, Paul Nightingale, and Andy Stirling. 2012. How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management. Research Policy 41: 1262–1282. doi: 10.1016/j.respol.2012.03.015.CrossRefGoogle Scholar
- Robertson, G. Philip, Vivien G. Allen, George Boody, Emery R. Boose, Nancy G. Creamer, E. Laurie, James R. Gosz, et al. 2008. Long-term Agricultural Research: A research, education, and extension imperative. BioScience 58: 640–645.Google Scholar
- Røttingen, John-Arne, Sadie Regmi, Mari Eide, Alison J. Young, Roderik F. Viergever, Christine Ardal, Javier Guzman, Danny Edwards, Stephen Matlin, and Robert F. Terry. 2013. Mapping of available health research and development data: What’s there, what’s missing, and what role is there for a global observatory? Lancet 382: 1286–1307. doi: 10.1016/S0140-6736(13)61046-6.CrossRefGoogle Scholar
- Sarewitz, Daniel. 1996. Frontiers of Illusion: Science, Technology and the Politics of Progress. Philadelphia: Temple University Press.Google Scholar
- Scientific Management Review Board. 2013. Draft Report on Approaches to Assess the Value of Biomedical Research Supported by NIH. National Institutes of Health.Google Scholar
- Souder, William E., and Tomislav Mandakovic. 1986. R&D Project Selection Models. Research Management 29: 36–42.Google Scholar
- Sponberg, Adrienne F. 2005. Streamlining the federal water research portfolio. BioScience 55.Google Scholar
- Stilgoe, Jack. 2014. Against excellence. The Guardian, December 19. http://www.theguardian.com/science/political-science/2014/dec/19/against-excellence.
- Stirling, Andy, and Ian Scoones. 2009. From Risk Assessment to Knowledge Mapping: Science, Precaution, and Participation in Disease Ecology. Ecology and Society 14: 14.Google Scholar
- Swedish Presidency of the European Union. 2009. The Lund Declaration. European Union.Google Scholar
- Waltman, Ludo, Nees Jan Van Eck, and Ed C. M. Noyons. 2009. A unified approach to mapping and clustering of bibliometric networks: 1–11.Google Scholar