Skip to main content

Advertisement

Log in

Research Portfolio Analysis in Science Policy: Moving from Financial Returns to Societal Benefits

  • Published:
Minerva Aims and scope Submit manuscript

Abstract

Funding agencies and large public scientific institutions are increasingly using the term “research portfolio” as a means of characterizing their research. While portfolios have long been used as a heuristic for managing corporate R&D (i.e. R&D aimed at gaining tangible economic benefits), they remain ill-defined in a science policy context where research is aimed at achieving societal outcomes. In this article we analyze the discursive uses of the term “research portfolio” and propose some general considerations for their application in science policy. We explore the use of the term in private R&D and related scholarly literature in existing science policy practices, and seek insight in relevant literature in science policy scholarship. While the financial analogy can in some instances be instructive, a simple transposition from the world of finance or of corporate R&D to public research is problematic. However, we do identify potentially fruitful uses of portfolio analysis in science policy. In particular, our review suggests that the concept of research portfolio can indeed be a useful analytical instrument for tackling complex societal challenges. Specifically, the strands of scholarship identified suggest that the use of research portfolio should: i) recognize the diversity of research lines relevant for a given societal challenge, given the uncertainty and ambiguity of research outcomes; ii) examine the relationships between research options of a portfolio and the expected societal outcomes; and iii) adopt a systemic perspective to research portfolios – i.e. examine a portfolio as a functional whole, rather than as the sum of its parts. We argue that with these considerations, portfolio-driven approaches may foster social inclusion in science policy decisions, help deliberation between “alternative” portfolios to tackle complex societal challenges, as well as promote cost-effectiveness and transparency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://ec.europa.eu/research/era/joint-programming-initiatives_en.html.

  2. http://www.grandchallenges.org/Pages/Default.aspx.

  3. Following Hicks (2014), we define grand challenges as multifaceted, multidisciplinary, large-scale and policy oriented problems with both an intellectual and practical component.

  4. See, for example, the illustration of (research) portfolios in terms of funding in the recent work of UberResearch. (http://www.uberresearch.com/visual-portfolios/) or in terms of publications in Rafols, Porter and Leydesdorff (2010) (http://www.idr.gatech.edu/maps).

  5. Documents were downloaded from the Web of Knowledge in May 2014. The search was limited to original articles, reviews and editorial material, and to any of the following search strings and their plural forms: “research portfolio”, “science portfolio”, “research and development portfolio”, “R&D portfolio”, “scientific portfolio”, “portfolio of research”, “portfolio of R&D”.

  6. Data retrieved June 2014 from http://wokinfo.com/.

  7. We thank Tommaso Ciarli for suggesting to include this table as a means to organize and summarize much of the reviewed information.

  8. See http://report.nih.gov and http://gtr.rcuk.ac.uk.

  9. There is even a new company, ÜberResearch (http://www.uberresearch.com/), specialized in the analysis of research portfolios of funding agencies.

  10. For example, see a typical description of such skills at: http://www.pmi.org/Professional-Development/Career-Central/Three-Must-Have-Skills-for-Portfolio-Managers.aspx (accessed June 20, 2014).

  11. Excerpts from a recent job vacancy posting at the National Health Service (italics added). http://jobs.gstt.nhs.uk/job/UK/London/London/Guys_St_Thomas_NHS_Foundation_Trust/Biomedical_Research_Centre/Biomedical_Research_Centre-v318965 (accessed June 20, 2014).

  12. We perform a Google search for “research portfolios” from websites ending in: .eu, .ca, .uk, .au, and .gov, then extract the context of each “research portfolio” result using the Outwit software package. Finally, we perform a manual cleaning, removing spurious and duplicate results, which leaves us with 1,186 distinct search contexts.

  13. Here, we keep all the relevant terms extracted, removing only “research” and “research portfolio” as terms which are common to the bulk of the results.

  14. Complementing the mapping approach, we perform a manual analysis of the results by looking for how often some of the most common strings occur. These strings are mainly related to employment (“job”, “vacancy”, etc.), planning and reporting (“accountability”, “outcome”, etc.), financial return (“investment”, “fund”, etc.), and general descriptions (“overview”, “profile”, etc.).

  15. To avoid confusion, let us stress that this definition of system is purely functional and, in principle, unrelated to the literature of national, regional or technological innovation systems.

References

  • Agarwal, Pankaj, and David B Searls. 2009. Can literature analysis identify innovation drivers in drug discovery? Nature Reviews. Drug Discovery 8. Nature Publishing Group: 865–78. doi:10.1038/nrd2973.

  • Altman, Edward I., and Anthony Saunders. 1998. Credit risk measurement: Developments over the last 20 years. Journal of Banking and Finance 21: 1721–1742.

    Article  Google Scholar 

  • Arnold, Erik. 2004. Evaluating research and innovation policy: A systems world needs systems evaluations. Research Evaluation 13: 3–17.

    Article  Google Scholar 

  • Awerbuch, Shimon. 2006. Portfolio-Based Electricity Generation Planning: Policy Implications For Renewables And Energy Security. Mitigation and Adaptation Strategies for Global Change 11: 693–710. doi:10.1007/s11027-006-4754-4.

    Article  Google Scholar 

  • Bazilian, Morgan, and Fabien Roque (eds.). 2008. No Analytical Methods for Energy Diversity and Security: Portfolio Optimization in the Energy Sector: A Tribute to the Work of Dr. Shimon Awerbuch. Amsterdam: Elsevier.

    Google Scholar 

  • Bernard Cohen, I. 1993. Analogy, Homology, and Metaphor in the Interactions between the Natural Sciences and the Social Sciences, Especially Economics. In Non-natural Social Science: Reflecting on the Enterprise of More Heat than Light, ed. Neil de Marchi, 7–44. Durham: Duke University Press.

    Google Scholar 

  • Boaz, Annette, Siobhan Fitzpatrick, and Ben Shaw. 2008. Assessing the impact of research on policy: A review of the literature for a project on bridging research and policy through outcome evaluation Final report with references and appendices, February 2008.

  • Boyack, Kevin W., and Paul Jordan. 2011. Metrics associated with NIH funding: A high-level view. JAMIA 18: 423–431. doi:10.1136/amiajnl-2011-000213.

    Google Scholar 

  • Bozeman, Barry, and Juan Rogers. 2001. Strategic management of government-sponsored R&D portfolios. Environment and Planning C: Government and Policy 19: 413–442. doi:10.1068/c1v.

    Article  Google Scholar 

  • Bozeman, Barry, and Daniel Sarewitz. 2005. Public values and public failure in US science policy. Science and Public Policy 32(2): 119–136.

    Article  Google Scholar 

  • Bozeman, Barry, and Daniel Sarewitz. 2011. Public Value Mapping and Science Policy Evaluation. Minerva 49(1): 1–23. doi:10.1007/s11024-011-9161-7.

    Article  Google Scholar 

  • Brooks, Harvey. 1978. The Problem of Research Priorities. Daedelus 107: 171–190.

    Google Scholar 

  • Buxton, Martin, Leonie Sundmacher, Jorge Mestre-Ferrandiz, Liz Allen, Nick Black, David Cox, Helen Munn, Briony Rayfield, Eddy Nason, and Jon Sussex. 2008. Medical Research: What's it worth? Estimating the economic benefits from medical research in the UK. London: Health Economics Research Group, Office of Health Economics, RAND Europe.

  • Calvert, Jane. 2006. What’s Special about Basic Research? Science, Technology & Human Values 31: 199–220. doi:10.1177/0162243905283642.

    Article  Google Scholar 

  • Chalmers, Iain, M.B. Bracken, and Ben Djulbegovic. 2014. How to increase value and reduce waste when research priorities are set. The Lancet: 7–16.

  • Chien, Chen–Fu. 2002. A portfolio–evaluation framework for selecting R&D projects. R&D Management 32: 359–368. doi:10.1111/1467-9310.00266.

    Article  Google Scholar 

  • Cozzens, Susan. 1997. The knowledge pool: Measurement challenges in evaluating fundamental research programs. Evaluation and Program Planning 20: 77–89. doi:10.1016/S0149-7189(96)00038-9.

    Article  Google Scholar 

  • Cozzens, Susan, and Michelle Snoeck. 2010. Knowledge to Policy Contributing to the Measurement of Social, Health, and Environmental Benefits. Paper prepared for the Workshop on the Science of Science Measurement: 1–39.

  • Dasgupta, Partha, and Eric Maskin. 2012. The Simple Economics of Research Portfolios. The Economic Journal 97: 581–595.

    Article  Google Scholar 

  • Devinney, Timothy M., and David W. Stewart. 1988. Rethinking the Product Portfolio: A Generalized Investment Model. Management Science 34: 1080–1095. doi:10.1287/mnsc.34.9.1080.

    Article  Google Scholar 

  • Dietz, James S., and Juan D. Rogers. 2012. Meanings and policy implications of “transformative research”: Frontiers, hot science, evolution, and investment risk. Minerva 50(1): 21–44. doi:10.1007/s11024-012-9190-x.

    Article  Google Scholar 

  • Dolby, Kevin, Jimmy Whitworth, Marta Tufet, Suzi Morris, Jessica Burnett, Lily Ickowitz-Seidler, Annie Sanderson, Dave Carr, and Jo Scott. 2012. Malaria 1990–2009. London: Wellcome Trust.

    Google Scholar 

  • Eikenberry, Angela M., and Jodie Drapal Kluver. 2004. The Marketization of the Nonprofit Sector: Civil Society at Risk. Public Administration Review 64: 132–140.

    Article  Google Scholar 

  • Ely, Adrian, Patrick Van Zwanenberg, and Andrew Stirling. 2014. Broadening out and opening up technology assessment: Approaches to enhance international development, co-ordination and democratisation. Research Policy 43: 505–518. doi:10.1016/j.respol.2013.09.004.

    Article  Google Scholar 

  • Ernst, Holger. 1998. Patent portfolios for strategic R&D planning. Journal of Engineering and Technology Management 15: 279–308. doi:10.1016/S0923-4748(98)00018-6.

    Article  Google Scholar 

  • European Commission. 2005. Impact assessment and ex ante evaluation. Brussels.

  • Evans, James A., Jae-Mahn Shim, and John P. Ioannidis. 2014. Attention to local health burden and the global disparity of health research. PloS One 9: e90147. doi:10.1371/journal.pone.0090147.

    Article  Google Scholar 

  • Feller, Irwin. 2012. Performance measures as forms of evidence for science and technology policy decisions. The Journal of Technology Transfer 38: 565–576. doi:10.1007/s10961-012-9264-9.

    Article  Google Scholar 

  • Fernandez, Eduardo, Edy Lopez, Gustavo Mazcorro, Rafael Olmedo, and Carlos Coello Coello. 2013. Application of the non-outranked sorting genetic algorithm to public project portfolio selection. Information Sciences 228: 131–149. doi:10.1016/j.ins.2012.11.018.

    Article  Google Scholar 

  • Fisher, Erik, Catherine P. Slade, Derrick Anderson, and Barry Bozeman. 2010. The public value of nanotechnology? Scientometrics 85: 29–39. doi:10.1007/s11192-010-0237-1.

    Article  Google Scholar 

  • Foray, Dominique, David C. Mowery, and Richard R. Nelson. 2012. Public R&D and social challenges: What lessons from mission R&D programs? Research Policy 41: 1697–1702. doi:10.1016/j.respol.2012.07.011.

    Article  Google Scholar 

  • Freeman, Christopher. 1991. Innovation, Changes of Techno-Economic Paradigm and Biological Analogies in Economics. Revue économique 42: 211. doi:10.2307/3502005.

    Google Scholar 

  • Frodeman, Robert, and Adam Briggle. 2012. The dedisciplining of peer review. Minerva 50(1): 3–19. doi:10.1007/s11024-012-9192-8.

    Article  Google Scholar 

  • Garfinkel, Michele S., Daniel Sarewitz, and Alan L. Porter. 2006. A societal outcomes map for health research and policy. American Journal of Public Health 96: 441–446. doi:10.2105/AJPH.2005.063495.

    Article  Google Scholar 

  • Geels, Frank W. 2004. From sectoral systems of innovation to socio-technical systems. Research Policy 33: 897–920. doi:10.1016/j.respol.2004.01.015.

    Article  Google Scholar 

  • Georghiou, Luke. 1998. Issues in the Evaluation of Innovation and Technology Policy. Evaluation 4: 37–51. doi:10.1177/13563899822208374.

    Article  Google Scholar 

  • Ghiselin, Michael T. 1978. The Economy of the Body. The American Economic Review 68: 233–237.

    Google Scholar 

  • Gläser, Jochen. 2012. Jochen Gläser on the possibility of a sociological middle-range theory linking science. TUTS-WP-1-2012. Technical University Technology Studies Working Papers. Berlin.

  • Golec, Joseph H. 1996. The effects of mutual fund managers’ characteristics on their portfolio performance, risk and fees. Financial Services Review 5: 133–147. doi:10.1016/S1057-0810(96)90006-2.

    Article  Google Scholar 

  • Guthrie, Susan, Benoit Guerin, Helen Wu, Sharif Ismail, and Steven Wooding. 2013. Alternatives to Peer Review in Research Project Funding.

  • Haak, Laurel L., Will Ferriss, Kevin Wright, Michael E. Pollard, Kirk Barden, Matt A. Probus, Michael Tartakovsky, and Charles J. Hackett. 2012. The electronic Scientific Portfolio Assistant: Integrating scientific knowledge databases to support program impact assessment. Science and Public Policy 39: 464–475. doi:10.1093/scipol/scs030.

    Article  Google Scholar 

  • Hage, Jerald, Gretchen B. Jordan, and Jonathan Mote. 2007. A theory-based innovation systems framework for evaluating diverse portfolios of research, part two: Macro indicators and policy interventions. Science and Public Policy 34: 731–741. doi:10.3152/030234207X265385.

    Article  Google Scholar 

  • Hammerstein, Peter, and Edward H. Hagen. 2005. The second wave of evolutionary economics in biology. Trends in Ecology & Evolution 20: 604–609. doi:10.1016/j.tree.2005.07.012.

    Article  Google Scholar 

  • Hanney, Stephen R. 2003. The utilisation of health research in policy-making: Concepts, examples and methods of assessment. Health Research 28: 1–28.

    Google Scholar 

  • Hanney, Stephen R., Iain Frame, Jonathan Grant, Martin Buxton, Tracey Young, and Grant Lewison. 2005. Using categorisations of citations when assessing the outcomes from health research. Scientometrics 65: 357–379.

    Article  Google Scholar 

  • Hausmann, Ricardo, César A. Hidalgo, Sebastián Bustos, Michele Coscia, Alexander Simoes, and Muhammed A. Yildirim. 2013. The atlas of economic complexity: Mapping paths to prosperity. Cambridge: Massachusetts Institute of Technology and Centre for International Development, Harvard University.

    Google Scholar 

  • Hicks, Diana. 2014. “What are grand challenges?” The selected works of Diana Hicks. http://works.bepress.com/diana_hicks/38. (unpublished).

  • Holbrook, J. Britt, and Robert Frodeman. 2011. Peer review and the ex ante assessment of societal impacts. Research Evaluation 20: 239–246. doi:10.3152/095820211X12941371876788.

  • Ioannidis, John P. 2011. Fund people not projects. Nature 477: 529–531.

    Article  Google Scholar 

  • Ioannidis, John P. 2014. How to Make More Published Research True. PLoS Medicine 11: e1001747. doi:10.1371/journal.pmed.1001747.

    Article  Google Scholar 

  • Ismail, Sharif, Jan Tiessen, and Steven Wooding. 2010. Strengthening Research Portfolio Evaluation at the Medical Research Council.

  • Jordan, Gretchen B., Jerald Hage, and Jonathon Mote. 2008. A theories-based systemic framework for evaluating diverse portfolios of scientific work, part 1: Micro and meso indicators. New Directions for Evaluation 2008: 7–24.

    Article  Google Scholar 

  • Kay, Luciano, Nils Newman, Jan Youtie, Alan L. Porter, and Ismael Rafols. 2014. Patent Overlay Mapping: Visualizing Technological Distance. Journal of the American Society for Information Science and Technology 65: 2432–2443. doi:10.1002/asi.23146.

    Article  Google Scholar 

  • Kuehn, Bridget M. 2012. US Reviews High-Risk Research Portfolio. JAMA 307: 1682.

    Article  Google Scholar 

  • Kuhn, Thomas S. 1979. Metaphor in science. In Metaphor and Thought, ed. A. Ortony. Cambridge: Cambridge University Press.

    Google Scholar 

  • Lakoff, George, and Mark Johnson. 1980. Metaphors we live by. Chicago: University of Chicago Press.

  • Largent, Mark A., and Julia Lane. 2012. STAR METRICS and the Science of Science Policy. Review of Policy Research 29: 431–438. doi:10.1111/j.1541-1338.2012.00567.x.

    Article  Google Scholar 

  • Laudel, Grit, and Jochen Gläser. 2014. Beyond breakthrough research: Epistemic properties of research and their consequences for research funding. Research Policy 43: 1204–1216. doi:10.1016/j.respol.2014.02.006.

    Article  Google Scholar 

  • Liggins, Charlene, Lisa Pryor, and Marie A. Bernard. 2010. Challenges and Opportunities in Advancing Models of Care for Older Adults: An Assessment of the National Institute on Aging Research Portfolio. Journal of the American Geriatrics Society 58: 2345–2349. doi:10.1111/j.1532-5415.2010.03157.x.

    Article  Google Scholar 

  • Linton, Jonathan D., Steven T. Walsh, and Joseph Morabito. 2002. Analysis, ranking and selection of R&D projects in a portfolio. R&D Management 32: 139–148. doi:10.1111/1467-9310.00246.

    Article  Google Scholar 

  • Luo, Lieh-Ming. 2011. Optimal diversification for R&D project portfolios. Scientometrics 91: 219–229. doi:10.1007/s11192-011-0537-0.

    Article  Google Scholar 

  • Marburger, John. 2005. Presentation to the Annual Meeting of the Consortium of Social Science Associations (Washington, D.C.).

  • Markowitz, Harry. 1952. Portfolio Selection. Journal of Finance 7: 77–91.

    Google Scholar 

  • Marres, Noortje, and Esther Weltevrede. 2013. SCRAPING THE SOCIAL? Journal of Cultural Economy 6:313–335. doi:10.1080/17530350.2013.772070.

  • Martin, Ben R. 2011. The Research Excellence Framework and the “impact agenda”: Are we creating a Frankenstein monster? Research Evaluation 20: 247–254. doi:10.3152/095820211X13118583635693.

    Article  Google Scholar 

  • McGeary, Michael, and Philip M. Smith. 1996. The R&D portfolio: A concept for allocating science and technology funds. Science 274: 1484–1485.

    Article  Google Scholar 

  • Meador, Kimford J., Jacqueline French, David W. Loring, and Page B. Pennell. 2011. Disparities in NIH funding for epilepsy research. Neurology 77: 1305–1307.

    Article  Google Scholar 

  • Mirowski, Philip. 1991. More Heat than Light: Economics as Social Physics, Physics as Nature’s Economics. Cambridge: Cambridge University Press.

    Google Scholar 

  • Molas-Gallart, Jordi, and Puay Tang. 2011. Tracing “productive interactions” to identify social impacts: An example from the social sciences. Research Evaluation 20: 219–226. doi:10.3152/095820211X12941371876706.

    Article  Google Scholar 

  • Moravcsik, Michael J. 1984. Life in a multidimensional world. Scientometrics 6: 75–85. doi:10.1007/BF02021280.

    Article  Google Scholar 

  • Moravcsik, Michael J. 1988. The limits of science and the scientific method. Research Policy 17: 293–299.

    Article  Google Scholar 

  • Mowery, David C. 2012. Defense-related R&D as a model for “Grand Challenges” technology policies. Research Policy 41: 1703–1715. doi:10.1016/j.respol.2012.03.027.

    Article  Google Scholar 

  • National Research Council. 2005. A prospective evaluation of applied energy research and development at DOE (Phase One). Washington.

  • National Research Council. 2012. A Review of NASA Human Research Program’s Scientific Merit Processes: Letter Report. Washington: The National Academies Press.

    Google Scholar 

  • National Science Board. 2001. Federal Research Resources: A Process for Setting Priorities. National Science Foundation.

  • Nicholson, Joshua M., and John P. Ioannidis. 2012. Research grants: Conform and be funded. Nature 492: 34–36.

    Google Scholar 

  • Perlitz, Manfred, Thorsten Peske, and Randolf Schrank. 1999. Real options valuation: The new frontier in R&D project evaluation? R&D Management 29: 255–270. doi:10.1111/1467-9310.00135.

    Article  Google Scholar 

  • Porter, Theodore M. 1995. Trust in numbers: The pursuit of objectivity in science and public life. Princeton: Princeton University Press.

  • Portfolio Review Group. 2014. Report of the Portfolio Review Group: 2012-2013 University of California Systemwide Research Portfolio: Cycle 1 Programs Findings and Recommendations. University of California.

  • Rafols, Ismael, Alan L. Porter, and Loet Leydesdorff. 2010. Science overlay maps: A new tool for research policy and library management. Journal of the American Society for Information Science and Technology 61: 1871–1887. doi:10.1002/asi.21368.

    Article  Google Scholar 

  • Rafols, Ismael, Loet Leydesdorff, Alice O’Hare, Paul Nightingale, and Andy Stirling. 2012. How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management. Research Policy 41: 1262–1282. doi:10.1016/j.respol.2012.03.015.

    Article  Google Scholar 

  • Reid, W.V., D. Chen, L. Goldfarb, H. Hackmann, Y.T. Lee, K. Mokhele, E. Ostrom, K. Raivio, H.J. Schellnhuber, and A. Whyte. 2010. Earth system science for global sustainability: Grand challenges. Science 330: 916–917.

    Article  Google Scholar 

  • Robertson, G. Philip, Vivien G. Allen, George Boody, Emery R. Boose, Nancy G. Creamer, E. Laurie, James R. Gosz, et al. 2008. Long-term Agricultural Research: A research, education, and extension imperative. BioScience 58: 640–645.

  • Røttingen, John-Arne, Sadie Regmi, Mari Eide, Alison J. Young, Roderik F. Viergever, Christine Ardal, Javier Guzman, Danny Edwards, Stephen Matlin, and Robert F. Terry. 2013. Mapping of available health research and development data: What’s there, what’s missing, and what role is there for a global observatory? Lancet 382: 1286–1307. doi:10.1016/S0140-6736(13)61046-6.

    Article  Google Scholar 

  • Ruegg, Rosalie T. 2007. Quantitative portfolio evaluation of US federal research and development programs. Science and Public Policy 34: 723–730. doi:10.3152/030234207X259021.

    Article  Google Scholar 

  • Salter, Ammon J., and Ben R. Martin. 2001. The economic benefits of publicly funded basic research: A critical review. Research Policy 30: 509–532. doi:10.1016/S0048-7333(00)00091-3.

    Article  Google Scholar 

  • Sarewitz, Daniel. 1996. Frontiers of Illusion: Science, Technology and the Politics of Progress. Philadelphia: Temple University Press.

    Google Scholar 

  • Sarewitz, Daniel, and Roger A. Pielke Jr. 2007. The neglected heart of science policy: Reconciling supply of and demand for science. Environmental Science & Policy 10: 5–16. doi:10.1016/j.envsci.2006.10.001.

    Article  Google Scholar 

  • Schwenk, Charles R. 1988. The Cognitive Perspective on Strategic Decision Making. Journal of Management Studies 25: 41–55. doi:10.1111/j.1467-6486.1988.tb00021.x.

    Article  Google Scholar 

  • Scientific Management Review Board. 2013. Draft Report on Approaches to Assess the Value of Biomedical Research Supported by NIH. National Institutes of Health.

  • Skupin, André, Joseph R. Biberstine, and Katy Börner. 2013. Visualizing the topical structure of the medical sciences: A self-organizing map approach. PloS One 8: e58779. doi:10.1371/journal.pone.0058779.

    Article  Google Scholar 

  • Smith, Richard. 1988. Peering into the bowels of the MRC. I: Setting priorities. British Medical Journal (Clinical research ed.) 296: 484–488.

    Article  Google Scholar 

  • Smith, Keith. 2000. Innovation as a Systemic Phenomenon: Rethinking the Role of Policy. Enterprise and Innovation Management Studies 1: 73–102. doi:10.1080/146324400363536.

    Article  Google Scholar 

  • Snellen, Ignatius Th.M. 1983. Social Merit as a Criterion of Scientific Choice: Its Application in Dutch Science Policy. Minerva 21: 16–36.

    Article  Google Scholar 

  • Souder, William E., and Tomislav Mandakovic. 1986. R&D Project Selection Models. Research Management 29: 36–42.

    Google Scholar 

  • Spaapen, Jack, and Leonie van Drooge. 2011. Introducing “productive interactions” in social impact assessment. Research Evaluation 20: 211–218. doi:10.3152/095820211X12941371876742.

    Article  Google Scholar 

  • Sponberg, Adrienne F. 2005. Streamlining the federal water research portfolio. BioScience 55.

  • Srivastava, Christina Viola, Nathaniel Deshmukh Towery, and Brian Zuckerman. 2007. Challenges and opportunities for research portfolio analysis, management, and evaluation. Research Evaluation 16: 152–156. doi:10.3152/095820207X236385.

    Article  Google Scholar 

  • Stilgoe, Jack. 2014. Against excellence. The Guardian, December 19. http://www.theguardian.com/science/political-science/2014/dec/19/against-excellence.

  • Stirling, Andy. 2007. A general framework for analysing diversity in science, technology and society. Journal of the Royal Society Interface 4: 707–719. doi:10.1098/rsif.2007.0213.

    Article  Google Scholar 

  • Stirling, Andy, and Ian Scoones. 2009. From Risk Assessment to Knowledge Mapping: Science, Precaution, and Participation in Disease Ecology. Ecology and Society 14: 14.

    Google Scholar 

  • Stummer, Christian, and Kurt Heidenberger. 2003. Interactive R&D Portfolio Analysis With Project Interdependencies and Time Profiles of Multiple Objectives. IEEE Transactions on Engineering Management 50(2): 175–183.

    Article  Google Scholar 

  • Swedish Presidency of the European Union. 2009. The Lund Declaration. European Union.

  • Van Bekkum, Sjoerd, Enrico Pennings, and Han Smit. 2009. A real options perspective on R&D portfolio diversification. Research Policy 38: 1150–1158.

    Article  Google Scholar 

  • Vonortas, Nicholas S., and Chintal A. Desai. 2007. “Real options” framework to assess public research investments. Science and Public Policy 34: 699–708. doi:10.3152/030234207X259012.

    Article  Google Scholar 

  • Waltman, Ludo, and Nees Jan van Eck. 2012. A new methodology for constructing a publication-level classification system of science. Journal of the American Society for Information Science and Technology 63: 2378–2392.

    Article  Google Scholar 

  • Waltman, Ludo, Nees Jan Van Eck, and Ed C. M. Noyons. 2009. A unified approach to mapping and clustering of bibliometric networks: 1–11.

  • Weinberg, Alvin M. 1963. The Criteria for Scientific Choice. Minerva 1: 159–171.

    Article  Google Scholar 

  • Woolf, Steven H. 2008. The meaning of translational research and why it matters. JAMA 299: 211–213. doi:10.1001/jama.2007.26.

    Google Scholar 

  • Wulf, William A. 1998. Balancing the research portfolio. Science (New York, N.Y.) 281: 1803.

    Article  Google Scholar 

Download references

Acknowledgements

We thank Tommaso Ciarli, Jochen Gläser, Jordi Molas-Gallart, Richard Wooley, and two anonymous referees for their insightful comments and suggestions. We acknowledge support from the UK Economic and Social Research Council (Grant RES-360-25-0076, Mapping the Development of Emergent Technologies) and the FP7 EU Marie Curie Integration Grant to IR (MapRePort).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Matthew L. Wallace.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wallace, M.L., Rafols, I. Research Portfolio Analysis in Science Policy: Moving from Financial Returns to Societal Benefits. Minerva 53, 89–115 (2015). https://doi.org/10.1007/s11024-015-9271-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11024-015-9271-8

Keywords

Navigation