In recent year, a growing attention is dedicated to the assessment of research’s social impact. While prior research has often dealt with results of research, the last decade has begun to generate knowledge on the assessment of health research’s social impact. However, this knowledge is scattered across different disciplines, research communities, and journals. Therefore, this paper analyzes the heterogeneous picture research has drawn within the past years with a focus on the health research’s social impact on different stakeholders through an interdisciplinary, systematic review. By consulting major research databases, we have analyzed 53 key journal articles bibliographically and thematically. We argued that the adoption of a multi-stakeholder could be an evolution of the existing methods used to assess impact of research. After presenting a model to assess the health research’s social impact with a multi stakeholder perspective, we suggest the implementation in the research process of three practice: a multi-stakeholder workshop on research agenda; a multi stakeholder supervisory board; a multi-stakeholder review process.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Adam, P., Solans-Domènech, M., Pons, J. M. V., Aymerich, M., Berra, S., Guillamon, I., et al. (2012). Assessment of the impact of a clinical and health services research call in Catalonia. Research Evaluation, 21(4), 319–328.
Ahmed, S., Berzon, R. A., Revicki, D. A., Lenderking, W. R., Moinpour, C. M., Basch, E., et al. (2012). The use of patient-reported outcomes (PRO) within comparative effectiveness research implications for clinical practice and health care policy. Medical Care, 50(12), 60–70.
Amara, N., Ouimet, M., & Landry, R. (2004). New evidence on the instrumental, conceptual, and symbolic utilization of university research in government agencies. Science Communication, 26(1), 75–106.
Anderson, A. R. (1998). Cultivating the Garden of Eden: environmental entrepreneuring. Journal of Organizational Change Management, 11(2), 135–144.
Anderson, M., & McCleary, K. K. (2016). On the path to a science of patient input. Science Translational Medicine, 8(336), 336.
Banzi, R., Moja, L., Pistotti, V., Facchini, A., & Liberati, A. (2011). Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews. Health Research Policy and Systems, 9, 26.
Barker, K. (2007). The UK Research Assessment Exercise: the evolution of a national research evaluation system. Research Evaluation, 16(1), 3–12.
Barré, R. (2005). S&T indicators for policy making in a changing science–society relationship. In H. Moed, W. Glänzel & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 115–131). Dordrecht: Springer.
Bell, S., Shaw, B., & Boaz, A. (2011). Real-world approaches to assessing the impact of environmental research on policy. Research evaluation, 20(3), 227–237.
Bensing, J. M., Caris-Verhallen, W. M., Dekker, J., Delnoij, D. M., & Groenewegen, P. P. (2003). Doing the right thing and doing it right: toward a framework for assessing the policy relevance of health services research. International Journal of Technology Assessment in Health Care, 19(04), 604–612.
Bloch, C., Sørensen, M. P., Graversen, E. K., Schneider, J. W., Schmidt, E. K., Aagaard, K., et al. (2014). Developing a methodology to assess the impact of research grant funding: A mixed methods approach. Evaluation and program planning, 43, 105–117.
Boaz, A., Fitzpatrick, S., & Shaw, B. (2009). Assessing the impact of research on policy: a literature review. Science & Public Policy (SPP), 36(4), 255–270.
Bollen, J., Van de Sompel, H., & Rodriguez, M. A. (2008, June). Towards usage-based impact metrics: first results from the mesur project. In Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries (pp. 231–240). ACM.
Bornmann, L. (2013a). Measuring the societal impact of research: research is less and less assessed on scientific impact alone—We should aim to quantify the increasingly important contributions of science to society. EMBO Reports, 13(8), 673–676.
Bornmann, L. (2013b). What is societal impact of research and how can it be assessed? a literature survey. Journal of the American Society for Information Science and Technology, 64(2), 217–233.
Bornmann, L., & Marx, W. (2014). How should the societal impact of research be generated and measured? A proposal for a simple and practicable approach to allow interdisciplinary comparisons. Scientometrics, 98(1), 211–219.
Boyd, A., Cole, D. C., Cho, D. B., Aslanyan, G., & Bates, I. (2013). Frameworks for evaluating health research capacity strengthening: a qualitative study. Health Research Policy and Systems, 11(1), 46.
Bozeman, B., & Sarewitz, D. (2011). Public value mapping and science policy evaluation. Minerva, 49(1), 1–23.
Brewer, J. D. (2011). The impact of impact. Research Evaluation, 20(3), 255–256.
Bridges, J. F., & Buttorff, C. (2010). What outcomes should US policy makers compare in comparative effectiveness research? Expert Review of Pharmacoeconomics & Outcomes Research, 10(3), 217–220.
Brody, H., Croisant, S. A., Crowder, J. W., & Banda, J. P. (2015). Ethical issues in patient-centered outcomes research and comparative effectiveness research: A Pilot study of community dialogue. Journal of Empirical Research on Human Research Ethics, 10(1), 22–30.
Burdge, R. J., & Vanclay, F. (1995). Social impact assessment. In F. Vanclay & D. A. Bronstein (Eds.), Environmental and social impact assessment (pp. 31–65). Chichester, UK: Wiley.
Buxton, M., Hanney, S., Packwood, T., Roberts, S., & Youll, P. (2000). Getting reearch into practice: Assessing benefits from department of health and national health service research & development. Public Money and Management, 20(4), 29–34.
Castelnuovo, G., Limonta, D., Sarmiento, L., & Molinari, E. (2010). A more comprehensive index in the evaluation of scientific research: the single researcher impact factor proposal. Clinical practice and epidemiology in mental health: CP & EMH, 6, 109.
Cohen, G., Schroeder, J., Newson, R., King, L., Rychetnik, L., Milat, A. J., et al. (2015). Does health intervention research have real world policy and practice impact s: testing a new impact assessment tool. Health research policy and systems, 13(1), 3.
Colugnati, F. A., Firpo, S., de Castro, P. F. D., Sepulveda, J. E., & Salles-Filho, S. L. (2014). A propensity score approach in the impact evaluation on scientific production in Brazilian biodiversity research: The BIOTA Program. Scientometrics, 101(1), 85–107.
Cooksey, D. A. (2006). Review of UK health research funding. Norwich: HM Treasury.
Council of Canadian Academies (2012). Expert Panel on Science Performance and Research Funding.
Cousins, J. B., Svensson, K., Szijarto, B., Pinsent, C., Andrew, C., & Sylvestre, J. (2015). Assessing the practice impact of research on evaluation. New Directions for Evaluation, 2015(148), 73–88.
Cyril, F. M. D., & Phil, M. (2009). Health research: Measuring the social, health and economic benefits. Canadian Medical Association Journal, 180(5), 528–534.
Czarnitzki, D., & Lopes-Bento, C. (2013). Value for money? New Microeconometric Evidence on Public R&D Grants in Flanders. Research Policy, 42, 76–89.
Dannenberg, A. L., Bhatia, R., Cole, B. L., Dora, C., Fielding, J. E., Kraft, K., et al. (2006). Growing the field of health impact assessment in the United States: an agenda for research and practice. American Journal of Public Health, 96(2), 262–270.
Darmoni, S. J., Roussel, F., Benichou, J., Thirion, B., & Pinhas, N. (2002). Reading factor: A new bibliometric criterion for managing digital libraries. Journal-Medical Library Association, 90, 323–326.
Davies, P. (2004). Is evidence-based government possible? Jerry Lee lecture to Campbell Collaboration Colloquium, Washington DC 19 February.
Davies, P., Walker, A. E., & Grimshaw, J. M. (2010). A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implementation Science, 5(1), 5–14.
De Jong, S. P., Van Arensbergen, P., Daemen, F., Van Der Meulen, B., & Van Den Besselaar, P. (2011). Evaluation of research in context: An approach and two cases. Research Evaluation, 20(1), 61–72.
Denholm, E. M., & Martin, W. J. (2008). Translational research in environmental health sciences. Translational research: The journal of laboratory and clinical medicine, 151(2), 57.
Department of Education, Science and Training. (2005). Research quality framework: Assessing the quality and impact of research in Australia (Issue paper). Canberra: Commonwealth of Australia.
Donovan, C. (2007). The qualitative future of research evaluation. Science and Public Policy, 34(8), 585–597.
Donovan, C. (2008). The Australian Research Quality Framework: A live experiment in capturing the social, economic, environmental, and cultural returns of publicly funded research. New Directions for Evaluation, 2008(118), 47–60.
Donovan, C. (2011). State of the art in assessing research impact: introduction to a special issue. Research Evaluation, 20(3), 175–179.
Drew, C. H., Pettibone, K. G., Finch Iii, F. O., Giles, D., & Jordan, P. (2016). Automated Research Impact Assessment: A new bibliometrics approach. Scientometrics, 106(3), 987–1005.
Ekboir, J. (2003). Why impact analysis should not be used for research evaluation and what the alternatives are. Agricultural Systems, 78(2), 166–184.
Eric. (2010). Evaluating the societal relevance of academic research: A guide. The Hague: Rathenau Institute.
Ernø-Kjølhede, E., & Hansson, F. (2011). Measuring research performance during a changing relationship between science and society. Research Evaluation, 20(2), 130–142.
European Commission. (2010). Assessing Europe’s university-based research. Expert group on assessment of university-based research. Brussels, Belgium: Publications Office of the European Union.
European Commission. (2011). Assessing Europe’s university-based research. Expert group on assessment of university-based research. Brussels: Publications Office of the European Union.
Evans, A., Strezov, V., & Evans, T. J. (2009). Assessment of sustainability indicators for renewable energy technologies. Renewable and Sustainable Energy Reviews, 13(5), 1082–1088.
Figueredo, A. J., & Sechrest, L. (2001). Approaches used in conducting health outcomes and effectiveness research. Evaluation and Program Planning, 24(1), 41–59.
Fink, A. (1998). Conducting research literature review: from paper to internet. Thousand Oaks: SagePublications.
Fox, R. J., Thompson, A., Baker, D., Baneke, P., Brown, D., Browne, P., et al. (2012). Setting a research agenda for progressive multiple sclerosis: The International Collaborative on Progressive MS. Multiple Sclerosis Journal, 18(11), 1534–1540.
Franceschini, F., Maisano, D., & Mastrogiacomo, L. (2015). Research quality evaluation: Comparing citation counts considering bibliometric database errors. Quality & Quantity, 49(1), 155–165.
Freeman, R. E. (1984). Strategic management, a stakeholder approach. Boston: Pitman.
Frey, B. S., & Rost, K. (2010). Do rankings reflect research quality? Journal of Applied Economics, 13(1), 1–38.
Furman, E., Kivimaa, P., Kuuppo, P., Nykänen, M., Väänänen, P., Mela, H., & Korpinen, P. (2006). Experiences in the management of research funding programmes for environmental protection. Including recommendations for best practice. Finnish Environment Institute.
Fusco, D., Barone, A. P., Sorge, C., D’Ovidio, M., Stafoggia, M., Lallo, A., et al. (2012). P. Re. Val. E.: Outcome research program for the evaluation of health care quality in Lazio, Italy. BMC Health Services Research, 12(1), 25.
Gershon, R., Rothrock, N. E., Hanrahan, R. T., Jansky, L. J., Harniss, M., & Riley, W. (2010). The development of a clinical outcomes survey research application: Assessment CenterSM. Quality of Life Research, 19(5), 677–685.
Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., & Trow, M. (1994). The new production of knowledge: The dynamics of science and research in contemporary societies. Thousand Oaks: Sage.
Gibson, T. B., Ehrlich, E. D., Graff, J., Dubois, R., Farr, A. M., Chernew, M., et al. (2014). Real-world impact of comparative effectiveness research findings on clinical practice. The American journal of managed care, 20(6), e208–e220.
Godin, B., & Dore, C. (2005). Measuring the impact s of science; beyond the economic dimension, INRS Urbanisation, Culture et Société. HIST Lecture, Helsinki Institute for Science and Technology Studies, Helsinki, Finland. Available at: http://www.csiic.ca/PDF/Godin_Dore_Impacts.pdf.
Göransson, B., Maharajh, R., & Schmoch, U. (2009). New activities of universities in transfer and extension: Multiple requirements and manifold solutions. Science and Public Policy, 36(2), 157–164.
Grimshaw, J. M., Eccles, M. P., Lavis, J. N., Hill, S. J., & Squires, J. E. (2012). Knowledge translation of research findings. Implementation science, 7(1), 50.
Guinea, J., Sela, E., Gómez-Núñez, A. J., Mangwende, T., Ambali, A., Ngum, N., et al. (2015). Impact oriented monitoring: A new methodology for monitoring and evaluation of international public health research projects. Research Evaluation, 24(2), 131–145.
Guthrie, S., Wamae, W., Diepeveen, S., Wooding, S., & Grant, J. (2013). Measuring research: A guide to research evaluation frameworks and tools. Santa Monica: RAND.
Haigh, F., Harris, P., & Haigh, N. (2012). Health impact assessment research and practice: A place for paradigm positioning? Environmental Impact Assessment Review, 33(1), 66–72.
Hall, J., & Wagner, M. (2012). Editorial: The challenges and opportunities of sustainable development for entrepreneurship and small business. Journal of Small Business & Entrepreneurship, 25(4), 409–416.
Hanney, S. R., Gonzalez-Block, M. A., Buxton, M. J., & Kogan, M. (2003). The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health research policy and systems, 1(1), 2.
Hanney, S., Packwood, T., & Buxton, M. (2000). Evaluating the benefits from health research and development centres: a categorization, a model and examples of application. Evaluation, 6(2), 137–160.
Harzing, A. W. (2010). The publish or perish book. Melbourne: Tarma Software Research Pty Ltd.
Healthcare Industries Task Force, (2004). Better health through partnership: a programme for action, Final report. London, England: Author.
Helming, K., Diehl, K., Kuhlman, T., Jansson, T., Verburg, P., Bakker, M., Morris, J. (2011). Ex ante impact assessment of policies affecting land use, part B: application of the analytical framework. Ecology and Society, 16(1), 1–29.
Henshall, C. (2011). The impact of payback research: Developing and using evidence in policy. Research Evaluation, 20(3), 257–258.
Hessels, L. K., & Van Lente, H. (2010). The mixed blessing of Mode 2 knowledge production. Science Technology and Innovation Studies, 6(1), 65–69.
Holbrook, J. B. (2012). Re-assessing the science-society relation: The case of the US National Science Foundation’s broader impact s merit review criterion (1997–2011). Techonology in Society, 27(4), 437–451.
Holbrook, J. B., & Frodeman, R. (2010, April). Comparative Assessment of Peer Review (CAPR). In EU/US workshop on peer review: Assessing ‘‘broader impact’’in research grant applications. Brussels: European Commission, Directorate-General for Research and Innovation.
Holbrook, J. B., & Frodeman, R. (2011). Peer review and the ex ante assessment of societal impacts. Research Evaluation, 20(3), 239–246.
Horton, K., Tschudin, V., & Forget, A. (2007). The value of nursing: A literature review. Nursing Ethics, 14(6), 716–740.
Inglesi-Lotz, R., & Pouris, A. (2011). Scientometric impact assessment of a research policy instrument: the case of rating researchers on scientific outputs in South Africa. Scientometrics, 88(3), 747–760.
Ippoliti, R., & Falavigna, G. (2014). Public health institutions, clinical research and protection system of patients’ rights: An impact evaluation of public policy. Public Organization Review, 14(2), 109–125.
Jammer, I., Wickboldt, N., Sander, M., Smith, A., Schultz, M. J., Pelosi, P., et al. (2015). Standards for definitions and use of outcome measures for clinical effectiveness research in perioperative medicine: European Perioperative Clinical Outcome (EPCO) definitions: a statement from the ESA-ESICM joint taskforce on perioperative outcome measures. European Journal of Anaesthesiology (EJA), 32(2), 88–105.
Jette, A. M., & Keysor, J. J. (2002). 3. Uses of evidence in disability outcomes and effectiveness research. Milbank Quarterly, 80(2), 325–345.
Kovacs, S. M., Turner-Bowker, D. M., Calarco, G., Mulberg, A. E., & Paty, J. (2016). Practical considerations for the use of clinical outcome assessments (COAs) in pediatric clinical research: examples from pediatric gastroenterology. Therapeutic Innovation & Regulatory Science, 50(1), 37–43.
Kryl, D., Allen, L., Dolby, K., Sherbon, B., & Viney, I. (2012). Tracking the impact of research on policy and practice: Investigating the feasibility of using citations in clinical guidelines for research evaluation. British Medical Journal Open, 2(2), e000897.
Lähteenmäki-Smith, K., Hyytinen, K., Kutinlahti, P., & Konttinen, J. (2006). Research with an impact. Evaluation practises in public research organisations. VTT Research Notes, 2336
LaKind, J. S., Goodman, M., Barr, D. B., Weisel, C. P., & Schoeters, G. (2015). Lessons learned from the application of BEES-C: Systematic assessment of study quality of epidemiologic research on BPA, neurodevelopment, and respiratory health. Environment International, 80, 41–71.
Lamm, G. M. (2006). Innovation works. A case study of an integrated pan-European technology transfer model. BIF Futura, 21(2), 86–90.
Lane, P. J., Koka, B. R., & Pathak, S. (2006). The reification of absorptive capacity: A critical review and rejuvenation of the construct. Academy of Management Review, 31(4), 833–863.
Leduc, P. (1994). Evaluation in the social sciences: The strategic context. Research Evaluation, 4(1), 2–5.
Lee, F. S. (2007). The Research Assessment Exercise, the state and the dominance of mainstream economics in British universities. Cambridge Journal of Economics, 31(2), 309–325.
Liebow, E., Phelps, J., Van Houten, B., Rose, S., Orians, C., Cohen, J., et al. (2009). Toward the assessment of scientific and public health impact s of the National Institute of Environmental Health Sciences Extramural Asthma Research Program using available data. Environmental Health Perspectives, 117(7), 1147.
Luukkonen, T. (1998). The difficulties in assessing the impact of EU framework programmes. Research Policy, 27(6), 599–610.
Manion, F. J., Harris, M. R., Buyuktur, A. G., Clark, P. M., An, L. C., & Hanauer, D. A. (2012). Leveraging EHR data for outcomes and comparative effectiveness research in oncology. Current oncology reports, 14(6), 494–501.
Maredia, M. K., & Byerlee, D. (2000). Efficiency of research investments in the presence of international spillovers: Wheat research in developing countries. Agricultural Economics, 22(1), 1–16.
Martin, B. R. (2007). Assessing the impact of basic research on society and the economy. In Paper presented at the rethinking the impact of basic research on society and the economy (WF-EST international conference, 11 May 2007), Vienna, Austria.
Martin, B. R. (2011). The research excellence framework and the’impact agenda’: Are we creating a Frankenstein monster? Research Evaluation, 20(3), 247–254.
Martin and Irvine. (1983). Assessing basic research: The case of the Isaac Newton telescope. Social Studies of Science, 13, 49–86.
Mayring, P. (2003). Qualitative inhaltsanalyse [Qualitative content analysis]. Qualitative Forschung, 3, 468–475.
Milat, A. J., Bauman, A. E., & Redman, S. (2015). A narrative review of research impact assessment models and methods. Health Research Policy and Systems, 13(1), 18.
Milat, A. J., Laws, R., King, L., Newson, R., Rychetnik, L., Rissel, C., et al. (2013). Policy and practice impact s of applied research: a case study analysis of the New South Wales Health Promotion Demonstration Research Grants Scheme 2000–2006. Health Research Policy and Systems, 11(1), 5.
Moed, H. F. (2007). The effect of “open access” on citation impact: An analysis of ArXiv’s condensed matter section. Journal of the American Society for Information Science and Technology, 58(13), 2047–2054.
Moed, H. F., & Halevi, G. (2015). Multidimensional assessment of scholarly research impact. Journal of the Association for Information Science and Technology, 66(10), 1988–2002.
Molas-Gallart, J., & Tang, P. (2011). Tracing “productive interactions” to identify social impacts: An example for the social sciences. Research Evaluation, 20(3), 219–226.
Molas-Gallart, J., Salter, A., Patel, P., Scott, A., & Duran, X. (2002). Measuring third stream activities. Final report to the Russell Group of Universities. Brighton: SPRU, University of Sussex
Moore, S. B., & Manring, S. L. (2009). Strategy development in small and medium sized enterprises for sustainability and increased value creation. Journal of Cleaner Production, 17(2), 276–282.
Morgan, M. M., & Grant, J. (2013). Making the grade: Methodologies for assessing and evidencing research impacts. Dean, A., Wykes, M. and Stevens, H.(eds), 7, 25–43
Morton, S. (2015). Progressing research impact assessment: A ‘contributions’ approach. Research Evaluation, rvv016.
Mostert, S. P., Ellenbroek, S. P., Meijer, I., Van Ark, G., & Klasen, E. C. (2010). Societal output and use of research performed by health research groups. Health Research Policy and Systems, 8(1), 30.
Mullins, C. D., Onukwugha, E., Cooke, J. L., Hussain, A., Baquet, C. R. (2010). The potential impact of comparative effectiveness research on the health of minority populations. Health Affairs, 29(11), 10–1377
Nallamothu, B. K., & Lüscher, T. F. (2012). Moving from impact to influence: Measurement and the changing role of medical journals. European Heart Journal, 33(23), 2892–2896.
New Philanthropy Capital. (2010). Social return on investment: Position paper. London: New Philanthropy Capital.
Newby, H. (1994). The challenge for social science: A new role in public policy-making. Research Evaluation, 4(1), 6–11.
Niederkrotenthaler, T., Dorner, T. E., & Maier, M. (2011). Development of a practical tool to measure the impact of publications on the society based on focus group discussions with scientists. BMC Public Health, 11(1), 588.
Nightingale, P., & Scott, A. (2007). Peer review and the relevance gap: ten suggestions for policy-makers. Science & Public Policy (SPP), 34(8), 543–553.
Nutley, S. M., Walter, I., & Davies, H. T. (2007). Using evidence: How research can inform public services. Bristol: Policy Press.
O’Connor, D. P., & Brinker, M. R. (2013). Challenges in outcome measurement: Clinical research perspective. Clinical Orthopaedics and Related Research®, 471(11), 3496–3503.
OECD. (2008). OECD science, technology and industry outlook. Paris: OECD.
OECD Report. (2016). OECD science, technology and industry outlook. Paris: OECD.
Penfield, T., Baker, M. J., Scoble, R., & Wykes, M. C. (2014). Assessment, evaluations, and definitions of research impact: A review. Research Evaluation, 23(1), 21–32.
Perrin, E. B. (2002). Some thoughts on outcomes research quality improvement, and performance measurement. Medical Care, 40(6), 89–91.
Pontille, D., & Torny, D. (2010). The controversial policies of journal ratings: Evaluating social sciences and humanities. Research evaluation, 19(5), 347–360.
Potì, B., & Cerulli, G. (2011). Evaluation of firm R&D and innovation support: New indicators and the ex-ante prediction of ex-post additionality-potential. Research Evaluation, 20(1), 19–29.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76.
Punt, A., Schiffelers, M. J. W., Horbach, G. J., van de Sandt, J. J., Groothuis, G. M., Rietjens, I. M., et al. (2011). Evaluation of research activities and research needs to increase the impact and applicability of alternative testing strategies in risk assessment practice. Regulatory Toxicology and Pharmacology, 61(1), 105–114.
Reeve, B. B., Burke, L. B., Chiang, Y. P., Clauser, S. B., Colpe, L. J., Elias, J. W., et al. (2007). Enhancing measurement in health outcomes research supported by Agencies within the US Department of Health and Human Services. Quality of Life Research, 16(1), 175–186.
Reichertz, J. (2010). Abduction: the logic of discovery of grounded theory. Forum Qualitative Social Research, 11, 1–12.
Roessner, D. (2000). Quantitative and qualitative methods and measures in the evaluation of research. Research Evaluation, 9(2), 125–132.
Rymer, L. (2011). Measuring the impact of research—The context for metric development. Turner, Australia: The Group of Eight.
Sarli, C. C., Dubinsky, E. K., & Holmes, K. L. (2010). Beyond citation analysis: A model for assessment of research impact. Journal of the American Medical Library Association, 98(1), 17.
Schaltegger, S. (2002). A framework for ecopreneurship e leading bioneers and environmental managers to ecopreneurship. Greener Management International Journal, 38, 45–58.
Schulz, K. F., Chalmers, I., Hayes, R. J., & Altman, D. G. (1995). Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA, 273(5), 408–412.
Seuring, S., & Müller, M. (2008). From a literature review to a conceptual framework for sustainable supply chain management. Journal of Cleaner Production, 16(15), 1699–1710.
Seuring, P. D. S., Müller, P. D. M., Westhaus, M., Morana, R. (2005). Conducting a literature review—the example of sustainability in supply chains. In H. Kotzab, S. Seuring, M. Muller & G. Reiner (Eds.), Research methodologies in supply chain management (pp. 91–106). Physica-Verlag HD.
Social Sciences and Humanities Scientific Committees. (2013). Humanities and social sciences in horizon 2020 societal challenges: Implementation and monitoring.
Sombatsompop, N., Markpin, T., Yochai, W., & Saechiew, M. (2005). An evaluation of research performance for different subject categories using Impact Factor Point Average (IFPA) index: Thailand case study. Scientometrics, 65(3), 293–305.
Spaapen, J., & van Drooge, L. (2011). Introducing “productive interactions” in social impact assessment. Research Evaluation, 30(3), 211–218.
Spaapen, J., Dijstelbloem, H., & Wamelink, F. (2007). Evaluating research in context. A method for comprehensive assessment (2nd ed.). The Hague: COS.
Stein, T. V., Anderson, D. H., & Kelly, T. (1999). Using stakeholders’ values to apply ecosystem management in an upper Midwest landscape. Environmental Management, 24(3), 399–413.
Stryer, D., Tunis, S., Hubbard, H., & Clancy, C. (2000). The outcomes of outcomes and effectiveness research: Impact s and lessons from the first decade. Health Services Research, 35(5 Pt 1), 977.
Taylor, J., & Bradbury-Jones, C. (2011). International principles of social impact assessment: Lessons for research? Journal of Research in Nursing, 16(2), 133–145.
Technopolis. (2009). Impact Europese Kaderprogramma’s in Nederland. Woluwe-Saint-Pierre: Technopolis Group.
Thorpe, R., Holt, R., Macpherson, A., & Pittaway, L. (2005). Using knowledge within small and medium-sized firms: A systematic review of the evidence. International Journal of Management Reviews, 7(4), 257–281.
Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222.
Tremblay, G., Zohar, S., Bravo, J., Potsepp, P., & Barker, M. (2010). The Canada Foundation for Innovation’s outcome measurement study: A pioneering approach to research evaluation. Research Evaluation, 19(5), 333–345.
United States Government Accountability Office. (2012). Designing evaluations. Washington, DC: Author.
Van den Besselaar, P., & Leydesdorff, L. (2009). Past performance, peer review and project selection: A case study in the social and behavioral sciences. Research Evaluation, 18(4), 273–288.
Van der Meulen, B., & Rip, A. (2000). Evaluation of societal quality of public sector research in the Netherlands. Research Evaluation, 9(1), 11–25.
Van Raan, A. F. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.
Van Vught, F., & Ziegele, F. (2011). Design and testing the feasibility of a multidimensional global university ranking. Final Report. European Community, Europe: Consortium for Higher Education and Research Performance Assessment, CHERPA Network.
Vanclay, F. (2003). International principles for social impact assessment. Impact assessment and project appraisal, 21(1), 5–12.
Westrich, K. D., Wilhelm, J. A., & Schur, C. L. (2016). Comparative effectiveness research in the U.S.A.: when will there be an impact on healthcare decision-making? Journal of Comparative Effectiveness Research, 5(2), 207–216.
Willis, T. A., Hartley, S., Glidewell, L., Farrin, A. J., Lawton, R., McEachan, R. R., et al. (2016). Action to Support Practices Implement Research Evidence (ASPIRE): Protocol for a cluster-randomised evaluation of adaptable implementation packages targeting ‘high impact’ clinical practice recommendations in general practice. Implementation Science, 11(1), 25.
Wooding, S., Hanney, S., Pollitt, A., Buxton, M., & Grantm, J. (2011). Project Retrosight. Understanding the Returns from Cardiovascular and Stroke Research: Policy Report. Cambridge: RAND Europe.
Wu, Z. (2015). Average evaluation intensity: A quality-oriented indicator for the evaluation of research performance. Library & Information Science Research, 37(1), 51–60.
Pastakia, C. M. R. (1998). The rapid impact assessment matrix (RIAM)—A new tool for environmental impact assessment. In K. Jensen (Ed.), Environmental impact assessment using the rapid impact assessment matrix (RIAM). Fredensborg, Denmark: Olsen & Olsen.
Yiend, J., Chambers, J. C., Burns, T., Doll, H., Fazel, S., Kaur, A., et al. (2011). Outcome measurement in forensic mental health research: An evaluation. Psychology, Crime & Law, 17(3), 277–292.
Zaratin, P., Battaglia, M. A., & Abbracchio, M. P. (2014). Nonprofit foundations spur translational research. Trends in Pharmacological Sciences, 35(11), 552–555.
Zaratin, P., Comi, G., Coetzee, T., Ramsey, K., Smith, K., Thompson, A., et al. (2016). Progressive MS Alliance Industry Forum: maximizing collective impact to enable drug development. Trends in Pharmacological Sciences, 37(10), 808–810.
Zelefsky, M. J., Lee, W. R., Zietman, A., Khalid, N., Crozier, C., Owen, J., et al. (2013). Evaluation of adherence to quality measures for prostate cancer radiotherapy in the United States: Results from the quality research in radiation oncology (QRRO) survey. Practical Radiation Oncology, 3(1), 2–8.
About this article
Cite this article
Pedrini, M., Langella, V., Battaglia, M.A. et al. Assessing the health research’s social impact: a systematic review. Scientometrics 114, 1227–1250 (2018). https://doi.org/10.1007/s11192-017-2585-6