Abstract
Communities of Practice (CoPs) are increasingly considered a part of ecohealth and other sectors such as health care, education, and business. However, there is little agreement on approaches to evaluate the influence and effectiveness of CoPs. The purpose of this review was to understand what frameworks and methods have been proposed or used to evaluate CoPs and/or knowledge networks. The review searched electronic databases in interdisciplinary, health, education, and business fields, and further collected references and forward citations from relevant articles. Nineteen articles with 16 frameworks were included in the synthesis. The purposes of the evaluation frameworks varied; while some focused on assessing the performance of CoPs, several frameworks sought to learn about CoPs and their critical success factors. Nine of the frameworks had been applied or tested in some way, most frequently to guide a case study. With limited applications of the frameworks, strong claims about generalizability could not be made. The review results can inform the development of tailored frameworks. However, there is a need for more detailed and targeted CoP evaluation frameworks, as many imperative CoP evaluation needs would be unmet by the available frameworks.
Similar content being viewed by others
References
Arksey H, O’Malley L (2005) Scoping studies: towards a methodological framework. International Journal of Social Research Methodology 8:19–32
Barragán-Ocaña A, Quijano-Solís Á, Vega-Díaz G, Sánchez-Lara B (2012) Communities of practice: A focus from complex systems. International Journal of General Systems 41:741-755
Barwick MA, Peters J, Boydell K (2009) Getting to uptake: Do communities of practice support the implementation of evidence-based practice? Journal of the Canadian Academy of Child and Adolescent Psychiatry 18:16
Bertone MP, Meessen B, Clarysse G, Hercot D, Kelley A, Kafando Y, Lange I, Pfaffmann J, Ridde V, Sieleunou I (2013) Assessing communities of practice in health policy: A conceptual framework as a first step towards empirical research. Health Research Policy and Systems 11:39
Botkin JW (1999). Smart business: How knowledge communities can revolutionize your company. New York: The Free Press
Charron DF (2012) Ecohealth Research in Practice: Innovative Applications of an Ecosystem Approach to Health, Insight and Innovation in Development, New York: Springer and Ottawa, Canada: International Development Research Centre
Chu M-T, Khosla R (2009) Index evaluations and business strategies on communities of practice. Expert Systems with Applications 36:1549-1558
Chu M-T, Khosla R, Nishida T (2012) Communities of practice model driven knowledge management in multinational knowledge based enterprises. Journal of Intelligent Manufacturing 23:1707-1720
Conklin J, Stolee P (2008) A model for evaluating knowledge exchange in a network context. CJNR (Canadian Journal of Nursing Research) 40:116-124
Cox A (2005) What are communities of practice? A comparative review of four seminal works. Journal of Information Science 31:527-540
Creech H, Ramji A (2004) Knowledge networks: Guidelines for assessment. International Institute for Sustainable Development. http://www.iisd.org/pdf/2004/networks_guidelines_for_assessment.pdf. Accessed 7 Oct, 2013.
Creech H, Laurie M, Paas L, Parry J-E (2012) Performance improvement and assessment of collaboration: Starting points for networks and communities of practice. International Institute for Sustainable Development. http://www.iisd.org/pdf/2012/performance_improvement_networks.pdf. Accessed 7 Oct 2013.
Davis K, Drey N, and Gould D (2009) What are scoping studies? A review of the nursing literature. International journal of nursing studies 46:1386-1400.
De Plaen R, Kilelu C (2004) From multiple voices to a common language: Ecosystem approaches to human health as an emerging paradigm. EcoHealth 1:SU8-SU15
Dubé L, Bourhis A, Jacob R (2006) Towards a typology of virtual communities of practice. Interdisciplinary Journal of Information, Knowledge, and Management 1:69–93
Fenton E, Harvey J, Sturt J (2007) Evaluating primary care research networks. Health Services Management Research 20:162-173
Fischer G (2001) Communities of interest: Learning through the interaction of multiple knowledge systems. In Proceedings of the 24th IRIS Conference.
Flynn-Dapaah K (2003) Ecosystem approaches to human health global community of practice: Report on the design phase consultations. International Development Research Centre, Ottawa
Grootveld A, Helms R (2008) Development and application of a factor framework to diagnose possible failure of communities of practice. In The 9th European Conference on Knowledge Management: Eckm 2008.
Haas PM (1992) Introduction: epistemic communities and international policy coordination. International organization 46(1):1-35.
Kim S, Hong J, Suh E (2012) A diagnosis framework for identifying the current knowledge sharing activity status in a community of practice. Expert Systems with Applications 39:130893
Kitson A, Harvey G, McCormack B (1998) Enabling the implementation of evidence based practice: a conceptual framework. Quality in Health care 7:149-158
Lave J, Wenger E (1991) Situated learning: Legitimate peripheral participation. Cambridge university press
Lee J, Suh E-h, Hong J (2010) A maturity model based cop evaluation framework: A case study of strategic cops in a Korean company. Expert Systems with Applications 37:2670-2681
Lesser EL, Storck J (2001) Communities of practice and organizational performance. IBM systems journal 40:831-841
Levac D, Colquhoun H, and O’Brien KK (2010) Scoping studies: advancing the methodology. Implementation Science 5:1-9
Li LC, Grimshaw JM, Nielsen C, Judd M, Coyte PC, Graham ID (2009a) Evolution of Wenger’s concept of community of practice. Implementation Science 4:11
Li LC, Grimshaw JM, Nielsen C, Judd M, Coyte PC, Graham ID (2009b) Use of communities of practice in business and health care sectors: A systematic review. Implementation Science 4:16
Loyarte E, Rivera O (2007) Communities of practice: A model for their cultivation. Journal of Knowledge Management 11:67-77
McDermott R (1999) Nurturing three-dimensional communities of practice. Knowledge Management Review: 11:26-29
McDermott R (2002) Measuring the impact of communities what happens when you need to show proof that the communities are working and are worth the investment? This article explains how to make sense of a chain of community activities, outcomes and value in the language of the business. Knowledge Management Review 5:26-29
Meessen B, Bertone MP (2012) Assessing performance of communities of practice in health policy.
Parkes MW (2011) Diversity, emergence, resilience: Guides for a new generation of ecohealth research and practice. EcoHealth 8:137-139
Parkes MW, Charron DF, Sanchez A (2012) Better together: field-building networks at the frontiers of ecohealth research. In: Ecohealth Research in Practice: Innovative Applications of an Ecosystem Approach to Health, Charron DF (editor), Insight and Innovation in Development, New York, NY: Springer and Ottawa, Canada: International Development Research Centre, pp 231–253.
Patton MQ (2011) Developmental evaluation: Applying coplexity concepts to enhance innovation and use. New York: The Guilford Press.
Perrin B (2002) How to—and how not to—evaluate innovation. Evaluation 8:13-28
Ranmuthugala G, Cunningham FC, Plumb JJ, Long J, Georgiou A, Westbrook JI, Braithwaite J (2011a) A realist evaluation of the role of communities of practice in changing healthcare practice. Implementation Science 6:49
Ranmuthugala G, Plumb JJ, Cunningham FC, Georgiou A, Westbrook JI, Braithwaite J (2011b) How and why are communities of practice established in the healthcare sector? A systematic review of the literature. BMC health services research 11:273
Rogers PJ (2008) Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation 14:29-48
Scarso E, Bolisani E (2008) Communities of practice as structures for managing knowledge in networked corporations. Journal of Manufacturing Technology Management 19:374-390
Scarso E, Bolisani E, Salvador L (2009) A systematic framework for analysing the critical success factors of communities of practice. Journal of Knowledge Management 13:431-447
Serrat O (2010) Knowledge solutions: tools, methods and approaches to drive development forward and enhance its effects. Asian Development Bank.
Spiegel J, Breilh J, Beltran E, Parra J, Solis F, Yassi A, et al (2011) Establishing a community of practice of researchers, practitioners, policy-makers and communities to sustainably manage environmental health risks in Ecuador. BMC international health and human rights 11:S5
Stein J, Stren R, Fitzgibbon J, MacLean M (2001) Networks of knowledge: Collaborative innovation in international learning. University of Toronto Press, Toronto
Stolee P, Conklin J (n.d.) Assessment of impact of SHRTN activities. http://seniorshealthknowledgenetwork.ca/sites/seniorshealthknowledgenetwork.ca/files/Model_to_guide_the_evaluation_of_SHRTN_2_.pdf. Accessed 7 Oct 2013.
Van Eerd D, Cole D, Keown K, Irvin E, Kramer D, Brenneman Gibson J, Kohn M, Mahood Q, Slack T, Amick III B, Phipps D, Garcia J, Morassaei S (2011) Report on knowledge transfer and exchange practices: A systematic review of the quality and types of instruments used to assess kte implementation and impact. Institute for Work & Health, Toronto
Verburg RM, Andriessen J (2006) The assessment of communities of practice. Knowledge and Process Management 13:13-25
Webb JC, Mergler D, Parkes MW, Saint-Charles J, Spiegel J, Waltner-Toews D, Yassi A, Woollard RF (2010) Tools for thoughtful action: The role of ecosystem approaches to health in enhancing public health. Canadian Journal of Public Health 101:439-441
Wenger E (1998) Communities of practice: Learning, meaning, and identity. Boston: Cambridge university press
Wenger E (2004) Knowledge management as a doughnut: Shaping your knowledge strategy through communities of practice. Ivey Business Journal 68:1-8
Wenger E, McDermott R, Snyder WM (2002) Cultivating communities of practice: A guide to managing knowledge. Boston: Harvard Business School Press
Wenger E, Trayner B, de Laat M (2011) Promoting and assessing value creation in communities and networks: a conceptual framework. http://wenger-trayner.com/wp-content/uploads/2011/12/11-04-Wenger_Trayner_DeLaat_Value_creation.pdf. Accessed 7 Oct 2013.
Yap R, Robben J (2012) Measuring the value of online communities and networks of practice for business. In: Proceedings of the 8th International Conference on Networked Learning.
Zhang J, Luna-Reyes LF, Nakashima M, Gil-Garcia JR, Sayogo DS, Mellouli S (2011) Building and sustaining a transnational and interdisciplinary research group: lessons learned from a North American experience. In: System Sciences (HICSS), 2011 44th Hawaii International Conference on System Sciences.
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
McKellar, K.A., Pitzul, K.B., Yi, J.Y. et al. Evaluating Communities of Practice and Knowledge Networks: A Systematic Scoping Review of Evaluation Frameworks. EcoHealth 11, 383–399 (2014). https://doi.org/10.1007/s10393-014-0958-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10393-014-0958-3