Abstract
Case studies are largely used for investigating software engineering practices. They are characterized by their flexible nature, multiple forms of data collection, and are mostly informed by qualitative data. Synthesis of case studies is necessary to build a body of knowledge from individual cases. There are many methods for such synthesis, but they are yet not well explored in software engineering. The objective of this research is to demonstrate the similarities and differences of the results and conclusions when applying three different methods of synthesis, and to discuss the challenges of synthesizing evidence from reported case studies in SE. We describe a worked example of three such methods where three independent teams synthesized two studies that investigated critical factors of trust in outsourced projects through thematic synthesis and cross-case analysis, and compared these to each other and also to an already published narrative synthesis. In addition, despite that the primary studies were well presented for synthesis, we identified challenges in the use of case studies synthesis methods related to the goals and research questions of the synthesis, the types and number of case studies, variations in context, limited access to raw data, and quality of the case studies. Thus, we recommend that the analysts should be aware of these challenges and try to account for them during the execution of the synthesis. We also recommend that analysts consider using more than one method of synthesis for sake of reliability of the results and conclusions.
Similar content being viewed by others
References
Ashrafian H, Darsi A, Athanasiou T (2011) Evidence synthesis: evolving methodologies to optimise patient care and enhance policy decisions in evidence synthesis in healthcare. Springer-Verlag, London
Babar MA, Verner JM, Nguyen PT (2007) Establishing and maintaining trust in software outsourcing relationships: an empirical investigation. J Syst Softw 80(9):1438–1449
Bethel EC, Bernard RM (2010) Developments and trends in synthesizing diverse forms of evidence: beyond comparisons between distance education and classroom instruction. Am J Dist Educ 31(3):231–256
Cruzes DS, Dybå T (2010) “Synthesizing evidence in software engineering research”, ESEM, 1–10
Cruzes DS, Dybå T (2011a) Research synthesis in software engineering: a tertiary study”. IST 53(5):440–455
Cruzes DS, Dybå T (2011b) “Recommended Steps for Thematic Synthesis in Software Engineering”, ESEM: 275–284
Cruzes DS, Dybå T, Runeson P, Höst M (2011) Case Studies Synthesis: Brief Experience and Challenges for the Future. ESEM: 343–346
Dybå T (2013) Contextualizing empirical evidence. IEEE Softw 30(1):81–83
Dybå T, Dingsøyr T (2008) Strength of Evidence in Systematic Reviews in Software Engineering, ESEM’, pp 178–187.
Dybå T, Sjøberg DIK, Cruzes DS (2012) What works for whom, where, when, and why?: on the role of context in empirical software engineering. ESEM: 19–28
Gerring J (2007) Case study research: principles and practices. Cambridge Univ. Press, New York
Khan S, VanWynsberghe R (2008) Cultivating the Under-Mined: Cross-Case Analysis as Knowledge Mobilization. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 9 (1), Art. 34, http://nbn-resolving.de/urn:nbn:de:0114-fqs0801348
Kitchenham BA, Dybå T, Jørgensen M (2004) Evidence-based Software Engineering, Proc. ICSE’04, Edinburgh, Scotland, 23–28 May, pp. 273–281
Kitchenham BA, Charters S (2007) Guidelines for performing Systematic Literature Reviews in Software Engineering, Version 2.3, Keele University, EBSE Technical Report, EBSE-2007-01
Kitchenham BA, Sjøberg DIK, Dybå T, Pfahl D, Brereton P, Budgen D, Höst M, Runeson P (2012) Three empirical studies on the agreement of reviewers about the quality of software engineering experiments. Inf Softw Technol 54(8):804–819
Larsson R (1993) Case survey methodology: quantitative analysis of patterns across case studies. Acad Manag J 36(6):1515–1546
Lucas WA (1974): The case survey method: Aggregating case experience. Santa Monica
Mays N, Pope C, Popay J (2005) Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 10:6–20
Miles MB, Huberman AM (1984) Qualitative Data Analysis: A Sourcebook of New Methods, Sage
Miles MB, Huberman AM (1994) Qualitative Data Analysis: An Expanded Source Book, Sage
Newig J, Fritsch O (2009) The case survey method and applications in political science. APSA 2009 Paper. Available at SSRN: ssrn.com/abstract = 1451643 (Toronto)
Nguyen PT, Babar MA, Verner JM (2006) Critical factors in establishing and maintaining trust in software outsourcing relationships. EASE, http://ewic.bcs.org/content/ConWebDoc/3709
Oza N, Hall T, Rainer A, Grey S (2005) Trust in software outsourcing relationships: an empirical investigation of Indian software companies. In: 9th International Conference on Empirical Assessment in Software Engineering (EASE 2005). Keele, UK
Oza N, Hall T, Rainer A, Grey S (2006) Trust in software outsourcing relationships: an empirical investigation of Indian software companies. IST 48:345–354
Popay J, Roberts H, Sowden A, Petticrew M, Britten N, Arai L, Roen K, Rodgers M (2010) Developing methods for the narrative synthesis of quantitative and qualitative data in systematic reviews of effects, Centre for Review and Dissemination, ESRC, http://www.york.ac.uk/inst/crd/projects/narrative_synthesis.htm. Assessed in Dec. 2012
Pope C, Mays N, Popay J (2007) Synthesizing Qualitative and Quantitative Health Evidence: A Guide to Methods, Open University Press
Ragin CC (1987) The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies, Univ. of California Press
Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering”. Empir Softw Eng 14(2):131–164
Runeson P, Höst M, Rainer A, Regnell B (2012) Case Study Research in Software Engineering – Guidelines and Examples. Wiley, first edition, ISBN-10: 1118104358, ISBN-13: 978–1118104354
Šmite D, Wohlin C, Gorschek T, Feldt R (2010) Empirical evidence in global software engineering: a systematic review”. Empir Softw Eng 15(1):91–118
Thomas J, Harden A (2008) Methods for the thematic synthesis of qualitative research in systematic reviews”. BMC Med Res Methodol 8:45
Yin RK (2014) Case study research. Design and methods, 5th edn. Sage, London
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Maurizio Morisio
Rights and permissions
About this article
Cite this article
Cruzes, D.S., Dybå, T., Runeson, P. et al. Case studies synthesis: a thematic, cross-case, and narrative synthesis worked example. Empir Software Eng 20, 1634–1665 (2015). https://doi.org/10.1007/s10664-014-9326-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-014-9326-8