Empirical Software Engineering

, Volume 20, Issue 6, pp 1634–1665 | Cite as

Case studies synthesis: a thematic, cross-case, and narrative synthesis worked example

  • Daniela S. CruzesEmail author
  • Tore Dybå
  • Per Runeson
  • Martin Höst


Case studies are largely used for investigating software engineering practices. They are characterized by their flexible nature, multiple forms of data collection, and are mostly informed by qualitative data. Synthesis of case studies is necessary to build a body of knowledge from individual cases. There are many methods for such synthesis, but they are yet not well explored in software engineering. The objective of this research is to demonstrate the similarities and differences of the results and conclusions when applying three different methods of synthesis, and to discuss the challenges of synthesizing evidence from reported case studies in SE. We describe a worked example of three such methods where three independent teams synthesized two studies that investigated critical factors of trust in outsourced projects through thematic synthesis and cross-case analysis, and compared these to each other and also to an already published narrative synthesis. In addition, despite that the primary studies were well presented for synthesis, we identified challenges in the use of case studies synthesis methods related to the goals and research questions of the synthesis, the types and number of case studies, variations in context, limited access to raw data, and quality of the case studies. Thus, we recommend that the analysts should be aware of these challenges and try to account for them during the execution of the synthesis. We also recommend that analysts consider using more than one method of synthesis for sake of reliability of the results and conclusions.


Evidence-based software engineering Systematic reviews Research synthesis Case study 


  1. Ashrafian H, Darsi A, Athanasiou T (2011) Evidence synthesis: evolving methodologies to optimise patient care and enhance policy decisions in evidence synthesis in healthcare. Springer-Verlag, LondonGoogle Scholar
  2. Babar MA, Verner JM, Nguyen PT (2007) Establishing and maintaining trust in software outsourcing relationships: an empirical investigation. J Syst Softw 80(9):1438–1449CrossRefGoogle Scholar
  3. Bethel EC, Bernard RM (2010) Developments and trends in synthesizing diverse forms of evidence: beyond comparisons between distance education and classroom instruction. Am J Dist Educ 31(3):231–256CrossRefGoogle Scholar
  4. Cruzes DS, Dybå T (2010) “Synthesizing evidence in software engineering research”, ESEM, 1–10Google Scholar
  5. Cruzes DS, Dybå T (2011a) Research synthesis in software engineering: a tertiary study”. IST 53(5):440–455Google Scholar
  6. Cruzes DS, Dybå T (2011b) “Recommended Steps for Thematic Synthesis in Software Engineering”, ESEM: 275–284Google Scholar
  7. Cruzes DS, Dybå T, Runeson P, Höst M (2011) Case Studies Synthesis: Brief Experience and Challenges for the Future. ESEM: 343–346Google Scholar
  8. Dybå T (2013) Contextualizing empirical evidence. IEEE Softw 30(1):81–83CrossRefGoogle Scholar
  9. Dybå T, Dingsøyr T (2008) Strength of Evidence in Systematic Reviews in Software Engineering, ESEM’, pp 178–187.Google Scholar
  10. Dybå T, Sjøberg DIK, Cruzes DS (2012) What works for whom, where, when, and why?: on the role of context in empirical software engineering. ESEM: 19–28Google Scholar
  11. Gerring J (2007) Case study research: principles and practices. Cambridge Univ. Press, New YorkGoogle Scholar
  12. Khan S, VanWynsberghe R (2008) Cultivating the Under-Mined: Cross-Case Analysis as Knowledge Mobilization. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 9 (1), Art. 34,
  13. Kitchenham BA, Dybå T, Jørgensen M (2004) Evidence-based Software Engineering, Proc. ICSE’04, Edinburgh, Scotland, 23–28 May, pp. 273–281Google Scholar
  14. Kitchenham BA, Charters S (2007) Guidelines for performing Systematic Literature Reviews in Software Engineering, Version 2.3, Keele University, EBSE Technical Report, EBSE-2007-01Google Scholar
  15. Kitchenham BA, Sjøberg DIK, Dybå T, Pfahl D, Brereton P, Budgen D, Höst M, Runeson P (2012) Three empirical studies on the agreement of reviewers about the quality of software engineering experiments. Inf Softw Technol 54(8):804–819CrossRefGoogle Scholar
  16. Larsson R (1993) Case survey methodology: quantitative analysis of patterns across case studies. Acad Manag J 36(6):1515–1546MathSciNetCrossRefGoogle Scholar
  17. Lucas WA (1974): The case survey method: Aggregating case experience. Santa MonicaGoogle Scholar
  18. Mays N, Pope C, Popay J (2005) Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 10:6–20CrossRefGoogle Scholar
  19. Miles MB, Huberman AM (1984) Qualitative Data Analysis: A Sourcebook of New Methods, SageGoogle Scholar
  20. Miles MB, Huberman AM (1994) Qualitative Data Analysis: An Expanded Source Book, SageGoogle Scholar
  21. Newig J, Fritsch O (2009) The case survey method and applications in political science. APSA 2009 Paper. Available at SSRN: = 1451643 (Toronto)Google Scholar
  22. Nguyen PT, Babar MA, Verner JM (2006) Critical factors in establishing and maintaining trust in software outsourcing relationships. EASE,
  23. Oza N, Hall T, Rainer A, Grey S (2005) Trust in software outsourcing relationships: an empirical investigation of Indian software companies. In: 9th International Conference on Empirical Assessment in Software Engineering (EASE 2005). Keele, UKGoogle Scholar
  24. Oza N, Hall T, Rainer A, Grey S (2006) Trust in software outsourcing relationships: an empirical investigation of Indian software companies. IST 48:345–354Google Scholar
  25. Popay J, Roberts H, Sowden A, Petticrew M, Britten N, Arai L, Roen K, Rodgers M (2010) Developing methods for the narrative synthesis of quantitative and qualitative data in systematic reviews of effects, Centre for Review and Dissemination, ESRC, Assessed in Dec. 2012
  26. Pope C, Mays N, Popay J (2007) Synthesizing Qualitative and Quantitative Health Evidence: A Guide to Methods, Open University PressGoogle Scholar
  27. Ragin CC (1987) The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies, Univ. of California PressGoogle Scholar
  28. Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering”. Empir Softw Eng 14(2):131–164CrossRefGoogle Scholar
  29. Runeson P, Höst M, Rainer A, Regnell B (2012) Case Study Research in Software Engineering – Guidelines and Examples. Wiley, first edition, ISBN-10: 1118104358, ISBN-13: 978–1118104354Google Scholar
  30. Šmite D, Wohlin C, Gorschek T, Feldt R (2010) Empirical evidence in global software engineering: a systematic review”. Empir Softw Eng 15(1):91–118CrossRefGoogle Scholar
  31. Thomas J, Harden A (2008) Methods for the thematic synthesis of qualitative research in systematic reviews”. BMC Med Res Methodol 8:45zbMATHCrossRefGoogle Scholar
  32. Yin RK (2014) Case study research. Design and methods, 5th edn. Sage, LondonGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Daniela S. Cruzes
    • 1
    Email author
  • Tore Dybå
    • 1
  • Per Runeson
    • 2
  • Martin Höst
    • 2
  1. 1.Department of Information and Communication Technology (ICT), SINTEFTrondheimNorway
  2. 2.Department Computer ScienceLund UniversityLundSweden

Personalised recommendations