An empirical study of architecting for continuous delivery and deployment

  • Mojtaba ShahinEmail author
  • Mansooreh Zahedi
  • Muhammad Ali Babar
  • Liming Zhu


Recently, many software organizations have been adopting Continuous Delivery and Continuous Deployment (CD) practices to develop and deliver quality software more frequently and reliably. Whilst an increasing amount of the literature covers different aspects of CD, little is known about the role of software architecture in CD and how an application should be (re-) architected to enable and support CD. We have conducted a mixed-methods empirical study that collected data through in-depth, semi-structured interviews with 21 industrial practitioners from 19 organizations, and a survey of 91 professional software practitioners. Based on a systematic and rigorous analysis of the gathered qualitative and quantitative data, we present a conceptual framework to support the process of (re-) architecting for CD. We provide evidence-based insights about practicing CD within monolithic systems and characterize the principle of “small and independent deployment units as an alternative to the monoliths. Our framework supplements the architecting process in a CD context through introducing the quality attributes (e.g., resilience) that require more attention and demonstrating the strategies (e.g., prioritizing operations concerns) to design operations-friendly architectures. We discuss the key insights (e.g., monoliths and CD are not intrinsically oxymoronic) gained from our study and draw implications for research and practice.


Software architecture Continuous delivery Continuous deployment DevOps Empirical study 



The authors would like to thank all participants in this study. This work is partially supported by Data61, a business unit of CSIRO, Australia. The first author is also supported by Australian Government Research Training Program Scholarship. We also greatly appreciate the hard work and time spent by the anonymous reviewers and the handling editor in providing insightful comments and help us to improve the manuscript.


  1. 2015 State of DevOps Report (2015) Available [Last accessed: 5 October 2015]Google Scholar
  2. Adams B, McIntosh S (2016) Modern release engineering in a nutshell -- why researchers should care. Presented at the IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), 14–18 March, 2016Google Scholar
  3. Adrian F (1996) Response bias, social desirability and dissimulation. Personal Individ Differ 7(3):385–400Google Scholar
  4. Andre van Hoorn PJ, Leitner P, Weber I (2017) Report from GI-Dagstuhl seminar 16394: software performance engineering in the DevOps world. Available at:
  5. Arun G (2015) Microservices, monoliths, and NoOps, Available at: [Last accessed: 8 November 2016]
  6. Balalaie A, Heydarnoori A, Jamshidi P (2016) Microservices architecture enables DevOps: migration to a cloud-native architecture. IEEE Softw 33(3):42–52CrossRefGoogle Scholar
  7. Bass L (2017) The software architect and DevOps. IEEE Softw 35(1):8–10CrossRefGoogle Scholar
  8. Bass L, Jeffery R, Wada H, Weber I, Liming Z (2013) Eliciting operations requirements for applications. In 1st International Workshop on Release Engineering (RELENG), pp. 5–8Google Scholar
  9. Bass L, Weber I, Zhu L (2015) DevOps: a software architect's perspective. Addison-Wesley ProfessionalGoogle Scholar
  10. Beijer P, de Klerk T (2010) IT Architecture - Essential Practice for IT Business Solutions.
  11. Bellomo S, Ernst N, Nord R, Kazman R (2014) Toward design decisions to enable deployability: empirical study of three projects reaching for the continuous delivery holy grail. Presented at the 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 23–26 June 2014Google Scholar
  12. Bosch J (2016) Speed, data, and ecosystems: the future of software engineering. Software, IEEE 33(1):82–88CrossRefGoogle Scholar
  13. Brandolini A (2013) Introducing event storming, Available at: [Last accessed: 8 July 2017]Google Scholar
  14. Brandtner M, Giger E, Gall H (2015) SQA-mashup: a mashup framework for continuous integration. Inf Softw Technol 65:97–113CrossRefGoogle Scholar
  15. Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77–101CrossRefGoogle Scholar
  16. Brown A (2015) What’s the best team structure for DevOps success? Available at: [Last accessed: 13 September 2017]
  17. Capilla R, Jansen A, Tang A, Avgeriou P, Babar MA (2016) 10 years of software architecture knowledge management: practice and future. J Syst Softw 116:191–205CrossRefGoogle Scholar
  18. Chen L (2015a) Continuous delivery: huge benefits, but challenges too. IEEE Softw 32(2):50–54MathSciNetCrossRefGoogle Scholar
  19. Chen L (2015b) Towards architecting for continuous delivery. In 12th Working IEEE/IFIP Conference on Software Architecture (WICSA), pp. 131–134Google Scholar
  20. Chris R. (2014). Pattern: monolithic architecture, Available at: [Last accessed: 4 November 2016]
  21. Cito J, Leitner P, Fritz T, Gall HC (2015) The making of cloud applications: an empirical study on software development for the cloud. In 10th Joint Meeting on Foundations of Software Engineering, Bergamo, Italy, pp. 393–403: ACMGoogle Scholar
  22. Claps GG, Berntsson Svensson R, Aurum A (2015) On the journey to continuous deployment: technical and social challenges along the way. Inf Softw Technol 57:21–31CrossRefGoogle Scholar
  23. Conway ME (1968) How do committees invent? Datamation 14(5)Google Scholar
  24. Cruzes DS, Dyba T (2011) Recommended steps for thematic synthesis in software engineering. In International Symposium on Empirical Software Engineering and Measurement (ESEM), pp. 275–284Google Scholar
  25. Debbiche A, Dienér M, Berntsson Svensson R (2014) Challenges when adopting continuous integration: a case study. Cham, pp. 17–32: Springer International PublishingGoogle Scholar
  26. Dooley PM (2015) The intersection of DevOps and ITIL, Available at: [Last accessed: 14 June 2017]. Global KnowledgeGoogle Scholar
  27. Dragoni N et al (2017) Microservices: yesterday, today, and tomorrow. In: Mazzara M, Meyer B (eds) Present and ulterior software engineering. Springer International Publishing, Cham, pp 195–216CrossRefGoogle Scholar
  28. Easterbrook S, Singer J, Storey M-A, Damian D (2008) Selecting empirical methods for software engineering research. In: Shull F, Singer J, Sjøberg DIK (eds) Guide to advanced empirical software engineering. Springer London, London, pp 285–311CrossRefGoogle Scholar
  29. Elbaum S, Rothermel G, Penix J (2014) Techniques for improving regression testing in continuous integration development environments. In 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, Hong Kong, China, pp. 235–245: ACMGoogle Scholar
  30. Erder M, Pureur P (2015) Continuous architecture: sustainable architecture in an agile and cloud-centric world. Morgan KaufmannGoogle Scholar
  31. Ernst N, Klein J, Mathew G, Menzies T (2017) Using stakeholder preferences to make better architecture decisions. In 2017 IEEE International Conference on Software Architecture Workshops (ICSAW), pp. 133–136Google Scholar
  32. Evans E (2004) Domain-driven design: tackling complexity in the heart of softwareT. Addison-Wesley ProfessionalGoogle Scholar
  33. Fitzgerald B, Stol K-J (2017) Continuous software engineering: a roadmap and agenda. J Syst Softw 123Google Scholar
  34. Ford N (2016) Architecture is abstract until operationalized, Available at: [Last accessed: 21 February 2016]Google Scholar
  35. Ford N, Parsons R (2016) Microservices as an Evolutionary Architecture, Available at: [Last accessed: 20 March 2016]
  36. Fowler M (2013) Continuous Delivery, Available at: [Last accessed: 21 October 2015]
  37. Fowler M (2015a) Continuous Integration, Available at: [Last accessed: 21 October 2015]Google Scholar
  38. Fowler M (2015b) MicroservicePremium, Available at: [Last accessed: 31 October 2016]
  39. Fowler Jr FJ (2013) Survey research methods. Sage publicationsGoogle Scholar
  40. Gabhart K (2014) Resilient IT Through DevOps, Available at: [Last accessed: 1 July 2017]
  41. Garousi V, Felderer M, Mäntylä MV (2016) The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In 20th International Conference on Evaluation and Assessment in Software Engineering, Limerick, Ireland, pp. 1–6, 2916008: ACMGoogle Scholar
  42. Gibson S (2016) Monoliths are bad design... and you know it, Available at: [Last accessed: 4 March 2016]Google Scholar
  43. Gitlevich V, Evans E (2015) What is Domain-driven design? Available at: [Last accessed: 21 June 2016]Google Scholar
  44. Goodman LA (1961) Snowball sampling. Ann Math Stat 32(1):148–170MathSciNetCrossRefzbMATHGoogle Scholar
  45. Gousios G, Storey M-A, Bacchelli A (2016) Work practices and challenges in pull-based development: the contributor's perspective. In 38th International Conference on Software Engineering, Austin, Texas, pp. 285–296: ACMGoogle Scholar
  46. Hasselbring W, Steinacker G (2017) Microservice architectures for scalability, agility and reliability in e-commerce. In 2017 IEEE International Conference on Software Architecture Workshops (ICSAW), pp. 243–246Google Scholar
  47. Hilton M, Nelson N, Tunnell T, Marinov D, Dig D (2017) Trade-offs in continuous integration: assurance, security, and flexibility. Presented at the Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, Paderborn, GermanyGoogle Scholar
  48. Hohpe G, Ozkaya I, Zdun U, Zimmermann O (2016) The software architect's role in the digital age. IEEE Softw 33(6):30–39CrossRefGoogle Scholar
  49. Hove SE, Anda B (2005) Experiences from conducting semi-structured interviews in empirical software engineering research. In 11th IEEE International Software Metrics Symposium, p. 23, 1092163: IEEE Computer SocietyGoogle Scholar
  50. Humble J (2011) Organize software delivery around outcomes, not roles: continuous delivery and cross-functional teams, Available at: [Last accessed: 10 August 2016]
  51. Humble J (2007) Continuous delivery vs continuous deployment, Available at: [Last accessed: 1 March 2016]Google Scholar
  52. Humble J, Farley D (2010) Continuous delivery: reliable software releases through build, test, and deployment automation, 1st ed. Addison-Wesley ProfessionalGoogle Scholar
  53. ISO/IEC/IEEE Systems and software engineering -- Architecture description (2011) ISO/IEC/IEEE 42010:2011(E) (Revision of ISO/IEC 42010:2007 and IEEE Std 1471–2000), pp. 1–46Google Scholar
  54. Jiang B, Zhang Z, Chan WK, Tse TH, Chen TY (2012) How well does test case prioritization integrate with statistical fault localization? Information and Software Technology,vol. 54, no. 7, pp. 739–758, 2012/07/01/ 2012Google Scholar
  55. Jong Md, Deursen Av, Cleve A (2017) Zero-downtime SQL database schema evolution for continuous deployment. In 39th International Conference on Software Engineering: Software Engineering in Practice Track, Buenos Aires, Argentina, pp. 143–152: IEEE PressGoogle Scholar
  56. Kim EH, Na JC, Ryoo SM (2009) Test automation framework for implementing continuous integration. In Sixth International Conference on Information Technology: New Generations, pp. 784–789Google Scholar
  57. Kitchenham BA, Pfleeger SL (2008) Personal opinion surveys. In: Shull F, Singer J, Sjøberg DIK (eds) Guide to advanced empirical software engineering. Springer London, London, pp 63–92CrossRefGoogle Scholar
  58. Kitchenham B, Pickard L, Pfleeger SL (1995) Case studies for method and tool evaluation. IEEE Softw 12(4):52–62CrossRefGoogle Scholar
  59. Laukkanen E, Lehtinen TOA, Itkonen J, Paasivaara M, Lassenius C (2016) Bottom-up adoption of continuous delivery in a stage-gate managed software organization. In 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, Ciudad Real, Spain, pp. 1–10: ACMGoogle Scholar
  60. Laukkanen E, Itkonen J, Lassenius C (2017) Problems, causes and solutions when adopting continuous delivery—a systematic literature review. Inf Softw Technol 82:55–79CrossRefGoogle Scholar
  61. Leppanen M et al (2015) The highways and country roads to continuous deployment. IEEE Softw 32(2):64–72MathSciNetCrossRefGoogle Scholar
  62. Lewis J, Fowler M (2010) Microservices: a definition of this new architectural term, Available at: [Last accessed: 05 January 2016]Google Scholar
  63. Luke E, Prince S (2016) No one agrees how to define CI or CD. Available at: [Last accessed: 1 August 2016]
  64. Mäkinen S et al (2016) Improving the delivery cycle: a multiple-case study of the toolchains in Finnish software intensive enterprises. Inf Softw Technol 80:175–194CrossRefGoogle Scholar
  65. Manotas I, et al (2016) An empirical study of practitioners' perspectives on green software engineering. In 38th International Conference on Software Engineering, Austin, Texas, pp. 237–248: ACMGoogle Scholar
  66. Mäntylä M, Adams B, Khomh F, Engström E, Petersen K (2015) On rapid releases and software testing: a case study and a semi-systematic literature review," (in English). Empir Softw Eng 20(5):1384–1425CrossRefGoogle Scholar
  67. Mårtensson T, Ståhl D, Bosch J (2017) Continuous integration impediments in large-scale industry projects. In 2017 IEEE International Conference on Software Architecture (ICSA), pp. 169–178Google Scholar
  68. Meade AW, Craig SB (2012) Identifying careless responses in survey data. Psychol Methods 17(3):437CrossRefGoogle Scholar
  69. Meho LI (2006) E-mail interviewing in qualitative research: a methodological discussion: research articles. J Am Soc Inf Sci Technol 57(10):1284–1295CrossRefGoogle Scholar
  70. Murphy-Hill E, Zimmermann T, Bird C, Nagappan N (2015) The design space of bug fixes and how developers navigate it. IEEE Trans Softw Eng 41(1):65–81CrossRefGoogle Scholar
  71. Newman S (2015) Building microservices. O'Reilly Media, IncGoogle Scholar
  72. Northrop L (2015) Trends and new directions in software architecture, Available at: Scholar
  73. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K (2015) Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health Ment Health Serv Res 42(5):533–544CrossRefGoogle Scholar
  74. Pauw Td (2017) Feature Branching is Evil, Available at: [Last accessed: 27 May 2017]
  75. Prewer L (2015) Smoothing the continuous delivery path – a tale of two teams, Available at: [Last accessed: 2 October 2016]
  76. Prince S (2016) The product managers’ guide to continuous delivery and DevOps, Available at: [Last accessed: 2 November 2016]
  77. Rahman MT, Querel LP, Rigby PC, Adams B (2016) Feature toggles: practitioner practices and a case study. In 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR), pp. 201–211Google Scholar
  78. Rodríguez P et al (2017) Continuous deployment of software intensive products and services: a systematic mapping study. J Syst Softw 123:263–291CrossRefGoogle Scholar
  79. Savor T, Douglas M, Gentili M, Williams L, Beck K, Stumm M (2016) Continuous deployment at Facebook and OANDA. In 38th International Conference on Software Engineering Companion, Austin, Texas, pp. 21–30: ACMGoogle Scholar
  80. Schauenberg D (2014) Development, deployment and collaboration at Etsy, Available at: [Last accessed: 1 September 2017]
  81. Schermann G, Cito J, Leitner P, Zdun U, Gall H (2016) An empirical study on principles and practices of continuous delivery and deployment. PeerJ Preprints 4:e1889v1Google Scholar
  82. Seaman CB (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557–572CrossRefGoogle Scholar
  83. Self-Contained Systems: Assembling Software from Independent Systems (2014) Available at: [Last accessed: 1 June 2017]
  84. Shahin M, Babar MA, Zhu L (2016) The intersection of continuous deployment and architecting process: practitioners' perspectives. In ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, Ciudad Real, Spain,, pp. 1–10: ACMGoogle Scholar
  85. Shahin M, Babar MA, Zhu L (2017a) Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE Access 5:3909–3943CrossRefGoogle Scholar
  86. Shahin M, Babar MA, Zahedi M, Zhu L (2017b) Beyond continuous delivery: an empirical investigation of continuous deployment challenges. In 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Toronto, Canada: IEEEGoogle Scholar
  87. Shahin M, Zahedi M, Babar MA, Zhu L (2017c) Adopting continuous delivery and deployment: impacts on team structures, collaboration and responsibilities. In 21st International Conference on Evaluation and Assessment in Software Engineering, Karlskrona, Sweden, pp. 384–393: ACMGoogle Scholar
  88. Skelton M (2016) How to break apart a monolithic system safely without destroying your team, Available at: [Last accessed: 4 November 2016]
  89. Skelton M, O'Dell C (2016) Continuous delivery with windows and .NET. O'ReillyGoogle Scholar
  90. Sokhan B (2016) Domain driven design for services architecture, Available at: [Last accessed: 10 January 2016]Google Scholar
  91. Ståhl D, Bosch J (2014) Modeling continuous integration practice differences in industry software development. J Syst Softw 87:48–59CrossRefGoogle Scholar
  92. Suneja S et al (2017) Safe inspection of live virtual machines. In 13th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Xi'an, China, pp. 97–111, 3050766: ACMGoogle Scholar
  93. Thiele A (2014) Continuous delivery: an easy must-have for agile development, Available at: [Last accessed: 10 July 2016]
  94. Vishal N (2015) Architecting for continuous delivery, Available at: [Last accessed: 15 March 2016]
  95. Wallgren A (2015) Continuous delivery of microservices: patterns and processes, Available at: [Last accessed: 10 February 2018]
  96. Waterman MG (2014) Reconciling agility and architecture: a theory of agile architecture. PhD Thesis, Victoria University of WellingtonGoogle Scholar
  97. What Team Structure is Right for DevOps to Flourish (2017) Available at: [Last accessed: 24 September 2017]Google Scholar
  98. Wnuk K (2017) Involving relevant stakeholders into the decision process about software components. In 2017 IEEE International Conference on Software Architecture Workshops (ICSAW), pp. 129–132Google Scholar
  99. Woods E (2016) Operational: the forgotten architectural view. IEEE Softw 33(3):20–23CrossRefGoogle Scholar
  100. Yaniv Y (2014) Closing the gap between database continuous delivery and code continuous delivery, Available at: [Last accessed: 21 August 2016]

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Mojtaba Shahin
    • 1
    Email author
  • Mansooreh Zahedi
    • 2
  • Muhammad Ali Babar
    • 1
  • Liming Zhu
    • 3
  1. 1.CREST - The Centre for Research on Engineering Software TechnologiesThe University of AdelaideAdelaideAustralia
  2. 2.CREST - The Centre for Research on Engineering Software TechnologiesIT University of CopenhagenCopenhagenDenmark
  3. 3.Data61, Commonwealth Scientific and Industrial Research OrganisationSydneyAustralia

Personalised recommendations