Feedback from Operations to Software Development—A DevOps Perspective on Runtime Metrics and Logs

  • Jürgen CitoEmail author
  • Johannes Wettinger
  • Lucy Ellen Lwakatare
  • Markus Borg
  • Fei Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11350)


DevOps achieve synergy between software development and operations engineers. This synergy can only happen if the right culture is in place to foster communication between these roles. We investigate the relationship between runtime data generated during production and how this data can be used as feedback in the software development process. For that, we want to discuss case study organizations that have different needs on their operations-to-development feedback pipeline, from which we abstract and propose a more general, higher-level feedback process. Given such a process, we discuss a technical environment required to support this process. We sketch out different scenarios in which feedback is useful in different phases of the software development life-cycle.


Software engineering DevOps Feedback 



This paper resulted from initial discussions at the GI-sDagstuhl Seminar: “Software Performance Engineering in the DevOps World” (seminar number 16394).


  1. 1.
    Barik, T., DeLine, R., Drucker, S., Fisher, D.: The bones of the system: a case study of logging and telemetry at Microsoft. In: Proceedings of the 38th International Conference on Software Engineering Companion, pp. 92–101 (2016)Google Scholar
  2. 2.
    Bass, L., Jeffery, R., Wada, H., Weber, I., Zhu, L.: Eliciting operations requirements for applications. In: Proceedings of the 1st International Workshop on Release Engineering, pp. 5–8 (2013)Google Scholar
  3. 3.
    Bass, L., Weber, I., Zhu, L.: DevOps: A Software Architect’s Perspective. Addison-Wesley Professional, Boston (2015)Google Scholar
  4. 4.
    Brunnert, A., et al.: Performance-oriented DevOps: a research agenda. arXiv preprint arXiv:1508.04752 (2015)
  5. 5.
    Cito, J., Leitner, P., Fritz, T., Gall, H.C.: The making of cloud applications: an empirical study on software development for the cloud. In: Proceedings of the 10th Joint Meeting on Foundations of Software Engineering, pp. 393–403 (2015)Google Scholar
  6. 6.
    Cito, J., Leitner, P., Gall, H.C., Dadashi, A., Keller, A., Roth, A.: Runtime metric meets developer: building better cloud applications using feedback. In: Proceedings of the ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!), pp. 14–27 (2015)Google Scholar
  7. 7.
    Dybå, T., Sjøberg, D., Cruzes, D.: What works for whom, where, when, and why? On the role of context in empirical software engineering. In: Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 19–28 (2012)Google Scholar
  8. 8.
    Olsson, H.H., Bosch, J.: Post-deployment data collection in software-intensive embedded products. In: Bosch, J. (ed.) Continuous Software Engineering, pp. 143–154. Springer, Cham (2014). Scholar
  9. 9.
    Humble, J., Farley, D.: Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation. Addison-Wesley Professional, Boston (2010)Google Scholar
  10. 10.
    Keim, D., Andrienko, G., Fekete, J.-D., Görg, C., Kohlhammer, J., Melançon, G.: Visual analytics: definition, process, and challenges. In: Kerren, A., Stasko, J.T., Fekete, J.-D., North, C. (eds.) Information Visualization: Human-Centered Issues and Perspectives. LNCS, vol. 4950, pp. 154–175. Springer, Heidelberg (2008). Scholar
  11. 11.
    Koschke, R.: Software visualization in software maintenance, reverse engineering, and re-engineering: a research survey. J. Softw. Maint. Evol.: Res. Pract. 15(2), 87–109 (2003)CrossRefGoogle Scholar
  12. 12.
    Murphy-Hill, E., Murphy, G.C.: Recommendation delivery. In: Robillard, M.P., Maalej, W., Walker, R.J., Zimmermann, T. (eds.) Recommendation Systems in Software Engineering, pp. 223–242. Springer, Heidelberg (2014). Scholar
  13. 13.
    Robillard, M.P., Walker, R.J.: An introduction to recommendation systems in software engineering. In: Robillard, M.P., Maalej, W., Walker, R.J., Zimmermann, T. (eds.) Recommendation Systems in Software Engineering, pp. 1–11. Springer, Heidelberg (2014). Scholar
  14. 14.
    Savor, T., Douglas, M., Gentili, M., Williams, L., Beck, K., Stumm, M.: Continuous deployment at Facebook and OANDA. In: Proceedings of the International Conference on Software Engineering Companion, pp. 21–30 (2016)Google Scholar
  15. 15.
    Schermann, G., Cito, J., Leitner, P., Zdun, U., Gall, H.C.: We’re doing it live: a multi-method empirical study on continuous experimentation. Inf. Softw. Technol. 99, 41–57 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Jürgen Cito
    • 1
    Email author
  • Johannes Wettinger
    • 2
  • Lucy Ellen Lwakatare
    • 3
  • Markus Borg
    • 4
  • Fei Li
    • 5
  1. 1.University of ZurichZürichSwitzerland
  2. 2.University of StuttgartStuttgartGermany
  3. 3.University of OuluOuluFinland
  4. 4.RISE SICS ABKistaSweden
  5. 5.Siemens AGViennaAustria

Personalised recommendations