Enabling continuous integration in a formal methods setting


In modern software development, the practices of continuous integration and DevOps are widely used to increase delivery speed and reduce the time it takes to deploy software changes to production. If formal method tools cannot be efficiently integrated in a DevOps paradigm, then their impact on software development will be reduced. In this paper, we present work addressing this issue through a series of extensions for the Overture tool supporting the Vienna Development Method. These extensions enable Overture to be used in a DevOps setting, through continuous integration and validation of models and generated code via integration with the Jenkins automation server. We frame the integration of formal methods and DevOps in a series of principles, demonstrate the value of this integration through a case study, and reflect on our experiences using formal methods and DevOps in an industrial setting. We hope that this work can help other formal method practitioners integrate their tools with DevOps.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9


  1. 1.



  1. 1.

    Aichernig, B.K., Larsen, P.G.: A proof obligation generator for VDM-SL. In: Fitzgerald, J.S., Jones, C.B., Lucas, P. (eds.) FME’97: Industrial Applications and Strengthened Foundations of Formal Methods (Proceedings of the 4th International Symposium of Formal Methods Europe, Graz, Austria, September 1997), Lecture Notes in Computer Science, vol. 1313, pp. 338–357. Springer (1997). ISBN 3-540-63533-5

  2. 2.

    Amazon.com Inc.: Amazon Simple Storage Service (S3). https://aws.amazon.com/s3/. Accessed 24 Oct 2019

  3. 3.

    Amazon.com Inc.: s2n. https://github.com/awslabs/s2n. Accessed 24 Oct 2019

  4. 4.

    Bass, L., Weber, I., Zhu, L.: DevOps: A Software Architect’s Perspective. Addison-Wesley Professional, Boston (2015)

    Google Scholar 

  5. 5.

    Chudnov, A., Collins, N., Cook, B., Dodds, J., Huffman, B., MacCárthaigh, C., Magill, S., Mertens, E., Mullen, E., Tasiran, S., Tomb, A., Westbrook, E.: Continuous formal verification of Amazon s2n. In: Chockler, H., Weissenbacher, G. (eds.) Computer Aided Verification, pp. 430–446. Springer, Cham (2018)

    Google Scholar 

  6. 6.

    Cook, B.: Formal reasoning about the security of amazon web services. In: Chockler, H., Weissenbacher, G. (eds.) Computer Aided Verification, pp. 38–47. Springer, Cham (2018)

    Google Scholar 

  7. 7.

    Couto, L.D., Larsen, P.G., Hasanagic, M., Kanakis, G., Lausdahl, K., Tran-Jørgensen, P.W.V.: Towards enabling overture as a platform for formal notation IDEs. In: Proceedings of the 2nd Workshop on Formal-IDE (F-IDE) (2015)

  8. 8.

    Couto, L.D., Tran-Jørgensen, P.W.V., Edwards, G.T.C.: Combining harvesting operation optimisations using strategy-based simulation. In: Proceedings of the 6th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH) (2016)

  9. 9.

    Couto, L.D., Tran-Jørgensen, P.W.V., Edwards, G.T.C.: Model-based development of a multi-algorithm harvest planning system. In: Simulation and Modeling Methodologies, Technologies and Applications: International Conference, SIMULTECH 2016 Lisbon, Portugal, July 29–31, 2016, Revised Selected Papers. Springer (2018)

  10. 10.

    Delegate Tutorial. https://github.com/ldcouto/delegate-tutorial (2019). Accessed 24 Oct 2019

  11. 11.

    Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1, 269–271 (1959)

    MathSciNet  Article  Google Scholar 

  12. 12.

    dSPACE TargetLink Product Management: TargetLink 4.1 Product Information (2016)

  13. 13.

    Fitzgerald, J.S., Larsen, P.G.: Formal specification techniques in the commercial development process. In: Wirsing, M. (ed.) Position Papers from the Workshop on Formal Methods Application in Software Engineering Practice, International Conference on Software Engineering (ICSE-17), Seattle (1995)

  14. 14.

    Fowler, M., Foemmel, M.: Continuous Integration. Thought-Works, Chicago (2006)

    Google Scholar 

  15. 15.

    Fraser, S.: Integrating VDM-SL into the continuous delivery pipelines of cloud-based software. In: Pierce, K., Verhoef, M. (eds.) The 16th Overture Workshop, pp. 123–138. Newcastle University, School of Computing, Oxford (2018) (TR-1524)

  16. 16.

    Fröhlich, B., Larsen, P.G.: Combining VDM-SL specifications with C++ code. In: Proceedings of the FME’96: Industrial Benefit and Advances in Formal Methods (1996)

  17. 17.

    Gallasch, G., Kristensen, L.M.: Comms/CPN: a communication infrastructure for external communication with design/CPN. In: Proceedings of the 3rd Workshop and Tutorial on Practical Use of Coloured Petri Nets and the CPN Tools (CPN’01) (2001)

  18. 18.

    Hasanagić, M., Tran-Jørgensen, P.W.V., Nilsson, R.S., Larsen, P.G.: Realization of distributed system models using code generation extensions. Software. Pract. Exp. 48(12), 478–497 (2018)

    Google Scholar 

  19. 19.

    Jenkins website. https://jenkins.io (2019). Accessed 24 Oct 2019

  20. 20.

    JGraphT website. http://www.jgrapht.org/ (2019). Accessed 24 Oct 2019

  21. 21.

    Jones, C.B.: Scientific decisions which characterize VDM. In: Proceedings of the FM’99—Formal Methods (1999)

  22. 22.

    Jørgensen, P.W.V., Couto, L.D., Larsen, M.: A code generation platform for VDM. In: Proceedings of the 12th Overture Workshop (2014)

  23. 23.

    Knight, J.C., Hanks, K.S., Travis, S.R.: Tool support for production use of formal techniques. In: Proceedings of the 12th International Symposium on Software Reliability Engineering (2001)

  24. 24.

    Larsen, P.G., Fitzgerald, J., Brookes, T.: Applying formal specification in industry. IEEE Softw. 13(3), 48–56 (1996)

    Article  Google Scholar 

  25. 25.

    Larsen, P.G., Fitzgerald, J., Wolff, S.: Are formal methods ready for agility? A reality check. In: Gruner, S., Rumpe, B. (eds.) 2nd International Workshop on Formal Methods and Agile Methods. Lecture Notes in Informatics, pp. 13–25 (2010) (ISSH 1617-5468)

  26. 26.

    Larsen, P.G., Lausdahl, K., Tran-Jørgensen, P.W.V., Ribeiro, A., Wolff, S., Battle, N.: Overture VDM-10 tool support: user guide. Technical report. The Overture Initiative (2010)

  27. 27.

    Macedo, H.D., Nilsson, R.S., Larsen, P.G.: The harvest coach architecture: embedding deviation-tolerance in a harvest logistic solution. Computers 8(2), 31 (2019)

    Article  Google Scholar 

  28. 28.

    Nielsen, C.B., Lausdahl, K., Larsen, P.G.: Combining VDM with executable code. In: Abstract State Machines, Alloy, B, VDM, and Z (2012)

  29. 29.

    O’Hearn, P.W.: Continuous reasoning: scaling the impact of formal methods. In: Proceedings of the 33rd annual ACM/IEEE symposium on logic in computer science, LICS ’18, pp. 13–25. ACM, New York, NY, USA. https://doi.org/10.1145/3209108.3209109 (2018)

  30. 30.

    Ribeiro, A., Larsen, P.G.: Proof obligation generation and discharging for recursive definitions in VDM. In: Song, J., Huibiao (eds.) The 12th International Conference on Formal Engineering Methods (ICFEM 2010). Springer (2010)

  31. 31.

    Sadowski, C., Aftandilian, E., Eagle, A., Miller-Cushon, L., Jaspan, C.: Lessons from building static analysis tools at Google. Commun. ACM 61(4), 58–66 (2018). https://doi.org/10.1145/3188720

    Article  Google Scholar 

  32. 32.

    Smith, P.R., Larsen, P.G.: Applications of VDM in banknote processing. In: Fitzgerald, J.S., Larsen, P.G. (eds.) VDM in Practice: Proceedings of the First VDM Workshop 1999 (1999)

  33. 33.

    The Apache Maven Project website. https://maven.apache.org (2019). Accessed 24 Oct 2019

  34. 34.

    The JUnit website. http://www.junit.org (2019). Accessed 24 Oct 2019

  35. 35.

    The Overture website. http://overturetool.org (2019). Accessed 24 Oct 2019

  36. 36.

    The VDM Tool Group: VDM Toolbox API. Technical report, CSK Systems (2008)

  37. 37.

    Tran-Jørgensen, P.W.V.: Enhancing System Realisation in Formal Model Development. Ph.D. thesis, Aarhus University (2016)

  38. 38.

    Tran-Jørgensen, P.W.V., Larsen, P.G., Leavens, G.T.: Automated translation of VDM to JML-annotated Java. Int. J. Softw. Tools Technol. Transf. 20(2), 211–235 (2018)

    Article  Google Scholar 

  39. 39.

    Tran-Jørgensen, P.W.V., Nilsson, R., Lausdahl, K.: Enhancing testing of VDM-SL models. In: Pierce, K., Verhoef, M. (eds.) The 16th Overture Workshop, pp. 7–22. Newcastle University, School of Computing, Oxford (2018) (TR-1524)

  40. 40.

    Vermolen, S., Hooman, J., Larsen, P.G.: Automating consistency proofs of VDM++ models using HOL. In: Proceedings of the 25th Symposium on Applied Computing (SAC 2010). ACM, Sierre, Switzerland (2010)

  41. 41.

    Wassyng, A., Lawford, M.: Software tools for safety-critical software development. Int. J. Softw. Tools Technol. Transf. 8(4), 337–354 (2005). https://doi.org/10.1007/s10009-005-0209-6

    Article  MATH  Google Scholar 

  42. 42.

    Westergaard, M., Kristensen, L.: The access/CPN framework: a tool for interacting with the CPN tools simulator. In: Proceedings of the 30th International Conference on Applications and Theory of Petri Nets (2009)

Download references


We thank the anonymous reviewers for their constructive comments, which helped us improve the manuscript. The authors would also like to thank Nick Battle for valuable feedback on earlier versions of this paper. Finally, we would like to thank Innovation Fund Denmark for supporting parts of this work.

Author information



Corresponding author

Correspondence to Peter W. V. Tran-Jørgensen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Couto, L.D., Tran-Jørgensen, P.W.V., Nilsson, R.S. et al. Enabling continuous integration in a formal methods setting. Int J Softw Tools Technol Transfer 22, 667–683 (2020). https://doi.org/10.1007/s10009-019-00546-y

Download citation


  • VDM
  • Modelling
  • Simulation
  • Code generation
  • Test automation
  • Continuous integration
  • DevOps