Automated Test Case Generation for the CTRL Programming Language Using Pex: Lessons Learned

  • Stefan KlikovitsEmail author
  • David P. Y. Lawrence
  • Manuel Gonzalez-Berges
  • Didier Buchs
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9823)


Over the last decade code-based test case generation techniques such as combinatorial testing or dynamic symbolic execution have seen growing research popularity. Most algorithms and tool implementations are based on finding assignments for input parameter values in order to maximise the execution branch coverage. In this paper we first present ITEC, a tool for automated test case generation in CTRL, as well as initial results of test cases executions on one of CERN’s SCADA frameworks. Our tool relies on Microsoft’s Pex for its code exploration. For the purpose of using this existing test generation tool, we have to translate the proprietary CTRL code into C#, one of Pex’s operating languages. Our main contribution lies in detailing a formal foundation for this step through source code decomposition and anonymization. We then propose a quality measure that is used to determine our confidence into the translation and the generated test cases.


Automated test case generation Resilience Software testing Translation validation Execution environment resilience 


  1. 1.
    Bernot, G., Gaudel, M., Marre, B.: Software testing based on formal specifications: a theory and a tool. Softw. Eng. J. 6(6), 387–405 (1991). CrossRefGoogle Scholar
  2. 2.
    CERN: The JCOP Framework, August 2014.
  3. 3.
    Csallner, C., Tillmann, N., Smaragdakis, Y.: DySy: dynamic symbolic execution for invariant inference. In: Proceedings of the 30th International Conference on Software Engineering, ICSE 2008, pp. 281–290. ACM, New York (2008).
  4. 4.
    ETM Professional Control: WinCC OA at a glance. Siemens AG (2012)Google Scholar
  5. 5.
    ETM Professional Control: Control script language (2015).
  6. 6.
    Fleurey, F., Steel, J., Baudry, B.: Validation in model-driven engineering: testing model transformations. In: First International Workshop on Model, Design and Validation, Proceedings, pp. 29–40, November 2004Google Scholar
  7. 7.
    Jörges, S. (ed.): Construction and Evolution of Code Generators. LNCS, vol. 7747, pp. 207–213. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  8. 8.
    Klikovits, S., Lawrence, D.P.Y., Gonzalez-Berges, M., Buchs, D.: Considering execution environment resilience: a white-box approach. In: Fantechi, A., Pelliccione, P. (eds.) SERENE 2015. LNCS, vol. 9274, pp. 46–61. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  9. 9.
    Küster, J.M., Abd-El-Razik, M.: Validation of model transformations – first experiences using a white box approach. In: Kühne, T. (ed.) MoDELS 2006. LNCS, vol. 4364, pp. 193–204. Springer, Heidelberg (2007). CrossRefGoogle Scholar
  10. 10.
    Malaiya, Y.K.: Antirandom testing: getting the most out of black-box testing. In: Sixth International Symposium on Software Reliability Engineering, ISSRE 1995, Toulouse, France, 24–27 October 1995, pp. 86–95. IEEE (1995).
  11. 11.
    Microsoft Research: Pex, Automated White box Testing for .NET.
  12. 12.
    Pnueli, A., Siegel, M.D., Singerman, E.: Translation validation. In: Steffen, B. (ed.) TACAS 1998. LNCS, vol. 1384, pp. 151–166. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  13. 13.
    Samet, H.: Automatically proving the correctness of translations involving optimized code. Memo AIM, Stanford University (1975).
  14. 14.
    Varró, D., Pataricza, A.: Automated formal verification of model transformations. In: Jürjens, J., Rumpe, B., France, R., Fernandez, E.B. (eds.) CSDUML 2003: Critical Systems Development in UML; Proceedings of the UML 2003 Workshop, pp. 63–78. No. TUM-I0323 in Technical report, Technische Universitüt München, Technische Universität München, September 2003.
  15. 15.
    Zuck, L.D., Pnueli, A., Goldberg, B.: VOC: a methodology for the translation validation of optimizing compilers. J. UCS 9(3), 223–247 (2003). Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Stefan Klikovits
    • 1
    • 2
    Email author
  • David P. Y. Lawrence
    • 1
  • Manuel Gonzalez-Berges
    • 2
  • Didier Buchs
    • 1
  1. 1.Centre Universitaire d’InformatiqueUniversité de GenèveCarougeSwitzerland
  2. 2.CERN, European Organization for Nuclear ResearchGenevaSwitzerland

Personalised recommendations