Advertisement

Automated Test Case Generation for the CTRL Programming Language Using Pex: Lessons Learned

  • Stefan Klikovits
  • David P. Y. Lawrence
  • Manuel Gonzalez-Berges
  • Didier Buchs
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9823)

Abstract

Over the last decade code-based test case generation techniques such as combinatorial testing or dynamic symbolic execution have seen growing research popularity. Most algorithms and tool implementations are based on finding assignments for input parameter values in order to maximise the execution branch coverage. In this paper we first present ITEC, a tool for automated test case generation in CTRL, as well as initial results of test cases executions on one of CERN’s SCADA frameworks. Our tool relies on Microsoft’s Pex for its code exploration. For the purpose of using this existing test generation tool, we have to translate the proprietary CTRL code into C#, one of Pex’s operating languages. Our main contribution lies in detailing a formal foundation for this step through source code decomposition and anonymization. We then propose a quality measure that is used to determine our confidence into the translation and the generated test cases.

Keywords

Automated test case generation Resilience Software testing Translation validation Execution environment resilience 

References

  1. 1.
    Bernot, G., Gaudel, M., Marre, B.: Software testing based on formal specifications: a theory and a tool. Softw. Eng. J. 6(6), 387–405 (1991). http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=120426 CrossRefGoogle Scholar
  2. 2.
    CERN: The JCOP Framework, August 2014. https://j2eeps.cern.ch/wikis/display/EN/JCOP+Framework
  3. 3.
    Csallner, C., Tillmann, N., Smaragdakis, Y.: DySy: dynamic symbolic execution for invariant inference. In: Proceedings of the 30th International Conference on Software Engineering, ICSE 2008, pp. 281–290. ACM, New York (2008). http://doi.acm.org/10.1145/1368088.1368127
  4. 4.
    ETM Professional Control: WinCC OA at a glance. Siemens AG (2012)Google Scholar
  5. 5.
    ETM Professional Control: Control script language (2015). http://etm.at/index_e.asp?id=2&sb1=54&sb2=118&sb3=&sname=&sid=&seite_id=118
  6. 6.
    Fleurey, F., Steel, J., Baudry, B.: Validation in model-driven engineering: testing model transformations. In: First International Workshop on Model, Design and Validation, Proceedings, pp. 29–40, November 2004Google Scholar
  7. 7.
    Jörges, S. (ed.): Construction and Evolution of Code Generators. LNCS, vol. 7747, pp. 207–213. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  8. 8.
    Klikovits, S., Lawrence, D.P.Y., Gonzalez-Berges, M., Buchs, D.: Considering execution environment resilience: a white-box approach. In: Fantechi, A., Pelliccione, P. (eds.) SERENE 2015. LNCS, vol. 9274, pp. 46–61. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  9. 9.
    Küster, J.M., Abd-El-Razik, M.: Validation of model transformations – first experiences using a white box approach. In: Kühne, T. (ed.) MoDELS 2006. LNCS, vol. 4364, pp. 193–204. Springer, Heidelberg (2007). http://dx.doi.org/10.1007/978-3-540-69489-2_24 CrossRefGoogle Scholar
  10. 10.
    Malaiya, Y.K.: Antirandom testing: getting the most out of black-box testing. In: Sixth International Symposium on Software Reliability Engineering, ISSRE 1995, Toulouse, France, 24–27 October 1995, pp. 86–95. IEEE (1995). http://dx.doi.org/10.1109/ISSRE.1995.497647
  11. 11.
    Microsoft Research: Pex, Automated White box Testing for .NET. http://research.microsoft.com/en-us/projects/pex/
  12. 12.
    Pnueli, A., Siegel, M.D., Singerman, E.: Translation validation. In: Steffen, B. (ed.) TACAS 1998. LNCS, vol. 1384, pp. 151–166. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  13. 13.
    Samet, H.: Automatically proving the correctness of translations involving optimized code. Memo AIM, Stanford University (1975). https://books.google.ch/books?id=1sI-AAAAIAAJ
  14. 14.
    Varró, D., Pataricza, A.: Automated formal verification of model transformations. In: Jürjens, J., Rumpe, B., France, R., Fernandez, E.B. (eds.) CSDUML 2003: Critical Systems Development in UML; Proceedings of the UML 2003 Workshop, pp. 63–78. No. TUM-I0323 in Technical report, Technische Universitüt München, Technische Universität München, September 2003. http://www.inf.mit.bme.hu/FTSRG/Publications/varro/2003/csduml2003_vp.pdf
  15. 15.
    Zuck, L.D., Pnueli, A., Goldberg, B.: VOC: a methodology for the translation validation of optimizing compilers. J. UCS 9(3), 223–247 (2003). http://dx.doi.org/10.3217/jucs-009-03-0223 Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Stefan Klikovits
    • 1
    • 2
  • David P. Y. Lawrence
    • 1
  • Manuel Gonzalez-Berges
    • 2
  • Didier Buchs
    • 1
  1. 1.Centre Universitaire d’InformatiqueUniversité de GenèveCarougeSwitzerland
  2. 2.CERN, European Organization for Nuclear ResearchGenevaSwitzerland

Personalised recommendations