Security Mutants for Property-Based Testing

  • Matthias Büchler
  • Johan Oudinet
  • Alexander Pretschner
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6706)


The last decade has witnessed impressive progress in terms of dedicated approaches to formally analyzing security properties of models. However, related approaches to generating tests generally rely on purely syntactic test selection criteria. In this paper, we consider models of protocols and describe an approach to generate tests from security properties. Security-specific mutation operators are defined and used to introduce potential security-specific leaks into the model. Then, if the leak is confirmed by a model analyzer, a test case for the security property is generated. We present examples for security-relevant mutants at the model level and show how they correspond to security-flawed implementations, thus providing evidence that model-level mutants are indeed useful for doing security testing.


Property-based testing Security protocols Mutation Test generation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ammann, P., Ding, W., Xu, D.: Using a model checker to test safety properties. In: International Conference on Engineering of Complex Computer Systems, pp. 212–221. IEEE, Los Alamitos (2001)Google Scholar
  2. 2.
    Ammann, P.E., Black, P.E., Majurski, W.: Using model checking to generate tests from specifications. In: ICFEM, pp. 46–54. IEEE, Los Alamitos (1998)Google Scholar
  3. 3.
    Armando, A., Basin, D., Boichut, Y., Chevalier, Y., Compagna, L., Cuellar, J., Drielsma, P.H., Heám, P.C., Kouchnarenko, O., Mantovani, J., Mödersheim, S., von Oheimb, D., Rusinowitch, M., Santiago, J., Turuani, M., Viganò, L., Vigneron, L.: The AVISPA tool for the automated validation of internet security protocols and applications. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 281–285. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Brucker, A.D., Brügger, L., Wolff, B.: Model-based firewall conformance testing. In: TestCom/FATES, pp. 103–118 (2008)Google Scholar
  5. 5.
    Dadeau, F., Héam, P.-C., Kheddam, R.: Mutation-based test generation from security protocols in HLPSL. In: ICST, p. 9. IEEE, Los Alamitos (2011)Google Scholar
  6. 6.
    Darmaillacq, V., Fernandez, J.-C., Groz, R., Mounier, L., Richier, J.-L.: Test generation for network security rules. In: Uyar, M.Ü., Duale, A.Y., Fecko, M.A. (eds.) TestCom 2006. LNCS, vol. 3964, pp. 341–356. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Program Mutation: A New Approach to Program Testing. In: Infotech State of the Art Report, Software Testing, pp. 107–126 (1979)Google Scholar
  8. 8.
    Fraser, G., Wotawa, F.: Complementary criteria for testing temporal logic properties. In: TAP, pp. 58–73 (2009)Google Scholar
  9. 9.
    Fraser, G., Wotawa, F., Ammann, P.: Testing with model checkers: a survey. Softw. Test., Verif. Reliab. 19(3), 215–261 (2009)CrossRefGoogle Scholar
  10. 10.
    García-Alfaro, J., Cuppens, F., Cuppens-Boulahia, N.: Towards filtering and alerting rule rewriting on single-component policies. In: Górski, J. (ed.) SAFECOMP 2006. LNCS, vol. 4166, pp. 182–194. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Goubault-Larrecq, J., Parrennes, F.: Cryptographic protocol analysis on real C code. In: Cousot, R. (ed.) VMCAI 2005. LNCS, vol. 3385, pp. 363–379. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Jia, Y., Harman, M.: An Analysis and Survey of the Development of Mutation Testing (2011); To appear in IEEE TSEGoogle Scholar
  13. 13.
    LeTraon, Y., Mouelhi, T., Baudry, B.: Testing security policies: Going beyond functional testing. In: ISSRE, pp. 93–102 (2007)Google Scholar
  14. 14.
    Mallouli, W., Morales, G., Cavalli, A.: Testing security policies for web applications. In: Proc. 1st Workshop on Security Testing (2008)Google Scholar
  15. 15.
    Martin, F., Xie, T.: A fault model and mutation testing of access control policies. In: Proc. 16th Intl. Conf. on the World Wide Web, pp. 667–676 (2007)Google Scholar
  16. 16.
    Mouelhi, T., LeTraon, Y., Baudry, B.: Mutation analysis for security tests qualification. In: Proc. 3rd Workshop on Mutation Analysis, pp. 233–242 (2007)Google Scholar
  17. 17.
    Pretschner, A., Mouelhi, T., LeTraon, Y.: Model-based tests for access control policies. In: ICST, pp. 338–347 (2008)Google Scholar
  18. 18.
    Senn, D., Basin, D.A., Caronni, G.: Firewall conformance testing. In: Khendek, F., Dssouli, R. (eds.) TestCom 2005. LNCS, vol. 3502, pp. 226–241. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  19. 19.
    The AVISPA Team. AVISPA User Manual, 1.1 edn. (2006),
  20. 20.
    Voas, J., McGraw, G.: Software Fault Injection: Innoculating Programs Against Errors. John Wiley & Sons, Chichester (1997)Google Scholar
  21. 21.
    Weiglhofer, M., Fraser, G., Wotawa, F.: Using coverage to automate and improve test purpose based testing. In: INFSOF, vol. 51(11), pp. 1601–1617 (2009)Google Scholar
  22. 22.
    Whalen, M.W., Rajan, A., Per Erik Heimdahl, M., Miller, S.P.: Coverage metrics for requirements-based testing. In: ISSTA, pp. 25–36 (2006)Google Scholar
  23. 23.
    Wimmel, G., Jürjens, J.: Specification-based test generation for security-critical systems using mutations. In: George, C.W., Miao, H. (eds.) ICFEM 2002. LNCS, vol. 2495, pp. 471–482. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  24. 24.
    Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Comput. Surv. 29, 366–427 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Matthias Büchler
    • 1
  • Johan Oudinet
    • 1
  • Alexander Pretschner
    • 1
  1. 1.Karlsruhe Institute of TechnologyKarlsruheGermany

Personalised recommendations