Testing Web Services with Model-Based Mutation

  • Faezeh SiavashiEmail author
  • Junaid Iqbal
  • Dragos Truscan
  • Jüri Vain
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 743)


One way of evaluating the robustness of a web service is to test it against invalid inputs. We introduce a model-based mutation technique which automatically generates faulty test inputs. From the specification of a Web service, a test model is designed using Uppaal Timed Automata and the conformance between the model and the implementation is validated via online model-based testing with the Uppaal Tron tool. A set of mutation operators is applied to the test model in order to generate mutant test models. We validate all generated mutants via verification rules and select those that are executable and introduce proper mutations. We employ bisimulation as a tool for detecting and eliminating equivalent mutants, that is those mutants which have identical input-output behavior with the original test model. The resulting mutants are used for online test generation against the service implementation in order to check whether the latter allows for unspecified behavior. We discuss tool support and present an experiment of applying our method for a case study of a blog web service with real-life properties. The experiment shows that the proposed approach of mutating the specifications is effective in detecting errors both in the system functionality and in the test model.


Web service Model-based mutation testing Uppaal Tron Bisimulation of Uppaal Timed Automata 


  1. 1.
    Budd, T.A., Gopal, A.S.: Program testing by specification mutation. Comput. Lang. 10, 63–73 (1985)CrossRefGoogle Scholar
  2. 2.
    Siavashi, F., Truscan, D., Vain, J.: On mutating UPPAAL timed automata to assess robustness of web services. In: Maciaszek, L., Cardoso, J., Ludwig, A., Sinderen, M.V., Cabello, E. (eds.) Proceedings of the 11th International Joint Conference on Software Technologies, vol. 1, pp. 15–26. SCITEPRESS-Science and Technology Publications (2016)Google Scholar
  3. 3.
    Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test. Verif. Reliab. 22, 297–312 (2012)CrossRefGoogle Scholar
  4. 4.
    Hessel, A., Larsen, K.G., Mikucionis, M., Nielsen, B., Pettersson, P., Skou, A.: Testing real-time systems using UPPAAL. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. LNCS, vol. 4949, pp. 77–117. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-78917-8_3CrossRefGoogle Scholar
  5. 5.
    Alur, R., et al.: Model-checking for real-time systems. In: Proceedings of Fifth Annual IEEE Symposium on e Logic in Computer Science, LICS 1990, pp. 414–425. IEEE (1990)Google Scholar
  6. 6.
    Behrmann, G., David, A., Larsen, K.G.: A tutorial on Uppaal. In: Bernardo, M., Corradini, F. (eds.) SFM-RT 2004. LNCS, vol. 3185, pp. 200–236. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-30080-9_7CrossRefGoogle Scholar
  7. 7.
    Larsen, K.G., Mikucionis, M., Nielsen, B.: Online testing of real-time systems using Uppaal. In: Grabowski, J., Nielsen, B. (eds.) FATES 2004. LNCS, vol. 3395, pp. 79–94. Springer, Heidelberg (2005). doi: 10.1007/978-3-540-31848-4_6CrossRefzbMATHGoogle Scholar
  8. 8.
    Larsen, K., et al.: Testing real-time embedded software using UPPAAL-TRON: an industrial case study. In: Proceedings of the 5th ACM International Conference on Embedded Software, pp. 299–306. ACM (2005)Google Scholar
  9. 9.
    Belli, F., et al.: Model-based mutation testing approach and case studies. Sci. Comput. Program. 120, 25–48 (2016)CrossRefGoogle Scholar
  10. 10.
    Aboutrab, M., et al.: Specification mutation analysis for validating timed testing approaches based on timed automata. In: 36th Annual IEEE Computer Software and Applications Conference, COMPSAC 2012, Izmir, Turkey, 16–20 July 2012, pp. 660–669 (2012)Google Scholar
  11. 11.
    Aichernig, B.K., Lorber, F., Ničković, D.: Time for mutants — model-based mutation testing with timed automata. In: Veanes, M., Viganò, L. (eds.) TAP 2013. LNCS, vol. 7942, pp. 20–38. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38916-0_2CrossRefGoogle Scholar
  12. 12.
    Gargantini, A., Heitmeyer, C.: Using model checking to generate tests from requirements specifications. In: Nierstrasz, O., Lemoine, M. (eds.) ESEC/SIGSOFT FSE -1999. LNCS, vol. 1687, pp. 146–162. Springer, Heidelberg (1999). doi: 10.1007/3-540-48166-4_10CrossRefGoogle Scholar
  13. 13.
    Bengtsson, J., Yi, W.: Timed automata: semantics, algorithms and tools. In: Desel, J., Reisig, W., Rozenberg, G. (eds.) ACPN 2003. LNCS, vol. 3098, pp. 87–124. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-27755-2_3CrossRefzbMATHGoogle Scholar
  14. 14.
    Čerāns, K.: Decidability of bisimulation equivalences for parallel timer processes. In: Bochmann, G., Probst, D.K. (eds.) CAV 1992. LNCS, vol. 663, pp. 302–315. Springer, Heidelberg (1993). doi: 10.1007/3-540-56496-9_24CrossRefGoogle Scholar
  15. 15.
    Richardson, L., Ruby, S.: RESTful Web Services. O’Reilly, Sebastopol (2008)Google Scholar
  16. 16.
    Grinberg, M.: Flask Web Development: Developing Web Applications with Python. O’Reilly Media, Inc., Sebastopol (2014)Google Scholar
  17. 17.
    Iqbal, J., Truscan, D., Vain, J., Porres, I.: TRON2UPPAAL backtracer tool from TRON logs to UPPAAL traces. Technical report 1138, Turku Centre for Computer Science (2015)Google Scholar
  18. 18.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37, 649–678 (2011)CrossRefGoogle Scholar
  19. 19.
    Lee, S.C., Offutt, J.: Generating test cases for XML-based web component interactions using mutation analysis. In: Proceedings of 12th International Symposium on Software Reliability Engineering, ISSRE 2001, pp. 200–209 (2001)Google Scholar
  20. 20.
    Li, J.H., Dai, G.X., Li, H.H.: Mutation analysis for testing finite state machines. In: Second International Symposium on Electronic Commerce and Security, ISECS 2009, vol. 1, pp. 620–624. IEEE (2009)Google Scholar
  21. 21.
    Lee, S., et al.: Automatic mutation testing and simulation on OWL-S specified web services. In: 41st Annual Simulation Symposium, ANSS 2008, pp. 149–156 (2008)Google Scholar
  22. 22.
    Wang, R., Huang, N.: Requirement model-based mutation testing for web service. In: 4th International Conference on Next Generation Web Services Practices, NWESP 2008, pp. 71–76 (2008)Google Scholar
  23. 23.
    Rusli, H.M., et al.: Testing web services composition: a mapping study. In: Communications of the IBIMA 2011, pp. 34–48 (2007)Google Scholar
  24. 24.
    Bozkurt, M., et al.: Testing web services: a survey. Department of Computer Science, King’s College London, Technical report TR-10-01 (2010)Google Scholar
  25. 25.
    Rauf, I., Siavashi, F., Truscan, D., Porres, I.: An integrated approach for designing and validating REST web service compositions. In: Monfort, V., Krempels, K.H. (eds.) 10th International Conference on Web Information Systems and Technologies, vol. 1, pp. 104–115. SCITEPRESS Digital Library (2014)Google Scholar
  26. 26.
    Cambronero, M.E., et al.: Validation and verification of web services choreographies by using timed automata. J. Logic Algebraic Program. 80, 25–49 (2011)CrossRefGoogle Scholar
  27. 27.
    Dıaz, G., et al.: Model checking techniques applied to the design of web services. CLEI Electron. J. 10, 5–11 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Faezeh Siavashi
    • 1
    Email author
  • Junaid Iqbal
    • 1
  • Dragos Truscan
    • 1
  • Jüri Vain
    • 2
  1. 1.Faculty of Science and EngineeringÅbo Akademi UniversityÅboFinland
  2. 2.Department of Computer ScienceTallinn University of TechnologyTallinnEstonia

Personalised recommendations