Abstract
Mutation Testing is an error-based criterion that provides mechanisms to evaluate the quality of a test set and/or to generate test sets. This criterion, originally proposed to program testing, has also been applied to specification testing. In this paper, we propose the application of Mutation Testing for testing SDL specifications. We define a mutant operator set for SDL that intends to model errors related to the behavioral aspect of the processes, the communication among processes, the structure of the specification and some intrinsic characteristics of SDL. A testing strategy to apply the mutant operators to test SDL specifications is proposed. We illustrate our approach using the Alternating-Bit protocol.
Chapter PDF
Similar content being viewed by others
References
Agrawal, H.: Design of mutant operators for the C programming language. Technical Report SERC-TR-41-P, Software Engineering Research Center/Purdue University (1989)
Ammann, P., Black, P.: A specification-based coverage metric to evaluate test sets. In: Proceedings of Fourth IEEE International High-Assurance Systems Engineering Symposium (HASE 1999), pp. 239–248. IEEE Computer Society, Los Alamitos (1999)
Ammann, P., Black, P., Majurski, W.: Using model checking to generate tests from specifications. In: Proceedings of 2nd IEEE International Conference on Formal Engineering Methods, Brisbane, Australia, pp. 46–54. IEEE Computer Society, Los Alamitos (1998)
Barbosa, E.F., Maldonado, J.C., Vincenzi, A.M.R.: Toward the determination of sufficient mutant operators for C. Software Testing, Verification and Reliability Journal 11(2), 113–136 (2001)
Black, P.E., Okun, V., Yesha, Y.: Mutation operators for specifications. In: Proceedings of 15th IEEE International Conference on Automated Software Engineering (ASE 2000), pp. 81–89 (2000)
Bousquet, L.D., Ramangalahy, S., Simon, S., Viho, C., Belinfante, A., Vries, R.G.: Formal test automation: The conference protocol with TGV/TORX. In: Ural, H., Probert, R.L., von Bochmann, G. (eds.) IFIP 13th International Conference on Testing of Communicating Systems(TestCom 2000), Kluwer Academic Publishers, Dordrecht (2000)
Bromstrup, L., Hogrefe, D.: TESDL: Experience with generating test cases from SDL specifications. In: Proceedings of Fourth SDL Forum, pp. 267–279 (1989)
Chow, T.S.: Testing software design modeled by Finite-State Machines. IEEE Transactions on Software Engineering 4(3), 178–187 (1978)
Delamaro, M.E., Maldonado, J.C.: Proteum: A tool for the assessment of test adequacy for C programs. In: Conference on Performability in Computing Systems, Brunswick, NJ, pp. 79–95 (1996)
Delamaro, M.E., Maldonado, J.C., Mathur, A.P.: Interface mutation: An approach for integration testing. IEEE Transactions on Software Engineering 27(3), 228–247 (2001)
DeMillo, R.A.: Mutation analysis as a tool for software quality assurance. In: Proceedings of COMPSAC 1980, Chicago, IL (1980)
DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: Help for the practicing programmer. IEEE Computer 11(4), 34–41 (1978)
DeMillo, R.A., Offutt, A.J.: Constraint-based automatic test data generation. IEEE Transactions on Software Engineering 17(9), 900–910 (1991)
Fabbri, S.C.P.F., Maldonado, J.C., Delamaro, M.E., Masiero, P.C.: Proteum/FSM: A tool to support Finite State Machine validation based on mutation testing. In: Proceedings of XIX SCCC - International Conference of the Chilean Computer Science Society, Talca, Chile, pp. 96–104 (1999a)
Fabbri, S.C.P.F., Maldonado, J.C., Masiero, P.C.: Mutation analysis in the context of reactive system specification and validation. In: 5th Annual International Conference on Software Quality Management, Bath, UK, pp. 247–258 (1997)
Fabbri, S.C.P.F., Maldonado, J.C., Masiero, P.C., Delamaro, M.E.: Mutation analysis testing for Finite State Machines. In: Proceedings of ISSRE 1994 - Fifth International Symposium on Software Reliability Engineering, Monterey, California, USA, pp. 220–229 (1994)
Fabbri, S.C.P.F., Maldonado, J.C., Masiero, P.C., Delamaro, M.E., Wong, E.: Mutation testing applied to validate specifications based on Petri nets. In: Proceedings of FORTE 1995 - 8th International IFIP Conference on Formal Description Techniques for Distributed Systems and Communications Protocol, Montreal, Canada, pp. 329–337 (1995)
Fabbri, S.C.P.F., Maldonado, J.C., Sugeta, T., Masiero, P.C.: Mutation testing applied to validate specifications based on Statecharts. In: ISSRE— International Symposium on Software Reliability Systems, Boca Raton, Flórida, EUA, pp. 210–219 (1999b)
Ghosh, S., Mathur, A.P.: Interface mutation. In: Mutation 2000 - A Symposium on Mutation Testing for the New Century, San José, California, pp. 112–123 (2000)
Kovács, G., Pap, Z., Viet, D.L., Wu-Hen-Chang, A., Csopaki, G.: Applying mutation analysis to sdl specifications. In: Reed, R., Reed, J. (eds.) SDL 2003. LNCS, vol. 2708, pp. 269–284. Springer, Heidelberg (2003)
Luo, G., Das, A., Bochmann, G.: Software test selection based on SDL specification with save. In: Proceedings of 5th SDL Forum, pp. 313–324. Elsevier, Glasgow (1991)
Maldonado, J.C., Delamaro, M.E., Fabbri, S.C.P.F., Simao, A.S., Sugeta, T., Vincenzi, A.M.R., Masiero, P.C.: Proteum: A family of tools to support specification and program testing based on mutation. In: Mutation 2000 - A Symposium on Mutation Testing for the New Century, San José, California, pp. 146–149 (2000)
Offutt, A.J., Lee, A., Rothermel, G., Untch, R.H., Zapf, C.: An experimental determination of sufficient mutant operators. ACMT ransactions on Software Engineering Methodology 5(2), 99–118 (1996)
Offutt, A.J., Pan, J.: Automatically detecting equivalent mutants and infeasible paths. The Journal of Software Testing, Verification, and Reliability 7(3), 165–192 (1997)
Petrenko, A., Bochmann, G.: On fault coverage of tests for Finite State specifications. Technical report, Département d’Informatique et de Recherche Opérationnelle, Université de Montreal (1996)
Probert, R.L., Guo, F.: Mutation testing of protocols: Principles and preliminary experimental results. In: Proceedings of the IFIP TC6 Third International Workshop on Protocol Teste Systems, pp. 57–76. North-Holland, Amsterdam (1991)
Simão, A.S., Maldonado, J.C.: Mutation based test sequence generation for Petri nets. In: Proceedings of III Workshop of Formal Methods, João Pessoa, PB, pp. 68–79 (2000)
Simão, A.S., Maldonado, J.C., Fabbri, S.C.P.F.: Proteum-RS/PN: A tool to support edition, simulation and validation of Petri nets based on mutation testing. In: Proceedings of XIV Brazilian Symposium of Software Engineering, João Pessoa, PB, Brazil, pp. 227–242 (2000)
Souza, S.R.S., Maldonado, J.C., Fabbri, S.C.P.F., Lopes de Souza, W.: Mutation testing applied to Estelle specifications. Quality Software Journal 8(4), 285–301 (2000) (Also published in the 33rdHawaii Internacional Conference on System Sciences (2000).
Sugeta, T., Maldonado, J.C., Masiero, P.C., Fabbri, S.C.P.F.: Proteum-RS/ST – A Tool to Support Statecharts Validation Based on Mutation Testing. In: 4th Workshop Iberoamericano de Engenharia de Requisitos e Ambientes de Software - IDEAS 2001, Santo Domingo, Costa Rica, pp. 370–384 (2001)
Tanenbaum, A.S.: Computer Networks, 3rd edn. Prentice Hall, Englewood Cliffs (1996)
Ural, H., Saleh, K., Williams, A.: Test generation based on control and data dependencies within system specifications in SDL. Computer Communications 23(7), 609–627 (2000)
Weyuker, E.J., Goradia, T., Singh, A.: Automatically generating test data from a boolean specification. IEEE Transactions on Software Engineering 20(5), 353–363 (1994)
Wong, W.E., Maldonado, J.C., Mathur, A.P.: Mutation versus alluses: An empirical evaluation of cost, strength and effectiveness. In: First IFIP/SQI International Conference on Software Quality and Productivity – Theory, Practice, Education and Training, Hong Kong (1994)
Wong, W.E., Sugeta, T., Li, J.J., Maldonado, J.C.: Coverage testing software architectural design in SDL. Computer Networks 42(3), 359–374 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 IFIP International Federation for Information Processing
About this paper
Cite this paper
Sugeta, T., Maldonado, J.C., Wong, W.E. (2004). Mutation Testing Applied to Validate SDL Specifications. In: Groz, R., Hierons, R.M. (eds) Testing of Communicating Systems. TestCom 2004. Lecture Notes in Computer Science, vol 2978. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24704-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-540-24704-3_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21219-5
Online ISBN: 978-3-540-24704-3
eBook Packages: Springer Book Archive