Model-Based Testing from Controlled Natural Language Requirements
Model-Based Testing (MBT) techniques usually take as input models that are not available in the very beginning of a development. Therefore, its use is postponed. In this work we present an approach to MBT that takes as input requirements described in a Controlled Natural Language. Initially, the requirements are syntactically analyzed according to a domain specific language for describing system requirements, and their informal semantics is depicted based on the Case Grammar theory. Then, the requirements semantics is automatically represented as a Transition Relation, which provides formal basis for MBT, and test cases are generated with the support of a solver. Our approach was evaluated considering four examples from different domains. Within seconds, our approach generated 94 % of the test vectors manually written by specialists. Moreover, considering a mutant-based strength analysis, our approach yielded a mutation score between 54 % and 98 %.
KeywordsNatural language Case grammar Solver Test case
We thank Augusto Sampaio for his valuable advice. This work has been partially funded by the EU FP7 COMPASS project (no.287829).
- 1.Allen, J.: Natural Language Understanding. Benjamin/Cummings, San Francisco (1995)Google Scholar
- 2.Andrews, J.H., Briand, L.C., Labiche, Y.: Is mutation an appropriate tool for testing experiments? In: International Conference on Software Engineering, pp. 402–411. ACM, New York (2005)Google Scholar
- 3.Boddu, R., Guo, L., Mukhopadhyay, S., Cukic, B.: RETNA: from requirements to testing in a natural way. In: International Requirements Engineering (2004)Google Scholar
- 4.Brottier, E., Baudry, B., Traon, Y.L., Touzet, D., Nicolas, B.: Producing a global requirement model from multiple requirement specifications. In: International Enterprise Distributed Object Computing Conference, pp. 390–404. USA (2007)Google Scholar
- 5.Carvalho, G., Falcão, D., Barros, F., Sampaio, A., Mota, A., Motta, L., Blackburn, M.: Test case generation from natural language requirements based on SCR specifications. In: ACM Symposium on Applied Computing (2013)Google Scholar
- 7.Larsen, K., Mikucionis, M., Nielsen, B.: Online testing of real-time systems using UPPAAL: status and future work. In: Dagstuhl Seminar Proceedings volume 04371: Perspectives of Model-Based Testing (2004)Google Scholar
- 10.Nogueira, S., Sampaio, A., Mota, A.: Test generation from state based use case models. Formal Aspects Comput. 1, 1–50 (2012)Google Scholar
- 11.Peleska, J., Honisch, A., Lapschies, F., Löding, H., Schmid, H., Smuda, P., Vorobev, E., Zahlten, C.: A real-world Benchmark model for testing concurrent real-time systems in the automotive domain. In: Wolff, B., Zaïdi, F. (eds.) ICTSS 2011. LNCS, vol. 7019, pp. 146–161. Springer, Heidelberg (2011)Google Scholar
- 12.Peleska, J., Vorobev, E., Lapschies, F., Zahlten, C.: Automated model-based testing with RT-Tester. Universität Bremen, Technical report (2011)Google Scholar
- 14.Schnelte, M.: Generating test cases for timed systems from controlled natural language specifications. In: International Conference on System Integration and Reliability Improvements, pp. 348–353 (2009)Google Scholar
- 15.Sinha, A., Suttan Jr., S.M., Paradkar, A.: Text2Test: automated inspection of natural language use cases. International Conference on Software Testing, Verification and Validation, pp. 155–164. IEEE Computer Society, Washington (2010)Google Scholar
- 16.Sneed, H.: Testing against natural language requirements. In: International Conference on Quality Software, pp. 380–387 (2007)Google Scholar