Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Fault-based refinement-testing for CSP

  • 147 Accesses

Abstract

The process algebra CSP has been studied as a notation for model-based testing. Theoretical and practical work has been developed using its trace and failure semantics, and their refinement notions as conformance relations. Two sets of tests have been defined and proved to be exhaustive, in the sense that they can identify any SUT that is non-conforming with respect to the relevant refinement relation. However, these sets are usually infinite, and in this case, it is obviously not possible to apply them to verify the conformity of an SUT. Some classical selection criteria based on models have been studied. In this paper, we propose a procedure for online test generation for selection of finite test sets for traces refinement from CSP models. It is based on the notion of fault domains, focusing on the set of faulty implementations of interest. We investigate scenarios where the verdict of a test campaign can be reached after a finite number of test executions. We illustrate the usage of the procedure with some case studies.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    Available at github.com/adenilso/CSP-FD-TGen/blob/master/case-studies/robot.csp.

  2. 2.

    Available at github.com/adenilso/CSP-FD-TGen/blob/master/case-studies/ers.csp.

References

  1. Alberto, A., Cavalcanti, A.L.C., Gaudel, M.-C., Simao, A. (2017). Formal mutation testing for Circus. Information and Software Technology, 81, 131–153.

  2. Andrews, Z., Payne, R., Romanovsky, A., Didier, A., Mota, A. (2013). Model-based development of fault tolerant systems of systems. In 2013 IEEE International Systems Conference (SysCon) (pp. 356–363).

  3. Cavalcanti, A.L.C., & Gaudel, M.-C. (2007). Testing for refinement in CSP. In 9th International Conference on Formal Engineering Methods, volume 4789 of Lecture Notes in Computer Science (pp. 151–170): Springer-Verlag.

  4. Cavalcanti, A.L.C., & Gaudel, M.-C. (2011). Testing for refinement in Circus. Acta Informatica, 48(2), 97–147.

  5. Cavalcanti, A.L.C., & Gaudel, M.-C. (2014). Data flow coverage for Circus-based testing. In Fundamental Approaches to Software Engineering, volume 8441 of Lecture Notes in Computer Science (pp. 415–429).

  6. Cavalcanti, A.L.C., & Gaudel, M.-C. (2015). Test selection for traces refinement. Theoretical Computer Science, 563(0), 1–42.

  7. Cavalcanti, A.L.C., & Hierons, R. M. (2013). Testing with inputs and outputs in CSP. In Fundamental Approaches to Software Engineering, volume 7793 of Lecture Notes in Computer Science (pp. 359–374).

  8. Cavalcanti, A.L.C., & Simão, A. (2017). Fault-based testing for refinement in CSP. In Yevtushenko, N., Cavalli, A. R., Yenigün, H. (Eds.) 29th IFIP WG 6.1 International Conference on Testing Software and Systems, volume 10533 of Lecture Notes in Computer Science (pp. 21–37): Springer.

  9. Cavalcanti, A.L.C., & Woodcock, J.C.P. (1999). ZRC—a refinement calculus for Z. Formal Aspects of Computing, 10(3), 267—289.

  10. Cavalcanti, A.L.C., Sampaio, A.C.A., Woodcock, J.C.P. (2003). A refinement strategy for Circus. Formal Aspects of Computing, 15(2 - 3), 146–181.

  11. Chow, T.S. (1978). Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering, 4(3), 178–187.

  12. Dijkstra, E.W. (1976). A discipline of programming. Upper Saddle River: Prentice-Hall.

  13. El-Fakih, K.A., Dorofeeva, R., Yevtushenko, N.V., Bochmann, G.V. (2012). FSM-based testing from user defined faults adapted to incremental and mutation testing. Programming and Computer Software, 38(4), 201–209.

  14. Fujiwara, S., & von Bochmann, G. (1991). Testing non-deterministic state machines with fault coverage. In: IFIP TC6/WG6.1 4th Int. Wshop on Protocol Test Systems IV, pages 267–280. North-Holland.

  15. Gibson-Robinson, T., Armstrong, P., Boulgakov, A., Roscoe, A.W. (2014). FDR3 — a modern refinement checker for CSP. In Tools and Algorithms for the Construction and Analysis of Systems (pp. 187–201).

  16. Hierons, R.M. (2002). Comparing test sets and criteria in the presence of test hypotheses and fault domains. ACM Transactions on Software Engineering and Methodology, 11(4), 427–448.

  17. Hierons, R.M., & Ural, H. (2006). Optimizing the length of checking sequences. IEEE Trans. on Computers, 55(5), 618–629.

  18. Huang, W.-L., & Peleska, J. (2013). Exhaustive model-based equivalence class testing. In: Testing software and systems - 25th IFIP WG 6.1 International Conference, ICTSS 2013, Istanbul, Turkey, November 13-15, 2013 Proceedings, pp. 49–64.

  19. Koufareva, I., Petrenko, A., Yevtushenko, N. (1999). Test generation driven by user-defined fault models. In: Testing of communicating systems method and applications, IFIP TC6 12th International Workshop on Testing Communicating Systems, September 1-3, 1999, Budapest, Hungary, pp 215–236.

  20. Luo, G., Bochmann, G.V., Petrenko, A. (1994). Test selection based on communicating nondeterministic finite-state machines using a generalized Wp-method. IEEE Transactions on Software Engineering, 20(2), 149–162.

  21. Milner, A.J.R.G. (1980). A calculus of communicating systems, volume 92. Springer Verlag.

  22. Moraes, A., de L. Andrade, W., Machado, P.D.L. (2016). A family of test selection criteria for timed input-output symbolic transition system models. Sci Comput. Program., 126, 52–72.

  23. Morell, L.J. (Aug 1990). A theory of fault-based testing. IEEE Transactions on Software Engineering, 16(8), 844–857.

  24. Morgan, C.C. (1994). Programming from specifications. Prentice-hall 2nd edition.

  25. Mota, A., Farias, A., Didier, A., Woodcock, J. (2014). Rapid prototyping of a semantically well founded Circus model checker. In: Software engineering and formal methods, volume 8702 of LNCS, pages 235–249. Springer.

  26. Nogueira, S., Sampaio, A.C.A., Mota, A.C. (2014). Test generation from state based use case models. Formal Aspects of Computing, 26(3), 441–490.

  27. Peleska, J., & Siegel, M. (1996). Test automation of safety-critical reactive systems. In: Formal methods Europe, industrial benefits and advances in formal methods, volume 1051 of lecture notes in computer science.

  28. Petrenko, A., & Yevtushenko, N. (2005). Testing from partial deterministic FSM specifications. IEEE Trans. on Computers, 54(9).

  29. Petrenko, A., Bochmann, G.V., Yao, M.Y. (1996). On fault coverage of tests for finite state specifications. Computer Networks and ISDN Systems, 29(1), 81–106.

  30. Probert, P.J., Djian, D., Hu, H. (1991). Transputer architectures for sensing in a robot controller formal methods for design. Concurrency: Practice and Experience, 3(4), 283–292.

  31. Qin, S., Dong, J.S., Chin, W.N. (2003). A semantic foundation for TCOZ in unifying theories of programming. In Araki, K., Gnesi, S., Mandrioli, D. (Eds.) FME2003: Formal Methods, volume 2805 of lecture notes in computer science (pp. 321–340).

  32. Roscoe, A.W. (2011). Understanding concurrent systems. Texts in computer science. Springer.

  33. Schneider, S. (2000). Concurrent and real-time systems: the CSP approach. Wiley.

  34. Schneider, S., & Treharne, H. (2002). Communicating B machines. In Bert, D., Bowen, J., Henson, M., Robinson, K. (Eds.) ZB’2002: Formal specification and development in Z and B, volume 2272 of lecture notes in computer science (pp. 416–435).

  35. Tretmans, J. (1996). Test generation with inputs, outputs, and quiescence. In TACAS’96, volume 1055 of LNCS, pages 127–146: Springer.

  36. Weiglhofer, M., Aichernig, B.K., Wotawa, F. (2009). Fault-based conformance testing in practice. Int. J Software and Informatics, 3(2-3), 375–411.

  37. Woodcock, J.C.P., & Davies, J. (1996). Using Z—specification, refinement, and proof. Prentice-Hall.

  38. Yu, Y.T., & Lau, M.F. (2012). Fault-based test suite prioritization for specification-based testing. Information and Software Technology, 54(2), 179–202.

Download references

Acknowledgements

The authors are thankful to Marie-Claude Gaudel, for useful discussions in an early version of this paper. No new primary data was generated.

Funding

The authors would like to thank financial support from Royal Society (Grant NI150186), FAPESP (Grant 2013/07375-0), EPSRC (Grants EP/M025756/1 and EP/R025134/1), and the Royal Academy of Engineering.

Author information

Correspondence to Ana Cavalcanti.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Refinement laws

Appendix: Refinement laws

Law

altI Alternation introduction

w : [pre, post]

\(\sqsubseteq altI\)

ifigi & w : [gipre, post] fi

provided \(\mathit {pre} \Rightarrow (\bigvee i \bullet g_{i})\)

Syntactic restrictions:

  • Each gi is a well-scoped predicate.

  • No gi has free dashed variables.

  • {igi} is non-empty.

Law

assigI Assignment introduction

w, VCL : [pre, post]

\(\sqsubseteq \mathit {assigI}\)

VCL := el

provided \(\mathit {pre} \Rightarrow \mathit {post}[el/vl^{\prime }][\_/^{\prime }]\)

Syntactic restrictions:

  • vl contains no duplicated variables.

  • vl and el have the same length.

  • el is well-scoped and well-typed.

  • el has no free dashed variables.

  • The corresponding variables of vl and expressions of el have the same type.

Law

cfR Contract frame

w, x : [pre, post]

\(\sqsubseteq ~cfR\)

x : [pre, post[w/w]]

Syntactic restrictions The variables of w are not in x.

Law

fassigI Following assignment introduction

w, VCL : [pre, post]

\(\sqsubseteq \mathit {fassigI}\)

w, VCL : [pre, post[el[w,vl/w, VCL]/VC]]; VCL := el

Syntactic restrictions:

  • vl contains no duplicated variables.

  • vl and el have the same length.

  • el is well-scoped and well-typed.

  • el has no free dashed variables.

  • The corresponding variables of vl and expressions of el have the same type.

Law

itI Iteration introduction

\(w :[inv,inv[w^{\prime }/w] \land \lnot (\bigvee i \bullet g_{i}[w^{\prime }/w])]\)

\(\sqsubseteq \mathit {itI}\)

doigi&w : [invgi,inv[w/w] ∧ 0 ≤ vrt[w/w] < vrt] od

Syntactic restrictions:

  • vrt is a well-scoped and well-typed integer.

  • Each gi and vrt have no free dashed variables. expression.

Law

vrbI Variable introduction

w : [pre, post]

= vrbI

|[vardvlVCL, w : [pre, post]]|

wheredvl declares the variables of vl.

Syntactic restrictions:

  • dvl is well-scoped and well-typed.

  • The variables of vl and vl are not free in w : [pre, post] and are not dashed.

Law

seqcI Sequential composition introduction

w, x : [pre, post]

\(\sqsubseteq \mathit {seqcI}\)

w : [pre, mid[w/w]]; w, x : [mid, post]

Syntactic restrictions:

  • mid is well-scoped and well-typed.

  • mid has no free dashed variables.

  • No free variable of post is in w.

Law

seqcI Sequential composition introduction

w, x, y!, z! : [pre, post]

\(\sqsubseteq \mathit {seqcI}\)

\(|[ \textbf {con} \mathit {dcl} \bullet w,y!:[\mathit {pre},\mathit {mid}];\vspace *{2pt} w,x,y!,z! :[\mathit {mid}[cl/w] [\_/^{\prime }],\mathit {post}[cl/w] ] ]|\)

wheredcl declares the constants of cl.

Syntactic restrictions:

  • mid is well-scoped and well-typed.

  • The names of cl and cl are not free in mid and w, x, y!,z! : [pre, post].

  • cl and w have the same length.

  • The constants of cl have the same type as the corresponding variables of w.

Law

sP Strengthen postcondition

w : [pre, post]

\(\sqsubseteq \mathit {sP}\)

w : [pre, npost]

providedprenpostpost

Syntactic restrictionsnpost is well-scoped and well-typed.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Cavalcanti, A., Simao, A. Fault-based refinement-testing for CSP. Software Qual J 27, 529–562 (2019). https://doi.org/10.1007/s11219-018-9431-9

Download citation

Keywords

  • Formal testing
  • Process algebra
  • Test generation