Software Quality Journal

, Volume 16, Issue 2, pp 185–211 | Cite as

Using communication coverage criteria and partial model generation to assist software integration testing

  • Christopher Robinson-Mallett
  • Robert M. Hierons
  • Jesse Poore
  • Peter Liggesmeyer


This paper considers the problem of integration testing the components of a timed distributed software system. We assume that communication between the components is specified using timed interface automata and use computational tree logic (CTL) to define communication-based coverage criteria that refer to send- and receive-statements and communication paths. The proposed method enables testers to focus during component integration on such parts of the specification, e.g. behaviour specifications or Markovian usage models, that are involved in the communication between components to be integrated. A more specific application area of this approach is the integration of test-models, e.g. a transmission gear can be tested based on separated models for the driver behaviour, the engine condition, and the mechanical and hydraulical transmission states. Given such a state-based specification of a distributed system and a concrete coverage goal, a model checker is used in order to determine the coverage or generate test sequences that achieve the goal. Given the generated test sequences we derive a partial test-model of the components from which the test sequences were derived. The partial model can be used to drive further testing and can also be used as the basis for producing additional partial models in incremental integration testing. While the process of deriving the test sequences could suffer from a combinatorial explosion, the effort required to generate the partial model is polynomial in the number of test sequences and their length. Thus, where it is not feasible to produce test sequences that achieve a given type of coverage it is still possible to produce a partial model on the basis of test sequences generated to achieve some other criterion. As a result, the process of generating a partial model has the potential to scale to large industrial software systems. While a particular model checker, UPPAAL, was used, it should be relatively straightforward to adapt the approach for use with other CTL based model checkers. A potential additional benefit of the approach is that it provides a visual description of the state-based testing of distributed systems, which may be beneficial in other contexts such as education and comprehension.


Integration Testing Distributed systems Coverage criteria Timed state-based specifications 


  1. Aizenbud-Reshef, N. (2001). Coverage Analysis for Message Flows, 12th International Symposium on Software Reliability Engineering (ISSRE 2001), pp. 276–286. IEEE.Google Scholar
  2. Alur, R., & Dill, L. (1994). A theory of timed automata. Theoretical Computer Science, 126(2), 183–235.MATHCrossRefMathSciNetGoogle Scholar
  3. Bengtsson, J., Larsen, K. G., Larsson, F., Pettersson, P., & Yi, W. (1995). UPPAAL—A Tool Suite for Automatic Verification of Real-Time Systems. Workshop on Verification and Control of Hybrid Systems, DIMACS.Google Scholar
  4. Bochmann, G. v., & Petrenko, A. (1994). Protocol testing: Review of methods and relevance for software testing. Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 1994), pp. 109–124. ACM Press.Google Scholar
  5. Cardell-Oliver, R. (2002). Conformance test experiments for distributed real-time systems. Proceedings of the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2002), pp. 159–163. ACM.Google Scholar
  6. Clarke, E. M., & Emerson, E. A. (1981). Design and synthesis of synchronization skeletons using branching time temporal logic. Proceedings of the Workshop on Logic of Programs. Yorktown Heights, NY, LNCS 131 pp. 52–71., Springer Press.Google Scholar
  7. Clarke, E. M., Grumberg, O., & Peled, D. A. (2000). Model Checking. Boston: MIT Press.Google Scholar
  8. de Alfaro, L., & Henzinger, T. A. (2001). Interface Automata. Proceedings of the 8th European Software Engineering Conference (ESEC 2001). pp. 109–120. ACM.Google Scholar
  9. de Alfaro, L., Henzinger, T. A., & Stroelinga, M. (2002). Timed interfaces. Proceedings of the Second International Conference on Embedded Software, (EMSOFT 2002), 2491 pp. 108–122. LNCS. Springer.Google Scholar
  10. Glasser, U., Gurevich, Y., & Veanes, M. (2004). Abstract communication model for distributed systems. IEEE Transactions on Software Engineering, 30(7), 458–472.CrossRefGoogle Scholar
  11. Harrold, M. J., & Rothermel, G. (1994). Performing dataflow testing on classes, Proceedings of the Symposium Foundations of Software Engineering, ACM.Google Scholar
  12. Hong, H. S., Cha, S. D., Lee, I., Sokolsky, O., & Ural, H. (2003). Data Flow Testing as Model Checking, Proceedings of International Conference on Software Engineering (ICSE ‘03), pp. 232–242, May 2003.Google Scholar
  13. Hong, H., Lee, I., Sokolsky, O., & Ural, H. (2002). A Temporal Logic Based Theory of Test Coverage and Generation, International Conference on Tools and Algorithms for Construction and Analysis of Systems (TACAS2002), April 8–11.Google Scholar
  14. Huhn, M., & Mücke, T. (2004). Generation of Optimized Test suites for UML Statecharts with Time. Testing of Communicating Systems (TestCom’04). Oxford: Springer.Google Scholar
  15. Khoumsi, A. (2002). A temporal approach for testing distributed systems. IEEE Transactions on Software Engineering, 28(11), 1085–1103.CrossRefGoogle Scholar
  16. Liu, W., & Dasiewicz, P. (2001). Component Interaction Testing Using Model Checking, Canadian Conference on Electrical and Computer Engineering, 1 pp. 41–46, IEEE.Google Scholar
  17. Moore, E. F. (1956). Gedanken-Experiments on Sequential Machines. Automata Studies (Annals of Mathematics Studies), 34.Google Scholar
  18. Offutt, A. J., Xiong, Y., & Liu, S. (1999). Criteria for Generating Specification-based Tests, Proceedings of 5th International Conference on Engineering of Complex Computer Systems, ACM.Google Scholar
  19. Prowell, S. J. (2003). JUMBL: A Tool for Model-Based Statistical Testing, Proceedings of the 36th Hawaii International Conference on System Sciences, IEEE.Google Scholar
  20. Prowell, S. J., & Poore, J. H. (2004). Computing system reliability using Markov chain usage models, Journal of Systems and Software, 73(2) pp. 219–225, Elsevier.Google Scholar
  21. Robinson-Mallett, C., Hierons, R. M., & Liggesmeyer, P., (2006a). Achieving Communication Coverage Criteria in Testing, Workshop on Advances in Model-Based Testing 2006. Raleigh, NC.Google Scholar
  22. Robinson-Mallett C., Mücke T., Liggesmeyer P., Goltz U. (2006b) Extended State Identification and Verification using a Model Checker. Journal on Information and Software Technology, 48 (10) pp. 981−992. Elsevier.Google Scholar
  23. Robinson-Mallett C., Hierons, R. M., Poore J., & Bauer T. (2007). Using Partial Models to support the Testing of Distributed Systems, IASTED International Conference on Software Engineering and Applications (SEA 2007). Boston:IASTED.Google Scholar
  24. Robinson-Mallett, C., Mücke, T., Liggesmeyer, P., & Goltz, U. (2005). Generating Optimal Distinguishing Sequences with a Model Checker. Workshop on Advances in Model-Based Software Testing (A-MOST’05). St. Louis.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Christopher Robinson-Mallett
    • 1
  • Robert M. Hierons
    • 2
  • Jesse Poore
    • 3
  • Peter Liggesmeyer
    • 4
  1. 1.Berner & Mattner Systemtechnik GmbHBerlinGermany
  2. 2.Brunel UniversityUxbridgeUK
  3. 3.University of TennesseeKnoxvilleUSA
  4. 4.University of KaiserslauternKaiserslauternGermany

Personalised recommendations