Testing abstract behavioral specifications

  • Peter Y. H. WongEmail author
  • Richard Bubel
  • Frank S. de Boer
  • Miguel Gómez-Zamalloa
  • Stijn de Gouw
  • Reiner Hähnle
  • Karl Meinke
  • Muddassar Azam Sindhu
Regular Paper


We present a range of testing techniques for the Abstract Behavioral Specification (ABS) language and apply them to an industrial case study. ABS is a formal modeling language for highly variable, concurrent, component-based systems. The nature of these systems makes them susceptible to the introduction of subtle bugs that are hard to detect in the presence of steady adaptation. While static analysis techniques are available for an abstract language such as ABS, testing is still indispensable and complements analytic methods. We focus on fully automated testing techniques including black-box and glass-box test generation as well as runtime assertion checking, which are shown to be effective in an industrial setting.


Automated testing Industrial case study Black-box testing  Glass-box testing  Runtime assertion checking 



We thank Andreas Kohn for proof reading this paper and anonymous referees for their useful suggestions and comments.


  1. 1.
    Albert, E., Arenas, P., Gómez-Zamalloa, M.: Towards testing concurrent objects in CLP. In: Agostino Dovier and Vítor Santos Costa, editors, Technical Communications of the 28th International Conference on Logic Programming (ICLP’12), vol. 17 of Leibniz International Proceedings in Informatics (LIPIcs), pp. 98–108, Dagstuhl, Germany. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2012)Google Scholar
  2. 2.
    Albert, E., Gómez-Zamalloa, M., Rojas, J.M., Puebla, G.: Compositional CLP-based test data generation for imperative languages. In: LOPSTR 2010 Revised Selected Papers, vol. 6564 of LNCS. Springer-Verlag (2011)Google Scholar
  3. 3.
    Allan, C., Avgustinov, P., Christensen, A.S., Hendren, L., Kuzins, S., Lhoták, O., de Moor, O., Sereni, D., Sittampalam, G., Tibble, J.: Adding trace matching with free variables to AspectJ. In: Proceedings of the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, OOPSLA ’05, pp. 345–364. ACM, New York (2005)Google Scholar
  4. 4.
    Bartetzko, Detlef, Fischer, Clemens, Möller, Michael, Wehrheim, Heike: Jass-Java with assertions. Electron. Notes Theor. Comput. Sci. 55(2), 103–117 (2001)CrossRefGoogle Scholar
  5. 5.
    Chen, F., Roşu, G.: Mop: an efficient and generic runtime verification framework. In: Proceedings of the 22nd annual ACM SIGPLAN conference on Object-oriented programming systems and applications, OOPSLA ’07, pp. 569–588. ACM, New York (2007)Google Scholar
  6. 6.
    Cimatti, A., Clarke, E.M., Giunchiglia, F., Roveri, M.: NuSMV: a new symbolic model verifier. In: Proceedings of CAV 1999, vol. 1633 of LNCS (1999)Google Scholar
  7. 7.
    Clarke, D., Diakov, N., Hähnle, R., Johnsen, E.B., Schaefer, I., Schäfer, J., Schlatte, R., Wong, P.Y.H.: Modeling spatial and temporal variability with the HATS abstract behavioral modeling language. In: Bernardo, M., Issarny, V. (eds.) ormal Methods for Eternal Networked Software Systems, vol. 6659 of Lecture Notes in Computer Science, pp. 417–457. Springer-Verlag, Berlin (2011)Google Scholar
  8. 8.
    Colombo, C., Pace, G.J., Schneider, G.: LARVA–safer monitoring of real-time Java Programs (Tool Paper). In: Proceedings of the 2009 Seventh IEEE International Conference on Software Engineering and Formal Methods, SEFM ’09, pp. 33–37, Washington, DC, USA. IEEE Computer Society (2009)Google Scholar
  9. 9.
    de Boer, F.S., de Gouw, S., Johnsen, E.B., Wong, P.Y.H.: Run-time assertion checking of data- and protocol-oriented properties of java programs: an industrial case study. LNCS Transactions on Aspect-Oriented Software Development (TAOSD). Special Issue on Runtime Verification and Analysis. To appear (2013)Google Scholar
  10. 10.
    de Boer, F.S., de Gouw, S., Wong, P.Y.H.: Run-time verification of coboxes. In: Proceedings of 11th International Conference on Software Engineering and Formal Methods, vol. 8137 of LNCS, pp. 259–273 (2013)Google Scholar
  11. 11.
    de Gouw, S., Vinju, J., de Boer, F.: Prototyping a tool environment for run-time assertion checking in JML with Communication Histories. In: Proceedings of the 12th Workshop on Formal Techniques for Java-Like Programs, FTFJP ’10, pp. 6:1–6:7. ACM, New York (2010)Google Scholar
  12. 12.
    Analysis Final Report. Deliverable 2.7 of project FP7-231620 (HATS), available at December 2012
  13. 13.
    Feng, L., Lundmark, S., Meinke, K., Niu, F., Sindhu, M.A., Wong, P.Y.H.: Case studies in learning-based testing. In: Proceedings Twenty Fifth IFIP International Conference on Testing Software and Systems (ICTSS 2013), vol. 8254 of LNCS, pp. 164–179. Springer, New York (2013)Google Scholar
  14. 14.
    Gómez-Zamalloa, M., Albert, E., Puebla, G.: Test Case Generation for Object-Oriented Imperative Languages in CLP. Theory and Practice of Logic Programming, 26th Int’l. Conference on Logic Programming (ICLP’10) Special Issue, 10 (4–6):659–674, July (2010)Google Scholar
  15. 15.
    Hähnle, R.: The abstract behavioral specification language: a tutorial introduction. In: Bonsangue, M., de Boer, F., Giachino, E., Hähnle, R. (eds.) International School on Formal Models for Components and Objects: Post Proceedings, vol. 7866 of Lecture Notes in Computer Science, pp. 1–37. Springer-Verlag, Berlin (2013)Google Scholar
  16. 16.
    Hähnle, R., Schaefer, I., Bubel, R.: Reuse in software verification by abstract method calls. In: Bonacina, M.P. (ed.) Proceedings 24th Conference on Automated Deduction (CADE), Lake Placid, USA, vol. 7898 of Lecture Notes in Computer Science, pp. 300–314. Springer-Verlag, Berlin (2013)Google Scholar
  17. 17.
    Hamill, P.: Unit Test Frameworks. O’Reilly Media (2004) Google Scholar
  18. 18.
    Johnsen, E.B., Hähnle, R., Schäfer, J., Schlatte, R., Steffen, M.: ABS: a core language for abstract behavioral specification. In: Aichernig, B., de Boer F.S., Bonsangue, M.M. (eds.) Proceedings 9th International Symposium on Formal Methods for Components and Objects (FMCO 2010), vol. 6957 of LNCS, pp. 142–164. Springer-Verlag, Berlin (2011)Google Scholar
  19. 19.
    Einar, B.J., Olaf, O.: An asynchronous communication model for distributed concurrent objects. Softw. Syst. Model. 6(1), 35–58 (2007)Google Scholar
  20. 20.
    King, J.C.: Symbolic execution and program testing. Commun. ACM 19(7), 385–394 (1976)CrossRefzbMATHGoogle Scholar
  21. 21.
    Klint, P., van der Storm, T., Vinju, J.: RASCAL: a domain specific language for source code analysis and manipulation. In: Proceedings of the 2009 Ninth IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM ’09, pp. 168–177, Washington, DC, USA. IEEE Computer Society (2009)Google Scholar
  22. 22.
    Martin, M., Livshits, B., Lam, M.S.: Finding application errors and security flaws using PQL: a program query language. In: Proceedings of the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, OOPSLA ’05, pp. 365–383. ACM, New York (2005)Google Scholar
  23. 23.
    Meinke, K., Niu, F., Sindhu, M.: Learning-based software testing: a tutorial. In: Reiner, H., Jens, K., Tiziana, M., Dietmar, S., Bernhard, S. (eds.) Leveraging Applications of Formal Methods, Verification, and Validation, Communications in Computer and Information Science, pp. 200–219. Springer-Verlag, Berlin (2012)Google Scholar
  24. 24.
    Meinke, K., Sindhu, M.: Incremental learning-based testing for reactive systems. In: Proceedings Fifth International Conference on Tests and Proofs (TAP2011), number 6706 in Lecture Notes in Computer Science, pp. 134–151. Springer-Verlag, Berlin (2011)Google Scholar
  25. 25.
    Meinke, K., Sindhu, M.A.: LBTest: a learning-based testing tool for reactive systems. In Proc ICST-13, Sixth IEEE International Conference on Software Testing, Verification and Validation, pp. 447–454. IEEE Computer Society (2013)Google Scholar
  26. 26.
    Nobakht, B., Bonsangue, M.M., de Boer, F.S., de Gouw, S.: Monitoring method call sequences using annotations. In: Proceedings of the 7th international conference on Formal Aspects of Component Software, FACS’10, pp. 53–70. Springer-Verlag, Berlin, Heidelberg (2012)Google Scholar
  27. 27.
    Terrence, P.: The Definitive ANTLR Reference. Pragmatic Bookshelf (2007)Google Scholar
  28. 28.
    Pohl, Klaus, Böckle, Günter, Van Der Linden, Frank: Software Product Line Engineering: Foundations, Principles, and Techniques. Springer, New York (2005)CrossRefGoogle Scholar
  29. 29.
    Schaefer, I., Bettini, L., Bono, V., Damiani, F., Tanzarella, N.: Delta-oriented programming of software product lines. In: Proceedings of 14th Software Product Line Conference (SPLC 2010), September (2010)Google Scholar
  30. 30.
    Schäfer, J., Poetzsch-Heffter, A.: JCoBox: Generalizing active objects to concurrent components. In: European Conference on Object-Oriented Programming (ECOOP’10), vol. 6183 of Lecture Notes in Computer Science, pp. 275–299. Springer-Verlag, June (2010)Google Scholar
  31. 31.
    Wong, P.Y.H., Albert, E., Muschevici, R., Proença, J., Schäfer, J., Schlatte, R.: The ABS tool suite: modelling, executing and analysing distributed adaptable object-oriented systems. J. Soft. Tools Technol. Trans. 14(5), 567–588 (2012)CrossRefGoogle Scholar
  32. 32.
    Wong, P.Y.H., Diakov, N., Schaefer, I.: Modelling Distributed Adaptable Object Oriented Systems using HATS Approach: A Fredhopper Case Study. In: Proceedings of FoVeOOS 2011, vol. 7421 of LNCS (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Peter Y. H. Wong
    • 1
    Email author
  • Richard Bubel
    • 2
  • Frank S. de Boer
    • 3
  • Miguel Gómez-Zamalloa
    • 4
  • Stijn de Gouw
    • 3
  • Reiner Hähnle
    • 2
  • Karl Meinke
    • 5
  • Muddassar Azam Sindhu
    • 6
  1. 1.SDLAmsterdamThe Netherlands
  2. 2.Department of Computer ScienceTechnische Universität DarmstadtDarmstadtThe Netherlands
  3. 3.CWIAmsterdamThe Netherlands
  4. 4.DSICComplutense University of MadridMadridSpain
  5. 5.School of Computer Science and CommunicationKTH Royal Institute of TechnologyStockholmSweden
  6. 6.Department of Computer ScienceQuaid-i-Azam UniversityIslamabadPakistan

Personalised recommendations