Skip to main content

Configurable Model-Based Test Generation for Distributed Controllers Using Declarative Model Queries and Model Checkers

  • Conference paper
  • First Online:
Formal Methods for Industrial Critical Systems (FMICS 2023)

Abstract

Distributed programmable controllers are getting prevalence in critical infrastructure, among others, in railway interlocking systems (RIS). Generally, such systems are integrated using various reactive components and must carry out critical tasks. Accordingly, their systematic testing is vital, which can be hindered by their complexity and distributed nature. This paper presents a model-based test generation approach using hidden formal methods. It is based on the collaborating statechart models of the system components and parametric test coverage criteria configurable by declarative model queries. Statecharts can be integrated using various composition modes (e.g., synchronous and asynchronous) and then automatically mapped into the inputs of model checker back-ends, namely UPPAAL, Theta and Spin. The model checkers generate tests by traversing the emergent analysis models to cover the elements of the test model as specified by pattern-based model queries. The returned diagnostic traces are then concretized to different execution environments. The approach is implemented in our open source Gamma Statechart Composition Framework and evaluated on a distributed RIS subsystem under development.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    More information about the framework (e.g., preprints) and the source code can be found at http://gamma.inf.mit.bme.hu/ and https://github.com/ftsrg/gamma/.

  2. 2.

    https://www.prolan.hu/en/products/PRORIS.

  3. 3.

    https://eclipse.org/viatra/.

References

  1. Adam, S., Larsen, M., Jensen, K., Schultz, U.P.: Rule-based dynamic safety monitoring for mobile robots. J. Softw. Eng. Robot. 7(1), 120–141 (2016)

    Google Scholar 

  2. Amendola, A., et al.: A model-based approach to the design, verification and deployment of railway interlocking system. In: Margaria, T., Steffen, B. (eds.) ISoLA 2020. LNCS, vol. 12478, pp. 240–254. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61467-6_16

    Chapter  Google Scholar 

  3. Basu, A., et al.: Rigorous component-based system design using the BIP framework. IEEE Softw. 28(3), 41–48 (2011). https://doi.org/10.1109/MS.2011.27

    Article  Google Scholar 

  4. Behrmann, G., et al.: UPPAAL 4.0. In: Proceedings of the 3rd International Conference on the Quantitative Evaluation of Systems, QEST 2006, pp. 125–126. IEEE Computer Society, USA (2006). https://doi.org/10.1109/QEST.2006.59

  5. Bitsch, F.: Safety Patterns—the key to formal specification of safety requirements. In: Voges, U. (ed.) SAFECOMP 2001. LNCS, vol. 2187, pp. 176–189. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45416-0_18

    Chapter  Google Scholar 

  6. Boulanger, J.L.: CENELEC 50128 and IEC 62279 Standards. Wiley, Hoboken (2015)

    Book  Google Scholar 

  7. Bouwman, M., Luttik, B., van der Wal, D.: A formalisation of SysML state machines in mCRL2. In: Peters, K., Willemse, T.A.C. (eds.) FORTE 2021. LNCS, vol. 12719, pp. 42–59. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78089-0_3

    Chapter  Google Scholar 

  8. Bouwman, M., Luttik, S., Rensink, A., Stoelinga, M., van der Wal, D.: Formal methods in railway signalling infrastructure standardisation processes. In: Margaria, T., Steffen, B. (eds.) ISoLA 2021. Lecture Notes in Computer Science, vol. 13036, pp. 500–501. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-89159-6

    Chapter  Google Scholar 

  9. Bozhinoski, D., Di Ruscio, D., Malavolta, I., Pelliccione, P., Tivoli, M.: FLYAQ: enabling non-expert users to specify and generate missions of autonomous multicopters. In: 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 801–806. IEEE (2015)

    Google Scholar 

  10. Bunte, O., et al.: The mCRL2 toolset for analysing concurrent systems. In: Vojnar, T., Zhang, L. (eds.) TACAS 2019. LNCS, vol. 11428, pp. 21–39. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17465-1_2

    Chapter  Google Scholar 

  11. Castillos, K.C., Dadeau, F., Julliand, J., Kanso, B., Taha, S.: A compositional automata-based semantics for property patterns. In: Johnsen, E.B., Petre, L. (eds.) IFM 2013. LNCS, vol. 7940, pp. 316–330. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38613-8_22

    Chapter  Google Scholar 

  12. Chechik, M., Păun, D.O.: Events in property patterns. In: Dams, D., Gerth, R., Leue, S., Massink, M. (eds.) SPIN 1999. LNCS, vol. 1680, pp. 154–167. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48234-2_13

    Chapter  Google Scholar 

  13. Childs, A., Greenwald, J., Jung, G., Hoosier, M., Hatcliff, J.: CALM and Cadena: metamodeling for component-based product-line development. IEEE Comput. 39(2), 42–50 (2006). https://doi.org/10.1109/MC.2006.51

    Article  Google Scholar 

  14. Ciccozzi, F., Di Ruscio, D., Malavolta, I., Pelliccione, P.: Adopting MDE for specifying and executing civilian missions of mobile multi-robot systems. IEEE Access 4, 6451–6466 (2016)

    Article  Google Scholar 

  15. da Costa Cavalheiro, S.A., Foss, L., Ribeiro, L.: Specification patterns for properties over reachable states of graph grammars. In: Gheyi, R., Naumann, D. (eds.) SBMF 2012. LNCS, vol. 7498, pp. 83–98. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33296-8_8

    Chapter  Google Scholar 

  16. Dwyer, M.B., Avrunin, G.S., Corbett, J.C.: Patterns in property specifications for finite-state verification. In: Proceedings of the 21st International Conference on Software Engineering, pp. 411–420 (1999)

    Google Scholar 

  17. Emerson, E.A., Halpern, J.Y.: “Sometimes’’ and “not never’’ revisited: on branching versus linear time temporal logic. J. ACM 33(1), 151–178 (1986). https://doi.org/10.1145/4904.4999

    Article  MathSciNet  Google Scholar 

  18. Enoiu, E.P., Čaušević, A., Ostrand, T.J., Weyuker, E.J., Sundmark, D., Pettersson, P.: Automated test generation using model checking: an industrial evaluation. Int. J. Softw. Tools Technol. Transf. 18(3), 335–353 (2016). https://doi.org/10.1007/s10009-014-0355-9

    Article  Google Scholar 

  19. Ferrari, A., Mazzanti, F., Basile, D., ter Beek, M.H.: Systematic evaluation and usability analysis of formal methods tools for railway signaling system design. IEEE Trans. Software Eng. 48(11), 4675–4691 (2022). https://doi.org/10.1109/TSE.2021.3124677

    Article  Google Scholar 

  20. Fraser, G., Wotawa, F., Ammann, P.E.: Testing with model checkers: a survey. Softw. Test. Verif. Reliab. 19(3), 215–261 (2009). https://doi.org/10.1002/stvr.402

    Article  Google Scholar 

  21. Garcia, L.A.: Automatic generation and verification of complex pattern-based software specifications. The University of Texas at El Paso (2007)

    Google Scholar 

  22. Golarits, Z., Sinka, D., Jávor, A.: Proris—a new interlocking system for regional and moderate-traffic lines. SIGNAL+DRAHT - Signal. Datacommun. (114), 28–36 (2022)

    Google Scholar 

  23. Graics, B.: Documentation of the Gamma Statechart composition framework v0.9. Technical report, Budapest University of Technology and Economics, Department of Measurement and Information Systems (2016). https://tinyurl.com/yeywrkd6

  24. Graics, B., Majzik, I.: Integration test generation and formal verification for distributed controllers. In: Renczes, B. (ed.) Proceedings of the 30th PhD Minisymposium. Budapest University of Technology and Economics, Department of Measurement and Information Systems (2023). https://doi.org/10.3311/minisy2023-001

  25. Graics, B., Molnár, V.: Formal compositional semantics for Yakindu statecharts. In: Pataki, B. (ed.) Proceedings of the 24th PhD Mini-Symposium, Budapest, Hungary, pp. 22–25 (2017)

    Google Scholar 

  26. Graics, B., Molnár, V., Majzik, I.: Integration test generation for state-based components in the Gamma framework. Preprint (2022). https://tinyurl.com/4dhubca4

  27. Graics, B., Molnár, V., Vörös, A., Majzik, I., Varró, D.: Mixed-semantics composition of statecharts for the component-based design of reactive systems. Softw. Syst. Model. 19(6), 1483–1517 (2020). https://doi.org/10.1007/s10270-020-00806-5

    Article  Google Scholar 

  28. Harel, D.: Statecharts: a visual formalism for complex systems. Sci. Comput. Program. 8(3), 231–274 (1987). https://doi.org/10.1016/0167-6423(87)90035-9

    Article  MathSciNet  Google Scholar 

  29. Hartman, A., Nagin, K.: The AGEDIS tools for model based testing. ACM Sigsoft Softw. Eng. Notes 29, 129–132 (2004). https://doi.org/10.1145/1007512.1007529

    Article  Google Scholar 

  30. Heineman, G.T., Councill, W.T.: Component-Based Software Engineering. Putting the Pieces Together. Addison Wesley (2001). https://doi.org/10.5555/379381

  31. Holzmann, G.: The SPIN Model Checker: Primer and Reference Manual, 1st edn. Addison-Wesley Professional (2011)

    Google Scholar 

  32. Huang, L.: The past, present and future of railway interlocking system. In: 2020 IEEE 5th International Conference on Intelligent Transportation Engineering (ICITE), pp. 170–174 (2020). https://doi.org/10.1109/ICITE50838.2020.9231438

  33. Jéron, T., Morel, P.: Test generation derived from model-checking. In: Halbwachs, N., Peled, D. (eds.) CAV 1999. LNCS, vol. 1633, pp. 108–122. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48683-6_12

    Chapter  Google Scholar 

  34. Kaliappan, P.S., Kaliappan, V.K.: Deriving the behavioral properties from UML designs as LTL for model checking. In: 2015 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES), pp. 1–5 (2015). https://doi.org/10.1109/SPICES.2015.7091419

  35. Ke, X., Sierszecki, K., Angelov, C.: COMDES-II: a component-based framework for generative development of distributed real-time control systems. In: 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), pp. 199–208 (2007). https://doi.org/10.1109/RTCSA.2007.29

  36. Lee, I., Sokolsky, O.: A graphical property specification language. In: Proceedings 1997 High-Assurance Engineering Workshop, pp. 42–47. IEEE (1997)

    Google Scholar 

  37. Legeard, B., Bouzy, A.: Smartesting CertifyIt: model-based testing for enterprise IT. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, pp. 391–397 (2013). https://doi.org/10.1109/ICST.2013.55

  38. Li, W., Le Gall, F., Spaseski, N.: A survey on model-based testing tools for test case generation. In: Itsykson, V., Scedrov, A., Zakharov, V. (eds.) TMPA 2017. CCIS, vol. 779, pp. 77–89. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-71734-0_7

    Chapter  Google Scholar 

  39. Lukács, G., Bartha, T.: Formal modeling and verification of the functionality of electronic urban railway control systems through a case study. Urban Rail Transit 8, 217–245 (2022). https://doi.org/10.1007/s40864-022-00177-8

    Article  Google Scholar 

  40. Martinez, S., Pereira, D.I.D.A., Bon, P., Collart-Dutilleul, S., Perin, M.: Towards safe and secure computer based railway interlocking systems. Int. J. Transp. Dev. Integr. 4(3), 218–229 (2020)

    Article  Google Scholar 

  41. Menghi, C., Tsigkanos, C., Pelliccione, P., Ghezzi, C., Berger, T.: Specification patterns for robotic missions. IEEE Trans. Softw. Eng. 47(10), 2208–2224 (2021). https://doi.org/10.1109/TSE.2019.2945329

    Article  Google Scholar 

  42. Mohalik, S., Gadkari, A.A., Yeolekar, A., Shashidhar, K., Ramesh, S.: Automatic test case generation from simulink/stateflow models using model checking. Softw. Test. Verif. Reliab. 24, 155–180 (2014). https://doi.org/10.1002/stvr.1489

    Article  Google Scholar 

  43. Molnár, V., Graics, B., Vörös, A., Majzik, I., Varró, D.: The Gamma statechart composition framework. In: 40th International Conference on Software Engineering (ICSE), pp. 113–116. ACM, Gothenburg (2018). https://doi.org/10.1145/3183440.3183489

  44. Mondragon, O.A., Gates, A.Q.: Supporting elicitation and specification of software properties through patterns and composite propositions. Int. J. Softw. Eng. Knowl. Eng. 14(01), 21–41 (2004)

    Article  Google Scholar 

  45. Paun, D.O., Chechik, M.: Events in linear-time properties. In: Proceedings IEEE International Symposium on Requirements Engineering (Cat. No. PR00188), pp. 123–132. IEEE (1999)

    Google Scholar 

  46. Radnai, B.: Integration of SCXML state machines to the Gamma framework. Technical report, Budapest University of Technology and Economics, Department of Measurement and Information Systems (2022). https://tinyurl.com/4mmtsw7v

  47. Remenska, D., Willemse, T.A.C., Templon, J., Verstoep, K., Bal, H.: Property specification made easy: harnessing the power of model checking in UML designs. In: Ábrahám, E., Palamidessi, C. (eds.) FORTE 2014. LNCS, vol. 8461, pp. 17–32. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-43613-4_2

    Chapter  Google Scholar 

  48. Ruscio, D.D., Malavolta, I., Pelliccione, P., Tivoli, M.: Automatic generation of detailed flight plans from high-level mission descriptions. In: Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems, pp. 45–55 (2016)

    Google Scholar 

  49. Salamah, S., Gates, A.Q., Kreinovich, V., Roach, S.: Verification of automatically generated pattern-based LTL specifications. In: 10th IEEE High Assurance Systems Engineering Symposium (HASE 2007), pp. 341–348 (2007). https://doi.org/10.1109/HASE.2007.37

  50. Smith, M.H., Holzmann, G.J., Etessami, K.: Events and constraints: a graphical editor for capturing logic requirements of programs. In: Proceedings Fifth IEEE International Symposium on Requirements Engineering, pp. 14–22. IEEE (2001)

    Google Scholar 

  51. Smith, R.L., Avrunin, G.S., Clarke, L.A., Osterweil, L.J.: PROPEL: an approach supporting property elucidation. In: Proceedings of the 24th International Conference on Software Engineering, ICSE 2002, pp. 11–21. Association for Computing Machinery, New York (2002). https://doi.org/10.1145/581339.581345

  52. Spanoudakis, G., Kloukinas, C., Androutsopoulos, K.: Towards security monitoring patterns. In: Proceedings of the 2007 ACM Symposium on Applied Computing, pp. 1518–1525 (2007)

    Google Scholar 

  53. Srinivas, S., Kermani, R., Kim, K., Kobayashi, Y., Fainekos, G.: A graphical language for LTL motion and mission planning. In: 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 704–709. IEEE (2013)

    Google Scholar 

  54. Steinberg, D., Budinsky, F., Merks, E., Paternostro, M.: EMF: Eclipse Modeling Framework. Pearson Education, London (2008)

    Google Scholar 

  55. Sztipanovits, J., Bapty, T., Neema, S., Howard, L., Jackson, E.: OpenMETA: a model- and component-based design tool chain for cyber-physical systems. In: Bensalem, S., Lakhneck, Y., Legay, A. (eds.) ETAPS 2014. LNCS, vol. 8415, pp. 235–248. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54848-2_16

    Chapter  Google Scholar 

  56. Tóth, T., Hajdu, A., Vörös, A., Micskei, Z., Majzik, I.: Theta: a framework for abstraction refinement-based model checking. In: Stewart, D., Weissenbacher, G. (eds.) Proceedings of the 17th Conference on Formal Methods in Computer-Aided Design, pp. 176–179 (2017). https://doi.org/10.23919/FMCAD.2017.8102257

  57. Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. LNCS, vol. 4949, pp. 1–38. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78917-8_1

    Chapter  Google Scholar 

  58. Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test. Verif. Reliab. 22(5), 297–312 (2012). https://doi.org/10.1002/stvr.456

    Article  Google Scholar 

  59. Varró, D., Bergmann, G., Hegedüs, Á., Horváth, Á., Ráth, I., Ujhelyi, Z.: Road to a reactive and incremental model transformation platform: three generations of the VIATRA framework. Softw. Syst. Model. 15(3), 609–629 (2016). https://doi.org/10.1007/s10270-016-0530-4

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the anonymous reviewers for their thorough and constructive feedback. Project no. 2019-1.3.1-KK-2019-00004 has been implemented with the support provided from the National Research, Development and Innovation Fund of Hungary, financed under the 2019-1.3.1-KK funding scheme.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Bence Graics .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Graics, B., Molnár, V., Majzik, I. (2023). Configurable Model-Based Test Generation for Distributed Controllers Using Declarative Model Queries and Model Checkers. In: Cimatti, A., Titolo, L. (eds) Formal Methods for Industrial Critical Systems. FMICS 2023. Lecture Notes in Computer Science, vol 14290. Springer, Cham. https://doi.org/10.1007/978-3-031-43681-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43681-9_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43680-2

  • Online ISBN: 978-3-031-43681-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics