Skip to main content
SpringerLink
Log in

Model-Based Testing for Avionic Systems Proven Benefits and Further Challenges

  • Conference paper
  • First Online:
Leveraging Applications of Formal Methods, Verification and Validation. Industrial Practice (ISoLA 2018)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11247))

Included in the following conference series:

Abstract

In this article, we report on the transition of model-based testing (MBT) from a widely discussed research discipline to an accepted technology that is currently becoming state of the art in industry; in particular, in the field of safety-critical systems testing. It is reviewed how focal points of MBT-related research in the past have found their way into today’s commercial MBT products. We describe the benefits of MBT that are – from our experience – most appreciated by practitioners. Moreover, some interesting open challenges are described, and potential future solutions are presented. The material presented in this paper is based on our practical experience with recent MBT campaigns performed for Airbus in Germany.

The work presented in this contribution has been partially funded by the German Federal Ministry for Economic Affairs and Energy (BMWi) in the context of project STEVE, grant application 20Y1301P.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    See https://www.ibm.com/support/knowledgecenter/SSB2MU_8.2.1/com.btc.tcatg.user.doc/topics/com.btc.tcatg.user.doc.html and ftp://public.dhe.ibm.com/software/uk/itsolutions/innovate2013/12.00_Udo_Brockmeyer-003.pdf.

  2. 2.

    A more extensive list of MBT tools is given in http://mit.bme.hu/~micskeiz/pages/modelbased_testing.html#tools.

  3. 3.

    A test suite is complete with respect to a given reference model M, conformance relation \(\le \), and fault domain \(\mathcal{D}\), if (1) every implementation conforming to M passes all test cases, and (2) every implementation whose behavior is reflected by a model \(M'\) in the fault domain \(\mathcal{D}\) fails at least one test case in the suite if \(M'\) does not conform to M. The fault domain \(\mathcal{D}\) contains a (possibly infinite) set of models that may or may not conform to the reference model. In black-box testing, completeness can only be guaranteed under the assumption that the true SUT behavior is captured by one of the models in \(\mathcal{D}\).

  4. 4.

    Formula \(\psi _1\mathbf {W}\psi _2\) uses the weak until operator which states that \(\psi _1\) will hold until \(\psi _2\) holds, but it is not guaranteed that \(\psi _2\) will ever become true. In this case, \(\psi _1\) will always hold, so \(\psi _1\mathbf {W}{\mathtt{false}}\equiv \mathbf {G}\psi _1\).

  5. 5.

    We use the term test scenario to denote a composite test case, exercising a larger fragment of SUT functionality in end-to-end fashion. Typically, a test scenario comprises several model coverage test cases in a specific order.

References

  1. Araujo, H.L.S., Carvalho, G., Mohaqeqi, M., Mousavi, M.R., Sampaio, A.: Sound conformance testing for cyber-physical systems: theory and implementation. Sci. Comput. Program. 162, 35–54 (2018). https://doi.org/10.1016/j.scico.2017.07.002

    Article  Google Scholar 

  2. Banci, M., Fantechi, A., Gnesi, S., Lombardi, G.: Model driven development and code generation: an automotive case study. In: Gaudin, E., Najm, E., Reed, R. (eds.) SDL 2007. LNCS, vol. 4745, pp. 19–34. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74984-4_2

    Chapter  Google Scholar 

  3. Biere, A., Heljanko, K., Junttila, T., Latvala, T., Schuppan, V.: Linear encodings of bounded LTL model checking. Log. Methods Comput. Sci. 2(5) (2006). arXiv:cs/0611029

  4. Du Bousquet, L., Ramangalahy, S., Simon, S., Viho, C., Belinfante, A., de Vries, R.G.: Formal test automation: the conference protocol with TGV/TorX. In: Ural, H., Probert, R.L., v. Bochmann, G. (eds.) Testing of Communicating Systems. IAICT, vol. 48, pp. 221–228. Springer, Boston, MA (2000). https://doi.org/10.1007/978-0-387-35516-0_14

    Chapter  Google Scholar 

  5. Brauer, J., Peleska, J., Schulze, U.: Efficient and trustworthy tool qualification for model-based testing tools. In: Nielsen, B., Weise, C. (eds.) ICTSS 2012. LNCS, vol. 7641, pp. 8–23. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34691-0_3

    Chapter  Google Scholar 

  6. Brauer, J., Schulze, U.: Model-based testing for avionics systems. In: Havelund, K., Peleska, J., Roscoe, B., de Vink, E. (eds.) FM 2018. LNCS, vol. 10951, pp. 657–661. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95582-7_40

    Chapter  Google Scholar 

  7. Cavarra, A.: Data flow analysis and testing of abstract state machines. In: Börger, E., Butler, M., Bowen, J.P., Boca, P. (eds.) ABZ 2008. LNCS, vol. 5238, pp. 85–97. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87603-8_8

    Chapter  MATH  Google Scholar 

  8. Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. SE–4(3), 178–186 (1978)

    Article  Google Scholar 

  9. Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999)

    MATH  Google Scholar 

  10. Hessel, A., Larsen, K.G., Mikucionis, M., Nielsen, B., Pettersson, P., Skou, A.: Testing real-time systems using UPPAAL. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. LNCS, vol. 4949, pp. 77–117. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78917-8_3

    Chapter  Google Scholar 

  11. Hou, Z., Sanán, D., Tiu, A., Liu, Y.: A formal model for the SPARCv8 ISA and a proof of non-interference for the LEON3 processor. Archive of Formal Proofs 2016 (2016). https://www.isa-afp.org/entries/SPARCv8.shtml

  12. Huang, W., Peleska, J.: Complete model-based equivalence class testing. Softw. Tools Technol. Transfer 18(3), 265–283 (2016). https://doi.org/10.1007/s10009-014-0356-8

    Article  Google Scholar 

  13. Huang, W., Peleska, J.: Complete model-based equivalence class testing for nondeterministic systems. Formal Aspects of Comput. 29(2), 335–364 (2017). https://doi.org/10.1007/s00165-016-0402-2

    Article  MathSciNet  MATH  Google Scholar 

  14. Huang, W., Peleska, J.: Model-based testing strategies and their (in)dependence on syntactic model representations. Int. J. Softw. Tools Technol. Transf. 20, 441–465 (2017). https://doi.org/10.1007/s10009-017-0479-9

    Article  Google Scholar 

  15. Hübner, F., Huang, W., Peleska, J.: Experimental evaluation of a novel equivalence class partition testing strategy. Softw. Syst. Model. (2017). https://doi.org/10.1007/s10270-017-0595-8

  16. Jensen, H.E., Larsen, K.G., Skou, A.: Modelling and analysis of a collision avoidance protocol using spin and UPPAAL. In: Grégoire, J., Holzmann, G.J., Peled, D.A. (eds.) The Spin Verification System, Proceedings of a DIMACS Workshop, New Brunswick, New Jersey, USA, August 1996. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 32, pp. 33–50. DIMACS/AMS (1996). http://dimacs.rutgers.edu/Volumes/Vol32.html

  17. Kästner, D., et al.: Timing validation of automotive software. In: Margaria, T., Steffen, B. (eds.) ISoLA 2008. CCIS, vol. 17, pp. 93–107. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88479-8_8

    Chapter  Google Scholar 

  18. Kuhn, D.R., Kacker, R.N., Lei, Y.: Introduction to Combinatorial Testing. CRC Press, Boca Raton (2013)

    MATH  Google Scholar 

  19. Larsen, K.G., Mikucionis, M., Nielsen, B., Skou, A.: Testing real-time embedded software using UPPAAL-TRON: an industrial case study. In: Proceedings of the 5th ACM International Conference on Embedded Software, EMSOFT 2005, pp. 299–306. ACM, New York (2005). http://doi.acm.org/10.1145/1086228.1086283

  20. Lee, J., Kang, S., Lee, D.: A survey on software product line testing. In: Proceedings of the 16th International Software Product Line Conference, SPLC 2012, vol. 1, pp. 31–40. ACM, New York (2012). http://doi.acm.org/10.1145/2362536.2362545

  21. Mohacsi, S., Felderer, M., Beer, A.: A case study on the efficiency of model-based testing at the European space agency. In: 8th IEEE International Conference on Software Testing, Verification and Validation, ICST 2015, Graz, Austria, 13–17 April 2015, pp. 1–2. IEEE Computer Society (2015). https://doi.org/10.1109/ICST.2015.7102618

  22. Mohacsi, S., Felderer, M., Beer, A.: Estimating the cost and benefit of model-based testing: a decision support procedure for the application of model-based testing in industry. In: 41st Euromicro Conference on Software Engineering and Advanced Applications, EUROMICRO-SEAA 2015, Madeira, Portugal, 26–28 August 2015, pp. 382–389. IEEE Computer Society (2015). https://doi.org/10.1109/SEAA.2015.18

  23. Neto, A.C.D., Travassos, G.H.: A picture from the model-based testing area: concepts, techniques, and challenges. Adv. Comput. 80, 45–120 (2010). https://doi.org/10.1016/S0065-2458(10)80002-6

    Article  Google Scholar 

  24. Object Management Group: OMG Systems Modeling Language (OMG SysML), Version 1.4. Technical report, Object Management Group (2015). http://www.omg.org/spec/SysML/1.4

  25. Peleska, J.: Industrial-strength model-based testing - state of the art and current challenges. In: Petrenko, A.K., Schlingloff, H. (eds.) Proceedings Eighth Workshop on Model-Based Testing, Rome, Italy, 17th March 2013. Electronic Proceedings in Theoretical Computer Science, vol. 111, pp. 3–28. Open Publishing Association (2013)

    Google Scholar 

  26. Peleska, J.: Model-based avionic systems testing for the airbus family. In: 23rd IEEE European Test Symposium, ETS 2018, Bremen, Germany, 28 May–1 June 2018, pp. 1–10. IEEE (2018). https://doi.org/10.1109/ETS.2018.8400703

  27. Peleska, J., Huang, W., Hübner, F.: A novel approach to HW/SW integration testing of route-based interlocking system controllers. In: Lecomte, T., Pinger, R., Romanovsky, A. (eds.) RSSRail 2016. LNCS, vol. 9707, pp. 32–49. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-33951-1_3

    Chapter  Google Scholar 

  28. Peleska, J., Vorobev, E., Lapschies, F.: Automated test case generation with SMT-solving and abstract interpretation. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) NFM 2011. LNCS, vol. 6617, pp. 298–312. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20398-5_22

    Chapter  Google Scholar 

  29. Petrenko, A.: Checking experiments for symbolic input/output finite state machines. In: Ninth IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2016, Chicago, IL, USA, 11–15 April 2016, pp. 229–237. IEEE Computer Society (2016). https://doi.org/10.1109/ICSTW.2016.9

  30. Petrenko, A., Simao, A., Maldonado, J.C.: Model-based testing of software and systems: recent advances and challenges. Int. J. Softw. Tools Technol. Transf. 14(4), 383–386 (2012). https://doi.org/10.1007/s10009-012-0240-3

    Article  Google Scholar 

  31. RTCA SC-205/EUROCAE WG-71: Software Considerations in Airborne Systems and Equipment Certification. Technical report, RTCA/DO-178C, RTCA Inc, 1140 Connecticut Avenue, N.W., Suite 1020, Washington, D.C. 20036, December 2011

    Google Scholar 

  32. Sistla, A.P.: Safety, liveness and fairness in temporal logic. Formal Aspects Comput. 6(5), 495–511 (1994)

    Article  Google Scholar 

  33. Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test. Verif. Reliab. 22(5), 297–312 (2012). https://doi.org/10.1002/stvr.456

    Article  Google Scholar 

  34. Vasilevskii, M.P.: Failure diagnosis of automata. Kibernetika (Transl.) 4, 98–108 (1973)

    MathSciNet  Google Scholar 

  35. Weißleder, S.: Test models and coverage criteria for automatic model-based test generation with UML state machines. Ph.D. thesis, Humboldt University of Berlin (2010). http://d-nb.info/1011308983

Download references

Author information

Authors and Affiliations

  1. Verified Systems International GmbH, Bremen, Germany

    Jan Peleska & Jörg Brauer

  2. Department of Mathematics and Computer Science, University of Bremen, Bremen, Germany

    Jan Peleska & Wen-ling Huang

Authors
  1. Jan Peleska
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jörg Brauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wen-ling Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan Peleska .

Editor information

Editors and Affiliations

  1. University of Limerick, Limerick, Ireland

    Tiziana Margaria

  2. TU Dortmund, Dortmund, Germany

    Bernhard Steffen

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Peleska, J., Brauer, J., Huang, Wl. (2018). Model-Based Testing for Avionic Systems Proven Benefits and Further Challenges. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation. Industrial Practice. ISoLA 2018. Lecture Notes in Computer Science(), vol 11247. Springer, Cham. https://doi.org/10.1007/978-3-030-03427-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03427-6_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03426-9

  • Online ISBN: 978-3-030-03427-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation