Skip to main content
SpringerLink
Log in

Learning-Based Testing of Cyber-Physical Systems-of-Systems: A Platooning Study

  • Conference paper
  • First Online:
Computer Performance Engineering (EPEW 2017)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10497))

Included in the following conference series:

Abstract

Learning-based testing (LBT) is a paradigm for fully automated requirements testing that combines machine learning with model-checking techniques. LBT has been shown to be effective for unit and integration testing of safety critical components in cyber-physical systems, e.g. automotive ECU software.

We consider the challenges faced, and some initial results obtained in an effort to scale up LBT to testing co-operative open cyber-physical systems-of-systems (CO-CPS). For this we focus on a case study of testing safety and performance properties of multi-vehicle platoons.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    See www.safecop.eu.

  2. 2.

    This architecture has been developed within the VINNOVA FFI project VIRTUES, http://www.csc.kth.se/~karlm/virtues/.

  3. 3.

    Recall that propositional LTL extends basic propositional logic with the temporal modalities G( \(\phi \) ) (always \(\phi \)), F( \(\phi \) ) (sometime \(\phi \)) and X( \(\phi \) ) (next \(\phi \)). Other derived operators and past operators may also be included. See e.g. [12] for details.

  4. 4.

    Infinite counter-examples to LTL liveness formulas are truncated around the loop, and the weaker test verdict warning may be issued.

  5. 5.

    It seems possible to theoretically explain this observation for certain types of formulas by considering their semantics. However, this is outside the scope of our present discussion.

  6. 6.

    In practise, GPS localisation would be relied upon for greater accuracy.

  7. 7.

    For the lead vehicle, CACC is disabled and accelerator and brake pedal values are used by BBW instead. See Fig. 3.

  8. 8.

    Thus \(a_{10}\) represents 100% accelerator depression, \(a_{9}\) represents 90% depression, etc. Simultaneous depression of both pedals is handled as a brake request by the BBW component.

  9. 9.

    It is also possible to use SUT observations between the output cycles by thresholding. This can yield greater accuracy, but this approach was not taken here.

  10. 10.

    Non-homogeneous platoons could also be tested using our approach.

  11. 11.

    The actual platform used was a 4-core MacBook Pro, Mid 2014, running Yosemite OS-X 10.10.5 with 2.8 GHz Intel Core i7, 16 GB 1600 MHz DDR3 and 1 TB static disk flash storage.

  12. 12.

    Based on 1 s sampling.

  13. 13.

    Unfortunately our limited computing platform did not provide an opportunity to evaluate this result.

References

  1. Aki, M., Zheng, R., Yamabe, S., Nakano, K., Suda, Y., Suzuki, Y., Ishizaka, H., Kawashima, H., Sakuma, A.: Safety testing of an improved brake system for automatic platooning of trucks. Int. J. Intell. Transp. Syst. Res. 12(3), 98–109 (2014)

    Google Scholar 

  2. Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bennaceur, A., Giannakopoulou, D., Hähnle, R., Meinke, K.: Machine learning for dynamic software analysis: potentials and limits (Dagstuhl seminar 16172). Dagstuhl Rep. 6(4), 161–173 (2016)

    Google Scholar 

  4. Bergenhem, C., Shladover, S., Coelingh, E., Englund, C., Shladover, S., Tsugawa, S.: Overview of platooning systems. In: Proceedings of the 19th ITS World Congress, Vienna, October 2012

    Google Scholar 

  5. van den Bleek, R.: Design of a hybrid adaptive cruise control stop-&-go system. Master’s thesis, Technische Universiteit Eindhoven, Department of Mechanical Engineering (2007)

    Google Scholar 

  6. Cassel, S., Howar, F., Jonsson, B., Steffen, B.: Active learning for extended finite state machines. Form. Asp. Comput. 28(2), 233–263 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  7. Cimatti, A., et al.: NuSMV 2: an opensource tool for symbolic model checking. In: Brinksma, E., Larsen, K.G. (eds.) CAV 2002. LNCS, vol. 2404, pp. 359–364. Springer, Heidelberg (2002). doi:10.1007/3-540-45657-0_29

    Chapter  Google Scholar 

  8. Colin, S., Lanoix, A., Kouchnarenko, O., Souquières, J.: Using CSP\(||\)B components: application to a platoon of vehicles. In: Cofer, D., Fantechi, A. (eds.) FMICS 2008. LNCS, vol. 5596, pp. 103–118. Springer, Heidelberg (2009). doi:10.1007/978-3-642-03240-0_11

    Chapter  Google Scholar 

  9. El-Zaher, M., Contet, J., Gruer, P., Gechter, F., Koukam, A.: Compositional verification for reactive multi-agent systems applied to platoon non collision verification. Stud. Inform. Univ. 10(3), 119–141 (2012)

    Google Scholar 

  10. Engelberg, S.: A Mathematical Introduction to Control Theory. Imperial College Press, London (2015)

    Book  MATH  Google Scholar 

  11. Feng, L., Lundmark, S., Meinke, K., Niu, F., Sindhu, M.A., Wong, P.Y.H.: Case studies in learning-based testing. In: Yenigün, H., Yilmaz, C., Ulrich, A. (eds.) ICTSS 2013. LNCS, vol. 8254, pp. 164–179. Springer, Heidelberg (2013). doi:10.1007/978-3-642-41707-8_11

    Chapter  Google Scholar 

  12. Fisher, M.: An Introduction to Practical Formal Methods Using Temporal Logic. Wiley, Hoboken (2011)

    Book  MATH  Google Scholar 

  13. De la Higuera, C.: Grammatical Inference: Learning Automata and Grammars. Cambridge University Press, Cambridge (2010)

    Book  MATH  Google Scholar 

  14. Hossen, K., Groz, R., Oriat, C., Richier, J.: Automatic model inference of web applications for security testing. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, pp. 22–23 (2014)

    Google Scholar 

  15. Howar, F., Bauer, O., Merten, M., Steffen, B., Margaria, T.: The teachers’ crowd: the impact of distributed oracles on active automata learning. In: Hähnle, R., Knoop, J., Margaria, T., Schreiner, D., Steffen, B. (eds.) ISoLA 2011. CCIS, pp. 232–247. Springer, Heidelberg (2012). doi:10.1007/978-3-642-34781-8_18

    Chapter  Google Scholar 

  16. Isberner, M., Howar, F., Steffen, B.: The TTT algorithm: a redundancy-free approach to active automata learning. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 307–322. Springer, Cham (2014). doi:10.1007/978-3-319-11164-3_26

    Google Scholar 

  17. Jorgensen, P.C.: Software testing (2008)

    Google Scholar 

  18. Kamali, M., Dennis, L.A., McAree, O., Fisher, M., Veres, S.M.: Formal verification of autonomous vehicle platooning. CoRR abs/1602.01718 (2016)

    Google Scholar 

  19. Kearns, M., Vazirani, U.: An Introduction to Computational Learning Theory. MIT Press, Cambridge (1994)

    Google Scholar 

  20. Khosrowjerdi, H., Meinke, K., Rasmusson, A.: Automated behavioral requirements testing for automotive ECU applications. In: Proceedings of the 5th International Workshop on Model Based Safety Analysis. In: IMBSA 2017. LNCS. Springer (2017, to appear)

    Google Scholar 

  21. Meinke, K.: Recent progress in learning-based testing. In: Bennaceur, A., Hähnle, R., Meinke, K. (eds.) Machine Learning for Dynamic Software Analysis: Potentials and Limits: Proceedings of Dagstuhl Workshop, vol. 16172. Springer (2017, to appear)

    Google Scholar 

  22. Meinke, K., Niu, F.: An incremental learning algorithm for hybrid automata (2013)

    Google Scholar 

  23. Meinke, K., Sindhu, M.A.: Incremental learning-based testing for reactive systems. In: Gogolla, M., Wolff, B. (eds.) TAP 2011. LNCS, vol. 6706, pp. 134–151. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21768-5_11

    Chapter  Google Scholar 

  24. Meinke, K.: Automated black-box testing of functional correctness using function approximation. In: Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2004), pp. 143–153. ACM Press (2004)

    Google Scholar 

  25. Meinke, K., Nycander, P.: Learning-based testing of distributed microservice architectures: correctness and fault injection. In: Bianculli, D., Calinescu, R., Rumpe, B. (eds.) SEFM 2015. LNCS, vol. 9509, pp. 3–10. Springer, Heidelberg (2015). doi:10.1007/978-3-662-49224-6_1

    Chapter  Google Scholar 

  26. Meinke, K., Sindhu, M.A.: Lbtest: a learning-based testing tool for reactive systems. In: Proceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation (ICST 2013), pp. 447–454. IEEE Computer Society (2013)

    Google Scholar 

  27. Peled, D., Vardi, M.Y., Yannakakis, M.: Black box checking. In: Wu, J., Chanson, S.T., Gao, Q. (eds.) Formal Methods for Protocol Engineering and Distributed Systems. IAICT, vol. 28, pp. 225–240. Springer, Boston (1999). doi:10.1007/978-0-387-35578-8_13

    Chapter  Google Scholar 

  28. Raffelt, H., Steffen, B., Margaria, T.: Dynamic testing via automata learning. In: Yorav, K. (ed.) HVC 2007. LNCS, vol. 4899, pp. 136–152. Springer, Heidelberg (2008). doi:10.1007/978-3-540-77966-7_13

    Chapter  Google Scholar 

  29. Sjöberg, K.: Platooning - challenges and opportunities (2016). https://docbox.etsi.org/Workshop/2016/201603_ITS_WORKSHOP/S04_TWDS_ACCIDENT_FREE_AUTOMATED_DRIVING

  30. Valiant, L.G.: A theory of the learnable. Commun. ACM 27(11), 1134–1142 (1984)

    Article  MATH  Google Scholar 

  31. Walkinshaw, N., Bogdanov, K., Derrick, J., Paris, J.: Increasing functional coverage by inductive testing: a case study. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 126–141. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16573-3_10

    Chapter  Google Scholar 

  32. Weyuker, E.: Assessing test data adequacy through program inference. ACM Trans. Program. Lang. Syst 5(4), 641–655 (1983)

    Article  MATH  Google Scholar 

  33. Willke, T., Tientrakool, P., Maxemchuk, N.: A survey of inter-vehicle communication protocols and their applications. IEEE Commun. Surv. Tutor. 11(2), 3–20 (2009)

    Article  Google Scholar 

  34. Özguner, U., Acarman, T., Redmill, K.: Autonomous Ground Vehicles. Artech House Publishers, Boston (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Communication, KTH Royal Institute of Technology, 100 44, Stockholm, Sweden

    Karl Meinke

Authors
  1. Karl Meinke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karl Meinke .

Editor information

Editors and Affiliations

  1. Bristol, Germany

    Philipp Reinecke

  2. University of L’Aquila, L’Aquila, Italy

    Antinisca Di Marco

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Meinke, K. (2017). Learning-Based Testing of Cyber-Physical Systems-of-Systems: A Platooning Study. In: Reinecke, P., Di Marco, A. (eds) Computer Performance Engineering. EPEW 2017. Lecture Notes in Computer Science(), vol 10497. Springer, Cham. https://doi.org/10.1007/978-3-319-66583-2_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66583-2_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66582-5

  • Online ISBN: 978-3-319-66583-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation