Gaining Certainty About Uncertainty

Testing Cyber-Physical Systems in the Presence of Uncertainties at the Application Level
  • Martin A. SchneiderEmail author
  • Marc-Florian Wendland
  • Leon Bornemann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10224)


A cyber-physical system (CPS) comprises several connected, embedded systems and is additionally equipped with sensors and actuators. Thus, CPSs can communicate with their cyber environment and measure and interact with their physical environment. Due to the complexity of their operational environment, assumptions the manufacturer have made may not hold in operation. During an unforeseen environmental situation, a CPS may expose behavior that negatively impactsits reliability. This may arise due to insufficiently considered environmental conditions during the design of a CPS, or – even worse – it is impossible to anticipate such conditions. In the U-Test project, we are developing a configurable search-based testing framework that exploits information from functional testing and from declarative descriptions of uncertainties. Itaims at revealing unintended behavior in the presence of uncertainties. This framework enables testing for different scenarios of uncertainty and thus, allows to achieve a certain coverage of those, and to find unknown uncertainty scenarios.


Cyber-Physical systems Reliability Search-based testing Uncertainty UML state machines 


  1. 1.
    Ramirez, A.J., Jensen, A.C., Cheng, B.H.C.: A taxonomy of uncertainty for dynamically adaptive systems. In: Proceedings of the 7th International Symposium on Software Engineering for Adaptive and Self-managing Systems, Piscataway, NJ, USA, pp. 99–108 (2012)Google Scholar
  2. 2.
    Goldsby, H.J., Cheng, B.H.C.: Automatically discovering properties that specify the latent behavior of UML models. In: Petriu, D.C., Rouquette, N., Haugen, Ø. (eds.) Model Driven Engineering Languages and Systems, pp. 316–330. Springer, Berlin Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Welsh, K., Sawyer, P.: Understanding the scope of uncertainty in dynamically adaptive systems. In: Wieringa, R., Persson, A. (eds.) Requirements Engineering: Foundation for Software Quality, pp. 2–16. Springer, Berlin Heidelberg (2010)CrossRefGoogle Scholar
  4. 4.
    Walker, W.E., et al.: Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support. Integr. Assess. 4(1), 5–17 (2003)Google Scholar
  5. 5.
    Refsgaard, J.C., van der Sluijs, J.P., Højberg, A.L., Vanrolleghem, P.A.: Uncertainty in the environmental modelling process – a framework and guidance. Environ. Model Softw. 22(11), 1543–1556 (2007)CrossRefGoogle Scholar
  6. 6.
    Erkoyuncu, J.A., Roy, R., Shehab, E., Cheruvu, K.: Understanding service uncertainties in industrial product–service system cost estimation. Int. J. Adv. Manuf. Technol. 52(9–12), 1223–1238 (2011)CrossRefGoogle Scholar
  7. 7.
    Brown, J.D.: Knowledge, uncertainty and physical geography: towards the development of methodologies for questioning belief. Trans. Inst. Br. Geogr. 29(3), 367–381 (2004)CrossRefGoogle Scholar
  8. 8.
    Faro, D., Rottenstreich, Y.: Affect, empathy, and regressive mispredictions of others’ preferences under risk. Manag. Sci. 52(4), 529–541 (2006)CrossRefGoogle Scholar
  9. 9.
    Knight, F.H.: Risk, Uncertainty and Profit. Courier Corporation (2012)Google Scholar
  10. 10.
    Emblemsvåg, J.: Life-Cycle Costing: Using Activity-Based Costing and Monte Carlo Methods to Manage Future Costs and Risks. Wiley, New Jersey (2003)Google Scholar
  11. 11.
    Krenn, W., Schlick, R., Tiran, S., Aichernig, B., Jobstl, E., Brandl, H.: MoMut::UML model-based mutation testing for UML. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1–8 (2015)Google Scholar
  12. 12.
    Fabbri, S.P.F., Delamaro, M.E., Maldonado, J.C., Masiero, P.C.: Mutation analysis testing for finite state machines. In: Proceedings of 5th International Symposium on Software Reliability Engineering, pp. 220–229 (1994)Google Scholar
  13. 13.
    DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Program mutation: a new approach to program testing. Infotech State Art Rep. Softw. Test. 2, 107–126 (1979)Google Scholar
  14. 14.
    Ammann, P.E., Black, P.E., Majurski, W.: Using model checking to generate tests from specifications. In: Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241), pp. 46–54 (1998)Google Scholar
  15. 15.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011)CrossRefGoogle Scholar
  16. 16.
    Luke, S.: Essentials of metaheuristics (2013).
  17. 17.
    McMinn, P.: Search-based software test data generation: a survey: research articles. Softw. Test Verif. Reliab. 14(2), 105–156 (2004)CrossRefGoogle Scholar
  18. 18.
    Harman, M., Zhang, Y., Mansouri, S.A.: Search based software engineering: a comprehensive analysis and review of trends techniques and applications. King’s College (2009)Google Scholar
  19. 19.
    Cheng, B.H.C., Sawyer, P., Bencomo, N., Whittle, J.: A goal-based modeling approach to develop requirements of an adaptive system with environmental uncertainty. In: Schürr, A., Selic, B. (eds.) Model Driven Engineering Languages and Systems, pp. 468–483. Springer, Berlin Heidelberg (2009)CrossRefGoogle Scholar
  20. 20.
    Tackling Uncertainty for Transportation Cyber-Physical Systems | CPS-VO. Accessed 25 Sep 2016
  21. 21.
    NIST Foundations for Innovation for Cyber-Physical Systems. Accessed 25 Sep 2016
  22. 22.
    Ramirez, A.J., Jensen, A.C., Cheng, B.H.C., Knoester, D.B.: Automatically exploring how uncertainty impacts behavior of dynamically adaptive systems. In: Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering, Washington, DC, USA, pp. 568–571 (2011)Google Scholar
  23. 23.
    Whittle, J., Sawyer, P., Bencomo, N., Cheng, B.H.C., Bruel, J.-M.: RELAX: a language to address uncertainty in self-adaptive systems requirement. Requir. Eng. 15(2), 177–196 (2010)CrossRefGoogle Scholar
  24. 24.
    DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: help for the practicing programmer. Computer 11(4), 34–41 (1978)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Martin A. Schneider
    • 1
    Email author
  • Marc-Florian Wendland
    • 1
  • Leon Bornemann
    • 1
  1. 1.Fraunhofer FOKUSBerlinGermany

Personalised recommendations