Towards Learning and Verifying Invariants of Cyber-Physical Systems by Code Mutation

  • Yuqi Chen
  • Christopher M. Poskitt
  • Jun Sun
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9995)


Cyber-physical systems (CPS), which integrate algorithmic control with physical processes, often consist of physically distributed components communicating over a network. A malfunctioning or compromised component in such a CPS can lead to costly consequences, especially in the context of public infrastructure. In this short paper, we argue for the importance of constructing invariants (or models) of the physical behaviour exhibited by CPS, motivated by their applications to the control, monitoring, and attestation of components. To achieve this despite the inherent complexity of CPS, we propose a new technique for learning invariants that combines machine learning with ideas from mutation testing. We present a preliminary study on a water treatment system that suggests the efficacy of this approach, propose strategies for establishing confidence in the correctness of invariants, then summarise some research questions and the steps we are taking to investigate them.


Support Vector Machine Sensor Data Water Treatment Plant Mutation Testing Programmable Logic Controller 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We thank Pingfan Kong for assisting us with the SWaT simulator, and the anonymous referees for their helpful comments and criticisms. This work was supported by NRF Award No. NRF2014NCR-NCR001-40.


  1. 1.
    Secure Water Treatment (SWaT). Accessed Sep 2016
  2. 2.
    Supplementary material.
  3. 3.
    Adepu, S., Mathur, A.: Distributed detection of single-stage multipoint cyber attacks in a water treatment plant. In: Proceedings of ACM Asia Conference on Computer and Communications Security (AsiaCCS 2016), pp. 449–460. ACM (2016)Google Scholar
  4. 4.
    Adepu, S., Mathur, A.: Using process invariants to detect cyber attacks on a water treatment system. In: Hoepman, J.-H., Katzenbeisser, S. (eds.) Proceedings of International Conference on ICT Systems Security and Privacy Protection (SEC 2016). IFIP AICT, vol. 471, pp. 91–104. Springer, New York (2016)CrossRefGoogle Scholar
  5. 5.
    Alves, T., Felton, D.: TrustZone: integrated hardware and software security. ARM white paper (2004)Google Scholar
  6. 6.
    Anati, I., Gueron, S., Johnson, S.P., Scarlata, V.R.: Innovative technology for CPU based attestation and sealing. Intel white paper (2013)Google Scholar
  7. 7.
    Cárdenas, A.A., Amin, S., Sastry, S.: Research challenges for the security of control systems. In: Proceedings of USENIX Workshop on Hot Topics in Security (HotSec 2008). USENIX Association (2008)Google Scholar
  8. 8.
    Castelluccia, C., Francillon, A., Perito, D., Soriente, C.: On the difficulty of software-based attestation of embedded devices. In: Proceedings of ACM Conference on Computer and Communications Security (CCS 2009), pp. 400–409. ACM (2009)Google Scholar
  9. 9.
    Choudhari, A., Ramaprasad, H., Paul, T., Kimball, J.W., Zawodniok, M.J., McMillin, B.M., Chellappan, S.: Stability of a cyber-physical smart grid system using cooperating invariants. In: Proceedings of IEEE Computer Software and Applications Conference (COMPSAC 2013), pp. 760–769. IEEE (2013)Google Scholar
  10. 10.
    Clarke, E.M., Zuliani, P.: Statistical model checking for cyber-physical systems. In: Bultan, T., Hsiung, P.-A. (eds.) ATVA 2011. LNCS, vol. 6996, pp. 1–12. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-24372-1_1 CrossRefGoogle Scholar
  11. 11.
    Frehse, G., Guernic, C., Donzé, A., Cotton, S., Ray, R., Lebeltel, O., Ripado, R., Girard, A., Dang, T., Maler, O.: SpaceEx: scalable verification of hybrid systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 379–395. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-22110-1_30 CrossRefGoogle Scholar
  12. 12.
    Gao, S., Kong, S., Clarke, E.M.: dReal: An SMT solver for nonlinear theories over the reals. In: Bonacina, M.P. (ed.) CADE 2013. LNCS (LNAI), vol. 7898, pp. 208–214. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38574-2_14 CrossRefGoogle Scholar
  13. 13.
    Hasuo, I., Suenaga, K.: Exercises in nonstandard static analysis of hybrid systems. In: Madhusudan, P., Seshia, S.A. (eds.) CAV 2012. LNCS, vol. 7358, pp. 462–478. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-31424-7_34 CrossRefGoogle Scholar
  14. 14.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011)CrossRefGoogle Scholar
  15. 15.
    Kang, E., Adepu, S., Jackson, D., Mathur, A.P.: Model-based security analysis of a water treatment system. In: Proceedings of International Workshop on Software Engineering for Smart Cyber-Physical Systems (SEsCPS 2016), pp. 22–28. ACM (2016)Google Scholar
  16. 16.
    Khaitan, S.K., McCalley, J.D.: Design techniques and applications of cyberphysical systems: a survey. IEEE Syst. J. 9(2), 350–365 (2015)CrossRefGoogle Scholar
  17. 17.
    Kong, P., Li, Y., Chen, X., Sun, J., Sun, M., Wang, J.: Towards concolic testing for hybrid systems. In: Fitzgerald, J., et al. (eds.) FM 2016. LNCS-FM, vol. 9995, pp. 460–478. Springer, Heidelberg (2016)CrossRefGoogle Scholar
  18. 18.
    Lee, E.A.: Cyber physical systems: design challenges. In: Proceedings of International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC 2008), pp. 363–369. IEEE (2008)Google Scholar
  19. 19.
    Maier, A.: Online passive learning of timed automata for cyber-physical production systems. In: Proceedings of IEEE International Conference on Industrial Informatics (INDIN 2014), pp. 60–66. IEEE (2014)Google Scholar
  20. 20.
    Mitsch, S., Platzer, A.: ModelPlex: verified runtime validation of verified cyber-physical system models. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 199–214. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-11164-3_17 Google Scholar
  21. 21.
    Offutt, A.J., Lee, A., Rothermel, G., Untch, R.H., Zapf, C.: An experimental determination of sufficient mutant operators. ACM Trans. Softw. Eng. Methodol. (TOSEM) 5(2), 99–118 (1996)CrossRefGoogle Scholar
  22. 22.
    Paul, T., Kimball, J.W., Zawodniok, M.J., Roth, T.P., McMillin, B.M., Chellappan, S.: Unified invariants for cyber-physical switched system stability. IEEE Trans. Smart Grid 5(1), 112–120 (2014)CrossRefGoogle Scholar
  23. 23.
    Platzer, A., Quesel, J.-D.: KeYmaera: a hybrid theorem prover for hybrid systems (system description). In: Armando, A., Baumgartner, P., Dowek, G. (eds.) IJCAR 2008. LNCS (LNAI), vol. 5195, pp. 171–178. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-71070-7_15 CrossRefGoogle Scholar
  24. 24.
    Quesel, J., Mitsch, S., Loos, S.M., Arechiga, N., Platzer, A.: How to model and prove hybrid systems with KeYmaera: a tutorial on safety. Int. J. Softw. Tools Technol. Transf. 18(1), 67–91 (2016)CrossRefGoogle Scholar
  25. 25.
    Roth, T., McMillin, B.: Physical attestation of cyber processes in the smart grid. In: Luiijf, E., Hartel, P. (eds.) CRITIS 2013. LNCS, vol. 8328, pp. 96–107. Springer, Heidelberg (2013). doi: 10.1007/978-3-319-03964-0_9 CrossRefGoogle Scholar
  26. 26.
    Seshadri, A., Perrig, A., van Doorn, L., Khosla, P.K.: SWATT: software-based ATTestation for embedded devices. In: Proceedings of IEEE Symposium on Security and Privacy (S&P 2004), p. 272. IEEE (2004)Google Scholar
  27. 27.
    Valente, J., Barreto, C., Cárdenas, A.A.: Cyber-physical systems attestation. In: Proceedings of IEEE International Conference on Distributed Computing in Sensor Systems (DCOSS 2014), pp. 354–357. IEEE (2014)Google Scholar
  28. 28.
    Vodencarevic, A., Kleine Büning, H., Niggemann, O., Maier, A.: Identifying behavior models for process plants. In: Proceedings of IEEE Conference on Emerging Technologies & Factory Automation (ETFA 2011), pp. 1–8. IEEE (2011)Google Scholar
  29. 29.
    Wang, J., Sun, J., Yuan, Q., Pang, J.: Should we learn probabilistic models for model checking? a new approach and an empirical study. CoRR abs/1605.08278 (2016).
  30. 30.
    Zheng, X., Julien, C., Kim, M., Khurshid, S.: Perceptions on the state of the art in verification and validation in cyber-physical systems. IEEE Syst. J. PP(99), 1–14 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Singapore University of Technology and DesignSingaporeSingapore

Personalised recommendations