Skip to main content

A Proposal for the Classification of Methods for Verification and Validation of Safety, Cybersecurity, and Privacy of Automated Systems

  • Conference paper
  • First Online:
Quality of Information and Communications Technology (QUATIC 2021)

Abstract

As our dependence on automated systems grows, so does the need for guaranteeing their safety, cybersecurity, and privacy (SCP). Dedicated methods for verification and validation (V&V) must be used to this end and it is necessary that the methods and their characteristics can be clearly differentiated. This can be achieved via method classifications. However, we have experienced that existing classifications are not suitable to categorise V&V methods for SCP of automated systems. They do not pay enough attention to the distinguishing characteristics of this system type and of these quality concerns. As a solution, we present a new classification developed in the scope of a large-scale industry-academia project. The classification considers both the method type, e.g., testing, and the concern addressed, e.g., safety. Over 70 people have successfully used the classification on 53 methods. We argue that the classification is a more suitable means to categorise V&V methods for SCP of automated systems and that it can help other researchers and practitioners.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Amalthea4public project: D3.1 - Analysis of state of the art V&V techniques (2015)

    Google Scholar 

  2. Arfelt, E., Basin, D., Debois, S.: Monitoring the GDPR. In: Sako, K., Schneider, S., Ryan, P.Y.A. (eds.) ESORICS 2019. LNCS, vol. 11735, pp. 681–699. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29959-0_33

    Chapter  Google Scholar 

  3. Avizienis, A., et al.: Fundamental concepts of dependability. University of Newcastle (2001)

    Google Scholar 

  4. Barbosa, R., et al.: The VALU3S ECSEL project: verification and validation of automated systems safety and security. In: DSD 2020 (2020)

    Google Scholar 

  5. Bartocci, E., Manjunath, N., Mariani, L., Mateis, C., Ničković, D.: Automatic failure explanation in CPS models. In: Ölveczky, P.C., Salaün, G. (eds.) SEFM 2019. LNCS, vol. 11724, pp. 69–86. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30446-1_4

    Chapter  Google Scholar 

  6. Belmonte, L., et al.: Feeling of safety and comfort towards a socially assistive unmanned aerial vehicle that monitors people in a virtual home. Sensors 21(3), 908 (2021)

    Article  Google Scholar 

  7. Bozzano, M., Cimatti, A., Griggio, A., Mattarei, C.: Efficient anytime techniques for model-based safety analysis. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 603–621. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21690-4_41

    Chapter  Google Scholar 

  8. Cassar, I., et al: A survey of runtime monitoring instrumentation techniques. PrePost@iFM (2017)

    Google Scholar 

  9. CENELEC: EN 50128 - Railway applications - Communication, signalling and processing systems - Software for railway control and protection systems (2020)

    Google Scholar 

  10. Cimatti, A., Tian, C., Tonetta, S.: Assumption-based runtime verification with partial observability and resets. In: Finkbeiner, B., Mariani, L. (eds.) RV 2019. LNCS, vol. 11757, pp. 165–184. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32079-9_10

    Chapter  Google Scholar 

  11. Clarke, E.M., et al.: Handbook of Model Checking. Springer, Heidelberg (2018). https://doi.org/10.1007/978-3-319-10575-8

    Book  MATH  Google Scholar 

  12. de la Vara, J.L., et al.: Assurance and certification of cyber-physical systems: the AMASS open source ecosystem. J. Syst. Softw. 171, 110812 (2021)

    Article  Google Scholar 

  13. Dias, R., et al.: Verifying concurrent programs using contracts. In: ICST 2017 (2017)

    Google Scholar 

  14. Duckham, M., Kulik, L.: Simulation of obfuscation and negotiation for location privacy. In: Cohn, A.G., Mark, D.M. (eds.) COSIT 2005. LNCS, vol. 3693, pp. 31–48. Springer, Heidelberg (2005). https://doi.org/10.1007/11556114_3

    Chapter  Google Scholar 

  15. Fonseca, J., et al.: Analysis of field data on web security vulnerabilities. IEEE Trans. Dependable Secure Comput. 11(2), 89–100 (2014)

    Article  Google Scholar 

  16. Gallina, B., et al.: Multi‐concern dependability‐centered assurance for space systems via ConcertoFLA. Ada-Europe (2018)

    Google Scholar 

  17. Halfind, W.G.J., et al.: A classification of SQL injection attacks and countermeasures. In: ISSSE 2006 (2006)

    Google Scholar 

  18. Herdt, V., et al.: Efficient cross-level testing for processor verification: a RISC-V case-study. In: FDL 2020 (2020)

    Google Scholar 

  19. Humbatova, N., et al.: Taxonomy of real faults in deep learning systems. In: ICSE 2020 (2020)

    Google Scholar 

  20. IEC: IEC 61508 - Functional safety of electrical/electronic/programmable electronic safety-related systems (2011)

    Google Scholar 

  21. Hähnle, R., Huisman, M.: Deductive software verification: from pen-and-paper proofs to industrial tools. In: Steffen, B., Woeginger, G. (eds.) Computing and Software Science. LNCS, vol. 10000, pp. 345–373. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-91908-9_18

    Chapter  Google Scholar 

  22. IEEE: IEEE Std 1012 - IEEE Standard for System, Software, and Hardware V&V (2016)

    Google Scholar 

  23. Kammueller, F.: Formal modeling and analysis of data protection for GDPR compliance of IoT healthcare systems. In: SMC 2018 (2018)

    Google Scholar 

  24. Khalastchi, E., Kalech, M.: On fault detection and diagnosis in robotic systems. ACM Comput. Surv. 51(1), 9 (2018)

    Article  Google Scholar 

  25. Kuhn, T., Antonino, P.O., Bachorek, A.: A simulator coupling architecture for the creation of digital twins. In: Muccini, H., et al. (eds.) ECSA 2020. CCIS, vol. 1269, pp. 326–339. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59155-7_25

    Chapter  Google Scholar 

  26. Kramer, A., Legeard, B.: Model-Based Testing Essentials. Wiley, Hoboken (2016)

    Book  Google Scholar 

  27. Laskey, M., et al.: DART: noise injection for robust imitation learning. In: CoRL 2017 (2017)

    Google Scholar 

  28. Luckcuck, M., et al.: Formal specification and verification of autonomous robotic systems: a survey. ACM Comput. Surv. 52(5), 100 (2019)

    Article  Google Scholar 

  29. Nair, S., et al.: An extended systematic literature review on provision of evidence for safety certification. Inf. Softw. Technol. 56(7), 689–717 (2014)

    Article  Google Scholar 

  30. Natella, R., et al.: Assessing dependability with software fault injection: a survey. ACM Comput. Surv. 48(3), 44 (2016)

    Article  Google Scholar 

  31. Oxford UK Dictionary: method (2021). https://www.lexico.com/definition/method

  32. Pan, L., et al.: Cyber security attacks to modern vehicular systems. J. Inf. Secur. Appl. 36, 30–100 (2017)

    Google Scholar 

  33. Pandit, H.J., O’Sullivan, D., Lewis, D.: Test-driven approach towards GDPR compliance. In: Acosta, M., Cudré-Mauroux, P., Maleshkova, M., Pellegrini, T., Sack, H., Sure-Vetter, Y. (eds.) SEMANTiCS 2019. LNCS, vol. 11702, pp. 19–33. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33220-4_2

    Chapter  Google Scholar 

  34. Parra, E., et al.: Advances in artefact quality analysis for safety-critical systems. In: ISSRE 2019 (2019)

    Google Scholar 

  35. Paz, A., El Boussaidi, G.: A requirements modelling language to facilitate avionics software verification and certification. In: RET 2019 (2019)

    Google Scholar 

  36. Rival, X., Yi, K.: Introduction to Static Analysis. An Abstract Interpretation Perspective. MIT Press (2020)

    Google Scholar 

  37. Sangchoolie, B., et al.: A study of the interplay between safety and security using model-implemented fault injection. In: EDCC 2018 (2018)

    Google Scholar 

  38. Savary, A., Frappier, M., Leuschel, M., Lanet, J.-L.: Model-based robustness testing in event-B using mutation. In: Calinescu, R., Rumpe, B. (eds.) SEFM 2015. LNCS, vol. 9276, pp. 132–147. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22969-0_10

    Chapter  Google Scholar 

  39. Skoglund, M., et al.: Black-box testing for security-informed safety of automated driving systems. In: VTC 2021-Spring (2021)

    Google Scholar 

  40. Timperley, C.S., et al.: Crashing simulated planes is cheap: Can simulation detect robotics bugs early? In: ICST 2018 (2018)

    Google Scholar 

  41. Tsachouridis, V.A., et al.: Formal analysis of the Schulz matrix inversion algorithm: a paradigm towards computer aided verification of general matrix flow solvers. Numer. Algebra Control Optim. 10(2), 177–206 (2020)

    Article  MathSciNet  Google Scholar 

  42. US DoD: Defense Modeling & Simulation Coordination Office, V&V Technique Taxonomy (2001). https://vva.msco.mil/default.htm?Ref_Docs/VVTechniques/

  43. VALU3S project: D3.1 - V&V methods for SCP evaluation of automated systems (2021)

    Google Scholar 

  44. Yang, Y., et al.: Man-in-the-middle attack test-bed investigating cyber-security vulnerabilities in smart grid SCADA systems. In: SUPERGEN 2012 (2012)

    Google Scholar 

Download references

Acknowledgments

The research leading to this paper has received funding from the VALU3S (H2020-ECSEL grant agreement no 876852; Spain’s MICINN ref. PCI2020-112001), iRel4.0 (H2020-ECSEL grant agreement no 876659; MICINN ref. PCI2020-112240), and Treasure (JCCM SBPLY/19/180501/000270; European Regional Development Fund) projects, and from the Ramon y Cajal Program (MICINN RYC-2017-22836; European Social Fund). We are also grateful to all the VALU3S partners that have provided input and feedback for the development of the classification.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jose Luis de la Vara .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

de la Vara, J.L. et al. (2021). A Proposal for the Classification of Methods for Verification and Validation of Safety, Cybersecurity, and Privacy of Automated Systems. In: Paiva, A.C.R., Cavalli, A.R., Ventura Martins, P., Pérez-Castillo, R. (eds) Quality of Information and Communications Technology. QUATIC 2021. Communications in Computer and Information Science, vol 1439. Springer, Cham. https://doi.org/10.1007/978-3-030-85347-1_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85347-1_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85346-4

  • Online ISBN: 978-3-030-85347-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics