Skip to main content
SpringerLink
Log in

Verification Complexity: An Initial Look at Verification Artifacts

  • Conference paper
  • First Online:
The Proceedings of the 2023 Conference on Systems Engineering Research (CSER 2023)

Part of the book series: Conference on Systems Engineering Research Series ((CSERS))

Included in the following conference series:

  • 106 Accesses

Abstract

There has been increasing interest in developing numerical methods to inform the design of verification strategies in recent years. Existing work has been applied to toy problems that significantly simplify the complexity of verification in real-life applications, both in terms of the size of the problem and the interdependencies between the different elements of a verification strategy. To our knowledge, there is no publicly available evidence of the complexity of the verification “problem” in terms of requirements that need to be verified, verification activities that are conducted, and their interrelationships. In this chapter, we use knowledge graphs to visualize the size of the verification problem for two industrial projects, as captured in their verification artifacts, which include requirements traceability matrices and verification matrices.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 279.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. P.D. Collopy, A research agenda for the coming renaissance in systems engineering, in 50th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, (2012)

    Google Scholar 

  2. P.D. Collopy, Report on the science of systems engineering workshop, in 53rd AIAA Aerospace Sciences Meeting, (American Institute of Aeronautics and Astronautics, 2015)

    Google Scholar 

  3. P.D. Collopy, Systems engineering theory: What needs to be done, in Systems Conference (SysCon), 2015 9th Annual IEEE International, (2015)

    Google Scholar 

  4. A. Salado, R. Nilchiani, D. Verma, A contribution to the scientific foundations of systems engineering: Solution spaces and requirements. J. Syst. Sci. Syst. Eng. 26(5), 549–589 (2017)

    Article  Google Scholar 

  5. A. Salado, R. Nilchiani, On the evolution of solution spaces triggered by emerging technologies. Procedia Comput. Sci. 44, 155–163 (2015)

    Article  Google Scholar 

  6. A. Salado, R. Nilchiani, The concept of order of conflict in requirements engineering. IEEE Syst. J. 10(1), 25–35 (2016)

    Article  Google Scholar 

  7. P.D. Collopy, P.M. Hollingsworth, Value-driven design. J. Aircr. 48(3), 749–759 (2011)

    Article  Google Scholar 

  8. B.D. Lee, C.J.J. Paredis, A conceptual framework for value-driven design and systems engineering. Procedia CIRP 21, 10–17 (2014)

    Article  Google Scholar 

  9. A.W. Wymore, Model-Based Systems Engineering (CRC Press, Boca Raton, 1993)

    Google Scholar 

  10. INCOSE, Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities, Version 4.0 ed (Wiley, Hoboken, 2015)

    Google Scholar 

  11. D.M. Buede, W.D. Miller (eds.), The Engineering Design of Systems: Models and Methods, 3rd edn. (Wiley, Hoboken, 2016)

    Google Scholar 

  12. A. Engel, in Verification, Validation, and Testing of Engineered Systems, Wiley Series on Systems Engineering and Management, ed. by A.P. Sage, (Wiley, Hoboken, 2010)

    Chapter  Google Scholar 

  13. A. Salado, H. Kannan, Elemental patterns of verification strategies. Syst. Eng. 5, 370–388 (2019)

    Article  Google Scholar 

  14. A. Salado, H. Kannan, A mathematical model of verification strategies. Syst. Eng. 21, 583–608 (2018)

    Article  Google Scholar 

  15. G.A. Hazelrigg, A framework for decision-based engineering design. J. Mech. Des. 120(4), 653–658 (1998)

    Article  Google Scholar 

  16. G.A. Hazelrigg, An axiomatic framework for engineering design. J. Mech. Des. 121(3), 342–347 (1999)

    Article  Google Scholar 

  17. D.M. Buede, The Engineering Design of Systems: Models and Methods (Wiley, 2009)

    Book  Google Scholar 

  18. M. Tribus, Rational Descriptions, Decisions and Designs (Pergamon Press Inc, Oxford, 1969)

    Google Scholar 

  19. S.C. Thompson, C.J.J. Paredis, An investigation into the decision analysis of design process decisions. J. Mech. Des. 132(12), 121009–121009-9 (2010)

    Article  Google Scholar 

  20. J.M. Aughenbaugh, C.J. Paredis, The value of using imprecise probabilities in engineering design. J. Mech. Des. 128(4), 969–979 (2005)

    Article  Google Scholar 

  21. A. Salado, H. Kannan, Properties of the utility of verification, in IEEE International Symposium in Systems Engineering, (Rome, Italy, 2018)

    Google Scholar 

  22. ECSS, Space Engineering – Verification (European Cooperation for Space Standardization, Noordwijk, 2009)

    Google Scholar 

  23. ECSS, Space Engineering – Testing (European Cooperation for Space Standardization, Noordwijk, 2012)

    Google Scholar 

  24. D.S. Hamermesh, Expectations, life expectancy, and economic behavior. Q. J. Econ. 100(2), 389–408 (1985)

    Article  MathSciNet  Google Scholar 

  25. V.K. Smith, D.H. Taylor, F.A. Sloan, Longevity expectations and death: Can people predict their own demise? Am. Econ. Rev. 91(4), 1126–1134 (2001)

    Article  Google Scholar 

  26. M. Hoppe, A. Engel, S. Shachar, SysTest: Improving the verification, validation, and testing process – Assessing six industrial pilot projects. Syst. Eng. 10(4), 323–347 (2007)

    Article  Google Scholar 

  27. M. Barad, A. Engel, Optimizing VVT strategies: A decomposition approach. J. Oper. Res. Soc. 57(8), 965–974 (2005)

    Article  Google Scholar 

  28. P. Xu, A. Salado, A mathematical approach to design verification strategies that incorporate corrective activities as dedicated decisions. IEEE Open J. Syst. Eng. 1, 1–10 (2022)

    Article  Google Scholar 

  29. A.U. Kulkarni et al., An evaluation of the optimality of frequent verification for vertically integrated systems. Syst. Eng. 24(1), 17–33 (2021)

    Article  Google Scholar 

  30. M. Messer et al., Model selection under limited information using a value-of-information-based indicator. J. Mech. Des. 132(12), 121008 (2010)

    Article  Google Scholar 

  31. A. Engel, S. Shachar, Measuring and optimizing systems’ quality costs and project duration. Syst. Eng. 9(3), 259–280 (2006)

    Article  Google Scholar 

  32. A. Salado, H. Kannan, F. Farkhondehmaal, Capturing the Information Dependencies of Verification Activities with Bayesian Networks (Springer, Cham, 2019)

    Book  Google Scholar 

  33. F. Farkhondehmaal, A. Salado, Efficient population of the verification tradespace using Bayesian inference. IEEE Syst. J. 14(3), 3225–3232 (2020)

    Article  Google Scholar 

  34. A.U. Kulkarni et al., Is verifying frequently an optimal strategy? A belief-based model of verification, in ASME 2020 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, (2020)

    Google Scholar 

  35. A.U. Kulkarni, C. Wernz, A. Salado, Coordination of verification activities with incentives: A two-firm model. Res. Eng. Des. 32, 31–47 (2021)

    Article  Google Scholar 

  36. P. Xu, A. Salado, A concept for set-based design of verification strategies, in INCOSE International Symposium, (Orlando, FL, USA, 2019)

    Google Scholar 

  37. P. Xu, A. Salado, G. Xie, A reinforcement learning approach to design verification strategies of engineered systems, in IEEE International Conference on Systems, Man, and Cybernetics, (Toronto, Canada, 2020)

    Google Scholar 

  38. P. Xu, A. Salado, X. Deng, A parallel tempering approach for efficient exploration of the verification tradespace in engineered systems. IEEE Trans. Syst. Man Cybern. Syst. 2022, 1–13 (2022)

    Google Scholar 

Download references

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant No. CMMI-2205468.

Author information

Authors and Affiliations

  1. The University of Arizona, Tucson, AZ, USA

    Sukhwan Jung & Alejandro Salado

Authors
  1. Sukhwan Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alejandro Salado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alejandro Salado .

Editor information

Editors and Affiliations

  1. Stevens Institute of Technology, Hoboken, NJ, USA

    Dinesh Verma

  2. Astronautics Aerospace and Mechanical Engineering Department, University of Southern California, Los Angeles, CA, USA

    Azad M. Madni

  3. Endevor, Wilmington, DE, USA

    Steven Hoffenson

  4. Stevens Institute of Technology, Hoboken, NJ, USA

    Lu Xiao

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jung, S., Salado, A. (2024). Verification Complexity: An Initial Look at Verification Artifacts. In: Verma, D., Madni, A.M., Hoffenson, S., Xiao, L. (eds) The Proceedings of the 2023 Conference on Systems Engineering Research. CSER 2023. Conference on Systems Engineering Research Series. Springer, Cham. https://doi.org/10.1007/978-3-031-49179-5_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49179-5_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49178-8

  • Online ISBN: 978-3-031-49179-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation