Modelling the Efficacy of Assurance Strategies for Better Integration, Interoperability and Information Assurance in Family-of-System-of-Systems Portfolios

  • Keith Joiner
  • Mahmoud EfatmaneshnikEmail author
  • Malcolm Tutty
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 878)


Military systems, and more broadly society’s, are increasingly complex and interconnected enabling hitherto only dreamed of capabilities and yet also humanity’s forays into wholesale malicious cyber-warfare. Loosely coupled families-of-systems of systems cooperate and evolve sporadically when using linear lifecycles and project-by-project development, defying capability control and assurance at that mesa-level. The U.S. Defense has evolved systematic ways for their families-of-systems to be progressively more integrated, interoperable and information assured and this is dramatically differentiating its capability assurance from its allies. This paper reports new Markovian testability modelling comparing the abstract efficacy of assurance experimentation and testing strategies employed by Australia Defence against the new U.S. strategies that are now able to quantitatively illustrate the widening gap between these allies. The modelling technique has potential to tailor Australian plans to keep pace with its ally and in modelling civilian families-of-system-of-systems in transportation, energy healthcare and the like.



Authors are gracious for rich and helpful dissuasions with CMDR Kate Miller, whose insights facilitated shaping this paper.


  1. 1.
    Jenner, S.: Why do projects ‘fail’ and more to the point what can we do about it? The case for disciplined, ‘fast and frugal’ decision-making. Management 45, 6–19 (2015)Google Scholar
  2. 2.
    Hecht, M.: Verification of software intensive system reliability and availability through testing and modeling. ITEA J. 36, 304–312 (2015)Google Scholar
  3. 3.
    Courtney, J., Merali, Y., Paradice, D., et al.: On the study of complexity in information systems. Int. J. Inf. Technol. Syst. Approach 1(1), 37–48 (2008)CrossRefGoogle Scholar
  4. 4.
    Flyvbjerg, B., Budzier, A.: Why your IT project may be riskier than you think. Harv. Bus. Rev. 24–27 (2011)Google Scholar
  5. 5.
    Ahner, D.K.: Better buying power, developmental testing, and scientific test and analysis techniques. ITEA J. 37, 286–290 (2016)Google Scholar
  6. 6.
    Joiner, K.F.: Six-sigma reform and education in Australian Defence: lessons-learned give rigour and efficiency to ordnance, aircraft and ship testing. In: 7th International Conference on Lean Six Sigma, Dubai, United Arab Emirates (2018)Google Scholar
  7. 7.
    Murphy, T., Leiby, L.D., Glaeser, K., et al.: How scientific test and analysis techniques can assist the chief developmental tester. ITEA J. 36, 96–101 (2015)Google Scholar
  8. 8.
    Cofer, D.: Taming the complexity beast. ITEA J. 36, 313–318 (2015)Google Scholar
  9. 9.
    Kuhn, D.R., Kacker, R.N., Feldman, L., et al.: Combinatorial testing for cybersecurity and reliability (2016)Google Scholar
  10. 10.
    Copeland, E.J., Holzer, T.H., Eveleigh, T.J., et al.: The effects of system prototype demonstrations on weapon systems. In: Defense Acquisition Univ FT Belvoir VA (2015)Google Scholar
  11. 11.
    Joiner, K.F.: How new test and evaluation policy is being used to de-risk project approvals through preview T&E. ITEA J. 36, 288–297 (2015)Google Scholar
  12. 12.
    Joiner, K.F.: How Australia can catch up to US cyber resilience by understanding that cyber survivability test and evaluation drives defense investment. Inf. Secur. J. Glob. Perspect. 26, 74–84 (2017)CrossRefGoogle Scholar
  13. 13.
    Mead, N.R., Woody, C.: Cyber Security Engineering: A Practical Approach for Systems and Software Assurance. Addison-Wesley Professional, Boston (2016)Google Scholar
  14. 14.
    Joiner, K.F.: Implementing the defence first principles review: two key opportunities to achieve best practice in capability development. In: Strategic Insights: Australian Strategic Policy Institute (2015).
  15. 15.
    Keating, C.B., Katina, P.F., Joiner, K.F., et al.: A method for identification, representation, and assessment of complex system pathologies in acquisition programs. In: 15th Annual Acquisition Research Symposium, Graduate School of Business & Public Policy at the Naval Postgraduate School, Monterey, California (2018)Google Scholar
  16. 16.
    Association IS: Standard for Fail-Safe Design of Autonomous and Semi-Autonomous Systems P7009 (under development) (2018)Google Scholar
  17. 17.
    Joiner, K.F., Tutty, M.G.: A tale of two allied defence departments: new assurance initiatives for managing increasing system complexity, interconnectedness and vulnerability. Aust. J. Multi-Discipl. Eng. 14, 4–25 (2018)CrossRefGoogle Scholar
  18. 18.
    Tutty, M.G.: The profession of arms in the information age: operational joint fires capability preparedness in a small-world. University of South Australia (2016)Google Scholar
  19. 19.
    Layton, P.: Fifth-generation air warfare. Aust. Def. Force J. 204, 23–32 (2018)Google Scholar
  20. 20.
    Efatmaneshnik, M., Shoval, S., Ryan, M.: A framework for testability analysis from system architecture perspective. In: INCOSE Annual Symposium, Washington, D.C., 18–22 July 2018 (2018)Google Scholar
  21. 21.
    Australian Senate and Australian National Audit Office (ANAO): Test and Evaluation of Major Defence Equipment Acquisitions (2015)Google Scholar
  22. 22.
    Elele, J.N., Hall, D.H., Davis, M.E., et al.: M&S requirements and VV&A requirements: what’s the relationship? ITEA J. 37, 333–341 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Keith Joiner
    • 1
  • Mahmoud Efatmaneshnik
    • 2
    Email author
  • Malcolm Tutty
    • 3
  1. 1.School of Engineering and Information Technology, Capability Systems CentreUniversity of New South Wales – Canberra, Australian Defence Force AcademyCampbell, ACTAustralia
  2. 2.Capability Systems CentreUniversity of New South Wales – Canberra, Australian Defence Force AcademyCampbell, ACTAustralia
  3. 3.Air Power Development Centre, Royal Australian Air Force, Fairbairn Offices, Department of DefenceCanberra, ACTAustralia

Personalised recommendations