Abstract
With backgrounds in the science of information fusion and information technology, a review of Systems Engineering (SE) for Artificial Intelligence (AI)-based systems is provided across time, first with a brief history of AI and then the systems’ perspective based on the lead author’s experience with information fusion processes. The different types of AI are reviewed, such as expert systems and machine learning. Then SE is introduced and how it has evolved and must evolve further to become fully integrated with AI, such that both disciplines can help each other move into the future and evolve together. Several SE issues are reviewed, including risk, technical debt, software engineering, test and evaluation, emergent behavior, safety, and explainable AI.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adadi, A., & Berrada, M. (2018). Peeking Inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6, 52138–52160.
Adcock, R., et al. (2020). Emergence, systems engineering body of knowledge WIKI page. https://www.sebokwiki.org/wiki/Emergence. Retrieved February 2020.
AI Next Campaign. https://www.darpa.mil/work-with-us/ai-next-campaign. Retrieved February 2020.
Amershi, S., et al. (2019). Software engineering for machine learning: A case study. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP).
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mane, D. (2016). Concrete problems in AI safety. arXiv:1606.06565.
Anderson, N., & Nolte, W. (2005). Systems engineering principles applied to basic research and development. In Georgia Tech Space Systems Engineering Conference, Paper No. GTSSEC.D.5, Retrieved November 8–10, 2005.
Arrieta, A., et al. (2020). Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AIO. Information Fusion, 58, 82–115.
Belani, H., Vuković, M., & Car, Z. (2019). Requirements engineering challenges in building AI-based complex systems. arXiv:1908.11791.
Bennington, H. D. (1983). Production of large computer programs. Annals of the History of Computing, 5(4), 350–361. October–December 1983. https://doi.org/10.1109/MAHC.1983.10102.
Bergstein, B. (2020). AI still gets confused about how the world works. MIT Technology Review, 123(2), pp. 62–65. March/April 2020.
Bloehdorn S., & Hotho A. (2009). Ontologies for machine learning. In S. Staab, R. Studer (Eds.), Handbook on Ontologies: International Handbooks on Information Systems. Springer, Berlin, Heidelberg.
Brooks, F. P., Jr. (1982). The mythical man-month: Essays on software engineering. Reading, Mass.: Addison-Wesley Pub. Co.
Chaillan, N., & Yasar, H. (2019). Waterfall to DevSecOps in DoD, YouTube presentation. Retrieved March 15, 2021, from https://www.youtube.com/watch?v=vZ6AT7u_BcM.
Chalmers, D. J. (2006). Strong and weak emergence. In P. Clayton & P. Davies (Eds.), The re-emergence of emergence (pp. 244–256). UK; Oxford University Press.
Carnegie-Mellon University online course materials, Machine Learning in Production/AI Engineering. https://ckaestne.github.io/seai/. Retrieved February 2020.
Cognitive Uncontainability (2019). In Arbital. Retrieved May 19, 2019, from https://arbital.com/p/uncontainability/.
Corea, F. AI knowledge map: How to classify AI technologies. https://www.forbes.com/sites/cognitiveworld/2018/08/22/ai-knowledge-map-how-to-classify-ai-technologies/#2ec3f6337773.
Cunningham, W. OOPSLA ‘92: Addendum to the proceedings on object-oriented programming systems, languages, and applications (Addendum), pp. 29–30. Retrieved December 1992. https://doi-org.gate.lib.buffalo.edu/. https://doi.org/10.1145/157709.157715.
DARPA releases AIMEE call for proposals, Intelligence Community News. https://intelligencecommunitynews.com/darpa-releases-aimee-call-for-proposals/.
DeBrusk, C. The risk of machine learning bias (and How to Prevent it). https://www.oliverwyman.com/our-expertise/insights/2018/mar/the-risk-of-machine-learning-bias--and-how-to-prevent-it.html.
Dwyer, M. (2019). AI principles and the challenge of implementation. https://www.csis.org/analysis/ai-principles-and-challenge-implementation.
Faria, J. M. (2018). Machine learning safety: An overview. In Proceedings of the 26th Safety Critical Systems Symposium. New York, UK. Retrieved February 2018.
Flasch, P. (2019). Performance evaluation in machine learning: The good, the bad, the ugly and the way forward. In Proceedings of the 33th AAAI Conference on Artificial Intelligence
Fromm, J. (2005). Types and forms of emergence, distributed systems group, Universität Kassel, Germany. http://arxiv.org/ftp/nlin/papers/0506/0506028.pdf.
Guizzardi, R., et al. (2020). Ethical requirements for AI systems. In C. Goutte & X. Zhu (Eds.), Advances in artificial intelligence (pp. 251–256). Springer International Publishing.
Gunning, D. (2016). Explainable artificial intelligence, online presentation. https://www.cc.gatech.edu/~alanwags/DLAI2016/(Gunning)%20IJCAI-16%20DLAI%20WS.pdf.
Hall, M., Harborne, D., Tomsett, R., Galetic, V., Quintana-Amate, S., Nottle, A., & Preece, A. (2019). A systematic method to understand requirements for explainable AI (XAI) systems. In Proceedings of the IJCAI Workshop on eXplainable Artificial Intelligence (XAI 2019). Macau, China.
Hawley, S. H. (2019). Challenges for an ontology of artificial intelligence, perspectives on science and christian faith (Vol. 71), Special Edition on A.I.
Hernández-Orallo, J. (2017). Evaluation in artificial intelligence: From task-oriented to ability-oriented measurement. Artificial Intelligence Review, 48, 397–447. https://doi.org/10.1007/s10462-016-9505-7
Horkoff, J., Aydemir, F. B., et al. (2019). Goal-oriented requirements engineering: An extended systematic mapping study. Requirements Engineering, 24(2), 133–160.
Kass, A., & Leake, D. (1987). Types of explanations, Technical Report ADA183253, DTIC Document.
Kruchten, P., Nord, R. L., & Ozkaya, I. (2012). Technical debt: From metaphor to theory and practice. IEEE Software, 29(6), 18–21.
Kumar, A. QA—Metamorphic testing for machine learning models. https://vitalflux.com/qa-metamorphic-testing-machine-learning/.
Lombardo, N., et al. (2015). A systems engineering framework for R&D organizations. In 25th Annual INCOSE International Symposium (IS2015) Seattle, WA. Retrieved July 13–16, 2015.
Military standard: Defense system software development. United States Department of Defense. Retrieved February 29, 1988.
Marr, B. (2010) The biggest challenges facing artificial intelligence (AI). In Business and society. https://www.forbes.com/sites/bernardmarr/2017/07/13/the-biggest-challenges-facing-artificial-intelligence-ai-in-business-and-society/#33ea75ef2aec [CMMI, 2010] Software Engineering for AI-Enabled Systems; https://ckaestne.github.io/seai/--CMMI Product Team, CMMI® for Development, Version 1.3, Carnegie-Mellon SEI Tech Report CMU/SEI-2010-TR-033, November 2010.
Martin, J., & Oxman, S. (1988). Building expert systems: A tutorial. Prentice-Hall, Inc.
Martini, A., Besker, T., & Bosch, J. (2018). Technical debt tracking: Current state of practice a survey and multiple case study in 15 large organizations. Science of Computer Programming.
Morales, J., Turner, R., Miller, S., Capell, P., Place, P., & Shepard, D. J. (2020). Guide to implementing DevSecOps for a system of systems in highly regulated environments. Technical report CMU/SEI-2020-TR-002. Retrieved April 2020.
Nandi, C., et al. (2017). Debugging probabilistic programs. In Proceedings of MAPL’17. Barcelona, Spain. Retrieved June 18, 2017.
Neace, K. S., & Chipkevich, M. B. A. (2018). Designed complex adaptive systems exhibiting weak emergence. In NAECON 2018—IEEE National Aerospace and Electronics Conference. Dayton, OH. Retrieved July 23–26, 2018.
Prasanna, S. (2020). How to debug machine learning models to catch issues early and often. https://towardsdatascience.com/how-to-debug-machine-learning-models-to-catch-issues-early-and-often-5663f2b4383b. Retrieved February 2020.
Rossi, F., & Mattei, M. (2019). Building ethically bounded AI. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI).
Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., & Crespo, J. F. (2015). Hidden technical debt in machine learning systems. In Neural Information Processing Systems (NIPS).
Shewan, D. 10 Companies using machine learning in cool ways. https://www.wordstream.com/blog/ws/2017/07/28/machine-learning-applications.
Subarna, D. (2020). Stages to develop an expert system. https://www.engineeringenotes.com/artificial-intelligence-2/expert-systems/stages-to-develop-an-expert-system-artificial-intelligence/35592. Retrieved February 2020.
The Mitre Corporation. (2020). The evolution of systems engineering. https://www.mitre.org/publications/systems-engineering-guide/systems-engineering-guide/the-evolution-of-systems. Retrieved February 2020.
Top 12 Software Development Methodologies & its Advantages & Disadvantages. (2015). https://www.tatvasoft.com/blog/top-12-software-development-methodologies-and-its-advantages-disadvantages/.
Trivedi, Mayank, T12 ways AI is transforming devops. Retrieved March 15, 2021, from https://www.tadigital.com/insights/perspectives/12-ways-ai-transforming-devops.
Turing, A. M. (1950). Computing machinery and intelligence. Mind, New Series, 59(236), 433–460.
Varhol, P. To agility and beyond: The history—and legacy—of agile development. https://techbeacon.com/app-dev-testing/agility-beyond-history-legacy-agile-development
Vinge, V. (1993). Technological singularity. In VISION-21 Symposium sponsored by NASA Lewis Research Center and the Ohio Aerospace Institute.
Vogelsang, A., & Borg, M. (2019). Requirements engineering for machine learning: Perspectives from data scientists. In IEEE 27th International Requirements Engineering Conference Workshops.
Vora, T. (2019). Design thinking for AI: Sustainable AI solution design. BLOG. https://www.cuelogic.com/blog/design-thinking-for-ai.
Walch, K. Rethinking weak versus. Strong AI. https://www.forbes.com/sites/cognitiveworld/2019/10/04/rethinking-weak-vs-strong-ai/?sh=7d8eb9c46da3.
Yampolskiy, R. V. (2019). Unexplainability and incomprehensibility of artificial intelligence. https://arxiv.org/ftp/arxiv/papers/1907/1907.03869.pdf.
Ways AI Transforms Software Development. (2018). https://www.metamaven.com/6-ways-machine-learning-transforms-software-development/.
Zeller, M. (2018). Leveraging AI and machine learning as competitive business drivers. Retrieved August 3, 2018, from https://www.dataversity.net/leveraging-ai-machine-learning-competitive-business-drivers/.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Llinas, J., Fouad, H., Mittu, R. (2021). Systems Engineering for Artificial Intelligence-based Systems: A Review in Time. In: Lawless, W.F., Mittu, R., Sofge, D.A., Shortell, T., McDermott, T.A. (eds) Systems Engineering and Artificial Intelligence . Springer, Cham. https://doi.org/10.1007/978-3-030-77283-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-77283-3_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77282-6
Online ISBN: 978-3-030-77283-3
eBook Packages: Computer ScienceComputer Science (R0)