Software & Systems Modeling

, Volume 17, Issue 2, pp 415–441 | Cite as

Model-driven performance prediction of systems of systems

  • Katrina Falkner
  • Claudia Szabo
  • Vanea Chiprianov
  • Gavin Puddy
  • Marianne Rieckmann
  • Dan Fraser
  • Cathlyn Aston
Theme Section Paper


Systems of systems exhibit characteristics that pose difficulty in modelling and predicting their overall performance capabilities, including the presence of operational independence, emergent behaviour, and evolutionary development. When considering systems of systems within the autonomous defence systems context, these aspects become increasingly critical, as constraints on the performance of the final system are typically driven by hard constraints on space, weight and power. System execution modelling languages and tools permit early prediction of the performance of model-driven systems; however, the focus to date has been on understanding the performance of a model rather than determining whether it meets performance requirements, and only subsequently carrying out analysis to reveal the causes of any requirement violations. Moreover, such an analysis is even more difficult when applied to several systems cooperating to achieve a common goal—a system of systems. In this article, we propose an integrated approach to performance prediction of model-driven real-time embedded defence systems and systems of systems. Our architectural prototyping system supports a scenario-driven experimental platform for evaluating model suitability within a set of deployment and real-time performance constraints. We present an overview of our performance prediction system, demonstrating the integration of modelling, execution and performance analysis, and discuss a case study to illustrate our approach.


Performance prediction Systems of systems Model-driven engineering System execution modelling 


  1. 1.
    Baker, P., Loh, S., Weil, F.: Model-driven engineering in a large industrial context - motorola case study. In: Briand, L., Williams, C. (eds.) Model Driven Engineering Languages and Systems. LNCS, vol. 3713, pp. 476–491. Springer, Berlin (2005)Google Scholar
  2. 2.
    Balasubramanian, K., Balasubramanian, J., Parsons, J., Gokhale, A., Schmidt, D.C.: Component modeling language for distributed real-time and embedded systems. In: Proceedings of the 11th IEEE Real Time on Embedded Technology and Applications Symposium (RTAS 05), pp. 190–199 (2005)Google Scholar
  3. 3.
    Balsamo, S., Marco, A.D., Inverardi, P., Simeoni, M.: Model-based performance prediction in software development: a survey. IEEE Trans. Softw. Eng. 30, 295–310 (2004)CrossRefGoogle Scholar
  4. 4.
    Barbierato, E., Gribaudo, M., Iacono, M.: Defining formalisms for performance evaluation with SIMTHESys. Electron. Notes Theor. Comput. Sci. 275, 37–51 (2011). doi: 10.1016/j.entcs.2011.09.004.
  5. 5.
    Beydeda, S., Book, M., Gruhn, V. (eds.): Model Driven Software Development. Spinger, Berlin (2010)MATHGoogle Scholar
  6. 6.
    Boardman, J., Sauser, B.: System of systems—the meaning of OF. In: IEEE/SMC International Conference on System of Systems Engineering (2006). doi: 10.1109/SYSOSE.2006.1652284
  7. 7.
    Boehm, B.: Some future software engineering opportunities and challenges. In: Nanz, S. (ed.) The Future of Software Engineering, pp. 1–32. Springer, Berlin (2011). doi: 10.1007/978-3-642-15187-3_1
  8. 8.
    Bown, R., Sahin, E.: A net-centric xml based system of systems architecture for human tracking. In: Proceedings of Fifth IEEE International Symposium on Service Oriented System Engineering (SoSE’2010), pp. 1–6 (2010)Google Scholar
  9. 9.
    Bryans, J., Fitzgerald, J., Payne, R., Miyazawa, A., Kristensen, K.: SysML contracts for systems of systems. In: IEEE Systems of Systems Engineering Conference (2014)Google Scholar
  10. 10.
    Chen, B., Zhang, L., Liu, X., Vangheluwe, H.: Activity-based simulation using devs: increasing performance by an activity model in parallel DEVS simulation. J. Zhejiang Univ. Sci. C 15(1), 13–30 (2014). doi: 10.1631/jzus.C1300121 CrossRefGoogle Scholar
  11. 11.
    Chiprianov, V., Falkner, K., Szabo, C., Puddy, G.: Architectural support for model-driven performance prediction of distributed real-time embedded systems of systems. In: Proceedings of the 2014 European Conference on Software Architecture. Lecture Notes in Computer Science, vol. 8627, pp. 357–364 (2014)Google Scholar
  12. 12.
    Cook, S.C.: 2.3.1 on the acquisition of systems of systems. INCOSE Int. Sympos. 11(1), 383–390 (2001). doi: 10.1002/j.2334-5837.2001.tb02318.x
  13. 13.
    Dagli, C.H., Kilicay-Ergin, N.: System of Systems Architecting. Wiley, Hoboken (2008)CrossRefGoogle Scholar
  14. 14.
    Denaro, G., Polini, A., Emmerich, W.: Early performance testing of distributed software applications. SIGSOFT Softw. Eng. Notes 29, 94–103 (2004)CrossRefGoogle Scholar
  15. 15.
    Deng, G., Balasubramanian, J., Otte, W., Schmidt, D., Gokhale, A.: Dance: a qos-enabled component deployment and configuration engine. In: Dearle, A., Eisenbach, S. (eds.) Component Deployment. Lecture Notes in Computer Science, vol. 3798, pp. 67–82. Springer. Berlin (2005). doi: 10.1007/11590712_6
  16. 16.
    Edwards, G., Malek, S., Medvidovic, N.: Scenario-driven dynamic analysis of distributed architectures. In: Proceedings of the 10th International Conference on Fundamental Approaches to Software Engineering, pp. 125–139 (2007)Google Scholar
  17. 17.
    Esponoza, H., Dubois, H., Gerard, S., Medina, J., Petriu, D., Woodside, M.: Annotating uml models with non-functional properties for quantitative analysis. In: Satellite Events at the MoDELS 2005 Conference, pp. 79–90 (2006)Google Scholar
  18. 18.
    Eusgeld, I., Nan, C., Dietz, S.: System-of-systems approach for interdependent critical infrastructures. Reliab. Eng. Syst. Saf. 96(6), 679–686 (2011). doi: 10.1016/j.ress.2010.12.010 CrossRefGoogle Scholar
  19. 19.
    Falkner, K., Chiprianov, V., Falkner, N., Szabo, C., Hill, J., Puddy, G., Fraser, D., Johnston, A., Rieckmann, M., Wallis, A.: Model-driven performance prediction of distributed real-time embedded defence systems. In: The 18th International Conference on Engineering of Complex Computer Systems, Singapore, pp. 155–158 (2013)Google Scholar
  20. 20.
    Falkner, K., Chiprianov, V., Falkner, N., Szabo, C., Puddy, G.: Modeling scenarios for the performance prediction of distributed real-time embedded systems. In: Military Communications and Information Systems Conference, Canberra, Australia, pp. 1–6 (2013)Google Scholar
  21. 21.
    Farcas, C., Farcas, E., Krueger, I., Menarini, M.: Addressing the integration challenge for avionics and automotive systems: from components to rick services. Proc. IEEE 98(4), 562–583 (2010)CrossRefGoogle Scholar
  22. 22.
    Franceschinis, G., Gribaudo, M., Iacono, M., Marrone, S., Moscato, F., Vittorini, V.: Interfaces and binding in component based development of formal models. ICST (2010). doi: 10.4108/ICST.VALUETOOLS2009.7677
  23. 23.
    Gaonkar, S., Keefe, K., Lamprecht, R., Rozier, E., Kemper, P., Sanders, W.H.: Performance and dependability modeling with Möbius. SIGMETRICS Perform. Eval. Rev. 36(4), 16–21 (2009). doi: 10.1145/1530873.1530878 CrossRefGoogle Scholar
  24. 24.
    Ge, B., Hipel, K.W., Yang, K., Chen, Y.: A data-centric capability-focused approach for system-of-systems architecture modeling and analysis. Syst. Eng. 16(3), 363–377 (2013). doi: 10.1002/sys.21253 CrossRefGoogle Scholar
  25. 25.
    Gokhale, A., Balasubramanian, K., Krishna, A.S., Balasubramanian, J., Edwards, G., Deng, G., Turkay, E., Parsons, J., Schmidt, D.C.: Model driven middleware: a new paradigm for developing distributed real-time and embedded systems. Sci. Comput. Program. 73(1), 39–58 (2008). doi: 10.1016/j.scico.2008.05.005 MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Gorod, A., Gove, R., Sauser, B., Boardman, J.: System of systems management: a network management approach. In: IEEE International Conference on System of Systems Engineering, pp. 1–5 (2007). doi: 10.1109/SYSOSE.2007.4304218
  27. 27.
    Graciano Neto, V.V., Guessi, M., Oliveira, L.B.R., Oquendo, F., Nakagawa, E.Y.: Investigating the model-driven development for systems-of-systems. In: Proceedings of the 2014 European Conference on Software Architecture Workshops (ECSAW ’14), pp. 22:1–22:8. ACM, New York, NY, USA (2014). doi: 10.1145/2642803.2642825
  28. 28.
    Green, T., Petre, M.: Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. J. Vis. Lang. Comput. 7(2), 131–174 (1996)CrossRefGoogle Scholar
  29. 29.
    Groenda, H.: Improving performance predictions by accounting for the accuracy of composed performance models. In: Proceedings of the international conference on the quality of software architectures (QoSA), pp. 111–116 (2012)Google Scholar
  30. 30.
    Happe, J., Koziolek, H., Reussner, R.: Facilitating performance predictions using software components. IEEE Softw. 28, 27–33 (2011)CrossRefGoogle Scholar
  31. 31.
    Hata, Y., Kamazaki, Y., Sawayama, T., Taniguchi, K., Nakajima, H.: A heart pulse monitoring system by air pressure and ultrasonic sensor systems. In: Proceedings of the Second IEEE International Symposium on Service Oriented System Engineering (SoSE ’2007), pp. 1–5 (2007)Google Scholar
  32. 32.
    Hill, J., Schmidt, D., Edmondson, J., Gokhale, A.: Tools for continuously evaluating distributed system qualities. IEEE Softw. 27(4), 65–71 (2010)CrossRefGoogle Scholar
  33. 33.
    Hill, J., Schmidt, D., Porter, A., Slaby, J.: CiCUTS: combining system execution modeling tools with continuous integration environments. In: David, W.B., Roy, S (eds.) Engineering of Computer Based System, pp. 66–75. IEEE, Washington DC, USA (2008). doi: 10.1109/ECBS.2008.20
  34. 34.
    Hill, J., Schmidt, D., Slaby, J.: Designing software-intensive systems: methods and principles. In: System Execution Modeling Tools for Evaluating the Quality of Service of Enterprise Distributed Real-time and Embedded Systems, pp. 335–371. IGI Global (2008)Google Scholar
  35. 35.
    Hill, J., Slaby, J., Baker, S., Schmidt, D.: System execution modeling tools to evaluate enterprise distributed real-time and embedded system qos. In: Proceedings of the 12th International Conference on Embedded and Real-Time Computing Systems and Applications (2006)Google Scholar
  36. 36.
    Ingram, C., Payne, R., Perry, S., Holt, J., Hansen, F., Couto, L.D.: Modelling patterns for systems of systems architectures. In: IEEE International Systems Conference (SysCon2014) (2014)Google Scholar
  37. 37.
    Jamshidi, M.: System of systems engineering—new challenges for the 21st century. IEEE Aerosp. Electron. Syst. Mag. 23(5), 4–19 (2008). doi: 10.1109/MAES.2008.4523909 CrossRefGoogle Scholar
  38. 38.
    Jasmine, K., Vasantha, R.: Design based performance prediction of component based software products. World Acad. Sci. Eng. Technol. 30, 266–269 (2007)Google Scholar
  39. 39.
    Kampert, D., Nazari, S., Sonntag, C.: D4.1: Dymasos Engineering Concept Specification. Technical Report, DYMASOS. Technische Universität Dortmund, Dortmund, Germany (2014).
  40. 40.
    Kewley, R., Cook, J., Goerger, N., Henderson, D., Teague, E.: Federated simulations for systems of systems integration. In: Proceedings of the 2008 Winter Simulation Conference, pp. 1121–1129 (2008)Google Scholar
  41. 41.
    Kieburtz, R.B., McKinney, L., Bell, J.M., Hook, J., Kotov, A., Lewis, J., Oliva, D.P., Sheard, T., Smith, I., Walton, L.: A software engineering experiment in software component generation. In: Proceedings of the 18th International Conference on Software engineering (ICSE), pp. 542–552 (1996)Google Scholar
  42. 42.
    Klein, J., van Vliet, H.: A systematic review of system-of-systems architecture research. In: Proceedings of the 9th International ACM Sigsoft Conference on Quality of Software Architectures, pp. 13–22 (2013). doi: 10.1145/2465478.2465490
  43. 43.
    Koziolek, H.: Performance evaluation of component-based software systems: a survey. Perform. Eval. 67(8), 634–658 (2010)CrossRefGoogle Scholar
  44. 44.
    Maier, M.W.: Architecting principles for systems-of-systems. Syst. Eng. 1(4), 267–284 (1998). doi: 10.1002/(SICI)1520-6858(1998)1:4<267:AID-SYS3>3.0.CO;2-D CrossRefGoogle Scholar
  45. 45.
    Manthorpe, W.H.J.: The emerging joint system of systems: a systems engineering challenge and opportunity for APL. John Hopkins APL Tech. Dig. 17(3), 305–310 (1996)Google Scholar
  46. 46.
    Merola, L.: The COTS software obsolescence threat. In: Fifth International Conference on Commercial-off-the-Shelf (COTS)-Based Software Systems (2006)Google Scholar
  47. 47.
    Michelson, B.M.: Event-Driven Architecture Overview. Technical Report, Patricia Seybold Group. Bridgewater, MA, USA (2006)Google Scholar
  48. 48.
    Mustafiz, S., Denil, J., Lúcio, L., Vangheluwe, H.: The ftg + pm framework for multi-paradigm modelling: An automotive case study. In: Proceedings of the 6th International Workshop on Multi-Paradigm Modeling, MPM ’12, pp. 13–18. ACM, New York, NY, USA (2012). doi: 10.1145/2508443.2508446
  49. 49.
    OMG: Data Distribution Service for Real-Time Systems Version 1.2. Object Management Group, Needham, MA, USA (2007)Google Scholar
  50. 50.
    OMG: UML Profile for MARTE: Modeling and Analysis of Real-Time Embedded Systems v 1.1. Object Management Group, Needham, MA, USA (2011)Google Scholar
  51. 51.
    Paunov, S., Hill, J., Schmidt, D., Baker, S., Slaby, J.: Domain-specific modeling languages for configuring and evaluating enterprise DRE system quality of service. In: 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (2006)Google Scholar
  52. 52.
    Pérez, J., Díaz, J., Garbajosa, J., Yagüe, A., Gonzalez, E., Lopez-Perea, M.: Large-scale smart grids as system of systems. In: Proceedings of the First International Workshop on Software Engineering for Systems-of-Systems, SESoS ’13, pp. 38–42. ACM, New York, NY, USA (2013). doi: 10.1145/2489850.2489858
  53. 53.
    Pisano, N.: Technical performance measurement earned value, and risk management: an integrated diagnostic tool for program management. In: Defense Acquisition University Acquisition Research Symposium, Ft. Belvoir, VA, USA (1995)Google Scholar
  54. 54.
    Rieckmann, M., Fraser, D., Chiprianov, V., Szabo, C., Falkner, K.: Demonstration of model-driven performance prediction of distributed real-time embedded systems of systems. In: Proceedings of the European Conference on Software Architecture Workshops, ECSAW ’14, pp. 34:1–34:4. ACM, New York, NY, USA (2014). doi: 10.1145/2642803.2642837
  55. 55.
    Rieckmann, M., Fraser, D., Chiprianov, V., Szabo, C., Falkner, K.: Demonstration of model-driven performance prediction of distributed real-time embedded systems of systems. In: Proceedings of the European Conference on Software Architecture Workshops, pp. 34–40 (2014)Google Scholar
  56. 56.
    Sage, A.P., Cuppan, C.D.: On the systems engineering and management of systems of systems and federation of systems. Inf. Knowl. Syst. Manag. 2, 325–345 (2001)Google Scholar
  57. 57.
    Schmidt, D.C., Stal, M., Rohnert, H., Bushmann, F.: Pattern-Oriented Software Architecture: Patterns for Concurrent and Networked Objects, vol. 2. Wiley, Hoboken (2000)Google Scholar
  58. 58.
    Schneider, J.P., Teodorov, C., Senn, E., Champeau, J.: Towards a dynamic infrastructure for playing with systems of systems. In: Proceedings of the European Conference on Software Architecture Workshops (ECSAW ’14), pp. 31:1–31:4. ACM, New York, NY, USA (2014). doi: 10.1145/2642803.2642834
  59. 59.
    Sharawi, A., Sala-Diakanda, S.N., Dalton, A., Quijada, S., Yousef, N., Rabelo, L., Sepulveda, J.: A distributed simulation approach for modeling and analyzing systems of systems. In: Proceedings of the Winter Simulation Conference, pp. 1028–1035 (2006)Google Scholar
  60. 60.
    SISO: Military Scenario Sefinition Language (MSDL). siso-std-007-2008. Simulation Interoperability Standards Organization, Orlando, FL, USA (2008)Google Scholar
  61. 61.
    Smith, C.: Introduction to software performance engineering: origins and outstanding problems. In: Proceedings of the 7th International Conference on Formal Methods for Performance Evaluation, pp. 395–428 (2007)Google Scholar
  62. 62.
    Szabo, C., Chen, Y.: A model-driven change traceability method in system modeling execution. In: Proceedings of the 22nd Australasian Software Engineering Conference (2013)Google Scholar
  63. 63.
    Trubiani, C., Meedeniya, I., Cortellessa, V., Aleti, A., Grunske, L.: Model-based performance analysis of software architectures under uncertainty. In: Proceedings of the International Conference on the Quality of Software Architectures (QoSA), pp. 69–78 (2013)Google Scholar
  64. 64.
    Vangheluwe, H., De Lara, J., Mosterman, P.J.: An introduction to multi-paradigm modelling and simulation. In: Proceedings of the AIS2002 Conference (AI, Simulation and Planning in High Autonomy Systems), Lisboa, Portugal, pp. 9–20 (2002)Google Scholar
  65. 65.
    Vierhauser, M., Rabiser, R., Grünbacher, P., Danner, C., Wallner, S.: Evolving systems of systems: industrial challenges and research perspectives. In: Proceedings of the First International Workshop on Software Engineering for Systems-of-Systems (SESoS ’13), pp. 1–4. ACM, New York, NY, USA (2013). doi: 10.1145/2489850.2489851
  66. 66.
    Volkert Richard, J.T.S., Yu, J.: A framework for performance prediction during development of systems of systems. Int. J. Syst. Syst. Eng. 3, 76–95 (2012)CrossRefGoogle Scholar
  67. 67.
    Weiderman, N., Bergey, J., Smith, D., Tilley, S.: Approaches to legacy system evolution. Technical Report CMU/SEI-97-TR-014, Software Engineering Institute, Carnegie-Mellon University. Pittsburgh, PA, USA (1997)Google Scholar
  68. 68.
    Wu, X., McMullen, D., Woodside, M.: Component based performance prediction. In: Proceedings of the 6th ICSE Workshop on Component-Based Software Engineering (2003)Google Scholar
  69. 69.
    You, G.r., Sun, X., Sun, M., Wang, J., Chen, Y.: Bibliometric and social network analysis of the SOS field. In: 9th International Conference on System of Systems Engineering, pp. 13–18 (2014). doi: 10.1109/SYSOSE.2014.6892456

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Katrina Falkner
    • 1
  • Claudia Szabo
    • 1
  • Vanea Chiprianov
    • 2
  • Gavin Puddy
    • 3
  • Marianne Rieckmann
    • 1
  • Dan Fraser
    • 1
  • Cathlyn Aston
    • 1
  1. 1.School of Computer ScienceUniversity of AdelaideAdelaideAustralia
  2. 2.LIUPPAUniversity of Pau and Pays AdourPauFrance
  3. 3.Department of DefenceDefence Science and Technology OrganisationCanberraAustralia

Personalised recommendations