On the Predictive Properties of Performance Models Derived through Input-Output Relationships

  • Mahmoud Awad
  • Daniel A. Menascé
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8721)


Building an analytical performance model is a challenge when little is known about the functionality and behavior of the system being modeled and/or when obtaining model parameters through measurements is difficult. This paper addresses this problem by presenting an approach that derives analytic model parameters by observing the input-output relationships of a real system. More specifically, input (i.e., arrival rates for each job class) and output (i.e., average response time for each job class) measurements are used to estimate the per-class service demands and number of servers for a Queuing Network model of the system. This model, called the computed model (CM), provides the same output values for the same input values used to derive the CM. The important question is whether the CM has predictive power, i.e., can the CM predict the output values that would be observed in the real system for different values of the input? The CM’s parameters are obtained by solving a non-linear optimization problem. The paper shows through experiments that the CM is relatively robust and has predictive power over a range of input values.


Queuing network models parameter estimation non-linear optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barna, C., Litoiu, M., Ghanbari, H.: Autonomic load-testing framework. In: Proc. 8th ACM Intl. Conf. Autonomic Computing, pp. 91–100 (2011)Google Scholar
  2. 2.
    Begin, T., Baynat, B., Sourd, F., Brandwajn, A.: A DFO technique to calibrate queuing models. Computers & Operations Research 37(2), 273–281 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Begin, T., Brandwajn, A., Baynat, B., Wolfinger, B.E., Fdida, S.: Towards an automatic modeling tool for observed system behavior. In: Wolter, K. (ed.) EPEW 2007. LNCS, vol. 4748, pp. 200–212. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Begin, T., Brandwajn, A., Baynat, B., Wolfinger, B.E., Fdida, S.: High-level approach to modeling of observed system behavior. ACM SIGMETRICS Performance Evaluation Review 35(3), 34–36 (2007)CrossRefGoogle Scholar
  5. 5.
    Bennani, M.N., Menascé, D.A.: Assessing the robustness of self-managing computer systems under highly variable workloads. In: Intl. Conf. Autonomic Computing, pp. 62–69 (2004)Google Scholar
  6. 6.
    Bennani, M.N., Menascé, D.A.: Resource Allocation for Autonomic Data Centers Using Analytic Performance Models. In: 2005 IEEE Intl. Conf. Autonomic Computing, Seattle, WA, June 13-16 (2005)Google Scholar
  7. 7.
    Brosig, F., Huber, N., Kounev, S.: Automated extraction of architecture-level performance models of distributed component-based systems. In: 26th IEEE/ACM Intl. Conf. Automated Software Engineering (ASE), pp. 183–192 (2011)Google Scholar
  8. 8.
    Desnoyers, P., Wood, T., Shenoy, P., Singh, R., Patil, S., Vin, H.: Modellus: Automated modeling of complex internet data center applications. ACM Tr. on the Web (TWEB) 6(2) (2012)Google Scholar
  9. 9.
    Kounev, S., Huber, N., Spinner, S., Brosig, F.: Model-based techniques for performance engineering of business information systems. In: Shishkov, B. (ed.) BMSD 2011. LNBIP, vol. 109, pp. 19–37. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Litoiu, M., Woodside, M., Zheng, T.: Hierarchical model-based autonomic control of software systems. ACM SIGSOFT Software Engineering Notes 30(4), 1–7 (2005)CrossRefGoogle Scholar
  11. 11.
    Menascé, D.: Computing missing service demand parameters for performance models. In: Proc. 34th Intl. Computer Measurement Group Conf., pp. 7–12 (2008)Google Scholar
  12. 12.
    Menascé, D., Dowdy, L., Almeida, V.: Performance by Design: Computer Capacity Planning By Example. Prentice Hall (2004)Google Scholar
  13. 13.
    Noorshams, Q., Rostami, K., Kounev, S., Tuma, P., Reussner, R.: I/O Performance Modeling of Virtualized Storage Systems. In: IEEE 21st Intl. Symp. Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 121–130 (2013)Google Scholar
  14. 14.
    Seidmann, A., Schweitzer, P., Shalev-Oren, S.: Computerized Closed Queueing Network Models of Flexible Manufacturing, Large Scale System. J. North Holland 12, 91–107 (1987)zbMATHMathSciNetGoogle Scholar
  15. 15.
    Woodside, M., Zheng, T., Litoiu, M.: The use of optimal filters to track parameters of performance models. In: Second Intnl. Conf. Quantitative Evaluation of Systems, pp. 74–83 (2005)Google Scholar
  16. 16.
    Zheng, T., Yang, J., Woodside, M., Litoiu, M., Iszlai, G.: Tracking time-varying parameters in software systems with extended Kalman filters. In: Proc. 2005 Conf. of the Centre for Advanced Studies on Collaborative Research, pp. 334–345 (2005)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Mahmoud Awad
    • 1
  • Daniel A. Menascé
    • 1
  1. 1.Computer Science DepartmentGeorge Mason UniversityFairfaxUSA

Personalised recommendations