Information Technology and Management

, Volume 10, Issue 2–3, pp 135–149 | Cite as

On the predictive performance of queueing network models for large-scale distributed transaction processing systems

Article

Abstract

Automated business processes running on distributed transaction processing (DTP) systems characterize the IT backbone of services industries. New web services standards such as BPEL have increased the importance of DTP systems in business practice. IT departments are forced to meet pre-defined quality-of-service metrics, therefore performance prediction is essential. Unfortunately, the complexity of multiple interacting services running on multiple hardware resources as well as the volatility in the demand for these services can make performance analysis extremely difficult. While business process automation has been a dominant topic in the recent years, surprisingly little has been published on performance modelling of large-scale DTP systems. In this paper, we will describe these systems with respect to the workloads and technical features, and compare the predictive accuracy of different types of queueing models and discrete event simulations experimentally. The experiments are based on two real-world DTP systems and respective data sets of a telecom company. Overall, we found that while the results for average utilization scenarios are quite similar, the effort to implement and run analytic solutions is much lower. As long as standard distributional assumptions of analytical models hold, they provide a reliable and fast methodology to explore different demand mix scenarios even for large-scale systems. The difficulty to estimate service and arrival time parameters and demand mix for the respective queueing network models can largely be reduced with appropriate tooling. Often, this information is missing in IT departments. Also, complex event conditions and error handling in DTP systems can make the analysis difficult. For many DTP applications, however, performance modelling could provide valuable decision support for service level management.

Keywords

Performance modelling IT service management Transaction processing Queueing network model Discrete event simulation 

References

  1. 1.
    IDC (2007) Solutions for the Datacenter’s thermal challenges, whitepaperGoogle Scholar
  2. 2.
    Qin M et al (1996) Automatic generation of performance models for distributed application systems. In: Proceedings of the 1996 conference of the centre for advanced studies on collaborative research. IBM Press, TorontoGoogle Scholar
  3. 3.
    Rukma Prabhu V, Varsha A (2006) A methodology and tool for performance analysis of distributed server systems. In: Proceedings of the 28th international conference on software engineering. ACM, ShanghaiGoogle Scholar
  4. 4.
    Chen Y et al (2007) SLA decomposition: translating service level objectives to system level thresholds. In: ICAC ‘07: Proceedings of the fourth international conference on autonomic computing. IEEE Computer Society, Washington, DCGoogle Scholar
  5. 5.
    Liu X, Heo J, Sha L (2005) Modeling 3-tiered web applications. In: MASCOTS ‘05: Proceedings of the 13th IEEE international symposium on modeling, analysis, and simulation of computer and telecommunication systems. IEEE Computer Society, Washington, DCGoogle Scholar
  6. 6.
    Ouyang C et al (2008) From business process models to process-oriented software systems. ACM transactions on software engineering and methodology (TOSEM)Google Scholar
  7. 7.
    Dehnert J, van der Aalst WMP (2004) Bridging the gap between business models and workflow specifications. Int J Cooperative Inf Syst 13(3):289–332CrossRefGoogle Scholar
  8. 8.
    Brandl R, Bichler M, Ströbel M (2007) Cost accounting for shared IT infrastructures: estimating resource utilization in distributed IT infrastructures. Wirtschaftsinformatik 49(2):83–94CrossRefGoogle Scholar
  9. 9.
    Fitzsimmons J, Fitzsimmons M (2004) Service management. McGraw-Hill/Irwin, New YorkGoogle Scholar
  10. 10.
    Neumann K, Morlock M (2002) Operations research. Hanser, MünchenGoogle Scholar
  11. 11.
    Whinston W (1994) Operations research: ITPGoogle Scholar
  12. 12.
    Noel E, Tang KW (2000) Performance modeling of multihop network subject to uniform and nonuniform geometric traffic. In: IEEE/ACM transactions on networking (TON)Google Scholar
  13. 13.
    Thakkar SS, Schweiger M (1990) Performance of an OLTP application on symmetry multiprocessor system. In: 17th Annual international symposium on computer architecture, Seattle, WAGoogle Scholar
  14. 14.
    Saavedra RH, Smith AJ (1996) Analysis of benchmark characteristics and benchmark performance prediction. ACM Trans Comput Syst 14(4):344–384CrossRefGoogle Scholar
  15. 15.
    SAP (SAP Standard Applications Benchmarks) (2009) [cited 2009; Benchmark Specification]. Available from http://www.sap.com/solutions/benchmark/index.epx
  16. 16.
    SPEC (2009) SPECjbb. [cited 2009; Benchmark Specification]. Available from http://www.spec.org/jbb2005/
  17. 17.
    Oracle (2009) Oracle Applications Standard Benchmark. [cited 2009; Benchmark Specification]. Available from http://www.oracle.com/apps_benchmark/
  18. 18.
    TPC (2009) TPC-E OLTP. [cited 2009; Benchmark Specification]. Available from http://www.tpc.org/tpce/tpc-e.asp
  19. 19.
    TPC (2009) TPC-C V5. [cited 2009; Benchmark Specification]. Available from http://www.tpc.org/tpcc/default.asp
  20. 20.
    Urgaonkar B et al (2005) An analytical model for multi-tier internet services and its applications. In: SIGMETRICS ‘05: Proceedings of the 2005 ACM SIGMETRICS international conference on measurement and modeling of computer systems. ACM, New YorkGoogle Scholar
  21. 21.
    Feitelson DG (2002) Workload modeling for performance evaluation. In: Performance evaluation of complex systems: techniques and tools, performance 2002, Tutorial Lectures. Springer, LondonGoogle Scholar
  22. 22.
    Feitelson DG (2002) The forgotten factor: facts on performance evaluation and its dependence on workloads. In: Euro-Par ‘02: Proceedings of the 8th international euro-par conference on parallel processing. Springer, LondonGoogle Scholar
  23. 23.
    Stewart C, Kelly T, Zhang A (2007) Exploiting nonstationarity for performance prediction. In: EuroSys ‘07: Proceedings of the 2nd ACM SIGOPS/EuroSys European conference on computer systems 2007. ACM, New YorkGoogle Scholar
  24. 24.
    Josuttis N (2007) Soa in practice: the art of distributed system Design. O’Reilly Media, IncGoogle Scholar
  25. 25.
    Bailey DH, Snavely A (2005) Performance modeling: understanding the present and predicting the future. In: Euro-Par 2005 parallel processing, Lisbon, PortugalGoogle Scholar
  26. 26.
    Baskett F et al (1975) open, closed, and mixed networks of queues with different classes of customers. J ACM 22(2)Google Scholar
  27. 27.
    Bolch G et al (2006) Queueing networks and markov chains - modeling and performance evaluation with computer science applications, 2nd edn. Wiley, HobokenGoogle Scholar
  28. 28.
    Kounev S, Buchmann A (2003) Performance modeling and evaluation of large-scale j2ee applications. In: Proceedings of the 29th international conference of the computer measurement group on resource management and performance evaluation of enterprise computing systems (CMG 2003), Dallas, Texas, USA, December 7–12, 2003, pp 273–283Google Scholar
  29. 29.
    Menasce DA, Dowdy LW, Almeida VAF (2004) Performance by design: computer capacity planning by example. Prentice Hall PTR, Upper Saddle RiverGoogle Scholar
  30. 30.
    Kendall DG (1953) Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded markov chain. Ann Math Stat 24(3):338–354CrossRefGoogle Scholar
  31. 31.
    Reiser M, Lavenberg SS (1980) Mean-value analysis of closed multichain queuing networks. J ACM 27(2):313–322CrossRefGoogle Scholar
  32. 32.
    Neuse D, Chandy K (1981) SCAT: a heuristic algorithm for queueing network models of computing systems. SIGMETRICS Perform Eval Rev 10(3):59–79CrossRefGoogle Scholar
  33. 33.
    Fishman GS (2001) Discrete-event simulation. Springer, LondonGoogle Scholar
  34. 34.
    Banks J et al (2005) Discrete-event simulation, Prentice HallGoogle Scholar
  35. 35.
    Hühn O, Markl C (2007). PerMoTo - Performance Modelling Tool suite. In: WITS 07 - Seventeenth annual workshop on information technologies and systems, Montreal, CanadaGoogle Scholar
  36. 36.
    Bertoli M, Casale G, Serazzi G (2007) The JMT simulator for performance evaluation of non-product-form queueing networks. In: ANSS ‘07: Proceedings of the 40th annual simulation symposium. IEEE Computer Society, Washington, DCGoogle Scholar
  37. 37.
    Fishman GS (1973) Statistical analysis for queueing simulations. Manag Sci 20(3):363–369CrossRefGoogle Scholar
  38. 38.
    Spratt SC (1998) Heuristics for the startup problem, Department of Systems Engineering, University of VirginiaGoogle Scholar
  39. 39.
    Heidelberger P, Welch PD (1981) A spectral method for confidence interval generation and run length control in simulations. Commun ACM 24(4):233–245CrossRefGoogle Scholar
  40. 40.
    Bolch G, Jung H, Gaebell M (1993) Entwicklung und Validierung der Schließmethode zur Analyse offener Warteschlangennetze. In: Operations Research Proceedings 1992. Springer, HeidelbergGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Department of Informatics, Boltzmannstraße 3Technische Universität MünchenGarchingGermany

Personalised recommendations