Abstract
This paper intends to present an overview of the evolution of performance evaluation since its first steps the Erlang works for modelling telephone networks, based on simple queues until the present current challenges in Green ICT that will require the development of new paradigms and mathematical tools, and rapidly passing across the modelling works of Khintchine and Pollaczeck; Jackson; Baskett, Chandy, Muntz and Palacios; Buzen; Reiser and Lavenberg; and many others, and benchmarking standards that have produced solutions to the problems appearing in these hundred of years. Finally, we analyze some of the challenges of computer performance evaluation appearing today, mainly those related to the energy consumption and sustainability, globally known as Green ICT.
Keywords
References
Erlang, A.K.: The Theory of Probabilities and Telephone Conversations. Nyt Tidsskrift for Matematik, B 20 (1909)
Erlang, A.K.: Solution of some Problems in the Theory of Probabilities of Significance in Automatic Telephone Exchanges. Elektroteknikeren 17 (1917)
Erlang, A.K.: Telephone Waiting Times. Matematisk Tidsskrift, B 31 (1920)
Cobham, A.: Priority Assignment in Waiting Line Problems. Journal of the Operations Research Society of America (1954)
Jackson, J.R.: Jobshop like Queueing Sysems. Management Science 10 (1963)
Gordon, W.J., Newell, G.F.: Closed Queueing Systems with Exponential Servers. Operations Research 15(2) (1967)
Buzen, J.P.: Computational Algorithms for Closed Queueing Networks with Exponential Servers. CACM 16(9) (1973)
Baskett, F.T., Chandy, K.M., Muntz, R.R., Palacios, F.G.: Open, Closed and Mixed Networks with Different Classes of Customers. JACM 22(2) (1975)
Reiser, M., Kobayashi, H.: Queueing networks with multiple closed chains: Theory and computational algorithms. 1BM J. Res. Develop. 19, 3 (1975)
Reiser, M., Lavenberg, S.S.: Mean Value Analysis of Closed Multichain Queueing Networks. JACM 27(2) (1980)
Denning, P.J., Buzen, J.P.: The Operational Analysis of Queueing Network Models. ACM Computing Surveys 10(3) (1978)
Buzen, J.P., Denning, P.J.: Operational Treatment of Queue Length Distributions and Mean Value Analysis. Computer Performance 1(1) (1980)
Courtois, P.J.: Decomposability: Queueing and Computer Systems Applications. Academic Press (1977)
Marie, R.: Modélisation par Réseaux de Files d’Attente. Thèse de Docteur ès Sciences Mathematiques. Université de Rennes (1977)
Gelenbe, E., Mitrani, I.: Analysis and Synthesis of Computer Systems. Academic Press (1981)
LINPACK: http://www.netlib.org/linpack/
SPEC: Standard Performance Evaluation Corporation, http://www.spec.org/
OPNET: Applications and Network Performance, http://www.opnet.com/
ns-2: The Network Simulator, http://www.isi.edu/nsnam/ns/
Barroso, L.A., Hözle: The case for energy proportional computing. IEEE Computer 40, 12 (2007)
COST IC0804. Energy efficiency in large scale distributed systems, http://www.cost804.org/
Chen, G., He, W., Liu, J., Nath, S., Rigas, L., Xiao, L., Zhao, F.: Energy-aware server provisioning and load dispatching for connection-intensive internet services. In: NSDI, San Francisco, CA (2008)
Standard Performance Evaluation Corporation (SPEC). SPEC power, http://www.spec.org/
Fan, X., Weber, W.D., Barroso, L.A.: Power provisioning for a warehouse-sized computer. In: Proceedings of the International Symposium on Computer Architecture, ISCA (2007)
Heath, T., Diniz, B., et al.: Energy conservation in heterogeneous server clusters. In: Proceedings of the Symposium on Principles and Practice of Parallel Programming, PPoPP (2005)
Chun, B.G., Iannaccone, G., Katz, R.H., Lee, G., Niccolini, L.: An Energy Case for Hybrid Datacenters. In: Workshop on Power Aware Computing and Systems, HotPower 2009 (2009)
Flinn, J., Satyanarayanan, M.: Energy-aware adaptation for mobile applications. In: SOSP 1999: Proceedings of the 17th ACM Symposium on Operating Systems Principles (1999)
Mankoff, J., Kravets, R., Blevis, E.: Some computer science issues in creating a sustainable world. IEEE Computer 41(8) (2008)
Im, C., Ha, S.: Energy optimization for latency- and quality-constrained video applications. IEEE Design and Test of Computers 21(5) (2004)
Iyer, S., Luo, L., Mayo, R., Ranganathan, P.: Energy-adaptive display system designs for future mobile environments. In: ACM MobiSys (2003)
Gilly, K., Alcaraz, S., Juiz, C., Puigjaner, R.: Analysis of burstiness monitoring and detection in an adaptive Web system. Computer Networks 53(5) (2009)
Executive Summary. Chapter 9: Projections of Future Climate Change. Climate Change 2001: The Scientific Basis (2005), http://www.grida.no/climate/ipcc_tar/wg1/339.htm
Feng, X., Ge, R., Cameron, K.W.: Power and Energy Profiling of Scientific Applications on Distributed Systems. In: Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2005 (2005)
Kumar, R., Farkas, K.I., Jouppi, N.P., Ranganathan, P., Tullsen, D.M.: Single-ISA heterogeneous multi-core architectures: The potential for processor power reduction. In: ACM/IEEE MICRO (2003)
Pinheiro, E., Bianchini, R., et al.: Load balancing and unbalancing for power and performance in cluster-based systems. In: Proceedings of the Workshop on Compilers and Operating Systems for Low Power, COLP (2001)
Ranganathan, P., Rivoire, S., Moore, J.: Power modeling and measurement. In: Advances in Computers. Elsevier (2009)
U.S. EPA. Report to congress on server and data center energy efficiency. Tech. Rep. (2007)
Kusic, D., Kephart, J.O., Hanson, J.E., Kandasamy, N., Jiang, G.: Power and performance management of virtualized computing environments via lookahead control. Cluster Computing 12(1) (2009)
Alcaraz, S., Gilly, K., Juiz, C., Puigjaner, R.: Handling HTTP flows over a DiffServ framework. In: LANC 2007 (2007)
Narayanan, D., Donnelly, A., Rowstron, A.: Write off-loading: Practical power management for enterprise storage. In: FAST (2008)
Nedevschi, S., Popa, L., Iannaccone, G., Ratnasamy, S., Wetherall, D.: Reducing network energy consumption via sleeping and rate-adaptation. In: NSDI (2008)
Park, S., Jiang, W., Zhou, Y., Adve, S.: Managing energy-performance tradeoffs for multithreaded applications on multiprocessor architectures. In: SIGMETRICS (2007)
Raghavendra, R., Ranganathan, P., Talwar, V., Wang, Z., Zhu, X.: No power struggles: A unified multi-level power management architecture for the data center. In: ASPLOS (2008)
Menasce, D.A., Almeida, V., Dowdy, L.: Performance by Design: Computer Capacity Planning by Example. Prentice Hall PTR (2004)
Standard Performance Evaluation Corporation (SPEC). SPECweb2009, http://www.spec.org/web2009/
Menasce, D.A., Almeida, V.: Capacity Planning for Web Services: metrics, models, and methods. Prentice Hall PTR (2001)
Killelea, P.: Web Performance Tuning. O’Reilly (2002)
National greenhouse gas inventory data for the period 1990-2007. UN FCCC (2009), http://unfccc.int/resource/docs/2009/sbi/eng/12.pdf
Xie, F., Martonosi, M., Malik, S.: Compile-time dynamic voltage scaling settings: opportunities and limits. SIGPLAN Not. 38(5) (2003)
Banerjee, P., Patel, C.D., Bash, C., Ranganathan, P.: Sustainable Data Centers: Enabled by Supply and Demand Side Management. In: DAC 2009. ACM Press (2009)
Katz, R.: Tech Titans Building Boom. IEEE Spectrum (February 2009)
Patel, C.D., Ranganathan, P.: Enterprise power and cooling. ASPLOS Tutorial (2006)
Laudon, J.: Performance/Watt: The new server focus. SIGARCH Computer Architecture News 33(4) (2005)
Sun Microsystems. SWaP (Space, Watts and Performance) metric, http://search.sun.com/main/
Rivoire, S., Shah, M.A., Ranganathan, P., Kozyrakis, C.: JouleSort: A Balanced Energy-Efficiency Benchmark, http://www.hpl.hp.com/environment/datacenters.html
Molero, X., Juiz, C., Rodeño, M.J.: Evaluación y Modelado del Rendimiento de Sistemas Informáticos. Pearson Prentice Hall (2004)
Deloitte España. Barómetro de Empresas (2009).
Pierson, J.M., Hlavacs, H. (eds.): Proceedings of the COST Action IC0804 on Large Scale Distributed Systems, 1st Year (2010)
Slack, N., Chambers, S., Betts, A., Johnston, R.: Operations and Process Management: Principles and practice for strategic impact. FT Prentice Hall (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 IFIP International Federation for Information Processing
About this chapter
Cite this chapter
Juiz, C., Puigjaner, R. (2011). From the Origins of Performance Evaluation to New Green ICT Performance Engineering. In: Hummel, K.A., Hlavacs, H., Gansterer, W. (eds) Performance Evaluation of Computer and Communication Systems. Milestones and Future Challenges. PERFORM 2010. Lecture Notes in Computer Science, vol 6821. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25575-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-25575-5_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-25574-8
Online ISBN: 978-3-642-25575-5
eBook Packages: Computer ScienceComputer Science (R0)