Abstract
This chapter introduces the foundations of experimental design. Starting with the case of one factor, the analysis of variance technique from statistics is introduced, followed by the method of contrasts for comparing subsets of alternatives. The analysis of variance technique is then generalized to two factors that can be varied independently, and after that, it is generalized to m factors. Following this, the Plackett–Burman fractional factorial design is introduced and compared with the full-factorial analysis of variance technique. Finally, a case study showing how experimental design can be applied in practice is presented.
No amount of experimentation can ever prove me right; a single experiment can prove me wrong.
—Albert Einstein (1879–1955)
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Amrhein, V., Greenland, S., & McShane, B. (2019). Scientists rise up against statistical significance. Nature, 567(7748), 305–307.
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
Lilja, D. J. (2000). Measuring computer performance: A practitioner’s guide. Cambridge: Cambridge University Press.
Noorshams, Q., Bruhn, D., Kounev, S., & Reussner, R. (2013). Predictive performance modeling of virtualized storage systems using optimized statistical regression techniques. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013), Prague (pp. 283–294). New York, NY: ACM.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12, 2825–2830.
Plackett, R. L., & Burman, J. P. (1946). The design of optimum multifactorial experiments. Biometrika, 33(4), 305–325. Oxford: Oxford University Press on behalf of Biometrika Trust.
Russell, S., & Norvig, P. (2009). Artificial intelligence: A modern approach (3rd ed.). Prentice Hall series in artificial intelligence. Upper Saddle River, NJ: Prentice Hall.
Walpole, R. E., Myers, R. H., Myers, S. L., & Ye, K. E. (2016). Probability & statistics for engineers & scientists (9th ed.). London: Pearson Education.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Kounev, S., Lange, KD., Kistowski, J.v. (2020). Experimental Design. In: Systems Benchmarking. Springer, Cham. https://doi.org/10.1007/978-3-030-41705-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-41705-5_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41704-8
Online ISBN: 978-3-030-41705-5
eBook Packages: Computer ScienceComputer Science (R0)