This chapter introduces the foundations of experimental design. Starting with the case of one factor, the analysis of variance technique from statistics is introduced, followed by the method of contrasts for comparing subsets of alternatives. The analysis of variance technique is then generalized to two factors that can be varied independently, and after that, it is generalized to m factors. Following this, the Plackett–Burman fractional factorial design is introduced and compared with the full-factorial analysis of variance technique. Finally, a case study showing how experimental design can be applied in practice is presented.
No amount of experimentation can ever prove me right; a single experiment can prove me wrong.
—Albert Einstein (1879–1955)
This is a preview of subscription content, access via your institution.
Unable to display preview. Download preview PDF.
Amrhein, V., Greenland, S., & McShane, B. (2019). Scientists rise up against statistical significance. Nature, 567(7748), 305–307.
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
Lilja, D. J. (2000). Measuring computer performance: A practitioner’s guide. Cambridge: Cambridge University Press.
Noorshams, Q., Bruhn, D., Kounev, S., & Reussner, R. (2013). Predictive performance modeling of virtualized storage systems using optimized statistical regression techniques. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013), Prague (pp. 283–294). New York, NY: ACM.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12, 2825–2830.
Plackett, R. L., & Burman, J. P. (1946). The design of optimum multifactorial experiments. Biometrika, 33(4), 305–325. Oxford: Oxford University Press on behalf of Biometrika Trust.
Russell, S., & Norvig, P. (2009). Artificial intelligence: A modern approach (3rd ed.). Prentice Hall series in artificial intelligence. Upper Saddle River, NJ: Prentice Hall.
Walpole, R. E., Myers, R. H., Myers, S. L., & Ye, K. E. (2016). Probability & statistics for engineers & scientists (9th ed.). London: Pearson Education.
Rights and permissions
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Kounev, S., Lange, KD., Kistowski, J.v. (2020). Experimental Design. In: Systems Benchmarking. Springer, Cham. https://doi.org/10.1007/978-3-030-41705-5_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41704-8
Online ISBN: 978-3-030-41705-5
eBook Packages: Computer ScienceComputer Science (R0)