Skip to main content

Experimental Design

  • Chapter
  • 1199 Accesses


This chapter introduces the foundations of experimental design. Starting with the case of one factor, the analysis of variance technique from statistics is introduced, followed by the method of contrasts for comparing subsets of alternatives. The analysis of variance technique is then generalized to two factors that can be varied independently, and after that, it is generalized to m factors. Following this, the Plackett–Burman fractional factorial design is introduced and compared with the full-factorial analysis of variance technique. Finally, a case study showing how experimental design can be applied in practice is presented.

No amount of experimentation can ever prove me right; a single experiment can prove me wrong.

—Albert Einstein (1879–1955)

This is a preview of subscription content, access via your institution.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  • Amrhein, V., Greenland, S., & McShane, B. (2019). Scientists rise up against statistical significance. Nature, 567(7748), 305–307.

    CrossRef  Google Scholar 

  • Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

    CrossRef  Google Scholar 

  • Lilja, D. J. (2000). Measuring computer performance: A practitioner’s guide. Cambridge: Cambridge University Press.

    CrossRef  Google Scholar 

  • Noorshams, Q., Bruhn, D., Kounev, S., & Reussner, R. (2013). Predictive performance modeling of virtualized storage systems using optimized statistical regression techniques. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013), Prague (pp. 283–294). New York, NY: ACM.

    CrossRef  Google Scholar 

  • Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12, 2825–2830.

    MathSciNet  MATH  Google Scholar 

  • Plackett, R. L., & Burman, J. P. (1946). The design of optimum multifactorial experiments. Biometrika, 33(4), 305–325. Oxford: Oxford University Press on behalf of Biometrika Trust.

    Google Scholar 

  • Russell, S., & Norvig, P. (2009). Artificial intelligence: A modern approach (3rd ed.). Prentice Hall series in artificial intelligence. Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Walpole, R. E., Myers, R. H., Myers, S. L., & Ye, K. E. (2016). Probability & statistics for engineers & scientists (9th ed.). London: Pearson Education.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations


Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Cite this chapter

Kounev, S., Lange, KD., Kistowski, J.v. (2020). Experimental Design. In: Systems Benchmarking. Springer, Cham.

Download citation

  • DOI:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-41704-8

  • Online ISBN: 978-3-030-41705-5

  • eBook Packages: Computer ScienceComputer Science (R0)