The Question-Design-Analysis Bridge

Abstract

This is the first of four chapters of Part I of this book. The focus of Part I lies on common questions in experimental research. In this first chapter, the question-design-analysis (QDA) heuristic is introduced: there is a bridge connecting research questions and hypotheses, experimental design and sampling procedures, and common statistical methods in that context. This heuristic runs throughout the parts and chapters of this book and is used to present statistical analysis plans for a wide variety of situations and which include alternative routes to deal with assumption departures. This chapter briefly introduces the different statistical methods that are covered in this book through the QDA heuristic. Both likely candidates and alternative methods are presented for different questions and designs. Further, common flaws that move away from the original research question or from a particular hypothesis and/or fail to appropriately account for one experimental design feature or another are discussed. Examples of flaws discussed in this chapter are two-sided testing where one-sided testing would be expected or vice versa, treating two-way data as one-way data, treating a mediating variable as a confounding variable, and treating a within-subjects effect as between-subjects. This chapter provides the foundation for the subsequent chapters in this first part of the book: different approaches to statistical testing and estimation (Chap. 2), important principles of measurement, validity, and reliability (Chap. 3), and methods to deal with missing data (Chap. 4).