What Data Gathering Strategies Should I Use?

Abstract

In this chapter, we review many of the data gathering strategies that can be used by postgraduates in social and behavioural research. We explore three major domains of data gathering strategies: strategies for connecting with people (encompassing interaction-based and observation-based strategies), exploring people’s handiworks (encompassing participant-centred and artefact-based strategies) and structuring people’s experiences (encompassing data-shaping and experience-focused strategies). In light of our pluralist perspective, we consider each data gathering strategy, not only as a distinct and self-contained strategy (which may encompass a range of more specific data gathering approaches), but also as part of a larger more interconnected and dynamic toolkit. Our goal is to highlight some key considerations and issues associated with each strategy that might be relevant to your decision making about which might be appropriate for you to use as part of your research journey, given your research frame, pattern(s) of guiding assumptions, contextualisations, positionings, research questions/hypotheses, scoping and shaping considerations and MU configuration.

Appendix: Clarifying Experimental/Quasi-experimental Design Jargon

These contrasting concepts provide insights into the way that researchers, who implement the Manipulative experience-focused strategy under the positivist pattern of guiding assumptions, talk or write about certain features of their research.

Between groups versus Within groups IVs

A between groups IV has categories that define groups which contain different samples of participants (e.g., a treatment group and a control group). A within groups IV defines groups or conditions, all of which are experienced by each participant or by matched sets of participants such as twins or participants matched on key characteristics. A within groups IV includes intervention time-aligned conditions such as a pre-test and a post-test, giving rise to a class of experiments called ‘repeated measures’ designs)

Factorial versus Nested designs

A factorial design involves groups defined by least two IVs where each category of one IV is combined with each category of another IV, such that the groups exhaust all possible combinations (e.g., a quasi-experiment involving the IVs of gender, with 2 categories—male and female, and an experimental IV, with 2 categories—treatment condition and control condition, yields a 2 × 2 factorial design involving four distinct pairings of IVs (male-treatment; male-control; female-treatment; female-control). If you had a between groups factorial design with four IVs and each IV had 2 categories (or ‘levels’), you would have a 2 × 2 × 2 × 2 factorial design and that design would have 16 distinct groups of participants. A nested design involves groups defined by the categories of one IV being hierarchically embedded inside each category of another IV (e.g., an IV defined by year levels for classes of students at the primary school level is embedded within a second IV defined by specific schools). Nesting means, for example, that a year 6 class in one school cannot be considered equivalent to a year 6 class in another school (different teachers, different curricular emphases, different classroom environments, …), so that classes must be considered as nested within schools. Another type of nested design is a multi-level design, which compares samples defined by IVs at different levels of analysis (e.g., departments within organisations within industries) both within and between those levels

Main effect versus Interaction effect

For causal-comparative designs, a comparison of the groups or conditions defined by the categories (or ‘levels’) of a single IV comprises the main effect of that IV on the DV. The comparison of groups simultaneously defined by combinations of the categories of two or more IVs is termed an interaction. An interaction yields a conditional interpretation, where the pattern of relationship between one IV and the DV differs depending upon which category of another IV you choose to look at. Technically speaking, a moderator IV is an interaction IV. Where two IVs define an interaction, this is called a 2-way interaction; three IVs define a 3-way interaction and so on. In a factorial design, there are as many main effects as there are IVs in the design, all possible pairs of IVs form 2-way interactions, all possible triplets of IVs form 3-way interactions and so on. For example, if you had a factorial design involving 4 IVs (call them A, B, C, and D): there would be 4 main effects (A, B, C and D main effects), six 2-way interactions (AB (read as ‘A by B interaction’), AC, AD, BC, BD, CD interactions;), four 3-way interactions (ABC, ABD, ACD, and BCD interactions) and one 4-way interaction (ABCD) to test

Incomplete or Fractional factorial designs

In some design circumstances, it may not be possible or feasible for you to include all possible factorial combinations of IV categories in a research design. For example, if you have four IVs, each with 3 categories, there would be 3 × 3 × 3 × 3 = 81 possible factorial combinations, which may be too many for you to find adequate samples to fill or to have participants rate or evaluate. As an alternative approach, you could employ an incomplete or fractional factorial design, which includes only a specific fraction or proportion of the possible combinations. In the previous example, a 1/3 fractional factorial design would require only 27 combinations instead of 81. The fractional combinations used are identified by sacrificing information about higher order interactions (e.g., three- and four-way interactions) in order to provide viable estimates of lower-order effects, such as main effects and two-way interactions (fractional factorial designs are often used in conjoint measurement designs, for example). One example of an incomplete design is a ‘Latin square’ design, which can control, using counterbalancing, for order effects between conditions or other extraneous/‘nuisance’ variables

Manipulated (usually categorical/group-based) versus Measured IVs

A manipulated IV is one where you can control who experiences a specific category of the IV (e.g., treatment or control conditions) using random assignment of participants to category. In contrast, a measured IV is one where you must take the IV as having a pre-existing value with respect to every participant and therefore you can only measure it (e.g., age, gender, ethnic background). Thus, in a true experiment, you seek to manipulate all IVs being evaluated whereas in a quasi-experiment, you generally have a mix of manipulated IVs and measured IVs