Skip to main content
SpringerLink
Log in

Randomness is problematic for social science research purposes

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Mathematically defined random chance requires that every possible alternative sampling or assignment outcome has an equal or known probability of occurring. This is very different than haphazard chance for the outcomes of which there are no such statistical expectations. Because random chance cannot be sufficiently ensured for social science research purposes is logically arguable, what is actually relied upon must be haphazard chance. This needs to be made clear for several key aspects of social science research methodology: generalizing from samples, making treatment contrasts unbiased by unmeasured on subject dimensions, meta-analysis, interpreting comparative experiments’ residual outcome variance and so their statistical significance testing, the meaning of psychological measurement unreliability and of latent variables, and the imputation of missing data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Resampling techniques treat samples as populations in order to estimate the robustness of sample estimates by multiply sub-sampling given samples in order to get a distribution of estimates, but this does not escape the problem of needing randomness for statistical significance testing: “If external validity is a goal of the analysis, then random sampling [to obtain a sample for repeated sub-sampling] is important. If internal validity is a goal (as in experimental design), random assignment is an important assumption.”, however, “If the sample that is used to generate the empirical sampling distribution does not adequately represent the population, then the elements it contains that are probabilistically different from their occurrence in the population can compromise the inference process. Thus, at a very fundamental level, all resampling procedures assume that the sample that is used to generate the empirical sampling distribution reflects the structure of the population from which it came” (Rodgers 1999, 444).

  2. Any device for approximating the results of random sampling (see Dattalo 2009, 20–92), when random sampling is infeasible, adds this device’s problems in trying to represent the target population’s distribution (and how close cannot be known) to those of a random sampling’s attempt to do this (and how close it does so is another unknown).

  3. Any device for approximating the results of random assignment (see Dattalo 2009, 93–148), when it is infeasible, for trying to equate the influence of unmeasured subject independent-variables on a comparative experiment’s dependent-variables across the experiment’s comparison groups adds this device’s problems in trying to do so (how close it does is one unknown) to those of a random assignment’s attempt to do this (and how close it does is another unknown).

References

  • Baker, J.R., Yardley, J.K., McCaul, K.: Characteristics of responding-, non-responding-, and refusing-parents in an adolescent lifestyle choice study. Eval. Rev. 25(6), 605–618 (2001)

    Article  Google Scholar 

  • Beltrami, E.: What is Random? Chance and Order in Mathematics and Life. Springer, New York (1999)

    Book  Google Scholar 

  • Bennett, D.J.: Randomness. Harvard University Press, Cambridge (1998)

    Google Scholar 

  • Brown, G.S.: Probability and Scientific Inference. Longmans, Green, London (1957)

    Google Scholar 

  • Chen, S.: Random number generators go public. Science 360, 1383–1384 (2018)

    Article  Google Scholar 

  • Dattalo, P.: Strategies to Approximate Random Sampling and Assignment. Oxford University Press, New York (2009)

    Book  Google Scholar 

  • Dattalo, P.: Ethical dilemmas in sampling. J. Soc. Work Values Ethics 7(1), 1–12 (2010)

    Google Scholar 

  • Di Franco, G.: An alternative procedure for imputing missing data based on principal components analysis. Qual. Quant. 48(3), 1149–1163 (2014)

    Article  Google Scholar 

  • Feller, W.: An Introduction to Probability Theory and Its Applications, vol. 1, 3rd edn. Wiley, New York (1968)

    Google Scholar 

  • Ganguli, M., Mendelsohn, A., Lytle, M., Dodge, H.: A follow-up comparison of study participants and refusers within a rural elderly population. J. Gerontol. Ser. A Biol. Sci. Med. Sci. 53(6), M465–M470 (1998)

    Article  Google Scholar 

  • Gellert, W., Küstner, H., Hellwich, M., Kästner, H. (eds.): The VNR Concise Encyclopedia of Mathematics. Van Nostrand Reinhold, New York (1977)

    Google Scholar 

  • Hacking, I.: Logic of Statistical Inference. Cambridge University Press, Cambridge (1965)

    Book  Google Scholar 

  • Hedges, L.V.: Estimation of effect size from a series of independent experiments. Psychol. Bull. 92, 490–499 (1982)

    Article  Google Scholar 

  • Howard, K.I., Krause, M.S., Orlinsky, D.E.: The attrition dilemma. J. Consult. Clin. Psychol. 54, 106–110 (1986)

    Article  Google Scholar 

  • Howard, K.I., Krause, M.S., Saunders, S.M., Kopta, S.M.: Trials and tribulations in the meta-analysis of treatment differences: comment on Wampold et al. (1997). Psychol. Bull. 122, 221–225 (1997)

    Article  Google Scholar 

  • Kish, L.: Survey Sampling. Wiley, New York (1965)

    Google Scholar 

  • Krause, M.S.: How the psychotherapy research community must work toward measurement validity and why. J. Clin. Psychol. 61, 269–283 (2005)

    Article  Google Scholar 

  • Krause, M.S.: Significance testing and clinical trials. Psychotherapy 48, 217–222, 234–236 (2011)

  • Krause, M.S.: Measurement validity is fundamentally a matter of definition, not correlation. Rev. Gen. Psychol. 16, 391–400 (2012)

    Article  Google Scholar 

  • Krause, M.S.: The incompatibility of fully specifying linear causal models and assuming that residual dependent variable variance is random. Qual. Quant. 47, 3201–3204 (2013a)

    Article  Google Scholar 

  • Krause, M.S.: The data analytic implications of human psychology’s dimensions being ordinally scaled. Rev. Gen. Psychol. 17, 318–325 (2013b)

    Article  Google Scholar 

  • Krause, M.S.: Case sampling for psychotherapy practice, theory, and policy guidance: qualities and quantities. Psychother. Res. 26, 530–544 (2016)

    Article  Google Scholar 

  • Krause, M.S.: The scientific study of the qualities of individual human lives, rather than of their average quantities in aggregations of lives. Qual. Quant. 52, 1315–1329 (2018a)

    Article  Google Scholar 

  • Krause, M.S.: Mathematical expression and sampling issues of treatment-contrasts: beyond significance testing and meta-analysis to clinically useful research synthesis. Psychother. Res. 28(1), 58–75 (2018b)

    Article  Google Scholar 

  • Krause, M.S.: Psychological measuring and measurements: their possible effects on persons measured and unmeasured: reactivity. J. Methods Meas. Soc. Sci. https://journals.uair.arizona.edu/index.php/jmmss/index (2018c)

  • Krause, M.S., Howard, K.I.: What random assignment does and does not do. J. Clin. Psychol. 59, 751–766 (2003)

    Article  Google Scholar 

  • Krause, M.S., Lutz, W.: How we really ought to be comparing treatments for clinical purposes. Psychotherapy 43, 359–361 (2006)

    Article  Google Scholar 

  • Krause, M.S., Lutz, W., Bőhnke, J.R.: The role of sampling in clinical trial design. Psychother. Res. 21, 143–151 (2011)

    Article  Google Scholar 

  • Little, R.J., D’Agostino, R., et al.: The prevention and treatment of missing data in clinical trials. N. Engl. J. Med. 367, 1355–1360 (2012)

    Article  Google Scholar 

  • Little, R.J., Rubin, D.B.: Statistical Analysis with Missing Data. Wiley, New York (1987)

    Google Scholar 

  • Markides, K.S., Dickson, H.D., Pappas, C.: Characteristics of dropouts in longitudinal research on aging: a study of Mexican Americans and Anglos. Exp. Aging Res. 8(3), 163–167 (1982)

    Article  Google Scholar 

  • Michell, J.: Is psychometrics pathological science? Meas. Interdiscip. Res. Perspect. 6, 7–24 (2008)

    Article  Google Scholar 

  • Michell, J.: The quantity/quality interchange: a blind spot on the highway of science. In: Valsiner, J., Toomela, A. (eds.) Methodological Thinking in Psychology: 60 Years Gone Astray?, pp. 45–68. Information Age Publishing, Charlotte (2010)

    Google Scholar 

  • Nickerson, R.S.: Null hypothesis significance testing: a review of an old and continuing controversy. Psychol. Methods 5, 241–301 (2000)

    Article  Google Scholar 

  • Nickerson, R.S.: The production and perception of randomness. Psychol. Rev. 109, 330–357 (2002)

    Article  Google Scholar 

  • Rodgers, J.L.: The bootstrap, the jackknife, and the randomization test: a sampling taxonomy. Multivar. Behav. Res. 34, 441–456 (1999)

    Article  Google Scholar 

  • Rosenthal, R.: Meta-analytic Procedures for Social Research. Sage, Beverly Hills (1984)

    Google Scholar 

  • Schafer, J.L.: Analysis of Incomplete Multivariate Data. Chapman & Hall, New York (1997)

    Book  Google Scholar 

  • Schafer, J.L., Graham, J.W.: Missing data: our view of the state of the art. Psychol. Methods 7, 147–177 (2002)

    Article  Google Scholar 

  • Seaman, S., Galati, J., Jackson, D., Carlin, J.: What is meant by” missing at random”? Stat. Sci. 28, 257–268 (2013)

    Article  Google Scholar 

  • Sechrest, L., Davis, M.F., Stickle, T.R., McKnight, P.: Understanding “method” variance. In: Bickman, L. (ed.) Research Design: Donald Campbell’s Legacy, vol. 2, pp. 63–87. Sage, Thousand Oaks (2000)

    Google Scholar 

  • Stuart, A., Ord, J.K., Arnold, S.: Kendall’s Advanced Theory of Statistics, Volume 2a: Classical Inference and the Linear Model, 6th edn. Arnold, London (1999)

    Google Scholar 

  • Thompson, M.G., Heller, K., Rody, C.A.: Recruitment challenges in studying late-life depression: do community samples adequately represent depressed older adults? Psychol. Aging 9(1), 121–125 (1994)

    Article  Google Scholar 

  • Volchan, S.B.: What is a random sequence? Am. Math. Mon. 109(1), 46–63 (2002)

    Article  Google Scholar 

  • Worrall, J.: Why there’s no cause to randomize. Br. J. Philos. Sci. 58, 451–488 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Evanston, USA

    Merton S. Krause

Authors
  1. Merton S. Krause
    View author publications

    You can also search for this author in PubMed Google Scholar

Ethics declarations

Conflict of interest

The author declares that he has no conflict of interest.

Additional information

The author is now a Ph.D. independent scholar and an emeritus fellow of AAAS, APA, and APS.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krause, M.S. Randomness is problematic for social science research purposes. Qual Quant 53, 1495–1504 (2019). https://doi.org/10.1007/s11135-018-0824-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-018-0824-4

Keywords

Search

Navigation