Advertisement

The Behavior Analyst

, Volume 21, Issue 1, pp 125–137 | Cite as

A Critique of the Usefulness of Inferential Statistics in Applied Behavior Analysis

  • B. L. Hopkins
  • Brian L. Cole
  • Tina L. Mason
Article

Abstract

Researchers continue to recommend that applied behavior analysts use inferential statistics in making decisions about effects of independent variables on dependent variables. In many other approaches to behavioral science, inferential statistics are the primary means for deciding the importance of effects. Several possible uses of inferential statistics are considered. Rather than being an objective means for making decisions about effects, as is often claimed, inferential statistics are shown to be subjective. It is argued that the use of inferential statistics adds nothing to the complex and admittedly subjective nonstatistical methods that are often employed in applied behavior analysis. Attacks on inferential statistics that are being made, perhaps with increasing frequency, by those who are not behavior analysts, are discussed. These attackers are calling for banning the use of inferential statistics in research publications and commonly recommend that behavioral scientists should switch to using statistics aimed at interval estimation or the method of confidence intervals. Interval estimation is shown to be contrary to the fundamental assumption of behavior analysis that only individuals behave. It is recommended that authors who wish to publish the results of inferential statistics be asked to justify them as a means for helping us to identify any ways in which they may be useful.

Key words

research methods data analysis inferential statistics 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abelson, R. P. (1997). On the surprising longevity of flogged horses: Why there is a case for the significance test. Psychological Science, 8, 12–15.CrossRefGoogle Scholar
  2. Atkinson, D. R., Furlong, M. J., & Wampold, B. E. (1982). Statistical significance, reviewer evaluations, and the scientific process: Is there a (statistically) significant relationship? Journal of Counseling Psychology, 29, 189–194.CrossRefGoogle Scholar
  3. Baer, D. M. (1977). Perhaps it would be better not to know everything. Journal of Applied Behavior Analysis, 10, 167–172.CrossRefPubMedPubMedCentralGoogle Scholar
  4. Bakan, D. (1966). The test of significance in psychological research. Psychological Research, 66, 423–437.Google Scholar
  5. Barlow, D. H. (Ed.). (1983). Journal of Applied Behavior Analysis, 16(2, 3).Google Scholar
  6. Bernard, C. (1927). An introduction to the study of experimental medicine. New York: Macmillan. (Original work published 1865)Google Scholar
  7. Branch, M. N. (Ed.). (1993). Journal of the Experimental Analysis of Behavior, 60.Google Scholar
  8. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312.CrossRefGoogle Scholar
  9. Cohen, J. (1994). The earth is round (p <.05). American Psychologist, 49, 997–1003.CrossRefGoogle Scholar
  10. Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Columbus, OH: Merrill.Google Scholar
  11. Costa, L., & Rourke, B. P. (Eds.). (1994). Journal of Clinical and Experimental Neuropsychology, 16(1).Google Scholar
  12. DeRouen, T. A. (1987). Comment on statistical testing and confidence intervals. American Journal of Public Health, 77, 237.CrossRefGoogle Scholar
  13. Doob, L. W. (Ed.). (1994). Journal of Social Psychology, 134(4).Google Scholar
  14. Dugbartey, A. T. (1994). The factor structure of traditional beliefs among Ghanian university students. Journal of Social Psychology, 134, 549–550.CrossRefGoogle Scholar
  15. Edgington, E. S. (1980). Random assignment and statistical tests for one-subject experiments. Behavioral Assessment, 2, 19–28.Google Scholar
  16. Edgington, E. S. (1982). Non-parametric tests for single-subject multiple schedule experiments. Behavioral Assessment, 4, 83–91.Google Scholar
  17. Fisher, R. A. (1925). Statistical methods for research workers. Edinburgh, Scotland: Oliver and Boyd.Google Scholar
  18. Fleiss, J. L. (1986). Significance tests have a role in epidemiologic research: Reactions to A. M. Walker. American Journal of Public Health, 76, 559–560.CrossRefPubMedPubMedCentralGoogle Scholar
  19. Gentile, J. R., Roden, A. H., & Klein, R. D. (1972). An analysis of variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 5, 193–198.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Gigerenzer, G., & Murray, D. J. (1987). Cognition as intuitive statistics. Hillsdale, NJ: Erlbaum.Google Scholar
  21. Gottman, J. M. (1981). Time-series analysis: A comprehensive introduction for social scientists. Cambridge, England: Cambridge University Press.Google Scholar
  22. Gottman, J. M., & Glass, G. V. (1978). Analysis of interrupted time-series experiments. In T. R. Kratochwill (Ed.), Single-subject research: Strategies for evaluating change (pp. 197–235). New York: Academic Press.CrossRefGoogle Scholar
  23. Hartmann, D. P. (1974). Forcing square pegs into round holes: Some comments on “An analysis of variance model for the intrasubject replication design.” Journal of Applied Behavior Analysis, 7, 635–638.CrossRefPubMedPubMedCentralGoogle Scholar
  24. Hartmann, D. P., Gottman, J. M., Jones, R. R., Gardner, W., Kazdin, A. E., & Vaught, R. (1980). Interrupted time-series analysis and its application to behavioral data. Journal of Applied Behavior Analysis, 13, 543–559.CrossRefPubMedPubMedCentralGoogle Scholar
  25. Hays, W. L. (1963). Statistics for psychologists. New York: Holt, Rinehart, and Winston.Google Scholar
  26. Hogben, L. (1957). Statistical theory: The relationship of probability, credibility, and error: An examination of the contemporary crisis in statistical theory from a behaviourist viewpoint. London: Allen and Unwin.Google Scholar
  27. Horne, G. P., Yang, M. C. K., & Ware, W. B. (1982). Time-series analysis for single-subject designs. Psychological Bulletin, 91, 178–189.CrossRefGoogle Scholar
  28. Howell, D. C. (1992). Statistical methods for psychology (3rd ed.). Boston: PWS-Kent.Google Scholar
  29. Huitema, B. E. (1986). Statistical analysis and single-subject designs. In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis (pp. 209–232). New York: Plenum.CrossRefGoogle Scholar
  30. Hulse, S. H. (Ed.). (1994). Journal of Experimental Psychology: Animal Behavior Processes, 20(3).Google Scholar
  31. Hunt, E. (Ed.). (1994). Journal of Experimental Psychology: General, 123(2).Google Scholar
  32. Hunter, J. E. (1997). Needed: A ban on the significance test. Psychological Science, 8, 3–7.CrossRefGoogle Scholar
  33. Hurlburt, R. T. (1994). Comprehending behavioral statistics. Pacific Grove, CA: Brooks/Cole.Google Scholar
  34. Johnston, J. M., & Pennypacker, H. S. (1993). Strategies and tactics of behavioral research (2nd ed.). Hillsdale, NJ: Erlbaum.Google Scholar
  35. Jones, R. R., Vaught, R. S., & Weinrott, M. (1977). Time-series analysis in operant research. Journal of Applied Behavior Analysis, 10, 151–166.CrossRefPubMedPubMedCentralGoogle Scholar
  36. Kazdin, A. E. (1976). Statistical analyses for single-case experimental designs. In M. Hersen & D. H. Barlow (Eds.), Single-case experimental designs: Strategies for studying behavior change (pp. 265–316). Oxford, England: Pergamon Press.Google Scholar
  37. Keselman, H. J., & Leventhal, L. (1974). Concerning the statistical procedures enumerated by Gentile et al.: Another perspective. Journal of Applied Behavior Analysis, 7, 643–645.CrossRefPubMedPubMedCentralGoogle Scholar
  38. Kratochwill, T. (Ed.). (1974). Single-subject research: Strategies for evaluating change. New York: Academic Press.Google Scholar
  39. Kratochwill, T., Alden, K., Demuth, D., Dawson, D., Panicucci, C., Arnston, P., McMurray, N., Hemstead, J., & Levin, J. (1974). A further consideration in the application of an analysis of variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 7, 629–633.CrossRefPubMedPubMedCentralGoogle Scholar
  40. Kratochwill, T., & Levin, J. R. (1980). On the applicability of various data analysis procedures to the simultaneous and alternating treatment designs in behavior therapy research. Behavioral Assessment, 2, 353–360.Google Scholar
  41. Lachenbruch, P. A., Clark, V. A., Cumberland, W. G., Chang, P. C., Afifi, A. A., Flack, V. F., & Elashoff, R. M. (1987). Comment on statistical testing and confidence intervals. American Journal of Public Health, 77, 237.CrossRefPubMedCentralGoogle Scholar
  42. Levin, J. R. (Ed.). (1994). Journal of Educational Psychology, 86(3).Google Scholar
  43. Mainstone, L. E., & Levi, A. S. (1987). Fundamentals of statistical process control. Journal of Organizational Behavior Management, 9, 5–21.CrossRefGoogle Scholar
  44. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806–834.CrossRefGoogle Scholar
  45. Michael, J. (1974). Statistical inference for individual organism research: Mixed blessing or curse? Journal of Applied Behavior Analysis, 7, 647–653.CrossRefPubMedPubMedCentralGoogle Scholar
  46. Neef, N. A. (Ed.). (1993). Journal of Applied Behavior Analysis, 26(2, 3).Google Scholar
  47. Nevin, J. A. (Ed.). (1983). Journal of the Experimental Analysis of Behavior, 40.Google Scholar
  48. Notz, W. W., Boschman, I., & Tax, S. T. (1987). Reinforcing punishment and extinguishing reward: On the folly of OBM without SPC. Journal of Organizational Behavior Management, 9, 33–46CrossRefGoogle Scholar
  49. Parsonson, B. S., & Baer, D. M. (1986). The graphic analysis of data. In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis (pp. 157–186). New York: Plenum.CrossRefGoogle Scholar
  50. Pfadt, A., Cohen, I. L., Sudhalter, V., Romanczyk, R. G., & Wheeler, D. J. (1992). Applying statistical process control to clinical data: An illustration. Journal of Applied Behavior Analysis, 25, 551–560.CrossRefPubMedPubMedCentralGoogle Scholar
  51. Pfadt, A., & Wheeler, D. J. (1995). Using statistical process control to make data-based clinical decisions. Journal of Applied Behavior Analysis, 28, 349–370.CrossRefPubMedPubMedCentralGoogle Scholar
  52. Poole, C. (1987). Beyond the confidence interval. American Journal of Public Health, 77, 195–199.CrossRefPubMedPubMedCentralGoogle Scholar
  53. Rayner, K. (Ed.). (1994). Journal of Experimental Psychology: Learning, Memory, and Cognition, 20(5).Google Scholar
  54. Rosnow, R. L., & Rosenthal, R. (1988). Focused tests of significance and effect size estimation in counseling psychology. Journal of Counseling Psychology, 35, 203–208.CrossRefGoogle Scholar
  55. Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276–1284.CrossRefGoogle Scholar
  56. Rozeboom, W. W. (1960). The fallacy of the null-hypothesis significance test. Psychological Bulletin, 57, 416–428.CrossRefPubMedGoogle Scholar
  57. Savitz, D. (1987). Comment on statistical testing and confidence intervals. American Journal of Public Health, 77, 237–238.CrossRefGoogle Scholar
  58. Schmitt, N. (Ed.). (1994). Journal of Applied Psychology, 79(3).Google Scholar
  59. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309–316.CrossRefGoogle Scholar
  60. Shrot, P. E. (1997). Should significance tests be banned? Introduction to a special section exploring the pros and cons. Psychological Science, 8, 1–2.CrossRefGoogle Scholar
  61. Sidman, M. (1960). Tactics of scientific research. New York: Basic Books.Google Scholar
  62. Somerville, S. C. (Ed.). (1994). Child Development 65(3).Google Scholar
  63. Spence, J. T., Cotton, J. W., Underwood, B. J., & Duncan, C. P. (1990). Elementary statistics (5th ed.). Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  64. Thompson, W. D. (1987). Statistical criteria in the interpretation of epidemiologic data. American Journal of Public Health, 77, 191–194.CrossRefPubMedPubMedCentralGoogle Scholar
  65. Thoresen, C. E., & Elashoff, J. D. (1974). “An analysis-of-variance model for intrasubject replication design”: Some additional comments. Journal of Applied Behavior Analysis, 7, 639–641.CrossRefPubMedPubMedCentralGoogle Scholar
  66. Van Meter, P., Yokoi, L., & Pressley, M. (1994). College students’ theory of note taking derived from their perceptions of note-taking. Journal of Educational Psychology, 86, 323–338.CrossRefGoogle Scholar
  67. Walker, A. M. (1986). Reporting the results of epidemiologic studies. American Journal of Public Health, 76, 556–558.CrossRefPubMedPubMedCentralGoogle Scholar
  68. Williams, D. G. (1994). Population density and mental illness. Journal of Social Psychology, 134, 545–546.CrossRefPubMedGoogle Scholar
  69. Wolff, A. B., Sass, K. J., & Keidan, J. (1994). Case report of an intracarotid amobarbital procedure performed for a deaf patient. Journal of Clinical and Experimental Neuropsychology, 16, 15–20.CrossRefPubMedGoogle Scholar

Copyright information

© Association for Behavior Analysis International 1998

Authors and Affiliations

  • B. L. Hopkins
    • 1
  • Brian L. Cole
    • 1
  • Tina L. Mason
    • 1
  1. 1.Department of PsychologyAuburn UniversityAuburnUSA

Personalised recommendations