Abstract
Although no one knows just how effective graduate training may be in creating effective practitioners of applied behavior analysis, there are plenty of logical and historical reasons to think that not all practitioners are equally competent. I detail some of those reasons and explain why practitioner effectiveness may be a more pressing worry now than in the past. Because ineffective practitioners harm the profession, rigorous mechanisms are needed for evaluating graduate training programs in terms of the field effectiveness of their practitioners. Accountability of this nature, while difficult to arrange, would make applied behavior analysis nearly unique among professions, would complement existing quality control processes, and would help to protect the positive reputation and vigorous consumer demand that the profession currently enjoys.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Given our discipline’s strong empirical roots, there is much more to say about this. Among hypotheses worthy of discussing and testing are that research training (a) prepares students to understand clinical realities by showing behavior change in real time as a function of planned interventions; (b) teaches general measurement and data-based decision skills that help with the evaluation of clinical practice; (c) contributes to future treatment integrity and transportability by highlighting why interventions work; (d) spurs future clinical innovation by exposing future practitioners to research advances; (e) creates scientist practitioners who are capable of conducting research in practice settings; and (f) supports the discipline’s science wing by revealing to some students, in ways they might not have suspected, that their reinforcers lie in conducting research.
These course sequences can be embedded in graduate degree programs but the certification board does not approve or disapprove programs. As I understand the process, approval verifies that courses address selected topics but does not imply anything about the effectiveness of instruction.
Because I will examine some ways that demand influences supply, it is useful to specify how a market analogy does and does not apply to graduate training. In typical economic systems, suppliers of goods may profit directly from meeting demand. For “producers” of new ABA practitioners the contingencies are less direct, because graduate programs do not “sell” their students to employers the way manufacturers sell goods to consumers. Rather, they sell training to interested students who later test the employment market. Nevertheless, because programs cannot exist without students, and students presumably are sensitive to how graduate training relates to employment contingencies, some link between employment market and demand for graduate training can be assumed. I also assume that graduating students and placing them into professional positions serves as a reinforcer for those who run graduate programs—but of course, not the only reinforcer. Contingencies within academic institutions are complex and multilayered, so sometimes a program with few students will survive or a program with strong student interest will be scuttled (e.g., see Wolf 2001, on Arizona State University’s celebrated but short-lived “Fort Skinner in the Desert”). As a general rule, however, programs and the people who run them fare better when they can graduate lots of students. It is in this sense that I label program graduates as “reinforcers.”
Here I comment on the modal contingencies under which new programs develop, not on the creativity or commitment to innovation of many fine colleagues who, I am well aware, labor tirelessly in attempt to keep good practitioners flowing into the marketplace.
Or, as Benjamin Franklin remarked (and I am not making this up), “He who lives upon hope, dies farting” (Poor Richard's Almanack 1736). Franklin later amended the aphorism to, “He who lives upon hope, dies fasting.” Both versions address the pitfalls of misplaced passiveness. The updated version better describes the potential consequences for the ABA profession if it does not set its graduate training house in order, but the original version is more fun.
But there are examples from other disciplines of credentialing processes that, in the context of field expertise, are essentially inert. For example, some research in teacher education suggests that formal credentialing is unrelated to student achievement (Kane et al. 2006).
Baseball fans may recognize this approach as sharing properties with the “wins above replacement” (WAR) statistic, which estimates how a team’s win total differs from normative win totals as a result of the accomplishments of a given player (Thom and Palmer 2015). WAR is based on multivariate models that consider how a player’s measured accomplishments compare to those of other players, and how various kinds of accomplishments correlate with team win totals.
References
Bailey, J., & Burch, M. (2011). Ethics for applied behavior analysis (2nd ed.). New York: Routledge.
Ballou, D., & Springer, M. (2015). Using student test scores to measure teacher performance: some problems in the design and implementation of evaluation systems. Educational Researcher, 44(2), 77–86.
Boyd, D., Grossman, P., Lankford, H., Loeb, S., & Wyckoff, J. (2008). Teacher preparation and student achievement. National Bureau of Economic Research Working Paper 14313. Retrieved July 8, 2015, from http://www.nber.org/papers/w14314
Chetty, R., Friedman, J.N., Rockoff, J.E. (2011). The long-term impacts of teachers: teacher value-added and student outcomes in adulthood. National Bureau of Economic Research Working Paper 17699. Retrieved July 19, 2015, from http://www.nber.org/papers/w17699
Chevalier, J., & Mayzlin, D. (2006). The effect of word of mouth on sales: online book reviews. Journal of Marketing Research, 43, 345–354.
Cooper, J. O., Heron, T. E., & Heward, W. L. (2013). Applied behavior analysis (2nd ed.). New York: Pearson.
Deitz, S. M. (1978). Current status of applied behavior analysis: science versus technology. American Psychologist, 33, 805–814.
Detrich, R. (1999). Increasing treatment fidelity by matching interventions to contextual variables within the educational setting. School Psychology Review, 28, 608–620.
Dixon, M. R., Belisle, J., Whiting, S. W., & Rowsey, K. E. (2014). Developing a normative sample of the PEAK relational training system—direct training module for comparison to individuals with autism. Research in Autism Spectrum Disorders, 8(11), 1597–1606.
Dixon, M. R., Reed, D. D., Smith, T., Belisle, J., & Jackson, R. E. (2015). Research rankings of behavior analytic graduate training programs and their faculty. Behavior Analysis in Practice, 8, 7–15.
Fixsen, D.,L., Naoom, S.F., Blase, K.A., Friedman, R.M., & Wallace, F. (2005). Implementation research: a synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Freiberger, P., & Swaine, M. (2000). Fire in the valley: the making of the personal computer. New York: McGraw Hill.
Gilley, J. W., & Galbraith, M. W. (1986). Examining professional certification. Training and Development Journal, 40(6), 60–61.
Green, G. (1996). Evaluating claims about treatments for autism. In C. Maurice, G. Green, & S. Luce (Eds.), Behavioral intervention for young children with autism: a manual for parents and professionals (pp. 15–28). Austin: Pro-Ed.
Hanushek, E. A. (2011). The economic value of higher teacher quality. Educational Research Review, 30, 477–479.
Haskell, T. (1977). The emergence of professional social science: The American Social Science Association and the Nineteenth Century crisis of authority. Urbana: University of Illinois Press.
Hayes, S. C. (1978). Theory and technology in behavior analysis. The Behavior Analyst, 1, 35–41.
Hayes, S. C., Rincover, A., & Solnick, J. V. (1980). The technical drift of applied behavior analysis. Journal of Applied Behavior Analysis, 13, 275–285.
Hopkins, B. L. (1991). ABA to begin accrediting graduate programs of studies in behavior analysis. ABA Newsletter, l4(3), 19–21.
Jarbrink, K., & Knapp, M. (2009). The economic impact of autism in Britain. Autism, 5, 7–22.
Kane, T.J., Rockoff, J.E., & Staiger, D.O. (2006). What does teacher certification tell us about teacher effectiveness. Evidence from New York City. National Bureau of Economic Research Working Paper 12155. Downloaded July 28, 2015, from http://www.nber.org/papers/w12155 .
Lilienfeld, S. O. (2002). The scientific review of mental health practice: our raison d’etre. The Scientific Review of Mental Health Practice, 1, 1–9.
Lo, C. P. (2011). Global outsourcing or foreign direct investment: why Apple chose outsourcing for the iPod. Japan and the World Economy, 23, 163–169.
Lovaas, I. O. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3–9.
Maher, W. J. (1988). Contexts for understanding professional certification: opening Pandora's box? American Archivist, 51, 408–427.
Makrygianni, M. K., & Reed, P. (2010). A meta-analytic review of the effectiveness of behavioural early intervention programs for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 4, 577–593.
Mas-Colell, A., Whinston, M., & Green, J. (1995). Principles of microeconomics. Oxford: Oxford University Press.
Maurice, C. (1993). Let me hear your voice. New York: Knopf.
Michael, J. (1980). Flight from behavior analysis. The Behavior Analyst, 3, 1–21.
Moore, J., & Shook, G. L. (2001). Certification, accreditation, and quality control in behavior analysis. The Behavior Analyst, 24, 44–55.
Mullen, E. M. (1995). Mullen scales of early learning. New York: Pearson.
Noell, G. H., & Burns, J. L. (2006). Value-added assessment of teacher preparation: an illustration of emerging technology. Journal of Teacher Education, 57, 37–50.
Parker, R. I., & Hagan-Burke, S. (2007). Useful effect-size interpretations for single-case research. Behavior Therapy, 38, 95–105.
Reinberg, S. (2007). FDA halts imports of farmed fish from China. Retrieved July 23, 2015, from http://abcnews.go.com/Health/Healthday/story?id=4507743
Satariano, A. (2014). Apple’s iPhone first responders. BloombergBusiness. Retrieved July 15, 2015, from http://www.bloomberg.com/bw/articles/2014-09-04/for-iphone-6-defects-apple-has-failure-analysis-team-ready.
Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness, transportability, and dissemination of interventions. What matters when? Psychiatric Services, 52, 1190–1197.
Shook, G. L. (1993). The professional credential in behavior analysis. The Behavior Analyst, 16, 87–101.
Shook, G. L., & Favell, J. (2008). The Behavior Analysis Certification Board and the profession of behavior analysis. Behavior Analysis in Practice, 1, 44–48.
Shook, G. L., Johnston, J. M., Cone, J., Thomas, D., Greer, D., Beard, J., & Herring, R. (1988). Credentialing, quality assurance and right to practice. Kalamazoo: Association for Behavior Analysis.
Shook, G. L., Ala'i-Rosales, S., & Glenn, S. S. (2002). Training and certifying behavior analysts. Behavior Modification, 26, 27–48.
Shook, G. L., Johnston, J. M., & Mellichamp, F. H. (2004). Determining essential content for applied behavior analysis practitioners. The Behavior Analyst, 27, 67–94.
Shute, N. (2010). Desperate for a cure. Scientific American, 330(4), 80–85.
Skinner, B. F. (1953). Science and human behavior. New York: Basic Books.
Sparrow, S. S., Chichetti, D. V., & Balla, D. A. (2005). Vineland adaptive behavior scales (2nd ed.). New York: Pearson.
Starin, S., Hemingway, M., & Hartsfield, F. (1993). Credentialing behavior analysts and the Florida Behavior Analysis Certification Program. The Behavior Analyst, 16, 153–166.
Strosahl, K. D., Hayes, S. C., Bergan, J., & Romano, P. (1998). Assessing the field effectiveness of acceptance and commitment therapy: an example of the manipulation training research method. Behavior Therapy, 29, 35–69.
Sundberg, M. L. (2008). Verbal behavior milestones assessment and placement program: the VB-MAPP. Concord: AVB Press.
Thom, J., & Palmer, P. (2015). The hidden game of baseball. Chicago: University of Chicago Press.
Ulman, J. (2014). The Ulman-Skinner letters. European Journal of Behavior Analysis, 15, 11–19.
Watkins, C. L. (1987). Project follow through: a case study of contingencies influencing instructional practices of the educational establishment. Cambridge: Cambridge Center for Behavioral Studies.
Wolf, M. M. (2001). Application of operant conditioning principles to the behavior of an autistic child: a 25-year follow-up and the development of the Teaching-Family Model. In W. O’Donohue, D. Henderson, S. Hayes, J. Fisher, & L. Hayes (Eds.), A history of the behavioral therapies: founders’ personal histories (pp. 289–294). Reno: Context Press.
Author information
Authors and Affiliations
Corresponding author
Additional information
Author Note: The title is based on W.B. Yeats’ epigram to the poem “Responsibilities 1914,” which deftly melds the wisdom of Spiderman (“With great power there must also come -- great responsibility!”; Amazing fantasy #15, August, 1962) and Benjamin Franklin (“By failing to prepare, you are preparing to fail;” source unverified).
Rights and permissions
About this article
Cite this article
Critchfield, T.S. In Dreams Begin Responsibility: Why and How to Measure the Quality of Graduate Training in Applied Behavior Analysis. Behav Analysis Practice 8, 123–133 (2015). https://doi.org/10.1007/s40617-015-0090-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40617-015-0090-z