Skip to main content
SpringerLink
Log in

Evaluating the Effectiveness of Psychotropic Medications: Principles, Experimental Designs, and Outcome Measures

  • Chapter
  • First Online:
Handbook of Autism and Pervasive Developmental Disorder

Part of the book series: Autism and Child Psychopathology Series ((ACPS))

  • 2880 Accesses

Abstract

The randomized clinical trial (RCT) is the gold-standard source of evidence for the efficacy of psychotropic medication. In order to interpret the results of RCTs, the informed consumer should appreciate the rationale for methodological decisions which may go unexplained by experienced trialists. To this end, this chapter reviews the foundational concepts involved in structural design, outcome assessment, and statistical analysis of RCTs, with special attention to issues which arise in the study of treatments for autism spectrum disorder.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Adams-Huet, B., & Ahn, C. (2009). Bridging clinical investigators and statisticians: writing the statistical methodology for a research proposal. Journal of Investigative Medicine, 57(8), 818–824. https://doi.org/10.2310/JIM.0b013e3181c2996c

    Article  PubMed  PubMed Central  Google Scholar 

  • Altman, D. G., & Schulz, K. F. (2001). Statistics notes: Concealing treatment allocation in randomised trials. British Medical Journal, 323(7310), 446–447. https://doi.org/10.1136/bmj.323.7310.446

    Article  PubMed  PubMed Central  Google Scholar 

  • Aman, M. G., Novotny, S., Samango-Sprouse, C., Lecavalier, L., Leonard, E., Gadow, K. D., King, B. H., Pearson, D. A., Gernsbacher, M. A., & Chez, M. (2004). Outcome measures for clinical drug trials in autism. CNS Spectrums, 9(1), 36.

    Article  PubMed  PubMed Central  Google Scholar 

  • Anagnostou, E., Jones, N., Huerta, M., Halladay, A. K., Wang, P., Scahill, L., Horrigan, J. P., Kasari, C., Lord, C., Choi, D., & Sullivan, K. (2015). Measuring social communication behaviors as a treatment endpoint in individuals with autism spectrum disorder. Autism, 19(5), 622–636.

    Article  PubMed  Google Scholar 

  • Assmann, S. F., Pocock, S. J., Enos, L. E., & Kasten, L. E. (2000). Subgroup analysis and other (mis)uses of baseline data in clinical trials. The Lancet, 355(9209), 1064–1069. https://doi.org/10.1016/s0140-6736(00)02039-0

    Article  Google Scholar 

  • Bishop, S., Farmer, C., Kaat, A., Georgiades, S., Kanne, S., & Thurm, A. (2019). The need for a developmentally based measure of social-communication skills. Journal of the Americam Acadamy of Child and Adolescent Psychiatry, 58(6), 555.

    Article  Google Scholar 

  • Broglio, K. (2018). Randomization in clinical trials: Permuted blocks and stratification. Journal of the American Medical Association, 319(21), 2223–2224.

    Article  PubMed  Google Scholar 

  • Brugha, T. S., Doos, L., Tempier, A., Einfeld, S., & Howlin, P. (2015). Outcome measures in intervention trials for adults with autism spectrum disorders; a systematic review of assessments of core autism features and associated emotional and behavioural problems. International Journal of Methods and Psychiatric Resesearch, 24(2), 99–115. https://doi.org/10.1002/mpr.1466

    Article  Google Scholar 

  • Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365–376.

    Article  PubMed  Google Scholar 

  • Chow, S. C. (2014). Adaptive clinical trial design. Annual Review of Medicine, 65, 405–415. https://doi.org/10.1146/annurev-med-092012-112310

    Article  PubMed  Google Scholar 

  • Christensen, D. L., Braun, K. V. N., Baio, J., Bilder, D., Charles, J., Constantino, J. N., Daniels, J., Durkin, M. S., Fitzgerald, R. T., Kurzius-Spencer, M., Lee, L.-C., Pettygrove, S., Robinson, C., Schulz, E., Wells, C., Wingate, M. S., Zahorodny, W., & Yeargin-Allsopp, M. (2018). Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2012. MMWR Surveillance Summaries, 65(13), 1.

    Article  PubMed Central  Google Scholar 

  • Emanuel, E. J., Wendler, D., & Grady, C. (2000). What makes clinical research ethical? Journal of the American Medical Asociation, 283(20), 2701–2711.

    Article  Google Scholar 

  • Farmer, C., Golden, C., & Thurm, A. (2016). Concurrent validity of the differential ability scales, with the Mullen Scales of Early Learning in young children with and without neurodevelopmental disorders. Child Neuropsychology, 22(5), 556–569.

    Article  PubMed  Google Scholar 

  • Farmer, C. A., Kaat, A. J., Thurm, A., Anselm, I., Akshoomoff, N., Bennett, A., Berry, L., Bruchey, A., Barshop, B. A., Berry-Kravis, E., & Bianconi, S. (2020). Person ability scores as an alternative to norm-referenced scores as outcome measures in studies of neurodevelopmental disorders. American Journal of Intellectua and Devevelopmental Disabilities, 125(6), 475–480.

    Article  Google Scholar 

  • Fisher, M., & Keil, F. C. (2018). The binary bias: a systematic distortion in the integration of information. Psychological Science, 29(11), 1846–1858. https://doi.org/10.1177/0956797618792256

    Article  PubMed  Google Scholar 

  • Food and Drug Administration. (2018). The Drug Development Process. Retrieved from https://www.fda.gov/patients/drug-development-process/step-3-clinical-research

  • Gelman, A. (2013). Commentary: P values and statistical practice. Epidemiology, 24(1), 69–72.

    Article  PubMed  Google Scholar 

  • Goodman, W., Price, L., Rasmussen, S., Riddle, M., & Rapoport, J. (1991). Children’s yale-brown obsessive compulsive scale (CY-BOCS). Department of Psychiatry, Yale University School of Medicine.

    Google Scholar 

  • Grabb, M. C., Cross, A. J., Potter, W. Z., & McCracken, J. T. (2016). Derisking psychiatric drug development: The NIMH’s fast fail program, a novel precompetitive model. Journal of Clinical Psychopharmacology, 36(5), 419–421. https://doi.org/10.1097/jcp.0000000000000536

    Article  PubMed  PubMed Central  Google Scholar 

  • Grzadzinski, R., Janvier, D., & Kim, S. H. (2020). Recent developments in treatment outcome measures for young children With Autism Spectrum Disorder (ASD). Seminars in Pediatric Neurology, 34, 100806. https://doi.org/10.1016/j.spen.2020.100806

    Article  PubMed  PubMed Central  Google Scholar 

  • Havdahl, K. A., Hus Bal, V., Huerta, M., Pickles, A., Øyen, A.-S., Stoltenberg, C., Lord, C., & Bishop, S. L. (2016). Multidimensional influences on autism symptom measures: Implications for use in etiological research. Journal fo the American Acadamy of Child and Adolescent Psychiatry, 55(12), 1054–1063.e1053. https://doi.org/10.1016/j.jaac.2016.09.490

    Article  Google Scholar 

  • Haynes, R. B. (2006). Forming research questions. Journal of Clinical Epidemiology, 59(9), 881.

    Article  Google Scholar 

  • Heneghan, C., Goldacre, B., & Mahtani, K. R. (2017). Why clinical trial outcomes fail to translate into benefits for patients. Trials, 18(1), 122. https://doi.org/10.1186/s13063-017-1870-2

    Article  PubMed  PubMed Central  Google Scholar 

  • Ioannidis, J. P. (2020). Why Most Research Findings Are False. Journal Club ReprodutcibiliTea Brasil, 1, 07.

    Google Scholar 

  • Jones, D. R., & Mandell, D. S. (2020). To address racial disparities in autism research, we must think globally, act locally: SAGE Publications Sage UK: .

    Google Scholar 

  • Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S. (2002). Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry, 59(10), 877–883.

    Article  PubMed  Google Scholar 

  • Lachin, J. M. (2004). The role of measurement reliability in clinical trials. Clinical Trials, 1(6), 553–566.

    Article  PubMed  Google Scholar 

  • Lang, T. A., & Stroup, D. F. (2020). Who knew? The misleading specificity of “double-blind” and what to do about it. Trials, 21(1), 697. https://doi.org/10.1186/s13063-020-04607-5

    Article  PubMed  PubMed Central  Google Scholar 

  • Lecavalier, L., Wood, J. J., Halladay, A. K., Jones, N. E., Aman, M. G., Cook, E. H., Handen, B. L., King, B. H., Pearson, D. A., Hallett, V., & Sullivan, K. A. (2014). Measuring anxiety as a treatment endpoint in youth with autism spectrum disorder. Journal of Autism & Developmental Disorders, 44(5), 1128–1143.

    Article  Google Scholar 

  • Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011). The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research, 45(5), 626–629. https://doi.org/10.1016/j.jpsychires.2010.10.008

    Article  PubMed  Google Scholar 

  • Levine, R. J. (1988). Ethics and regulation of clinical research. Yale University Press.

    Google Scholar 

  • Little, R. J., D’Agostino, R., Cohen, M. L., Dickersin, K., Emerson, S. S., Farrar, J. T., Frangakis, C., Hogan, J. W., Molenberghs, G., Murphy, S. A., & Neaton, J. D. (2012). The prevention and treatment of missing data in clinical trials. New England Journal of Medicine, 367(14), 1355–1360.

    Article  PubMed  Google Scholar 

  • Mallinckrod, C. H., Lane, P. W., Schnell, D., Peng, Y., & Mancuso, J. P. (2008). Recommendations for the primary analysis of continuous endpoints in longitudinal clinical trials. Drug Information Journal, 42(4), 303–319.

    Article  Google Scholar 

  • McConachie, H., Parr, J. R., Glod, M., Hanratty, J., Livingstone, N., Oono, I. P., Robalino, S., Baird, G., Beresford, B., Charman, T., Garland, D., Green, J., Gringras, P., Jones, G., Law, J., Le Couteur, A. S., Macdonald, G., … Williams, K. (2015). Systematic review of tools to measure outcomes for young children with autism spectrum disorder. Health Technol Assess, 19(41), 1–506.

    Article  PubMed  PubMed Central  Google Scholar 

  • National Institutes of Health. (2017). NIH’s Definition of a Clinical Trial. Retrieved from https://grants.nih.gov/policy/clinical-trials/definition.htm

  • Pawlik, T. M., & Sosa, J. A. (2014). Success in academic surgery: Clinical trials. Springer.

    Book  Google Scholar 

  • Ratto, A. B., Anthony, B. J., Pugliese, C., Mendez, R., Safer-Lichtenstein, J., Dudley, K. M., Kahn, N. F., Kenworthy, L., Biel, M., Martucci, J. L., & Anthony, L. G. (2017). Lessons learned: Engaging culturally diverse families in neurodevelopmental disorders intervention research. Autism, 21(5), 622–634. https://doi.org/10.1177/1362361316650394

    Article  PubMed  Google Scholar 

  • Ruberg, S. J., Harrell, F. E., Jr., Gamalo-Siebers, M., LaVange, L., Jack Lee, J., Price, K., & Peck, C. (2019). Inference and decision making for 21st-century drug development and approval. The American Statistician, 73(sup1), 319–327.

    Article  Google Scholar 

  • Sansone, S. M., Schneider, A., Bickel, E., Berry-Kravis, E., Prescott, C., & Hessl, D. (2014). Improving IQ measurement in intellectual disabilities using true deviation from population norms. Journal of Neurodevelopmental Disorders, 6(1), 1–15.

    Article  Google Scholar 

  • Scahill, L., Aman, M. G., Lecavalier, L., Halladay, A. K., Bishop, S. L., Bodfish, J. W., Grondhuis, S., Jones, N., Horrigan, J. P., & Cook, E. H. (2015). Measuring repetitive behaviors as a treatment endpoint in youth with autism spectrum disorder. Autism, 19(1), 38–52.

    Article  PubMed  Google Scholar 

  • Senn, S. (2005). Dichotomania: an obsessive compulsive disorder that is badly affecting the quality of analysis of pharmaceutical trials. Proceedings of the International Statistical Institute, 55th Session, Sydney.

    Google Scholar 

  • Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107.

    Article  PubMed  Google Scholar 

  • Snapinn, S. M., & Jiang, Q. (2007). Responder analyses and the assessment of a clinically relevant treatment effect. Trials, 8(1), 31. https://doi.org/10.1186/1745-6215-8-31

    Article  PubMed  PubMed Central  Google Scholar 

  • Soorya, L., Leon, J., Trelles, M. P., & Thurm, A. (2018). Framework for assessing individuals with rare genetic disorders associated with profound intellectual and multiple disabilities (PIMD): The example of Phelan McDermid Syndrome. Clinical Neuropsychology, 32(7), 1226–1255. https://doi.org/10.1080/13854046.2017.1413211

    Article  Google Scholar 

  • Stedman, A., Taylor, B., Erard, M., Peura, C., & Siegel, M. (2019). Are children severely affected by autism spectrum disorder underrepresented in treatment studies? An analysis of the literature. Journal of Autism & Developmental Disorders, 49(4), 1378–1390.

    Article  Google Scholar 

  • Sturm, A., Kuhfeld, M., Kasari, C., & McCracken, J. T. (2017). Development and validation of an item response theory-based Social Responsiveness Scale short form. Journal of Child Psychology & Psychiatry, 58(9), 1053–1061. https://doi.org/10.1111/jcpp.12731

    Article  Google Scholar 

  • Vickers, A. J. (2001). The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: A simulation study. BMC Medical Research Methodology, 1(1), 6. https://doi.org/10.1186/1471-2288-1-6

    Article  PubMed  PubMed Central  Google Scholar 

  • Vickers, A. J. (2003). How many repeated measures in repeated measures designs? Statistical issues for comparative trials. BMC Medical Research Methodology, 3(1), 22. https://doi.org/10.1186/1471-2288-3-22

    Article  PubMed  PubMed Central  Google Scholar 

  • Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a World Beyond “p < 0.05”. The American Statistician, 73(sup1), 1–19. https://doi.org/10.1080/00031305.2019.1583913

    Article  Google Scholar 

  • West, E. A., Travers, J. C., Kemper, T. D., Liberty, L. M., Cote, D. L., McCollow, M. M., & Stansberry Brusnahan, L. L. (2016). Racial and ethnic diversity of participants in research supporting evidence-based practices for learners with autism spectrum disorder. The Journal of Special Education, 50(3), 151–163.

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Intramural Research Program of the NIMH (1ZICMH002961).

Author information

Authors and Affiliations

  1. Neurodevelopmental and Behavioral Phenotyping Service, Intramural Research Program, NIMH, NIH, Bethesda, MD, USA

    Cristan Farmer

Authors
  1. Cristan Farmer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cristan Farmer .

Editor information

Editors and Affiliations

  1. Department of Psychology, Louisiana State University, Baton Rouge, LA, USA

    Johnny L. Matson

  2. The Graduate Center and Queens College, City University of New York, New York, NY, USA

    Peter Sturmey

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Farmer, C. (2022). Evaluating the Effectiveness of Psychotropic Medications: Principles, Experimental Designs, and Outcome Measures. In: Matson, J.L., Sturmey, P. (eds) Handbook of Autism and Pervasive Developmental Disorder. Autism and Child Psychopathology Series. Springer, Cham. https://doi.org/10.1007/978-3-030-88538-0_48

Download citation

Publish with us

Policies and ethics

Search

Navigation