Abstract
With the objective of increasing the magnitude of treatment effects in behavioral health, there is steadily growing interest in tailoring assessments and interventions to better match individual needs. This aligns with the central idea that behavior can be adequately understood by considering the unique characteristics of the individual and context. Thus, data collected at an individual level provides critical evidence that can be used to inform health care decisions, improve treatment, or refine theories. Yet, the majority of research in behavioral health is based on group-level analyses. Recent developments in the field of single-case experimental design (SCED) has provided new opportunities to utilize individual data. The present article provides a state-of-the art overview regarding key aspects of SCED, including a historical background to why and how SCED emerged, declined, and recently reemerged as well as methodological aspects such as design issues, challenges related to reliability and validity of repeated observations, innovations in visual and statistical analyses of individual data, strategies to deal with missing values, methodology to examine effect size, and approaches to summarize data from a large number of SCEDs using multilevel models and meta-analyses of replication data. Finally, the article discusses key concerns and actions needed to move the field forward.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
It should be noted that there is an ongoing discussion within the field regarding the definitions of and relationships among terms used to label study designs and methods utilizing singe case data. In biomedical research, the term “N-of-1 trial” is commonly used for a multiple crossover evaluation performed in a single individual (Guyatt & Jaeschke, 1990). N-of-1 trials can be considered a subset of SCEDs.
CONSORT=CONsolidated Standards Of Reporting Trials
References
Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. The American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191.
Baker, D. B., & Benjamin Jr., L. T. (2000). The affirmation of the scientist-practitioner. A look back at Boulder. The American Psychologist, 55(2), 241–247 Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/10717972.
Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single case experimental designs: Strategies for studying behavior change (3rd ed.). New York, NY: Pearson.
Barlow, D. H., Farchione, T. J., Bullis, J. R., Gallagher, M. W., Murray-Latin, H., Sauer-Zavala, S., et al. (2017). The unified protocol for transdiagnostic treatment of emotional disorders compared with diagnosis-specific protocols for anxiety disorders: A randomized clinical trial. JAMA Psychiatry, 74(9), 875–884. https://doi.org/10.1001/jamapsychiatry.2017.2164.
Baron, A., & Perone, M. (1998). Experimental design and analysis in the laboratory study of human operant behavior. In K. A. Lattal & M. Perone (Eds.), Handbook of research methods in human operant behavior (pp. 45–91). Plenum Press.
Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D., Olkin, I., et al. (1996). Improving the quality of reporting of randomized controlled trials: The CONSORT statement. Journal of the American Medical Association, 276(8), 637–639. https://doi.org/10.1001/jama.276.8.637.
Bernard, C. (1957). An introduction to the study of experimental medicine. Courier Corporation Dover. (Original published 1865)
Bulte, I., & Onghena, P. (2008). An R package for single-case randomization tests. Behavior Research Methods, 40(2), 467–478 https://doi.org/10.3758/BRM.40.2.467.
Bulté, I., & Onghena, P. (2012). When the truth hits you between the eyes: A software tool for the visual analysis of single-case experimental data. Methodology, 8(3), 104–114.
Campbell, B. A. (1956). The reinforcement difference limen (RDL) function for shock reduction. Journal of Experimental Psychology, 52(4), 258–262 Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/13367349.
Clark, D. M. (2011). Implementing NICE guidelines for the psychological treatment of depression and anxiety disorders: The IAPT experience. International Review of Psychiatry, 23(4), 318–327. https://doi.org/10.3109/09540261.2011.606803.
DEcIDE, & Panel, M. C. N.-o.-G (2014). Design and implementation of n-of-1 trials: A user’s guide (AHRQ Publication No. 13(14)-EHC122-EF). Retrieved from http://www.effectivehealthcare.ahrq.gov/N-1-Trials.cfm., Issue.
Denis, J. L. (2014). Accountability in healthcare organizations and systems. Healthc Policy, 10(Spec issue), 8–11. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/25305384
Dunsmoor, J. E., Martin, A., & LaBar, K. S. (2012). Role of conceptual knowledge in learning and retention of conditioned fear. Biological Psychology, 89(2), 300–305. https://doi.org/10.1016/j.biopsycho.2011.11.002.
Evans, J. G. (1995). Evidence-based and evidence-biased medicine. Age and Ageing, 24(6), 461–463. https://doi.org/10.1093/ageing/24.6.461.
Fisher, R. A. (1925). Statistical methods for research workers. Oliver & Boyd.
Fontanarosa, P. B., & DeAngelis, C. D. (2002). Basic science and translational research in JAMA. Journal of the American Medical Association, 287(13), 1728. https://doi.org/10.1001/jama.287.13.1728.
Grossman, J., & Mackenzie, F. J. (2005). The randomized controlled trial: Gold standard, or merely standard? Perspectives in Biology and Medicine, 48(4), 516–534. https://doi.org/10.1353/pbm.2005.0092.
Group, O. L. o. E. W (2011). The Oxford levels of evidence 2. Retrieved from www.cebm.net/index.aspx?o=5653.
Guyatt, G. H., & Jaeschke, R. (1990). N-of-1 randomized trials: Where do we stand? West J Med, 152(1), 67–68. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/2309475
Hartmann, D. P. (1974). Forcing square pegs into round holes: some comments on "An analysis-of-variance model for the intrasubject replication design.". Journal of Applied Behavior Analysis, 7(4), 635–638. https://doi.org/10.1901/jaba.1974.7-635.
Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2012). A standardized mean difference effect size for single case designs. Research Synthesis Methods, 3(3), 224–239. https://doi.org/10.1002/jrsm.1052.
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179.
Houle, T. T. (2009). Statistical analyses for single-case experimental designs. In D. H. Barlow, M. K. Nock, & M. Hersen (Eds.), Single case experimental designs: Strategies for studying behavior change (Vol. 3; pp. 271−305). Allyn & Bacon.
Jacob, E. K. (2004). Classification and categorization: A difference that makes a difference. Library Trends, 52(3), 515–540.
Kaptchuk, T. J. (2001). The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf? Journal of Clinical Epidemiology, 54(6), 541–549 Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/11377113.
Kazdin, A. E. (1983). Single-case research designs in clinical child psychiatry. Journal of the American Academy of Child Psychiatry, 22(5), 423–432 Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/6630801.
Kazdin, A. E. (2011). Single-case research designs. 2nd edn. New York, Oxford University Press.
Koch, G. G., & Gillings, D. B. (1984). Inference, design based vs. model based. In N. L. Johnson & S. Kotz (Eds.), Encyclopedia of statistical sciences (Vol. 4; pp. 84–88). New York, NY: Wiley.
Kratochwill, T. R., & Levin, J. R. (1992). Single-case research design and analysis: New directions for psychology and education. Lawrence Erlbaum Associates.
Kratochwill, T. R., Hitchcock, J. R., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. Retrieved from https://ies.ed.gov/ncee/wwc/Docs/ReferenceResources/wwc_scd.pdf
Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34(1), 26–38.
Kronish, I. M., Cheung, Y. K., Shimbo, D., Julian, J., Gallagher, B., Parsons, F., & Davidson, K. W. (2019). Increasing the precision of hypertension treatment through personalized trials: A pilot study. Journal of General Internal Medicine, 34(6), 839–845. https://doi.org/10.1007/s11606-019-04831-z.
Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2018). Comparison of randomization-test procedures for single-case multiple-baseline designs. Developmental Neurorehabilitation, 21(5), 290–311. https://doi.org/10.1080/17518423.2016.1197708.
Maggin, D. M., Briesch, A. M., & Chafouleas, S. M. (2013). An application of the What Works Clearinghouse Standards for evaluating single-subject research: Synthesis of the self-management literature base. Remedial and Special Education, 34(1), 44–58.
Manolov, R. (2016). List of single-case data analysis software tools. Open Science Framework. https://osf.io/t6ws6/#!
Manolov, R., & Moeyaert, M. (2017). How can single-case data be analyzed? Software resources, tutorial, and reflections on analysis. Behavior Modification, 41(2), 179–228. https://doi.org/10.1177/0145445516664307.
McGill, R. J. (2017). Single-case design and evaluation in R: An introduction and tutorial for school psychologists. International Journal of School and Educational Psychology, 5, 39–51. https://doi.org/10.1080/21683603.2016.1173610.
McQuaid, E. L., Aosved, A. C., & Belanger, H. G. (2018). Integrating research into postdoctoral training in health service psychology: Challenges and opportunities. Training & Education in Professional Psychology, 12(2), 82–89.
Michiels, B., Heyvaert, M., Meulders, A., & Onghena, P. (2017). Confidence intervals for single-case effect size measures based on randomization test inversion. Behavior Research Methods, 49(1), 363–381. https://doi.org/10.3758/s13428-016-0714-4.
Mirza, R. D., Punja, S., Vohra, S., & Guyatt, G. (2017). The history and development of N-of-1 trials. Journal of the Royal Society of Medicine, 110(8), 330–340. https://doi.org/10.1177/0141076817721131.
Moeyaert, M., Ferron, J. M., Beretvas, S. N., & Van den Noortgate, W. (2014). From a single-level analysis to a multilevel analysis of single-case experimental designs. Journal of School Psychology, 52(2), 191–211. https://doi.org/10.1016/j.jsp.2013.11.003.
Moore, D. S., & Notz, W. I. (2017). Statistics: Concepts and controversies (9th ed.). Freeman.
Morley, S. (2018). Single case methods in clinical psychology: A practical guide. New York, NY: Routledge.
Morris, Z. S., Wooding, S., & Grant, J. (2011). The answer is 17 years, what is the question: Understanding time lags in translational research. Journal of the Royal Society of Medicine, 104(12), 510–520. https://doi.org/10.1258/jrsm.2011.110180.
Nikles, J., & McDonald, S. (2017). International Collaborative Network (ICN) for N-of-1 Clinical Trials and Single-Case Experimental Designs. https://www.nof1sced.org/.
Onghena, P., & Edgington, E. S. (2005). Customization of pain treatments: Single-case design and analysis. The Clinical Journal of Pain, 21(1), 56–68 discussion 69–72. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=15599132.
Onghena, P., Michiels, B., Jamshidi, L., Moeyaert, M., & Van den Noortgate, W. (2018). One by one: Accumulating evidence by using meta-analytical procedures for single-case experiments. Brain Impairment, 19(1), 33–58.
Parker, R. I., & Vannest, K. (2009). An improved effect size for single-case research: nonoverlap of all pairs. Behavior Therapy, 40(4), 357–367. https://doi.org/10.1016/j.beth.2008.10.006.
Pavlov, I. (1927). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex. (Translated and edited by G.V. Anrep ed.). New York. Dover Publications.
Petersen, S., Van Staeyen, K., Vogele, C., von Leupoldt, A., & Van den Bergh, O. (2015). Interoception and symptom reporting: disentangling accuracy and bias. Frontiers in Psychology, 6, 732. https://doi.org/10.3389/fpsyg.2015.00732.
Pothos, E. M., & Chater, N. (2005). Unsupervised categorization and category learning. Quarterly Journal of Experimental Psychology A, 58(4), 733–752 Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/16104104.
Punja, S., Bukutu, C., Shamseer, L., Sampson, M., Hartling, L., Urichuk, L., & Vohra, S. (2016a). N-of-1 trials are a tapestry of heterogeneity. Journal of Clinical Epidemiology, 76, 47–56. https://doi.org/10.1016/j.jclinepi.2016.03.023.
Punja, S., Xu, D., Schmid, C. H., Hartling, L., Urichuk, L., Nikles, C. J., & Vohra, S. (2016b). N-of-1 trials can be aggregated to generate group mean treatment effects: A systematic review and meta-analysis. Journal of Clinical Epidemiology, 76, 65–75. https://doi.org/10.1016/j.jclinepi.2016.03.026.
Reichow, B., Barton, E. E., & Maggin, D. M. (2018). Development and applications of the single-case design risk of bias tool for evaluating single-case design research study reports. Research in Developmental Disabilities, 79, 53–64. https://doi.org/10.1016/j.ridd.2018.05.008.
Richards, S. B. (2019). Single subject research: Applications in educational settings. Cengage.
Rothwell, P. M. (2005). External validity of randomised controlled trials: "To whom do the results of this trial apply?". Lancet, 365(9453), 82–93. https://doi.org/10.1016/S0140-6736(04)17670-8.
Rubin, M., & Badea, C. (2012). They’re all the same! … but for several different reasons: A review of the multicausal nature of perceived group variability. Current Directions in Psychological Science, 21(6), 367–372.
Schork, N. J. (2015). Personalized medicine: Time for one-person trials. Nature, 520(7549), 609–611. https://doi.org/10.1038/520609a.
Senn, S. (2018). Statistical pitfalls of personalized medicine. Nature, 563(7733), 619–621. https://doi.org/10.1038/d41586-018-07535-2.
Shamseer, L., Sampson, M., Bukutu, C., Schmid, C. H., Nikles, J., Tate, R., et al. (2015). CONSORT extension for reporting N-of-1 trials (CENT) 2015: Explanation and elaboration. BMJ, 350, h1793. https://doi.org/10.1136/bmj.h1793.
Shapiro, M. B. (1966). Generality of psychological processes and specificity of outcomes. Perceptual and Motor Skills, 23(1), 16. https://doi.org/10.2466/pms.1966.23.1.16.
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annu Rev Clin Psychol, 4, 1–32. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/18509902
Sidman, M. (1952). A note on functional relations obtained from group data. Psychological Bulletin, 49(3), 263–269 Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/14930162.
Sidman, M. (1960). The tactics of scientific research. New York, NY: Basic Books.
Simons, L., Vlaeyen, J. W. S., Declercq, L., Smith, M. A., Beebe, J., Hogan, M., Li, E., et al. (2020). Avoid or engage? Outcomes of graded exposure in youth with chronic pain using a sequential replicated single-case randomized design. Pain, 161(3), 520–531. https://doi.org/10.1097/j.pain.0000000000001735.
Smith, J. D. (2012). Single-case experimental designs: a systematic review of published research and current standards. Psychological Methods, 17(4), 510–550. https://doi.org/10.1037/a0029312.
Smith, L. S., & Wilkins, N. (2018). Mind the gap: Approaches to addressing the research-to-practice, practice-to-research chasm. Journal of Public Health Management and Practice, 24(Suppl. 1), S6–S11. https://doi.org/10.1097/PHH.0000000000000667.
Sniehotta, F. F., Presseau, J., Hobbs, N., & Araujo-Soares, V. (2012). Testing self-regulation interventions to increase walking using factorial randomized N-of-1 trials. Health Psychology, 31(6), 733–737. https://doi.org/10.1037/a0027337.
Solmi, F., & Onghena, P. (2014). Combining p-values in replicated single-case experiments with multivariate outcome. Neuropsychological Rehabilitation, 24(3–4), 607–633. https://doi.org/10.1080/09602011.2014.881747.
Solomon, B. G. (2014). Violations of assumptions in school-based single-case data: Implications for the selection and interpretation of effect sizes. Behavior Modification, 38(4), 477–496. https://doi.org/10.1177/0145445513510931.
Sung, N. S., Crowley Jr., W. F., Genel, M., Salber, P., Sandy, L., Sherwood, L. M., et al. (2003). Central challenges facing the national clinical research enterprise. Journal of the American Medical Association, 289(10), 1278–1287. https://doi.org/10.1001/jama.289.10.1278.
Tajfel, H., & Wilkes, A. L. (1963). Classification and quantitative judgement. British Journal of Psychology, 54, 101–114 Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/13980241.
Tanious, R., De, T. K., Michiels, B., Van den Noortgate, W., & Onghena, P. (2019). Assessing consistency in single-case A-B-A-B phase designs. Behavior Modification. https://doi.org/10.1177/0145445519837726.
Tarlow, K. R., & Brossart, D. F. (2018). A comprehensive method of single-case data analysis: Interrupted Time-Series Simulation (ITSSIM). School Psychology Quarterly, 33(4), 590–603. https://doi.org/10.1037/spq0000273.
Tate, R. L., & Perdices, M. (2019). Single-case experimental designs for clinical research and neurorehabilitation settings. Routledge. London and New York.
Tate, R. L., Perdices, M., Rosenkoetter, U., Wakim, D., Godbee, K., Togher, L., & McDonald, S. (2013). Revision of a method quality rating scale for single-case experimental designs and n-of-1 trials: The 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale. Neuropsychological Rehabilitation, 23(5), 619–638. https://doi.org/10.1080/09602011.2013.824383.
Tate, R. L., Perdices, M., McDonald, S., Togher, L., & Rosenkoetter, U. (2014). The design, conduct and report of single-case research: Resources to improve the quality of the neurorehabilitation literature. Neuropsychological Rehabilitation, 24(3–4), 315–331. https://doi.org/10.1080/09602011.2013.875043.
Tate, R. L., Perdices, M., Rosenkoetter, U., Shadish, W., Vohra, S., Barlow, D. H., Horner, R., Kazdin, A., Kratochwill, T., McDonald, S., Sampson, M., Shamseer, L., Togher, L., Albin, R., Backman, C., Douglas, J., Evans, J. J., Gast, D., Manolov, R., Mitchell, G., Nickels, L., Nikles, J., Ownsworth, T., Rose, M., Schmid, C. H., & Wilson, B. (2016a). The Single-Case Reporting Guideline In BEhavioural Interventions (SCRIBE) 2016 statement. Archives of Scientific Psychology, 4(1), 1–9. https://doi.org/10.1037/arc0000026
Tate, R. L., Perdices, M., Rosenkoetter, U., McDonald, S., Togher, L., Shadish, W., Horner, R., Kratochwill, T., Barlow, D. H., Kazdin, A., Sampson, M., Shamseer, L., & Vohra, S. (2016b). The Single-Case Reporting Guideline In BEhavioural Interventions (SCRIBE) 2016: Explanation and elaboration. Archives of Scientific Psychology, 4(1), 10–31. https://doi.org/10.1037/arc0000027.
Tate, R., McDonald, S., Moseley, A., Perdices, M., & Togher, L. (2019). NeuroBite: NeuroRehab Evidence Resource. https://neurorehab-evidence.com/web/cms/content/home
Ter Kuile, M. M., Bulte, I., Weijenborg, P. T. M., Beekman, A., Melles, R., & Onghena, P. (2009). Therapist-aided exposure for women with lifelong vaginismus: A replicated single-case design. Journal of Consulting and Clinical Psychology, 77(1), 149–159. https://doi.org/10.1037/a0014273.
Todman, J. B., & Dugard, P. (2001). Single-case and small-n experimental designs: A practical guide to randomization tests. Mahwah, N.J Lawrence Erlbaum Associates.
van der Meulen, M. A., Anton, F., & Petersen, S. (2017). Painful decisions: How classifying sensations can change the experience of pain. European Journal of Pain, 21(9), 1602–1610. https://doi.org/10.1002/ejp.1061.
Vlaeyen, J. W., & Morley, S. (2005). Cognitive-behavioral treatments for chronic pain: What works for whom? The Clinical Journal of Pain, 21(1), 1–8.
Vlaeyen, J. W., de Jong, J., Geilen, M., Heuts, P. H., & van Breukelen, G. (2001). Graded exposure in vivo in the treatment of pain-related fear: A replicated single-case experimental design in four patients with chronic low back pain. Behaviour Research and Therapy, 39(2), 151–166. https://doi.org/10.1016/s0005-7967(99)00174-6.
Vlaeyen, J. W., Morley, S., Linton, S., Boersma, K., & De Jong, J. (2012). Pain-related fear: Exposure-based treatment for chronic pain. IASP Press.
Vohra, S., & Punja, S. (2019). A case for n-of-1 Trials. JAMA Internal Medicine, 179(3), 452. https://doi.org/10.1001/jamainternmed.2018.7166.
Vohra, S., Shamseer, L., Sampson, M., Bukutu, C., Schmid, C. H., Tate, R., et al. (2015). CONSORT extension for reporting N-of-1 trials (CENT) 2015 statement. British Medical Journal, 350, h1738. https://doi.org/10.1136/bmj.h1738.
Walburn, J., Sarkany, R., Norton, S., Foster, L., Morgan, M., Sainsbury, K., et al. (2017). An investigation of the predictors of photoprotection and UVR dose to the face in patients with XP: A protocol using observational mixed methods. BMJ Open, 7(8), e018364. https://doi.org/10.1136/bmjopen-2017-018364.
Acknowledgements
We wish to acknowledge the people who were instrumental for and during the symposium: the participants in the strategic meeting during day 1 (in addition to the authors of the article): Maria Tillfors, Ida Flink, Katja Boersma, Mike Kemani, Lance McCracken, Julia Glombiewski, and Allen Finley, as well as the attendants of the open meeting during day 2.
Availability of Data and Materials
NA/This is a theoretical review article.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest
None
Research involving human participants and/or animals
NA
Informed Consent
NA
Additional information
Authors Note:
“Replication, replication, replication” (Morley, 2018). This article is dedicated to Stephen Morley (1950–2017), a wonderful friend and colleague who made significant contributions to the applications of single-case experimental design (SCED). It also summarizes the contributions of the Stockholm symposium, “Small is Beautiful,” held October 2018, on single-case experimental designs. The objective of the meeting was to ignite discussions among researchers and clinicians actively using or interested in applying this methodology in their work, and to discuss strategies to disseminate, implement, and further develop the SCED approach.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Vlaeyen, J.W.S., Wicksell, R.K., Simons, L.E. et al. From Boulder to Stockholm in 70 Years: Single Case Experimental Designs in Clinical Research. Psychol Rec 70, 659–670 (2020). https://doi.org/10.1007/s40732-020-00402-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40732-020-00402-5