Abstract
Evaluating social validity of interventions is paramount to the applied dimension of applied behavior analysis, but in research contexts, social validity is frequently assessed only after an intervention has concluded. The present study sought to evaluate the social validity of a proposed intervention prior to intervention development through qualitative analysis of interviews with stakeholders. We interviewed ten caregivers of children with autism about their experiences and challenges engaging in play with their children. We also solicited their opinions on the proposed components of the intervention, including (1) video models; (2) smart speaker prompts; and (3) feedback and coaching via video chat. Qualitative analysis of the interview transcripts revealed several domains related to caregivers’ experiences, including challenges with social communication, openness to learning skills, and desires to build connections with one’s child. Opinions about the proposed intervention components were largely positive, citing the convenience of using technology to access the mostly asynchronous intervention. Some participants expressed hesitation or discomfort surrounding feedback and coaching via video chat. Participants made several helpful suggestions regarding the content and features of the intervention. We discuss our findings in the context of an interactive social validation process.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets analyzed during the current study are available from the corresponding author on reasonable request.
References
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Association.
Anderson, R., Taylor, S., Taylor, T., & Virues-Ortega, J. (2021). Thematic and textual analysis methods for developing social validity questionnaires in applied behavior analysis. Behavioral Interventions, 37(3), 732–753. https://doi.org/10.1002/bin.1832
Bachmeyer-Lee, M. H., Kirkwood, C. A., Sheehan, C. M., Gibson, A. L., Shuler, N. J., & Keane, J. A. (2020). Utilizing in-vivo feedback for caregiver training of pediatric feeding protocols. Journal of Applied Behavior Analysis, 53(3), 1622–1637. https://doi.org/10.1002/jaba.693
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1), 91–97. https://doi.org/10.1901/jaba.1968.1-91
Baker, S., & Edwards, R. (2017). How many qualitative interviews is enough? Expert voices and early career reflections on sampling and cases in qualitative research. National Centre for Research Methods Review Paper.
Behavior Analyst Certification Board. (2020). Ethics code for behavior analysts. https://bacb.com/wp-content/ethics-code-for-behavior-analysts/
Behavior Analyst Certification Board. (2023). BCBA test content outline (6th ed.). https://www.bacb.com/wp-content/uploads/2022/01/BCBA-6th-Edition-Test-Content-Outline-230206-a.pdf
Brantlinger, E., Jimenez, R., Klingner, J., Pugach, M., & Richardson, V. (2005). Qualitative studies in special education. Exceptional Children, 71(2), 195–207. https://doi.org/10.1177/001440290507100205
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676X.2019.1628806
Braun, V., & Clarke, V. (2021). One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 18(3), 328–352. https://doi.org/10.1080/14780887.2020.1769238
Burney, V., Arnold-Saritepe, A., & McCann, C. M. (2023). Rethinking the place of qualitative methods in behavior analysis. Perspectives on Behavior Science, 46, 185–200. https://doi.org/10.1007/s40614-022-00362-x
Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013). Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement. Sociological Methods & Research, 42(3), 294–320. https://doi.org/10.1177/0049124113500475
Ferguson, J. L., Cihon, J. H., Leaf, J. B., Van Meter, S. M., McEachin, J., & Leaf, R. (2018). Assessment of social validity trends in the Journal of Applied Behavior Analysis. European Journal of Behavior Analysis, 20(1), 146–157. https://doi.org/10.1080/15021149.2018.1534771
Finney, J. W. (1991). On further development of the concept of social validity. Journal of Applied Behavior Analysis, 24(2), 245–249. https://doi.org/10.1901/jaba.1991.24-245
Ginn, N. C., Clionsky, L. N., Eyberg, S. M., Warner-Metzger, C., & Abner, J. P. (2017). Child-directed interaction training for young children with autism spectrum disorders: Parent and child outcomes. Journal of Clinical Child & Adolescent Psychology, 46(1), 101–109. https://doi.org/10.1080/15374416.2015.1015135
Hamberger, R. J., Evmenova, A. S., Coogle, C. G., & Regan, K. S. (2022). Parent coaching in natural communication opportunities through bug-in-ear technology. Topics in Early Childhood Special Education, 42(3), 234–245. https://doi.org/10.1177/02711214221119031
Hill, C. E., Thompson, B. J., & Williams, E. N. (1997). A guide to conducting consensual qualitative research. The Counseling Psychologist, 25(4), 517–572. https://doi.org/10.1177/0011000097254001
Hill, C. E., Knox, S., Thompson, B. J., Williams, E. N., Hess, S. A., & Ladany, N. (2005). Consensual qualitative research: An update. Journal of Counseling Psychology, 52(2), 196–205. https://doi.org/10.1037/0022-0167.52.2.196
Institute for Education Sciences. (2023). Education research grants program: Request for applications. U.S. Department of Education https://ies.ed.gov/funding/pdf/2024_84305A.pdf
Kemmerer, A. R., Vladescu, J. C., DeBar, R. M., Sidener, T. M., & Bell, M. C. (2023). A scoping review of the caregiver training literature for individuals with autism spectrum disorder. Behavioral Interventions, 38(3), 767–792. https://doi.org/10.1002/bin.1939
Kennedy, C. H. (1992). Trends in the measurement of social validity. The Behavior Analyst, 15, 147–156. https://doi.org/10.1007/BF03392597
Ledford, J. R., Hall, E., Conder, E., & Lane, J. D. (2016). Research for young children with autism spectrum disorders: Evidence of social and ecological validity. Topics in Early Childhood Special Education, 35(4), 223–233. https://doi.org/10.1177/0271121415585956
Leung, L. (2015). Validity, reliability, and generalizability in qualitative research. Journal of Family Medicine & Primary Care, 4(3), 324–327. https://doi.org/10.4103/2249-4863.161306
McLaughlin, D. M., & Carr, E. G. (2005). Quality of rapport as a setting event for problem behavior: Assessment and intervention. Journal of Positive Behavior Interventions, 7(2), 68–91. https://doi.org/10.1177/10983007050070020401
Schwartz, I. S., & Baer, D. M. (1991). Social validity assessments: Is current practice state of the art? Journal of Applied Behavior Analysis, 24(2), 189–204. https://doi.org/10.1901/jaba.1991.24-189
Schwartz, I. S., & Olswang, L. B. (1996). Evaluating child behavior change in natural settings: Exploring alternative strategies for data collection. Topics in Early Childhood Special Education, 16(1), 82–101. https://doi.org/10.1177/027112149601600108
Schwartz, I. S., Staub, D., Gallucci, C., & Peck, C. A. (1995). Blending qualitative and behavior analytic research methods to evaluate outcomes in inclusive schools. Journal of Behavioral Education, 5(1), 93–106. https://doi.org/10.1007/BF02110216
Shillingsburg, M. A., Bowen, C. N., & Shapiro, S. K. (2014). Increasing social approach and decreasing social avoidance in children with autism spectrum disorder during discrete trial training. Research in Autism Spectrum Disorders, 8(11), 1443–1453. https://doi.org/10.1016/j.rasd.2014.07.013
Spiegel, H. J., Kisamore, A. N., Vladescu, J. C., & Karsten, A. M. (2016). The effects of video modeling with voiceover instruction and on-screen text on parent implementation of guided compliance. Child & Family Behavior Therapy, 38(4), 299–317. https://doi.org/10.1080/07317107.2016.1238690
Unholz-Bowden, E., McComas, J. J., McMaster, K. L., Girtler, S. N., Kolb, R. L., & Shipchandler, A. (2020). Caregiver training via telehealth on behavioral procedures: A systematic review. Journal of Behavioral Education, 29(2), 246–281. https://doi.org/10.1007/s10864-020-09381-7
Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11(2), 203–214. https://doi.org/10.1901/jaba.1978.11-203
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Disclosure of Potential Conflicts of Interest
The authors declare that they have no conflicts of interest.
Ethical Approval
This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Oregon Research Institute Institutional Review Board. Protocol # ORI_BP4_G1022.
Informed Consent
Informed consent was obtained from all individual participants in this study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Guinness, K.E., Atkinson, R.S. & Feil, E.G. Evaluating Social Validity to Inform Intervention Development: Qualitative Analysis of Caregiver Interviews. Behav Analysis Practice (2024). https://doi.org/10.1007/s40617-023-00899-6
Accepted:
Published:
DOI: https://doi.org/10.1007/s40617-023-00899-6