A Comparison of Paper and Computer Administered Strengths and Difficulties Questionnaire
- 291 Downloads
The Strengths and Difficulties Questionnaire (SDQ) is one of the most widely used measures of young people’s mental health difficulties in research and clinical decision-making. Although the SDQ is available in both paper and computer survey formats, cross-format equivalences have yet to be established. The current study aimed to assess the measure’s equivalence across paper- and computer-based survey formats in a community-based school setting. The study examined self-reported measures completed by a matched sample of 11–14 year olds in secondary schools in England (589 completed paper version; 589 online version). Analyses demonstrate that the factor structure, although did not vary by survey format, resulted in poorly fitting models limiting the use of model based invariance testing. Results indicate that the measure does not operate similarly across different formats, with scale-level mean differences observed for the hyperactivity scale, which also affects the total difficulties score, with higher scores seen in the paper version. Responses to the impact supplement were also influenced by survey format, with higher impact in specific domains disclosed on the computer-based measure. Item-level differential item functioning was observed for four items in the measure; two from the prosocial scale where the DIF is large enough to affect the scale (DTF, ν2 = 0.14). The inconsistency across survey formats highlights the need for more assessment of influences of different survey formats on young people, their perceived privacy and their mental health disclosures via different media. The findings also highlight the potential confounding effect of format when different methods of data collection are used, with a potentially substantive impact on cross-sample comparisons and within child clinical review.
KeywordsSDQ Computer Psychometric properties Validation DIF Format effects
We would like to thank members of the wider research group who are part of the larger study from which data are drawn and the Department for Children, Schools and Families (now Department of Education) for funding the research. We are grateful to the schools and young people who participated in the study.
Conflict of Interest
Praveetha Patalay, Daniel Hayes, Jessica Deighton, and Miranda Wolpert declare that they have no conflict of interest.
Ethics permissions for collecting data in this and the wider study from which data are drawn, were received from the Research Ethics Committee at University College London.
- Achenbach, T. M., Becker, A., Dopfner, M., Heiervang, E., Roessner, V., Steinhausen, H. C., & Rothenberger, A. (2008). Multicultural assessment of child and adolescent psychopathology with ASEBA and SDQ instruments: research findings, applications, and future directions. Journal of Child Psychology and Psychiatry, 49, 251–275.CrossRefPubMedGoogle Scholar
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC:American Psychological Association.Google Scholar
- Brown, T. A. (2012). Confirmatory factor analysis for applied research: Guilford Press.Google Scholar
- Department for Education (2010). Schools, pupils and their characteristics. London: HMSO Retrieved from https://www.gov.uk/government/publications/schools-pupils-and-their-characteristics-january-2010.Google Scholar
- Department of health. (2013). Annual report of the Chief Medical Officer 2012: Our children deserve better: prevention pays. Available from https://www.gov.uk/government/publications/chief-medical-officers-annual-report-2012-our-children-deserve-better-prevention-pays.
- Fink, E., Patalay, P., Sharpe, H., Holley, S., Deighton, J., & Wolpert, M. (2015). Mental health difficulties in early adolescence: a comparison of two cross-sectional studies in England from 2009 to 2014. Journal of Adolescent Health, 56, 502–507. doi: 10.1016/j.jadohealth.2015.01.023.CrossRefPubMedGoogle Scholar
- Giannakopoulos, G., Tzavara, C., Dimitrakaki, C., Kolaitis, G., Rotsika, V., & Tountas, Y. (2009). The factor structure of the strengths and difficulties questionnaire (SDQ) in Greek adolescents. Annals General Psychiatry, 8.Google Scholar
- Hale, D., Patalay, P., Fitzgerald-Yau, N., Hargreaves, D., Bond, L., Görzig, A.,... Viner, R. (2014). School-level variation in health outcomes in adolescence: analysis of three longitudinal studies in England. Prevention Science, 15, 600–610. doi: 10.1007/s11121-013-0414-6.
- Holländare, F., Andersson, G., & Engström, I. (2010). A comparison of psychometric properties between internet and paper versions of two depression instruments (BDI-II and MADRS-S) administered to clinic patients. Journal of Medical Internet Research, 12. doi: 10.2196/jmir.1392.
- Hu, L. t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. doi: 10.1080/10705519909540118.
- Kelley, K., & Lai, K. (2012). MBESS R Package version 3.3.2: Retrieved from http://CRAN.R-project.org/package=MBESS.
- Klasen, H., Woerner, W., Wolke, D., Meyer, R., Overmeyer, S., Kaschnitz, W.,... Goodman, R. (2000). Comparing the German versions of the strengths and difficulties questionnaire (SDQ-Deu) and the child behavior checklist. European Child & Adolescent Psychiatry, 9, 271–276.Google Scholar
- Leuven, E., & Sianesi, B. (2003). PSMATCH2: Stata module to perform full mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing (version revised 19 Jul 2012): Boston College Department of Economics.Google Scholar
- Muthén, L. K., & Muthén, B. O. (2012). Mplus User's Guide, 7th Edn. Los Angeles, CA:Muthén & Muthén.Google Scholar
- Obel, C., Heiervang, E., Rodriguez, A., Heyerdahl, S., Smedje, H., Sourander, A.,... Olsen, J. (2004). The strengths and difficulties questionnaire in the Nordic countries. European Child & Adolescent Psychiatry, 13, ii32-ii39. doi: 10.1007/s00787-004-2006-2.
- Patalay, P., Deighton, J., Fonagy, P., & Wolpert, M. (2015). Equivalence of paper and computer formats of a child self-report mental health measure. European Journal of Psychological Assessment, 31, 54-61.Google Scholar
- Rosenbaum, P. R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician, 39, 33–38.Google Scholar
- Scientific advisory committee of the medical outcomes trust. (2002). Assessing health status and quality of life instruments: attributes and review criteria. Quality of Life Research, 11, 193–205. http://www.jstor.org/stable/4038039.
- StataCorp (2011). Stata Statistical Software: Release 12. College Station:StataCorp LP.Google Scholar
- Wijndaele, K., Matton, L., Duvigneaud, N., Lefevre, J., Duquet, W., Thomis, M.,... Philippaerts, R. (2007). Reliability, equivalence and respondent preference of computerized versus paper-and-pencil mental health questionnaires. Computers in Human Behavior, 23, 1958–1970. doi: 10.1016/j.chb.2006.02.005.
- Wolpert, M., Cheng, H., & Deighton, J. (2015). Measurement issues: review of four patient reported outcome measures: SDQ, RCADS, CORS and GBO–their strengths and limitations for clinical use and service evaluation. Child and Adolescent Mental Health, 20, 63–70.Google Scholar
- Wolpert, M., Görzig, A., Deighton, J., Fugard, A. J. B., Newman, R., & Ford, T. (2015). Comparison of indices of clinically meaningful change in child and adolescent mental health services: Difference scores, reliable change, crossing clinical thresholds and ‘added value’ – an exploration using parent rated scores on the SDQ. Child and Adolescent Mental Health, 20, 94–101. doi: 10.1111/camh.12080.CrossRefGoogle Scholar