Skip to main content

Investigating the Measurement Invariance and Method-Trait Effects of Parent and Teacher SNAP-IV Ratings of Preschool Children

Child Psychiatry & Human Development volume 53, pages 489–501 (2022)Cite this article

Abstract

The Swanson, Nolan, and Pelham scale version IV (SNAP-IV) is widely used to assess symptoms of attention deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD) in children and adolescents. Nevertheless, there is insufficient data to support its use in preschool children. The study had three goals: First, to test the factorial validity of the three correlated-factors model of ADHD and ODD items of the SNAP-IV. Second, to investigate the measurement invariance of the items over time (6-month longitudinal interval) and by sex. Third, to investigate the convergent validity and method-specific influences on ADHD/ODD assessments with respect to multiple raters (parents/teachers) of children’s symptoms. Participants were 618 preschool children (3.5–6 years) at baseline and 6-month follow-up. For model testing, we used confirmatory factor analysis for categorical observed variables. Method and trait effects were examined using the CT-C(M-1) model. The analyses showed partial measurement invariance over time and according to sex. Moreover, strong rater-specific effects were detected. The implication of the results for construct validation of the instrument and clinical assessment of ADHD and ODD traits are discussed.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

Notes

  1. The parcels of the different rater groups differ in the items that were assigned to one parcel. This is due to the fact that the parcels were originally created independently for the different rating groups using the item loadings of separate factor analyses for the different rater groups. A reviewer asked the question whether this difference in the parcels might question the validity of the conclusions about convergent validity. Therefore, in all analyses were redone using the same parcels across raters. The majority of the results for parcels containing identical items differed in the second decimal value, what do not change the interpretability of our results (data upon request).

  2. The mean difference refers to the standardized solution, in which the factor variances equal 1. It expresses the mean differences in proportions of a standard deviation. It should not be interpreted as Cohen’s d, which is based on a latent difference model in which the mean differences are divided by the standard deviation of the difference variable. Because we do not analyze a latent difference model, we used this other form of standardization which is automatically provided by Mplus.

References

  1. Posner J, Polanczyk GV, Sonuga-Barke E (2020) Attention-deficit hyperactivity disorder. Lancet 395:450–562. https://doi.org/10.1016/S0140-6736(19)33004-1

    Article  PubMed  PubMed Central  Google Scholar 

  2. Burke JD, Rowe R, Boylan K (2014) Functional outcomes of child and adolescent oppositional defiant disorder symptoms in young adult men. J Child Psychol Psychiatry 55:264–272. https://doi.org/10.1111/jcpp.12150

    Article  PubMed  Google Scholar 

  3. Franke B, Michelini G, Asherson P et al (2018) Live fast, die young? A review on the developmental trajectories of ADHD across the lifespan. Eur Neuropsychopharmacol 28:1059–1088. https://doi.org/10.1016/j.euroneuro.2018.08.001

    Article  PubMed  PubMed Central  Google Scholar 

  4. Nock MK, Kazdin AE, Hiripi E, Kessler RC (2007) Lifetime prevalence, correlates, and persistence of oppositional defiant disorder: results from the National Comorbidity Survey Replication. J Child Psychol Psychiatry 48:703–713. https://doi.org/10.1111/j.1469-7610.2007.01733.x

    Article  PubMed  Google Scholar 

  5. Van De Schoot R, Schmidt P, De Beuckelaer A, Lek K, Zondervan-Zwijnenburg M (2015) Measurement invariance. Front Psychol 6:1064. https://doi.org/10.3389/978-2-88919-650-0

    Article  Google Scholar 

  6. Eid M, Nussbeck FW, Geiser C, Cole DA, Gollwitzer M, Lischetzke T (2008) Structural equation modeling of multitrait-multimethod data: different models for different types of methods. Psychol Methods 13:230–253. https://doi.org/10.1037/a0013219

    Article  PubMed  Google Scholar 

  7. Eid M, Lischetzke T, Nussbeck FW, Trierweiler LI (2003) Separating trait effects from trait-specific method effects in multitrait-multimethod models: a multiple-indicator CT-C (M-1) model. Psychol Methods 8:38–60. https://doi.org/10.1037/1082-989X.8.1.38

    Article  PubMed  Google Scholar 

  8. Campbell DT, Fiske DW (1959) Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull 56:81–105. https://doi.org/10.1037/h0046016

    Article  PubMed  Google Scholar 

  9. Swanson JM, Kraemer HC, Hinshaw SP et al (2001) Clinical relevance of the primary findings of the MTA: success rates based on severity of ADHD and ODD symptoms at the end of treatment. J Am Acad Child Adolesc Psychiatry 40:168–179. https://doi.org/10.1097/00004583-200102000-00011

    Article  PubMed  Google Scholar 

  10. Bussing R, Fernandez M, Harwood M et al (2008) Parent and teacher SNAP-IV ratings of attention deficit hyperactivity disorder symptoms: psychometric properties and normative ratings from a school district sample. Assessment 15:317–328. https://doi.org/10.1177/1073191107313888

    Article  PubMed  PubMed Central  Google Scholar 

  11. Gau SSF, Shang CY, Liu SK et al (2008) Psychometric properties of the Chinese version of the Swanson, Nolan, and Pelham, version IV scale–parent form. Int J Methods Psychiatr Res 17:35–44. https://doi.org/10.1002/mpr.237

    Article  PubMed  PubMed Central  Google Scholar 

  12. Inoue Y, Ito K, Kita Y et al (2014) Psychometric properties of Japanese version of the Swanson, Nolan, and Pelham, version-IV Scale-Teacher Form: a study of school children in community samples. Brain Dev 36:700–706. https://doi.org/10.1016/j.braindev.2013.09.003

    Article  PubMed  Google Scholar 

  13. Costa DS, de Paula JJ, Malloy-Diniz LF, Romano-Silva MA, Miranda DM (2018) Parent SNAP-IV rating of attention-deficit/hyperactivity disorder: accuracy in a clinical sample of ADHD, validity, and reliability in a Brazilian sample. J Pediatr 95:736–743. https://doi.org/10.1016/j.jped.2018.06.014

    Article  Google Scholar 

  14. Ullebø AK, Breivik K, Gillberg C, Lundervold AJ, Posserud MB (2012) The factor structure of ADHD in a general population of primary school children. J Child Psychol Psychiatry 53:927–936. https://doi.org/10.1037/a0020975

    Article  PubMed  Google Scholar 

  15. Hall CL, Guo B, Valentine AZ et al (2019) The validity of the SNAP-IV in children displaying ADHD symptoms. Assessment. https://doi.org/10.1177/1073191119842255

    Article  PubMed  Google Scholar 

  16. Leopold DR, Christopher ME, Olson RK, Petrill SA, Willcutt EG (2018) Invariance of ADHD symptoms across sex and age: a latent analysis of ADHD and impairment ratings from early childhood into adolescence. J Abnorm Child Psychol 47:21–34. https://doi.org/10.1007/s10802-018-0434-6

    Article  Google Scholar 

  17. King KM, Luk JW, Witkiewitz K et al (2018) Externalizing behavior across childhood as reported by parents and teachers: a partial measurement invariance model. Assessment 25:744–758. https://doi.org/10.1177/1073191116660381

    Article  PubMed  Google Scholar 

  18. Burns GL, Servera M, del Mar BM, Carrillo JM, Geiser C (2014) Ratings of ADHD symptoms and academic impairment by mothers, fathers, teachers, and aides: construct validity within and across settings as well as occasions. Psychol Assess 26:1247–1258. https://doi.org/10.1037/pas0000008

    Article  PubMed  Google Scholar 

  19. Eich D, Angst J, Frei A et al (2012) A new rating scale for adult ADHD based on the Symptom Checklist 90 (SCL-90-R). Eur Arch Psychiatry Clin Neurosci 262:519–528. https://doi.org/10.1007/s00406-011-0288-1

    Article  PubMed  Google Scholar 

  20. Willoughby MT, Pek J, Greenberg MT, the Family Life Project Investigators (2012) Parent-reported attention deficit/hyperactivity symptomatology in preschool-aged children: factor structure, developmental change, and early risk factors. J Abnorm Child Psychol 40:1301–1312. https://doi.org/10.1007/s10802-012-9641-8

    Article  Google Scholar 

  21. Burns GL, Walsh JA, Servera M et al (2013) Construct validity of ADHD/ODD rating scales: recommendations for the evaluation of forthcoming DSM-V ADHD/ODD scales. J Abnorm Child Psychol 41:15–26. https://doi.org/10.1007/s10802-012-9660-5

    Article  PubMed  Google Scholar 

  22. Leopold DR, Christopher ME, Burns GL et al (2016) Attention-deficit/hyperactivity disorder and sluggish cognitive tempo throughout childhood: temporal invariance and stability from preschool through ninth grade. J Child Psychol Psychiatry 57:1066–1074. https://doi.org/10.1111/jcpp.12505

    Article  PubMed  PubMed Central  Google Scholar 

  23. Lahey BB, Pelham WE, Loney J et al (2004) Three-year predictive validity of DSM-IV attention deficit hyperactivity disorder in children diagnosed at 4–6 years of age. Am J Psychiatry 161:2014–2020. https://doi.org/10.1176/appi.ajp.161.11.2014

    Article  PubMed  Google Scholar 

  24. Puglisi ML, Cogo-Moreira H, Eid M, Polanczyk GV (2020) Comparison of curricular preschool programmes to promote language and executive functions development: a pragmatic, cluster, randomised controlled trial. [Manuscript submitted for publication]. Department of Psychiatry, University of Sao Paulo

  25. Fricke S, Bowyer-Crane C, Haley AJ, Hulme C, Snowling MJ (2013) Efficacy of language intervention in the early years. J Child Psychol Psychiatry 54:280–290. https://doi.org/10.1111/jcpp.12010

    Article  PubMed  Google Scholar 

  26. Dias NM, Seabra AG (2013) PIAFEx programa de intervenção em autorregulação e funções executivas [PIAFEx intervention program in self-regulation and executive functions]. Memnon, São Paulo

    Google Scholar 

  27. Fricke S, Burgoyne K, Bowyer-Crane C et al (2017) The efficacy of early language intervention in mainstream school settings: a randomized controlled trial. J Child Psychol Psychiatry 58:1141–1151. https://doi.org/10.1111/jcpp.12737

    Article  PubMed  Google Scholar 

  28. Mattos P, Serra-Pinheiro MA, Rohde LA, Pinto D (2006) A Brazilian version of the MTA-SNAP-IV for evaluation of symptoms of attention-deficit/hyperactivity disorder and oppositional-defiant disorder. Rev Psiquiatr Rio Gd Sul 28:290–297. https://doi.org/10.1590/S0101-81082006000300008

    Article  Google Scholar 

  29. Geiser C, Burns GL, Servera M (2014) Testing for measurement invariance and latent mean differences across methods: interesting incremental information from multitrait-multimethod studies. Front Psychol 5:1216. https://doi.org/10.3389/fpsyg.2014.01216

    Article  PubMed  PubMed Central  Google Scholar 

  30. Nussbeck FW, Eid M, Lischetzke T (2006) Analysing multitrait–multimethod data with structural equation models for ordinal variables applying the WLSMV estimator: what sample size is needed for valid results? Br J Math Stat Psychol 59:195–213. https://doi.org/10.1348/000711005X67490

    Article  PubMed  Google Scholar 

  31. Liu Y, Millsap RE, West SG et al (2017) Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychol Methods 22:486–506. https://doi.org/10.1037/met0000075

    Article  PubMed  Google Scholar 

  32. Millsap RE, Yun-Tein J (2004) Assessing factorial invariance in ordered-categorical measures. Multivar Behav Res 39:479–515. https://doi.org/10.1207/S15327906MBR3903_4

    Article  Google Scholar 

  33. Landis RS, Beal DJ, Tesluk PE (2000) A comparison of approaches to forming composite measures in structural equation models. Organ Res Methods 3:186–207. https://doi.org/10.1177/109442810032003

    Article  Google Scholar 

  34. Little TD, Cunningham WA, Shahar G, Widaman KF (2002) To parcel or not to parcel: exploring the question, weighing the merits. Struct Equ Model 9:151–173. https://doi.org/10.1207/S15328007SEM0902_1

    Article  Google Scholar 

  35. Little TD, Rhemtulla M, Gibson K, Schoemann AM (2013) Why the items versus parcels controversy needn’t be one. Psychol Methods 18:285–300. https://doi.org/10.1037/a0033266

    Article  PubMed  PubMed Central  Google Scholar 

  36. Schermelleh-Engel K, Moosbrugger H, Müller H (2003) Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods Psychol Res Online 8:23–74

    Google Scholar 

  37. Hu LT, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 6:1–55. https://doi.org/10.1080/10705519909540118

    Article  Google Scholar 

  38. Muthén LK, Muthén B (2015) Mplus.The comprehensive modeling program for applied researchers: user’s guide. Muthén & Muthén, Virginia

    Google Scholar 

  39. Whittaker TA (2012) Using the modification index and standardized expected parameter change for model modification. J Exp Educ 80:26–44. https://doi.org/10.1080/00220973.2010.531299

    Article  Google Scholar 

  40. Dobrean A, Păsărelu CR, Balazsi R, Predescu E (2019) Measurement invariance of the ADHD Rating Scale–IV home and school versions across age, gender, clinical status, and informant. Assessment. https://doi.org/10.1177/1073191119858421

    Article  PubMed  Google Scholar 

  41. Takayanagi N, Yoshida S, Yasuda S et al (2016) Psychometric properties of the Japanese ADHD-RS in preschool children. Res Dev Disabil 55:268–278. https://doi.org/10.1016/j.ridd.2016.05.002

    Article  PubMed  Google Scholar 

  42. Bauermeister JJ, Canino G, Polanczyk G, Rohde LA (2010) ADHD across cultures: is there evidence for a bidimensional organization of symptoms? J Clin Child Adolesc Psychol 39:362–372. https://doi.org/10.1080/15374411003691743

    Article  PubMed  Google Scholar 

  43. Tabachnick BG, Fidell LS (2013) Using multivariate statistics, 6th edn. Pearson, New Jersey

    Google Scholar 

  44. Munkvold L, Lundervold A, Lie SA, Manger T (2009) Should there be separate parent and teacher-based categories of ODD? Evidence from a general population. J Child Psychol Psychiatry 50:1264–1272. https://doi.org/10.1111/j.1469-7610.2009.02091.x

    Article  PubMed  Google Scholar 

  45. Eid M, Holtmann J, Santangelo P, Ebner-Priemer U (2017) On the definition of latent-state-trait models with autoregressive effects. Eur J Psychol Assess 33:285–295. https://doi.org/10.1027/1015-5759/a000435

    Article  Google Scholar 

  46. Burns GL, Walsh JA, Servera M, Lorenzo-Seva U, Cardo E, Rodríguez-Fornells A (2013) Construct validity of ADHD/ODD rating scales: recommendations for the evaluation of forthcoming DSM-V ADHD/ODD scales. J Abnorm Child Psychol 41:15–26. https://doi.org/10.1007/s10802-012-9660-5

    Article  PubMed  Google Scholar 

  47. Litson K, Geiser C, Burns GL, Servera M (2018) Trait and state variance in multi-informant assessments of ADHD and academic impairment in Spanish first-grade children. J Clin Child Adolesc Psychol 47:699–712. https://doi.org/10.1080/15374416.2015.1118693

    Article  PubMed  Google Scholar 

  48. Burns GL, Geiser C, Servera M, Becker SP, Beauchaine TP (2020) Application of the bifactor S–1 model to multisource ratings of ADHD/ODD Symptoms: an appropriate bifactor model for symptom ratings. J Abnorm Child Psychol 48:881–894. https://doi.org/10.1007/s10802-019-00608-4

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This work was supported by Fundação de Amparo à Pesquisa de São Paulo (FAPESP) [Grant Numbers 12/51624-1].

Author information

Authors and Affiliations

  1. Departamento de Psicologia e Psicanálise, Universidade Estadual de Londrina, Rodovia Celso Garcia Cid, PR 445, km 380, Campus Univeristário, Londrina, PR, CEP 86.057-970, Brazil

    Patrícia Silva Lúcio

  2. Psychology and Educational Sciences, Free University of Berlin, Berlin, Germany

    Michael Eid

  3. School of Public Health, The University of Hong Kong, Hong Kong, Hong Kong Special Administrative Region, China

    Hugo Cogo-Moreira

  4. Department of Communication Sciences, Universidade Federal de São Paulo, São Paulo, SP, Brazil

    Marina Leite Puglisi

  5. Faculdade de Medicina FMUSP, Universidade de São Paulo, São Paulo, SP, Brazil

    Guilherme V. Polanczyk

Authors
  1. Patrícia Silva Lúcio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Eid
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hugo Cogo-Moreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marina Leite Puglisi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guilherme V. Polanczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Patrícia Silva Lúcio.

Ethics declarations

Conflict of interest

This study was registered with ClinicalTrials.gov (Identifier NCT02807831). In the past 3 years, Guilherme V. Polanczyk has been active as a consultant, member of advisory board, and/or speaker for Shire/Takeda, Medice, Ache and Novo Nordisk. He received travel expenses for continuing education support from Shire/Takeda and royalties from Editora Manole. The other authors declare no conflict of interests.

Ethical Approval

The present study received approval by the ethical committee on human experimentation (Protocol Number 171.933/2012). In accordance with the Declaration of Helsinki, parents gave written consent for children’s participation.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

For the parent ratings, the following additional loadings are suggested: Item 15 should be loaded on the ATT factor (MI = 14.571, SEPC = − 0.400), item 3 on the HI factor (MI = 14.672, SEPC = 0.326), and item 1 on the ODD factor (MI = 10.670, SEPC = − 0.282). For the teacher ratings: Item 10 (MI = 31.600, SEPC = 0.291), item 16 (MI = 27.910, SEPC = − 0.313), item 20 (MI = 10.707, SEPC = − 0.198), and item 22 (MI = 11.336, SEPC = 0.164) should be loaded on the ATT factor; item 1 (MI = 12.805, SEPC = − 0.183), item 5 (MI = 13.346, SEPC = − 0.211), item 8 (MI = 11.363, SEPC = 0.186), item 22 (MI = 18.994, SEPC = 0.355), and item 25 (MI = 11.621, SEPC = 0.331) on the HI factor; and item 1 (MI = 11.698, SEPC = − 0.166), item 5 (MI = 11.008, SEPC = − 0.178), and item 18 (MI = 12.791, SEPC = 0.290) on the ODD factor.

Appendix 2

Estimated factor loadings and their respective standard errors for the baseline assessment

Domains

Items

Parents

Teachers

Estimate

SE

Estimate

SE

 

Item 1

0.542

0.038

0.932

0.007

 

Item 2

0.680

0.043

0.949

0.010

 

Item 3

0.541

0.048

0.907

0.018

ATT

Item 4

0.666

0.035

0.921

0.016

 

Item 5

0.652

0.028

0.909

0.017

 

Item 6

0.422

0.045

0.830

0.030

 

Item 7

0.455

0.054

0.861

0.024

 

Item 8

0.595

0.053

0.914

0.021

 

Item 9

0.642

0.040

0.897

0.018

 

Item 10

0.631

0.027

0.916

0.013

 

Item 11

0.676

0.036

0.936

0.010

 

Item 12

0.603

0.035

0.939

0.010

HI

Item 13

0.528

0.037

0.932

0.013

 

Item 14

0.643

0.033

0.975

0.006

 

Item 15

0.443

0.049

0.896

0.014

 

Item 16

0.446

0.042

0.843

0.025

 

Item 17

0.713

0.026

0.926

0.016

 

Item 18

0.652

0.032

0.924

0.014

 

Item 19

0.682

0.032

0.911

0.018

 

Item 20

0.637

0.024

0.910

0.014

 

Item 21

0.777

0.028

0.939

0.013

 

Item 22

0.655

0.032

0.968

0.009

ODD

Item 23

0.418

0.037

0.865

0.030

 

Item 24

0.404

0.052

0.760

0.036

 

Item 25

0.547

0.045

0.850

0.024

 

Item 26

0.682

0.049

0.888

0.022

Appendix 3

Factor correlations (IN, HI, and ODD) across parents and teachers at time 1 and time 2 (diagonals are variances)

Time/Rater

Trait

Pre-test

Post-test

ATT

HI

ODD

ATT

HI

ODD

Teachers

Pre-test

ATT

6.996

     

HI

0.664

18.341

    

ODD

0.569

0.816

8.045

   

Post-test

ATT

0.768

0.424

0.376

8.137

  

HI

0.586

0.752

0.69

0.693

15.152

 

ODD

0.527

0.702

0.828

0.564

0.848

8.397

Parents

Pre-test

ATT

0.869

     

HI

0.614

1.000

    

ODD

0.551

0.659

1.489

   

Post-test

ATT

0.612

0.575

0.471

1.003

  

HI

0.410

0.685

0.478

0.645

1.328

 

ODD

0.427

0.554

0.67

0.602

0.733

1.842

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lúcio, P.S., Eid, M., Cogo-Moreira, H. et al. Investigating the Measurement Invariance and Method-Trait Effects of Parent and Teacher SNAP-IV Ratings of Preschool Children. Child Psychiatry Hum Dev 53, 489–501 (2022). https://doi.org/10.1007/s10578-021-01145-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10578-021-01145-2

Keywords

  • SNAP-IV
  • Measurement invariance
  • Inter-rater agreement
  • ADHD
  • ODD