European Child & Adolescent Psychiatry

, Volume 17, Issue 8, pp 491–497

Parent and child agreement on reports of problem behaviour obtained from a screening questionnaire, the SDQ

Authors

  • Marije van der Meer
    • Department of Child and Adolescent PsychiatryVrije Universiteit Medical Centre
    • Department of Psychological MedicineChildren’s Hospital at Westmead
  • Donna Rose
    • Department of Psychological MedicineChildren’s Hospital at Westmead
ORIGINAL CONTRIBUTION

DOI: 10.1007/s00787-008-0691-y

Cite this article as:
van der Meer, M., Dixon, A. & Rose, D. Eur Child Adolesc Psychiatry (2008) 17: 491. doi:10.1007/s00787-008-0691-y

Abstract

Background and objectives

This study examined the level of agreement between parents and children on the Strengths and Difficulties Questionnaire (SDQ) in a clinical sample in Sydney, Australia.

Methods

Parent and child SDQ reports were collected from 379 parents-child pairs. Children were aged between 11 and 18 years and met criteria for at least one psychiatric diagnosis.

Results

Overall agreement between children and parents was low to modest. Sixty nine percent of parent-child pairs agreed that the child’s problems were either clinically significant or not (“neither”, “both”), while in 27% of pairs only the parents regarded the problems as clinically significant (“parent only”), and in 4% of pairs only the children regarded the problems as clinically significant. There was higher agreement for children with mood, anxiety or somatoform disorders. Children with mood disorders were over-represented in the “child only” group, and those with conduct disorders were more likely to be in the “parent only” group. Children with anxiety and somatoform disorders were more likely to be in the “neither” group. Age was not associated with rates of parent-child agreement, however more girls agreed with their parents that either they did not have a problem (“neither”) or they did have a problem without parental endorsement (“child only”).

Conclusions

This study highlights the limited agreement between parent and child reports of problem behaviour and the importance of integrating discrepant information into child and adolescent mental health assessments, formulations and treatments.

Keywords

strengths and difficulties questionnaire (SDQ)parent-child agreementdiagnosis

Introduction

While an estimated 14% of children and adolescents in Australia suffer from mental health problems, only about one in five of these children receive professional help [14]. Mental health problems at an early age can create considerable distress for the child and family and can have a significant impact on the child’s social, emotional and psychological development [3, 11]. Early and accurate identification of mental health problems in children is critical in order to reduce this disruption and to minimise the likelihood that psychological problems will persist into adulthood [3].

A multi-informant approach to the evaluation of psychological problems in children and adolescents is the preferred, best-practice approach [3, 4]. This most commonly involves interviews with the child, family and school, completion of rating scales and questionnaires and observations in a diagnostic setting [3]. Questionnaires completed by both the parent and child can serve as valuable adjuncts to the clinical interview and are being increasingly used as screening instruments in the assessment process [8]. In Australia, the Strengths and Difficulties Questionnaire (SDQ) is gaining popularity with public health services as a standardised screening instrument for mental health clients. With its short administration time and sound psychometric properties [8], the SDQ is now a mandatory component of the state-wide Mental Health Outcomes and Assessment Training (MHOAT) protocol required by all mental health services in the state of New South Wales (NSW).

A substantial body of research suggests, however, that parents and children do not always agree on problem reports [1, 3, 4, 7, 9, 10]. Low parent-child agreement has been demonstrated across several different assessment formats including structured and semi-structured interviews, rating scales and questionnaires [1, 4]. Several studies examining Achenbach’s widely used Child Behaviour Checklist (CBCL) and Youth Self Report (YSR), for example, have reported low to moderate parent-child correlations of between 0.20 and 0.30 [1, 7, 12, 17]. Overall, correlations between parents and children for continuous measures range from 0.15 to 0.35 and kappa for diagnostic agreement range from 0.10 to 0.40 [4].

What implications do such findings have for the utility of such screening instruments in clinical practice, particularly given their use is becoming increasingly widespread? How should clinicians manage such differences in parent-child reports in regards to the overall understanding of the presenting problem and subsequent treatment recommendations?

Several factors have been shown to influence the rate of parent-child agreement on questionnaires, including type of behaviour, population studied, gender and age. In a meta-analysis of 119 studies examining cross-informant agreement, Achenbach et al. [1] found that in clinical samples, parent-child agreement tends to be lower for internalising problems compared to externalising. While factors such as the visibility and intrusiveness of externalising behaviours compared to the subjectivity and invisibility of internalising behaviours (i.e. the child’s private, subjective thoughts) have been proposed to contribute to this difference [1, 9], the reverse appears true for community samples where more parent-child agreement has been found for internalising problems [3, 10].

Contrasting findings are also evident between clinical and community samples with respect to age, with more parent-child agreement for younger children than older children in clinical samples, and less parent-child agreement for younger than older children in community samples [10]. Gender of either the child or the parent does not appear to have a significant effect on agreement levels [1, 7, 15], though Achenbach et al. [1] reported slightly higher parent-child agreement for externalising behaviours in girls, but not in boys.

Given that low to moderate parent-child general agreement is a consistent finding, it has often been assumed that we need to take the parent’s view as primary [10]. Rather than low agreement reflecting a lack of reliable judgements by one or other informants, however, an alternative view is that informants may be reporting uniquely different information [9]. Martin and colleagues [10] examined the patterns of parent-child responding to the CBCL and found that children whose self-ratings fell in the clinical range, but whose parent ratings did not, were more likely to be depressed or anxious. This study highlighted the importance of taking the child’s report into consideration when assessing young people, particularly when the parent was not reporting clinically abnormal behaviour.

The goal of this study is to examine parent-child agreement on the SDQ and to investigate the effects of age, gender and type of disorder on agreement level. Overall, we expect to find a low to moderate parent-child agreement, with higher correspondence for externalising than internalising disorders [10]. Younger children are expected to agree more with their parents than older children, and parent-child agreement is expected to be higher in females then in males for internalising problems rather than externalising ones.

Methods

Participants

Participants were 379 parent-child pairs who attended the Department of Psychological Medicine at the Children’s Hospital at Westmead, Sydney Australia, between January 2002 and January 2007. Psychological Medicine provides tertiary in- and out-patient mental health services to children and adolescents in metropolitan and rural NSW. Children were between the ages of 11 and 18 years (X = 13.9 years, SD = 1.62) and 47% were males. The participants were primarily from the Sydney metropolitan region and the sample comprised of 73% outpatients and 27% inpatients.

The sample was taken from a total sample of 470 parent-child pairs; 91(19.4%) were excluded because of missing data or no designated psychiatric diagnosis. The study sample was representative of the total sample with respect to age (t (468) = −0.48, P = 0.64) and gender (χ= 0.02, P = 0.9). Ethics approval for this study was granted by the Hospital Research and Ethics Committee.

Measures

Children and their parents completed the SDQ as part of the standard clinical intake evaluation. This instrument is a component of the NSW Mental Health Data Collection Strategy (SCI-MHOAT) and is routinely completed by all families accessing NSW Health mental health services.

The SDQ is a 36 item questionnaire that seeks parent and child comment on the child’s social-emotional functioning. Each item is answered on a three-point Likert scale ranging from “not true”, “somewhat true” to “certainly true”. The SDQ provides four problem subscales (emotional symptoms, conduct problems, hyperactivity-inattention and peer problems) as well as a personal strengths subscale (prosocial behavior) and an impact subscale (impairment/distress). The personal strengths scale measures the child’s ability to act prosocially, independent of the difficulties measured by the other subscales [6]. The impact scale rates the extent of distress or impairment in daily functioning at home and school, in peer relationships and recreational activities, as per the DSM-IV [2] requirement for the presence of distress or impairment in role functioning for a diagnosis of mental illness [5].

All sub scores except for the prosocial and impact scale score are summed to provide a total difficulties score. Scores for each sub-scale and overall score can be categorised as “abnormal”, “borderline” or “normal”. The SDQ has demonstrated sound psychometric properties cross-culturally in both clinical and non-clinical settings [3, 6]. Estimated completion time is 5–10 min.

Psychiatric diagnoses

All children were assigned psychiatric diagnoses according to ICD-10 criteria by either a child and adolescent psychiatrist or clinical psychologist. Diagnoses were cross-referenced with patient file information (e.g. assessment reports). Children with no clear diagnoses were excluded. All children received a primary diagnosis and 51% also met criteria for other diagnoses. For this study, children were allocated to diagnostic groups according to their primary diagnosis, i.e. that which was causing the greatest distress and disruption to the child’s functioning. The primary diagnoses were clustered into nine categories including Anxiety Disorders (e.g. generalised anxiety disorder, obsessive-compulsive disorder, reaction and adjustment disorders, post traumatic stress disorder), Mood/Depressive Disorders (e.g. major and mild depressive disorder, bipolar mood disorder, mixed anxiety and depression disorder, hypomania), Psycho-affective Disorders (e.g. psychosis, schizophrenia, schizo-affective disorders), Somatoform Disorders, Eating disorders, Conduct and Hyperactivity Disorders (e.g. oppositional defiant disorder, conduct disorder, ADHD), Pervasive Developmental Disorders (autism, asperger’s syndrome, mild mental retardation) and Other Disorders (e.g. enuresis, tic disorder).

Statistical analyses

Means and reliabilities were calculated for parent and child SDQ scores on all sub-scales and the total difficulties scale. Kappa co-efficients and percentage of agreement scores were used to examine overall agreement between parents and children, as well as agreement within each diagnostic category. χ2 analyses were used to investigate the impact of age and gender on agreement. All statistical analyses were performed with SPSS version 13.0 using an alpha of 0.05.

Results

Parent-child agreement

The SDQ provides bandings for normal, borderline or abnormal classifications. To examine the agreement between parent and child scores, these bandings were further categorised as either “clinical” or “non-clinical”, where clinical scores were those in the SDQ “abnormal” range. This was based on Martin et al.'s [10] methodology using the CBCL. This produced four possible combinations of parent and child reports on the SDQ (“both” parent and child rate child as having clinically significant problems, “neither” parent nor child rate child as having clinically significant problems, “parent only” rates child as having clinically significant problems, “child only” rates child as having clinically significant problems). Table 1 shows the age and mean SDQ scores for each of the four groups.
Table 1

Age and mean SDQ scores

 

Neither (n = 174)

Both (n = 86)

Child only (n = 17)

Parent only (n = 102)

Total sample (n = 379)

Mean

SD

Mean

SD

Mean

SD

Mean

SD

Mean

SD

Age (mean)

13.7

1.6

13.8

1.6

14.2

1.4

14.2

1.6

13.9

1.6

SDQ parent score

10.34

4.21

25.15

4.36

11.88

3.28

21.63

4.04

19.32

6.91

SDQ self report score

11.27

4.45

24.64

3.81

24.53

3.67

14.22

3.64

16.46

6.46

A kappa coefficient was then computed to estimate the agreement between parent and child scores on a case-by-case basis. Kappa was used as the initial measure of agreement as it provides an agreement estimate in excess of that which would occur by chance [9, 13]. Kappa co-efficients range from 0 to 1, with 1 indicating perfect agreement. In general, values less than 0.40 indicate poor agreement, values between 0.41 and 0.60 indicate moderate agreement, values between 0.61 and 0.80 indicate good agreement and those above 0.81 indicate very good agreement [13]. In this study, the kappa statistic for overall scores indicated a poor level of agreement (k = 0.277, P < 0.00001). Due to low base rates in some cells, however, this result needs to be interpreted with caution.

The overall percentage of parent-child agreement in this sample was good (68.6%). Interestingly, while 22% of parents and children agreed that the child’s behaviour was in the clinical range, in almost half (46%) of the sample, both the parent and child rated the child in the non-clinical rage despite the fact that all children met criteria for at least one psychiatric diagnosis.

Diagnostic categories

Table 2 shows the proportion of diagnostic groups in each of the parent-child response groups. Overall, the most prevalent disorders in the sample were mood, anxiety, conduct and somatoform, respectively. When looking at the distribution across the four parent-child response groups, mood/depressive disorders were over-represented in the “child only” group comprising over half of cases (58.8%), while children with conduct disorder and ADHD were over-represented in the “parent only” group, comprising over one quarter of that group (26.5%). Children with anxiety and somatoform disorders were more likely to be rated as non-clinical by both parent and child (32.8 and 25.3%, respectively).
Table 2

Percentage of diagnostic groups in each of the parent-child response categories

Diagnostic groups

Percentage (%)

Neither (n = 174)

Both (n = 86)

Child only (n = 17)

Parent only (n = 102)

Total sample (n = 379)

Mood/depressive disorders

28.2

36.0

58.8

23.5

29.4

Anxiety disorders

32.8

19.8

5.9

18.6

21.1

Conduct disorders and ADHD

2.3

20.9

11.8

26.5

19.1

Somatoform disorders

25.3

3.5

5.9

7.8

10.6

Psychotic disorders

2.9

5.8

5.9

8.8

6.7

PDD, other developmental disorders

2.3

12.8

5.9

5.9

6.6

Eating disorders

4.0

0.0

5.9

5.9

4.2

Other disorders

2.3

1.2

0.0

2.9

2.2

Further investigation of the percentage of agreement between parent and child reports within the different diagnostic categories (Table 3) shows that overall, high levels of agreement were particularly evident for children who are anxious, depressed, or have a somataform disorder. Of note, however, most of this agreement was in the “neither” category, i.e. the parent and child agreed that the child did not have the disorder. Less overall agreement was evident for children with conduct problems and ADHD. Importantly, examination of the “both” category, where parent and child agree that the child does have the disorder, shows that rates of agreement were higher for the externalising disorders compared to the internalising ones. Further, parents were also much more likely to identify the externalising problems than internalising problems in their children when their children did not. Agreement in the last four categories was moderate, though this interpretation warrants caution due to low base rates in these groups.
Table 3

Percentages of agreement

Diagnosis

Neither

Both

Child only

Parent only

Total agreement

Depression

43.0

27.2

8.8

21.1

70.2

Anxiety

60.6

18.1

1.1

20.2

78.7

Conduct

7.8

35.3

3.9

52.9

43.1

Somatoform

78.6

5.4

1.8

14.3

84.0

Psychosis

25.0

25.0

5.0

45.0

50.0

PDD

18.2

50.0

4.5

27.3

68.2

Eating

50.0

0.0

7.1

42.9

50.0

Other

50.0

12.5

0.0

37.5

62.5

To further examine the nature of the (dis)agreement within diagnostic groups, means and differences plots for the four primary diagnostic groups are presented in Figs. 1 and 2. Means and differences plots show whether the measurement error as estimated by the differences is related to the size of the measurement as estimated by the mean [13]. Consistent with the high percentages of parent-child agreement, the scatter for children with anxiety and somatoform disorders is evenly distributed above and below the zero line of no difference, suggesting that there is no systematic bias between parent and child reports [13]. For children with conduct disorder and ADHD, however, parents are consistently rating their children higher on the SDQ than the child reports, as demonstrated by the wider scatter of scores above the zero line. Similarly, children with mood disorders are rating themselves higher on the SDQs than their parents ratings, as indicated by the wider spread of scores below the zero line.
https://static-content.springer.com/image/art%3A10.1007%2Fs00787-008-0691-y/MediaObjects/787_2008_691_Fig1_HTML.gif
Fig. 1

Mean versus differences plot for children with anxiety and mood disorders

https://static-content.springer.com/image/art%3A10.1007%2Fs00787-008-0691-y/MediaObjects/787_2008_691_Fig2_HTML.gif
Fig. 2

Means versus differences plot for children with somatoform and conduct disorders

Finally, χ2 analyses were conducted on the four response groups to examine whether age (±14.5 years) or gender differences were evident in rates of parent-child agreement. No age differences were apparent (χ2 (3) = 12.86, P = 0.41) though there were gender differences (χ2 (3) = 8.44, P < 0.04). There were more females in the “child only” and “neither” groups, and more males in the “parent only” and “both” groups (Table 4).
Table 4

Gender differences in percentages

Gender

Neither

Both

Child only

Parent only

Total

Male

39.7

57.0

41.2

52.0

47.0

Female

60.3

43.0

58.8

48.0

53.0

Kappa value = −0.042

Discussion

In this study, a moderate level of parent-child agreement was found overall. Almost a quarter of parents and children (23%) agreed that the children’s problems fell in the clinical range. Interesting, almost a half (46%) of parents and children agreed that the problems fell in the non-clinical range, despite all children meeting ICD-10 criteria for at least one psychiatric disorder. This meant that only a third of parents and children disagreed (31%). The direction of this disagreement was mostly that the parents reported problems in the clinical range when their children did not (27%). However, four percent of children reported their problems to be in the clinical range, though their parents did not.

Further, our results suggest that level of agreement varies with type of disorder. In cases where parents and children agreed that the child has a disorder, higher levels of agreement were apparent with externalising disorders (conduct and ADHD) than internalising disorders (anxiety, depression, somatoform). These findings are consistent with many other studies that have demonstrated more parent-child agreement for externalising than internalising disorders [1, 9, 10]. Interestingly, these findings were reversed when considering the false negative group. Parents and children who agreed that the child did not have a disorder were more likely to be from the internalising group rather than externalising. While this is consistent with previous reports that parents are not as good at detecting internalising symptoms in their children compared to externalising ones, it also suggests that children may not always be accurate reporters of these symptoms as well.

While more children with conduct problems and ADHD were in the “parent only” group, the “child only” group was over-represented by children with mood disorder diagnoses. The large proportion of depressed children in the “child only” group is a reminder that while parents and children may often be poorer reporters of internalising than externalising symptoms overall, in some instances, the child is a better reporter of her own subjective and internal experiences. This finding is consistent with that of Martin et al. [10] on the CBCL, and further reflects the likelihood that many child depression symptoms are concealed from parents’ view.

While no age differences were apparent, there were gender differences with more males being in the “both” and “parent only” group, and more females in the “neither” and “child only” groups. This may indicate that parents consider boys to have more, or at least more obvious problems, and is consistent with higher rates of externalising disorders in boys. Similarly, the higher proportion of girls in the “neither” and “child only” groups may reflect the higher rates of internalising disorders among girls (i.e. those that are less detectable by parents), particularly given the “child only” group was over-represented by children with mood disorders.

No significant differences with age were found in this study. It is possible that the age range of this sample (11–17 years) was not wide enough to detect differences. It may be that age differences found in other studies included younger children than this group.

Finally, the finding that 46% of parent-child pairs agreed that the child’s presentation fell in the non-clinical range is worth comment. Given that all children had received at least one psychiatric diagnosis by a clinician, the high rate of false negatives reported by parents and children brings into question the utility of the SDQ as a screening instrument. Such findings are in contrast to studies examining the predictive utility of the SDQ, where it has been shown to detect up to two-thirds of child psychiatric diagnoses [6].

Several factors may have contributed to our findings. For our analyses, the SDQ categories of “normal” and “borderline” were combined to form one “non-clinical” group, to compare with the “abnormal” category, or “clinical” group. It is likely, therefore, that a proportion of the false negatives in our study were partial rather than complete (i.e. fell in the borderline range). Likewise, impact scores were not included in the analysis due to significant missing data (items pertaining to impact fall at the end of the questionnaire, which many parents and children had left blank). If impact scores had been included, this may have also increased the number of abnormal scores reported by parents and children. Of note, scores used to define the three categories, normal, borderline and abnormal, were in accordance with the normal, borderline and abnormal bandings provided in the SDQ scoring instructions. Australian norms were not used. It is likely, however, that this would have resulted in a one-point higher threshold for abnormality on four of the parent sub-scales and two of the child subscales, meaning that the rate of abnormal scores reported by parents and children would likely be even lower.

Other factors contributing to the discrepancy between clinician and parent-child ratings are that the screening efficacy of the SDQ has been shown to differ depending on the diagnosis, with poorer sensitivity to some anxiety disorders (specific phobias, panic disorder, separation anxiety) and eating disorders [6]. This may be partially reflected in our study, where almost a third of the false negative group comprised anxiety disorders.

Limitations

While all children met criteria for at least one psychiatric diagnosis, 51% presented with co-morbid conditions. This study did not control for the number or type of co-morbidities, which may have affected the severity of presentation (and subsequent parent-child reports). Assessing differences in parent-child agreement between children with and without a co-morbid condition is an area for future research.

There were also differences in group sizes, with some diagnostic groups having only a few cases which limited our ability to examine parent-child agreement rates fully. A larger sample would improve reliability.

In addition, the cognitive functioning of the child may impact the parent/child agreement on symptom reporting. Psychometric testing is not routinely administered in this public hospital setting, however investigation of this factor would be another area for future research.

Finally, the sample was obtained from a large metropolitan public hospital with psychiatric diagnoses assigned following clinical assessments by different clinicians. Reliability of diagnoses between clinicians was difficult to measure, and future research would benefit from the use of structured or semi-structured clinical interviews and inter-rater reliability measures, both of which were not practical in this setting.

Clinical implications

These findings raise issues about how such differences should be dealt with in regards to the overall understanding of the presenting problem and subsequent treatment recommendations. It has been suggested that the extent to which parents and children agree on the problems for which they want help may affect their ability to work together in pursuit of therapy goals [16]. Given this possibility, a clinician faced with discrepant information from parent and child may seek to incorporate this in the process of formulation and treatment, including addressing the parent-child discrepancies as part of the treatment [4]. This is particularly relevant given the finding in this study that 4% of children recognised their own emotional-behavioural difficulties to be in the clinical range, whereas their parents did not. If only the parents reports are considered, or even heavily weighted, then this group of children may risk going untreated. These findings provide additional support to the importance of multi-informant assessment of child and adolescent mental health problems.

Acknowledgments

The authors would like to thank Professor Jennifer Peat for her statistical advice and comments on the manuscript and Ms Malini Sivabalan for her assistance with data mining and retrieval.

Copyright information

© Steinkopff Verlag 2008