Background

Many children have psychological problems, and 10–20% of children and adolescents worldwide have mental health problems [1, 2]. Early detection and treatment of mental health problems is crucial, because if these troubles are undetected and untreated, they can increase in severity with age, and lead to chronic, complex, disabling, and expensive complications in adulthood [3,4,5]. However, the assessment of mental health problems in children and adolescents can be difficult, because diverse methodological approaches are used. Children have different exposures to risk factors and protective factors, and the cultural context in which mental health problems occur may alter estimates of their prevalence, which range from 1.81% to 39.4% according to previous studies [1, 2, 6]. Current screening methods rely on the expression of certain symptoms or impairments in everyday functioning to identify at-risk individuals who need further evaluation and potential treatment [7].

Although many instruments are available to screen children with mental health problems, the Strengths & Difficulties Questionnaire (SDQ) has become one of the most-used instruments because it is relatively short, user-friendly, has good validity in different cultures, and has more positive wording than other common questionnaires [8]. The SDQ uses a multi-informant approach, and is suitable for studies of populations in the general community, in which most children are healthy. The use of SDQ results from multiple informants is valuable, due to the situational nature of psychosocial problems [9,10,11]. In the absence of a gold standard measure for assessing mental health problems in children and youths, a multi-informant multimodal approach is considered best [12,13,14,15,16].

Previous studies reported significant differences in the prevalence of mental health problems when children and adolescents were rated by different informants or different combinations of informants [17]. Good mental health is more than simply the lack of symptoms or low levels of symptoms, but is also the ability to function adaptively within important life domains [18]. Psychiatric symptoms lead to distress and functional impairment, and assessment of the impact of symptoms on the lives of children is central to clinical practice, and a key indicator of the need for mental health care [19]. An impact supplement of the SDQ enables informants to report on possible burden and distress [20]. When distress or impairment criteria are used for diagnosis, the prevalence of mental health problems is significantly lower [17, 21]. The DSM-IV and ICD-10 coding implicitly assume that impact is sufficiently separable from symptoms, and can be considered individually. This suggests that distress and impairment resulting from symptoms should be adequately measurable on their own, and should add predictive value to a diagnosis based on symptoms alone [21].

Although use of multiple informants and impact score are crucial for making valid assessments of the prevalence of psychological problems in children and adolescents, most studies have used the SDQ to screen for mental health problems using only one informant, and have not considered the impact score. There is little research on the influence of considering the impact score on the estimated prevalence of mental health problems in children [17]. For example, 9 previous studies used the SDQ to evaluate the prevalence of mental health problems in Chinese schoolchildren using 1 informant [22,23,24,25,26,27,28,29,30]; eight of these studies used a parental version of the SDQ, and the rates ranged from 8.2 to 19.3%, and 1 study used a student version of the SDQ, and the rate was 10.7%. However, none of these 9 studies considered the impact score. Another previous study used 3 informants to assess the prevalence of emotional problems in Chinese schoolchildren [31], and reported that the prevalence rated by parents (8.2%), students (8.3%), and teachers (8.5%) were similar. However, the prevalence based on different combinations of 3 informants has not yet been determined. Therefore, the true prevalence of mental health problems in Chinese schoolchildren, based on different individual informants, different combinations of informants, and impact scores, remains to be established.

The purpose of the present study is to measure the influence of different combinations of multiple informants and the impact score on the estimated prevalence of mental health problems in Chinese schoolchildren.

Methods

Sample

The specifics of the sampling procedure were previously described [32]. This study was a cross-sectional survey of Chinese schoolchildren who were 6–17 years-old, and attended primary, junior, or senior schools. These children lived in 3 cities and 3 rural counties of Liaoning Province, in northeastern China. A 2-stage sampling procedure was conducted: schools were randomly selected, and a random sample of students was selected from each school. After obtaining written consent from parents, an informant-rated version of the SDQ was completed by the parents (P), teachers (T), and students (S; 11–17 years-old). The study protocol was reviewed and approved by the Institutional Review Board of the Liaoning Provincial Center for Disease Control and Prevention.

There were 9806 students eligible for the study, 9298 students returned the forms, and qualified SDQs were available for 8488 students. This included 8055 P-SDQs from parents (94.90%), 8418T-SDQs from teachers (99.18%), and 5446 S-SDQs from 5451 students aged 11–17 (99.91%). For this study, students were only included if SDQs from all 3 sources were available for students aged 11–17. By these criteria, 4986 students (91.5%) had complete data, and 465 students (8.5%) had incomplete data. The mean age of the included students was 13.9 years (SD: 1.9 years), and boys made up 48.7% of the sample.

Instruments

The SDQ is an instrument used to screen for mental health problems that asks about the occurrence of 25 behaviors in the past 6 months (0 = not true, 1 = somewhat true, 2 = certainly true) in the following dimensions: emotional problems, conduct problems, hyperactivity/inattention, peer relationship problems, and prosocial behavior. The items in the first 4 subscales generate a “total difficulties score” (range 0–40). The SDQ impact score is generated by the sum of 5 items: 1 item about distress and 4 items about social impairment (in family life, friendships, learning activities, and leisure activities) [20]. The Chinese version of the SDQ has been validated [33].

Statistical analysis

Descriptive statistics (means and standard deviations) for the SDQ subscales, total difficulties score, and impact score, are presented. A paired t-test was used to determine the significance of differences in SDQ total score and sub-scores from the parent, teacher, and student. The corresponding values for the schoolchildren aged 11–17 from Shanghai [33] and UK [34] were also presented and compared. Because Chinese norms for these scales are not yet available, we used the SDQ cut-off points of ‘abnormal’ from a previous study in the United Kingdom to define students with mental health problems [34]. To investigate the role of symptoms versus impact in assessment of mental health problems, we combined information from two parts of the multiple informant-reported SDQ (symptoms and impact) by identification of children with abnormal symptom scores and impairment (impact score ≥ 1 or impact score ≥ 2).

To determine differences in the prevalence of mental health problems assessed by different individual informants and different combinations of informants, we calculated the rates by 1 informant (S, P, or T), 2 informants (S + P, S + T, or P + T) and all 3 informants (S + P + T). When combining 2 or 3 informants, we defined the student as ‘positive’ if any one classified the child as ‘abnormal’, according to the corresponding cut-off values [34]. We only analyzed the prevalence of overall mental health problems and not specific problems, in this paper.

Results

We initially recruited 5451 schoolchildren who were 11–17 years-old (Table 1). A total of 49.1% were male, 52.7% were under 15 years-old, 66.1% were from single-child families, and 7.3% had divorced parents. The percentages of completed SDQs were 93.5% for parents, 98.0% for teachers, and 100.0% for students. A total of 4986 students (91.5%) had completed SDQs from all 3 three informants, and we further analyzed these students below.

Table 1 General information of 5451 schoolchildren aged 11–17
Full size table

Table 2 shows that the 3 individual informants had significantly different mean scores in all SDQ domains except for hyperactivity in the total SDQ score, and in the impact score. The students’ scores were the highest in total problems, impact, emotional symptoms, and prosocial behavior, followed by parents and then teachers. Teacher scores were the highest for peer problems, followed by parents and then students. Parent scores were the lowest for conduct problems.

Table 2 Paired t-test of the means of SDQ scores for 4986 schoolchildren with complete information rated by three informants
Full size table

The general pattern of our SDQ scores for Liaoning schoolchildren is similar to that previously reported for schoolchildren aged 11–17 years-old from Shanghai [33] and the UK [34] (Table 2). However, the student impact score in our study (0.52) was much greater than the parent impact score (0.26) and teacher impact score (0.25); in contrast, the student impact score in the UK study (0.2) was much less than the parent impact score (0.4) and teacher impact score (0.4). Impact scores were not available for the study in Shanghai.

Table 3 compares the prevalence of overall mental health problems defined by different criteria: (a) abnormal symptoms using different combinations of 3 informants; (b) abnormal symptoms using different combinations of 3 informants and an impact score of 1 or more; and (c) abnormal symptoms using different combinations of 3 informants and an impact score of 2 or more. In all cases, we used the UK cut off values for abnormal symptoms and impairment [34]. The rate of mental health problems increased significantly with increasing number of informants, but these numbers were much smaller when we also considered impact score. More specifically, when we considered 1 informant, the prevalence of mental health problems was greater when rated by the students themselves (5.9% to 14.2%) than by teachers (3.3% to 14.0%) or parents (2.5% to 8.2%). When we considered 2 informants, the prevalence was greater when rated by students + teachers (8.7% to 24.7%), than students + parents (7.5% to 18.8%) or parents + teachers (5.5% to 19.8%). The prevalence was greatest when we considered all 3 informants together (10.1% to 28.3%). When we included impact scores in the assessment of all 3 informants, the prevalence of overall mental health problems was 10.1% (impact score ≥ 2) and 14.5% (impact score ≥ 1), much lower than when we only considered symptom scores alone (28.3%).

Table 3 Comparison of mental health problems by different combinations of abnormal symptoms and impact scores with UK cut off value for SDQ and impact scores [34]
Full size table

Discussion

Consistent with previous findings in Shanghai [33] and the UK [34], we found that the 3 individual informants had significant differences in SDQ scores in most domains. The low to moderate correlations among the 3 informants for total score (0.219 to 0.443) and impact score (0.101 to 0.296) confirms previous findings that the 3 informants are inconsistent in their assessment of mental health problems in children and adolescents [8, 35, 36]. The students reported significantly higher average scores than adults and teachers in most SDQ domains, except for peer problems and hyperactivity problems. This is consistent with the interpretation that schoolchildren are more cognitively or psychologically aware of subjective symptoms and internal burdens, and that adults are more aware of observable or objective phenomenon [37]. In general, when considering all 3 informants, we found similar mean SDQ subscale scores in Chinese and British children and adolescents (Table 2). However, comparisons of the impact scores of Chinese and British parents (0.26 vs. 0.40) and Chinese and British teachers (0.25 vs. 0.40) indicated significantly lower scores for the Chinese [34]. In contrast, Chinese children assigned significantly higher impact scores than British children (0.52 vs. 0.20). This may suggest that Chinese adults tend to undervalue the impact of such problems, or that the impairment is less obvious to observers in China, possibly due to a difference in the knowledge or attitude of adults to mental health problems of schoolchildren in these countries.

In general, we observed a similar prevalence of overall mental health problems (8.2%-14.2%) based on a previously established SDQ cut-off score [34] from individual informants without considering impact. This agrees with previous studies of Chinese schoolchildren using the SDQ scores from 1 informant (8.2–19.3%) [22,23,24,25,26,27,28,29,30]. As expected, we observed a significantly greater prevalence of mental problems as the number of informants increased, and a decreased prevalence when considering the impact score. The prevalence of mental health problems was greater when rated by students (5.9% to 14.2%) than parents (2.5–8.2%) or teachers (3.3–14.0%). In addition, consideration of 2 informants indicated the prevalence was greatest for students + teachers, followed by students + parents, and then teachers + parents. This confirms the presence of a big difference in perceiving and rating children’s behavior and mental health problems between students and observers (parents and teachers) [13]. These results also suggest that schoolchildren themselves have an important role in assessing the prevalence of overall mental health problems [21, 38].

However, a previous study reported that self-reported information from students provided poorer screening than parents or teachers, the combinations of parent + teacher and teacher + student provided better screening than parent + student, and parent + teacher + student provided the most sensitive screening [10]. Although there is no ‘gold standard’ about which informants to use to assess mental health problems in schoolchildren, our results support the view that multiple informants provide better information than any individual informant. In addition, the large variations between the students and observers, as well as their combinations, suggests that the type and number informants must be considered when comparing the prevalence of mental health problems among studies.

Similar to the results of the BELLA study [17], we found that the prevalence of overall mental health problems rated by individual informants and different combinations of informants declined substantially when additional impairment criteria are considered. For example, the prevalence assessed by all 3 informants was 28.3% when impact score was not considered, but was 14.5% for an impact score of 1 or more and 10.1% for an impact score of 2 or more. The prevalence of 28.3% represents children who met the symptom criteria alone, but included those who were not necessarily impaired by the symptoms; this may provide a greater sensitivity, but may also overestimate the mental burden, and lead to false positive diagnoses and unnecessary additional screening and treatment [10]. We believe the prevalence based on an abnormal SDQ score plus an impact score of 1 or more (14.5%) could reflect ‘real mental health problems’, because this combination considers symptoms and perceived functional damage. Interestingly, the prevalence determined by three informants that considers SDQ score and an impact score of 2 or more (10.1%) was very close to the overall prevalence of DSM-IV disorders (9.49%) assessed using the Developmental and Well-Being Assessment (DAWBA) in the same population [32]. This combination reflects a higher diagnostic quality (discrimination between respondents with and without a psychiatric diagnosis) when screening for a psychiatric disorder than the symptom scales of the SDQ [10, 17].

The present results should be viewed in the context of several limitations. First, this was a study of schoolchildren for whom full information was available from all 3 informants. Children who left school or could not get information from their parents or teachers were excluded. This may have led to an underestimate of the prevalence of mental health problems, because such children have greater risk of mental health problems in developed and developing countries. Second, we used the criteria and cutoff values from the UK, and there were big differences in the impact scores between Chinese and British children and adults. However, the purpose of the present study was to assess the influence of using multiple informants and impact scores on the prevalence of mental health problems in Chinese schoolchildren. Although it would be preferable if regional norms were available, we believe the internal comparisons among different subgroups, defined by different combinations of informants and impact scores, are comparable and valid. Third, because each teacher must rate all the students in a class (about 40), the teacher may not have sufficient time to carefully assess each child. It is unclear whether this is related to the lower average SDQ scores from teachers than from students and parents.

Conclusions

The results of the present paper confirm the presence of substantial differences in the prevalence of mental health problems in Chinese schoolchildren when rated by different informants, different combinations of informants, and when impairment scores are considered. The assessment of mental health problems using the SDQ depends on the purpose of the survey or screening, the types, numbers, and combinations of informants, the cut-off values, and the use of impact score [10]. It is preferable to use multiple informants and consider impact score to better assess the actual burden of mental health problems in children and adolescents.