To measure the effects of using different combinations of multiple informants and the impact score on the estimated prevalence of mental health problems in Chinese schoolchildren.
Complete information on the Strengths and Difficulties Questionnaire (SDQ) were obtained from students (S), parents (P), and teachers (T) for 4986 schoolchildren (11–17 years-old). We used 3 criteria to determine the prevalence of mental health problems: SDQ cut-off value (previously established in the United Kingdom), SDQ cut-off value plus an impact score of 1 or more, or plus an impact score of 2 or more. A student was defined as having a mental health problem if any informant (S, P, or T) classified the child as ‘abnormal’. We compared the prevalence of mental health problems determined from 1 informant, 2 informants, and 3 informants.
The prevalence of overall mental health problems increased with rising number of informants, but decreased with increasing impact score. When the impact score was not considered, the prevalence was 8.2% to 14.2% when rated by 1 informant, 18.8% to 24.7% when rated by 2 informants, and 28.3% when rated by all 3 informants. Failure to measure the impact score led to a two to threefold greater estimate of the prevalence of mental health problems.
The types, number, and combinations of multiple informants and use of the impact score must be considered when comparing the results of different studies. It is preferable to use multiple informants and have the impact score taken the impact into account to reflect the real burden of mental health burden in children and adolescent.
Many children have psychological problems, and 10–20% of children and adolescents worldwide have mental health problems [1, 2]. Early detection and treatment of mental health problems is crucial, because if these troubles are undetected and untreated, they can increase in severity with age, and lead to chronic, complex, disabling, and expensive complications in adulthood [3,4,5]. However, the assessment of mental health problems in children and adolescents can be difficult, because diverse methodological approaches are used. Children have different exposures to risk factors and protective factors, and the cultural context in which mental health problems occur may alter estimates of their prevalence, which range from 1.81% to 39.4% according to previous studies [1, 2, 6]. Current screening methods rely on the expression of certain symptoms or impairments in everyday functioning to identify at-risk individuals who need further evaluation and potential treatment .
Although many instruments are available to screen children with mental health problems, the Strengths & Difficulties Questionnaire (SDQ) has become one of the most-used instruments because it is relatively short, user-friendly, has good validity in different cultures, and has more positive wording than other common questionnaires . The SDQ uses a multi-informant approach, and is suitable for studies of populations in the general community, in which most children are healthy. The use of SDQ results from multiple informants is valuable, due to the situational nature of psychosocial problems [9,10,11]. In the absence of a gold standard measure for assessing mental health problems in children and youths, a multi-informant multimodal approach is considered best [12,13,14,15,16].
Previous studies reported significant differences in the prevalence of mental health problems when children and adolescents were rated by different informants or different combinations of informants . Good mental health is more than simply the lack of symptoms or low levels of symptoms, but is also the ability to function adaptively within important life domains . Psychiatric symptoms lead to distress and functional impairment, and assessment of the impact of symptoms on the lives of children is central to clinical practice, and a key indicator of the need for mental health care . An impact supplement of the SDQ enables informants to report on possible burden and distress . When distress or impairment criteria are used for diagnosis, the prevalence of mental health problems is significantly lower [17, 21]. The DSM-IV and ICD-10 coding implicitly assume that impact is sufficiently separable from symptoms, and can be considered individually. This suggests that distress and impairment resulting from symptoms should be adequately measurable on their own, and should add predictive value to a diagnosis based on symptoms alone .
Although use of multiple informants and impact score are crucial for making valid assessments of the prevalence of psychological problems in children and adolescents, most studies have used the SDQ to screen for mental health problems using only one informant, and have not considered the impact score. There is little research on the influence of considering the impact score on the estimated prevalence of mental health problems in children . For example, 9 previous studies used the SDQ to evaluate the prevalence of mental health problems in Chinese schoolchildren using 1 informant [22,23,24,25,26,27,28,29,30]; eight of these studies used a parental version of the SDQ, and the rates ranged from 8.2 to 19.3%, and 1 study used a student version of the SDQ, and the rate was 10.7%. However, none of these 9 studies considered the impact score. Another previous study used 3 informants to assess the prevalence of emotional problems in Chinese schoolchildren , and reported that the prevalence rated by parents (8.2%), students (8.3%), and teachers (8.5%) were similar. However, the prevalence based on different combinations of 3 informants has not yet been determined. Therefore, the true prevalence of mental health problems in Chinese schoolchildren, based on different individual informants, different combinations of informants, and impact scores, remains to be established.
The purpose of the present study is to measure the influence of different combinations of multiple informants and the impact score on the estimated prevalence of mental health problems in Chinese schoolchildren.
The specifics of the sampling procedure were previously described . This study was a cross-sectional survey of Chinese schoolchildren who were 6–17 years-old, and attended primary, junior, or senior schools. These children lived in 3 cities and 3 rural counties of Liaoning Province, in northeastern China. A 2-stage sampling procedure was conducted: schools were randomly selected, and a random sample of students was selected from each school. After obtaining written consent from parents, an informant-rated version of the SDQ was completed by the parents (P), teachers (T), and students (S; 11–17 years-old). The study protocol was reviewed and approved by the Institutional Review Board of the Liaoning Provincial Center for Disease Control and Prevention.
There were 9806 students eligible for the study, 9298 students returned the forms, and qualified SDQs were available for 8488 students. This included 8055 P-SDQs from parents (94.90%), 8418T-SDQs from teachers (99.18%), and 5446 S-SDQs from 5451 students aged 11–17 (99.91%). For this study, students were only included if SDQs from all 3 sources were available for students aged 11–17. By these criteria, 4986 students (91.5%) had complete data, and 465 students (8.5%) had incomplete data. The mean age of the included students was 13.9 years (SD: 1.9 years), and boys made up 48.7% of the sample.
The SDQ is an instrument used to screen for mental health problems that asks about the occurrence of 25 behaviors in the past 6 months (0 = not true, 1 = somewhat true, 2 = certainly true) in the following dimensions: emotional problems, conduct problems, hyperactivity/inattention, peer relationship problems, and prosocial behavior. The items in the first 4 subscales generate a “total difficulties score” (range 0–40). The SDQ impact score is generated by the sum of 5 items: 1 item about distress and 4 items about social impairment (in family life, friendships, learning activities, and leisure activities) . The Chinese version of the SDQ has been validated .
Descriptive statistics (means and standard deviations) for the SDQ subscales, total difficulties score, and impact score, are presented. A paired t-test was used to determine the significance of differences in SDQ total score and sub-scores from the parent, teacher, and student. The corresponding values for the schoolchildren aged 11–17 from Shanghai  and UK  were also presented and compared. Because Chinese norms for these scales are not yet available, we used the SDQ cut-off points of ‘abnormal’ from a previous study in the United Kingdom to define students with mental health problems . To investigate the role of symptoms versus impact in assessment of mental health problems, we combined information from two parts of the multiple informant-reported SDQ (symptoms and impact) by identification of children with abnormal symptom scores and impairment (impact score ≥ 1 or impact score ≥ 2).
To determine differences in the prevalence of mental health problems assessed by different individual informants and different combinations of informants, we calculated the rates by 1 informant (S, P, or T), 2 informants (S + P, S + T, or P + T) and all 3 informants (S + P + T). When combining 2 or 3 informants, we defined the student as ‘positive’ if any one classified the child as ‘abnormal’, according to the corresponding cut-off values . We only analyzed the prevalence of overall mental health problems and not specific problems, in this paper.
We initially recruited 5451 schoolchildren who were 11–17 years-old (Table 1). A total of 49.1% were male, 52.7% were under 15 years-old, 66.1% were from single-child families, and 7.3% had divorced parents. The percentages of completed SDQs were 93.5% for parents, 98.0% for teachers, and 100.0% for students. A total of 4986 students (91.5%) had completed SDQs from all 3 three informants, and we further analyzed these students below.
Table 2 shows that the 3 individual informants had significantly different mean scores in all SDQ domains except for hyperactivity in the total SDQ score, and in the impact score. The students’ scores were the highest in total problems, impact, emotional symptoms, and prosocial behavior, followed by parents and then teachers. Teacher scores were the highest for peer problems, followed by parents and then students. Parent scores were the lowest for conduct problems.
The general pattern of our SDQ scores for Liaoning schoolchildren is similar to that previously reported for schoolchildren aged 11–17 years-old from Shanghai  and the UK  (Table 2). However, the student impact score in our study (0.52) was much greater than the parent impact score (0.26) and teacher impact score (0.25); in contrast, the student impact score in the UK study (0.2) was much less than the parent impact score (0.4) and teacher impact score (0.4). Impact scores were not available for the study in Shanghai.
Table 3 compares the prevalence of overall mental health problems defined by different criteria: (a) abnormal symptoms using different combinations of 3 informants; (b) abnormal symptoms using different combinations of 3 informants and an impact score of 1 or more; and (c) abnormal symptoms using different combinations of 3 informants and an impact score of 2 or more. In all cases, we used the UK cut off values for abnormal symptoms and impairment . The rate of mental health problems increased significantly with increasing number of informants, but these numbers were much smaller when we also considered impact score. More specifically, when we considered 1 informant, the prevalence of mental health problems was greater when rated by the students themselves (5.9% to 14.2%) than by teachers (3.3% to 14.0%) or parents (2.5% to 8.2%). When we considered 2 informants, the prevalence was greater when rated by students + teachers (8.7% to 24.7%), than students + parents (7.5% to 18.8%) or parents + teachers (5.5% to 19.8%). The prevalence was greatest when we considered all 3 informants together (10.1% to 28.3%). When we included impact scores in the assessment of all 3 informants, the prevalence of overall mental health problems was 10.1% (impact score ≥ 2) and 14.5% (impact score ≥ 1), much lower than when we only considered symptom scores alone (28.3%).
Consistent with previous findings in Shanghai  and the UK , we found that the 3 individual informants had significant differences in SDQ scores in most domains. The low to moderate correlations among the 3 informants for total score (0.219 to 0.443) and impact score (0.101 to 0.296) confirms previous findings that the 3 informants are inconsistent in their assessment of mental health problems in children and adolescents [8, 35, 36]. The students reported significantly higher average scores than adults and teachers in most SDQ domains, except for peer problems and hyperactivity problems. This is consistent with the interpretation that schoolchildren are more cognitively or psychologically aware of subjective symptoms and internal burdens, and that adults are more aware of observable or objective phenomenon . In general, when considering all 3 informants, we found similar mean SDQ subscale scores in Chinese and British children and adolescents (Table 2). However, comparisons of the impact scores of Chinese and British parents (0.26 vs. 0.40) and Chinese and British teachers (0.25 vs. 0.40) indicated significantly lower scores for the Chinese . In contrast, Chinese children assigned significantly higher impact scores than British children (0.52 vs. 0.20). This may suggest that Chinese adults tend to undervalue the impact of such problems, or that the impairment is less obvious to observers in China, possibly due to a difference in the knowledge or attitude of adults to mental health problems of schoolchildren in these countries.
In general, we observed a similar prevalence of overall mental health problems (8.2%-14.2%) based on a previously established SDQ cut-off score  from individual informants without considering impact. This agrees with previous studies of Chinese schoolchildren using the SDQ scores from 1 informant (8.2–19.3%) [22,23,24,25,26,27,28,29,30]. As expected, we observed a significantly greater prevalence of mental problems as the number of informants increased, and a decreased prevalence when considering the impact score. The prevalence of mental health problems was greater when rated by students (5.9% to 14.2%) than parents (2.5–8.2%) or teachers (3.3–14.0%). In addition, consideration of 2 informants indicated the prevalence was greatest for students + teachers, followed by students + parents, and then teachers + parents. This confirms the presence of a big difference in perceiving and rating children’s behavior and mental health problems between students and observers (parents and teachers) . These results also suggest that schoolchildren themselves have an important role in assessing the prevalence of overall mental health problems [21, 38].
However, a previous study reported that self-reported information from students provided poorer screening than parents or teachers, the combinations of parent + teacher and teacher + student provided better screening than parent + student, and parent + teacher + student provided the most sensitive screening . Although there is no ‘gold standard’ about which informants to use to assess mental health problems in schoolchildren, our results support the view that multiple informants provide better information than any individual informant. In addition, the large variations between the students and observers, as well as their combinations, suggests that the type and number informants must be considered when comparing the prevalence of mental health problems among studies.
Similar to the results of the BELLA study , we found that the prevalence of overall mental health problems rated by individual informants and different combinations of informants declined substantially when additional impairment criteria are considered. For example, the prevalence assessed by all 3 informants was 28.3% when impact score was not considered, but was 14.5% for an impact score of 1 or more and 10.1% for an impact score of 2 or more. The prevalence of 28.3% represents children who met the symptom criteria alone, but included those who were not necessarily impaired by the symptoms; this may provide a greater sensitivity, but may also overestimate the mental burden, and lead to false positive diagnoses and unnecessary additional screening and treatment . We believe the prevalence based on an abnormal SDQ score plus an impact score of 1 or more (14.5%) could reflect ‘real mental health problems’, because this combination considers symptoms and perceived functional damage. Interestingly, the prevalence determined by three informants that considers SDQ score and an impact score of 2 or more (10.1%) was very close to the overall prevalence of DSM-IV disorders (9.49%) assessed using the Developmental and Well-Being Assessment (DAWBA) in the same population . This combination reflects a higher diagnostic quality (discrimination between respondents with and without a psychiatric diagnosis) when screening for a psychiatric disorder than the symptom scales of the SDQ [10, 17].
The present results should be viewed in the context of several limitations. First, this was a study of schoolchildren for whom full information was available from all 3 informants. Children who left school or could not get information from their parents or teachers were excluded. This may have led to an underestimate of the prevalence of mental health problems, because such children have greater risk of mental health problems in developed and developing countries. Second, we used the criteria and cutoff values from the UK, and there were big differences in the impact scores between Chinese and British children and adults. However, the purpose of the present study was to assess the influence of using multiple informants and impact scores on the prevalence of mental health problems in Chinese schoolchildren. Although it would be preferable if regional norms were available, we believe the internal comparisons among different subgroups, defined by different combinations of informants and impact scores, are comparable and valid. Third, because each teacher must rate all the students in a class (about 40), the teacher may not have sufficient time to carefully assess each child. It is unclear whether this is related to the lower average SDQ scores from teachers than from students and parents.
The results of the present paper confirm the presence of substantial differences in the prevalence of mental health problems in Chinese schoolchildren when rated by different informants, different combinations of informants, and when impairment scores are considered. The assessment of mental health problems using the SDQ depends on the purpose of the survey or screening, the types, numbers, and combinations of informants, the cut-off values, and the use of impact score . It is preferable to use multiple informants and consider impact score to better assess the actual burden of mental health problems in children and adolescents.
Availability of data and materials
The datasets generated and analysed during the current study are not publicly available because of our agreement with the parents of schoolchildren, but are available from the corresponding author on reasonable request.
Strengths & Difficulties Questionnaires
Diagnostic and Statistical Manual of Mental Disorders
International Classification of Diseases 10th Revision
Kieling C, Baker-Henningham H, Belfer M, Conti G, Ertem I, Omigbodun O, et al. Child and adolescent mental health worldwide: evidence for action. Lancet. 2011;378(9801):1515–25. https://doi.org/10.1016/S0140-6736(11)60827-1.
Kato N, Yanagawa T, Fujiwara T, Morawska A. Prevalence of Children's Mental Health problems and the effectiveness of population-level family interventions. J Epidemiol. 2015;25(8):507–16.
Bricker D, Davis MS, Squires J. Mental health screening in young children. Infants Young Child. 2004;17:129–44.
Patel V, Flisher AJ, Hetrick S, McGorry P. Mental health of young people: a global public health challenge. Lancet. 2007;369:1302–13.
Parks RJ, Kalberg RJ, Carter EW. Systematic screening at the middle school level: score reliability and validity of the student risk screening scale. J Emot Behav Disord. 2007;15:209–22.
Canino G, Alegría M. Psychiatric diagnosis-is it universal or relative to culture? J Child Psychol Psyc. 2008;49:237–50.
Levitt JM, Saka N, Romanelli LH, Hoagwood K. Early identification of mental health problems in schools: the status of instrumentation. J School Psychol. 2007;45:163–91.
Stone LL, Otten R, Engels RCME, Vermulst AA, Janssens JMAM. Psychometric properties of the parent and teacher versions of the strengths and difficulties questionnaire for 4- to 12-year-olds: a review. Clin Child Fam Psychol Rev. 2010;13(3):254–74.
Achenbach TM, McConaughy SH, Howell CT. Child/adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. Psychol Bull. 1987;101:213–32.
Goodman R, Ford T, Simmons H, Gatward R, Meltzer H. Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Brit J Psychiat. 2000;177:534–9.
Goodman R, Renfrew D, Mullick M. Predicting type of psychiatric disorder from Strengths and Difficulties Questionnaire (SDQ) scores in child mental health clinics in London and Dhaka. Eur Child Adoles Psy. 2000;9:129–34.
De Los RA, Kazdin AE. Measuring informant discrepancies in clinical child research. Psychol Assess. 2004;16:330–4.
De Los RA, Kazdin AE. Informant discrepancies in the assessment of childhood psychopathology: a critical review, theoretical framework, and recommendations for further study. Psychol Bull. 2005;131:483–509.
Verhulst FC, Dekker MC, Van der Ende J. Parent, teacher and self-reports as predictors of signs of disturbance in adolescents: whose information carries the most weight? Acta Psychiat Scand. 1997;96:75–81.
Achenbach TM. Advancing assessment of children and adolescents: commentary on evidence-based assessment of child and adolescent disorders. J Clin Child Adolesc Psychol. 2005;34(3):541–7.
Mash EJ, Hunsley J. Evidence-based assessment of child and adolescent disorders: issues and challenges. J Clin Child Adolesc. 2005;34(3):362–79.
Ravens-Sieberer U, Wille N, Erhart M, Bettge S, Wittchen HU, Rothenberger A, et al. Prevalence of mental health problems among children and adolescents in Germany: results of the BELLA study within the National Health Interview and Examination Survey. Eur Child Adoles Psy. 2008;17(Suppl 1):22–33. https://doi.org/10.1007/s00787-008-1003-2.P.
Rapee RM, Bogels SM, Van der Sluis CM, Michelle GC, Ollendick T. Annual research review: conceptualizing functional impairment in children and adolescents. J Child Psychol Psyc. 2012;53(5):454–68. https://doi.org/10.1111/j.1469-7610.2011.02479.x.
Dang HM, Weiss B, Trung LT. Functional impairment and mental health functioning among Vietnamese children. Soc Psych Psych Epid. 2016;51(1):39–47. https://doi.org/10.1007/s00127-015-1114-2.
Goodman R. The extended version of the Strengths and Difficulties Questionnaire as a guide to child psychiatric caseness and consequent burden. J Child Psychol Psyc. 1999;40(5):791–9.
Stringaris A, Goodman R. The value of measuring impact alongside symptoms in children and adolescents: a longitudinal assessment in a community sample. J Abnorm Child Psychol. 2013;41(7):1109–20.
Wang JN, Liu L, Wang L. Prevalence and associated factors of emotional and behavioural problems in Chinese school adolescents: a cross-sectional survey. Child Care Health Dev. 2014;40(3):319–26. https://doi.org/10.1111/cch.12101.
Sun L, Guo X, Zhang J, Liu HH, Xu SJ, Xu YY, et al. Gender specific associations between early puberty and behavioral and emotional characteristics in children. Zhonghua Liu Xing Bing Xue Za Zhi. 2016;37(1):35–9. https://doi.org/10.3760/cma.j.issn.0254-6450.2016.01.007.
Huang Y, Yan Q, Li Y, Liu J, Yao H, Yan Y. Strengths and Difficulties Questionnaire in 737 primary and middle school students aged 6–17 in Changsha. Zhong Nan Da Xue Xue Bao Yi Xue Ban. 2012;37(8):860–4. https://doi.org/10.3969/j.issn.1672-7347.2012.08.018.
Hu XM, Luan XY, Rou ZWG, Gai GQ. Investigations of psychological status in middle school students in Shihezi city. Lin Chuang Shen Xin Za Zhi. 2015;21(5):101–4. https://doi.org/10.3969/j.issn.1672-187X.2015.05.034-0101-04.
Chen Q, Ding K, Lu HC, Shi BB, Zhang WW. A cross-sectional study on psychological and behavioral problems of primary and secondary school students. Zheiiang Prev Med. 2013;25(10):30–3. https://d.wanfangdata.com.cn/periodical/zjyfyx201310007(in Chinese).
Wang XJ, Chen ZM. Investigation of 155 children aged 10–15 using SDQ parent version. Xian Dai Shi Yong Yi Xue. 2006;18(9):662–5. https://d.wanfangdata.com.cn/periodical/nbyx200609031(in Chinese).
Gao X, Liang J, Wang SF, Zai Y, Lu YB, Shi WH, et al. Analysis on pupils’ current mental health situation and their guardians’ attitude in eight provinces in China. Zhong Hua Ji Bing Kong Zhi Za Zhi. 2013;17(7):592–5. https://d.wanfangdata.com.cn/periodical/jbkzzz201307013(in Chinese).
WANG JN, Chang Y, Fu JL, Wang L. Relationship between mental health and personality characteristics among junior and senior school students in Shenyang. Zhong Guo Gong Gong Wei Sheng. 2010;26(11):1424–5. https://d.wanfangdata.com.cn/periodical/zgggws201011041(in Chinese).
Lu AJ, Wang YQ, Wang YF, Zhao ZM. Investigation about emotional and behavioral problems in students aged 7 to 16 years old in Songjiang Shanghai. Jing Shen Yi Xue Za Zhi. 2015;28(2):99–101. doi: https://doi.org/10.1007/BF00165590. https://d.wanfangdata.com.cn/periodical/sdjsyx201502009(in Chinese).
Yin L, Huang Y, Li Y, Shitu MJ, Xiang Y, Luo Y, et al. Epidemiological investigation on emotion difficulties among elementary and middle school students in Chengdu. 2012;12(3):238–40. https://doi.org/10.3969/j.issn.1009-6574.2012.03.007.
Xiaoli Y, Chao J, Wen P, Wenming X, Fang L, Ning L, et al. Prevalence of psychiatric disorders among children and adolescents in northeast China. PLoS ONE. 2014;9(10):e111223. https://doi.org/10.1371/journal.pone.0111223.
Du Y, Kou J, Coghill D. The validity, reliability and normative scores of the parent, teacher and self report versions of the Strengths and Difficulties Questionnaire in China. Child Adol Psych Men. 2008;2(1):8. https://doi.org/10.1186/1753-2000-2-8.
SDQ: Information for researchers and professionals about the Strengths & Difficulties Questionnaires 2008. https://www.sdqinfo.com. Accessed 04 Feb 2020.
Goodman R. The strengths and difficulties questionnaire: a research note. J Child Psychol Psyc. 1997;38:581–6.
Nguyen N, Whittlesey S, Scimeca K, Digiacomo D, Bui B, et al. Parent-child agreement in prepubertal depression: findings with a modified assessment method. J Am Acad Child Psy. 1994;33:1275–83.
Piacentini JC, Cohen P, Cohen J. Combining discrepant diagnostic information from multiple sources: are complex algorithms better than simple ones? J Abnorm Child Psychol. 1992;20:51–63.
Kuhn C, Aebi M, Jakobsen H, Banaschewski T, Poustka L, Grimmer Y, et al. Effective mental health screening in adolescents: should we collect data from youth, parents or both? Child Psychiat Hum D. 2017;48:385–92.
Thanks for the support of Science and Technology Department of Liaoning Province. The authors thank the schoolmasters, school teachers and parents of schoolchildren for their cooperation and support.
This study was supported by a grant from Science and Technology Department of Liaoning Province (No. 2005225012). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Ethics approval and consent to participate
The study was conducted in accordance with the Declaration of Helsinki on ethical principles for medical research involving human subjects. The ethics committee of Liaoning Provincial Center for Disease Control and Prevention (LNCDCP) approved the study. All subjects gave written informed consent after the study objectives were explained, and all subjects were free to withdraw at any time without giving any reason.
Consent for publication
The authors have declared that no competing interests exist.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Liu, L., Li, S., Pan, W. et al. Prevalence of mental health problems in Chinese schoolchildren: The influence of measuring impact score and combining information from multiple informants. Child Adolesc Psychiatry Ment Health 14, 44 (2020). https://doi.org/10.1186/s13034-020-00346-2
- Mental health problems
- Multiple informants
- Impact score
- Combined information