The current study estimated the diagnostic performance and optimal ‘caution’ cut-off scores of the MAYSI-2 in a sample of Swiss youths (i.e., French, German and Italian language regions) in welfare and juvenile justice institutions. As it has been reported [17, 26] that gender may influence the accuracy of self-report measures (e.g., girls tend to reveal their feelings on self-report scales more readily than boys), the other main purpose of the current study was to identify potential gender differences in the diagnostic performance and optimal ‘caution’ cut-off scores of the MAYSI-2 in a Swiss sample.
Diagnostic performance
Within all of the subsamples, except in the French/Italian-speaking subsample, the MAYSI-2 scales alcohol/drug use and suicide ideation revealed significant AUC values above .70 indicating these scales are adequately precise [28]. These results indicate that the MAYSI-2 is able to identify youths who may be a danger to themselves and are in need of direct attention (i.e., youths with alcohol/drug problems and/or youths with suicidal ideation).
The results of the ROC analyses on the total sample, the German-speaking and the boy’s subsamples demonstrate that nearly all homotypic mappings of MAYSI-2 scales onto (cluster of) psychiatric disorders revealed above chance level accuracy (i.e., significant AUC values greater than .50).
Surprisingly, within the French/Italian-speaking and girl’s subsample, no significant AUC value higher than .50 was found for the MAYSI-2 scales depressed-anxious and somatic complaints. However, this finding should be interpreted with caution as low power (due to the relatively small number of youths from the French/Italian-speaking subsample (n = 105) and girls (n = 149) may have influenced the results. In addition, the MAYSI-2 somatic complaints scale did not reveal a significant AUC value for any affective disorder within any of the subsamples (i.e., total sample, German-speaking subsample, French/Italian-speaking subsample, boys and girls). The deviant diagnostic performance of this scale may be explained by the relatively small number of youths in some of the subsamples (i.e., French/Italian-speaking subsample and girls), as above mentioned. In addition, although the MAYSI-2 scale somatic complaints asks about bodily aches and pains that tend to occur with anxiety or affective disorders, it is possible that the MAYSI-2 scale somatic complaints does not discriminate well between bodily aches and pains that tend to occur with anxiety or affective disorders and bodily aches and pains related to a physical illness in a sample of Swiss youth in welfare and juvenile justice institutions.
Optimal ‘caution’ cut-off scores
The optimal ‘caution’ cut-off scores derived from the ROC curve for predicting (cluster of) psychiatric disorders were, for several MAYSI-2 scales, comparable to the USA norm-based ‘caution’ cut-off scores (e.g., alcohol/drug use in all subsamples, angry-irritable in total sample, German-speaking and boys’ subsample). For some MAYSI-2 scales, however, higher optimal ‘caution’ cut-off scores were found. For example; compared to the USA norm-based ‘caution’ cut-off score, in the girls’ subsample a higher optimal ‘caution’ cut-off score was found for the MAYSI-2 scales angry-irritable and suicide ideation. The optimal ‘caution’ cut-off scores in the girls’ subsample for these MAYSI-2 scales were also higher than the optimal ‘caution’ cut-off scores for these scales in the boys’ subsample. This finding, which is consistent with earlier research [17, 26], raises questions about the response style of boys versus girls. It could be that Swiss girls have a higher threshold to report clinically relevant angry-irritable and suicidal behavior than American girls and boys. For example, the target population for which the MAYSI-2 was intended differs between the USA and Switzerland (e.g., due to a higher level of violence, harsh punishment, and more incarceration for youths [7, 25]). As a consequence, it may be that girls in the Swiss welfare and juvenile justice institutions have less severe mental health problems than girls in the USA juvenile justice system and therefore need to report more of their problems to reach the clinically relevant ‘caution’ cut-off score. In addition, it has been found that girls have higher rates of self-reported suicidal behavior than boys [29]; however, mortality from suicide is typically lower for girls than for boys [30]. In addition, as girls tend to reveal their feelings on self-report scales more readily than boys [17, 26], raising the ‘caution’ cut-off score for girls, compared to boys, on the abovementioned MAYSI-2 scales seems preferable in order to detect their clinically relevant potential emotional or behavioral problems adequately.
When determining the optimal ‘caution’ cut-off scores for the MAYSI-2 scale somatic complaints, it would be appropriate to select the optimal ‘caution’ cut-off scores based on the homotypic mappings that revealed above chance level accuracy (i.e., significant AUC values greater than 0.50). For example, in the total sample the optimal ‘caution’ cut-off score for somatic complaints with regard to any anxiety disorder would be appropriate. When determining the optimal ‘caution’ cut-off scores for the MAYSI-2 scale depressed-anxious, it would be appropriate to select the optimal ‘caution’ cut-off scores based on the lowest difference between the sensitivity and specificity value. For example, in the German-speaking subsample the optimal ‘caution’ cut-off score for depressed-anxious with regard to any affective disorder would be appropriate.
Furthermore, lowering the ‘caution’ cut-off score on the MAYSI-2 scale alcohol/drug use from 4 to 3 for the French/Italian subsample would increase the sensitivity rate (from .71 to .79). Consequently, more youths with the presence of any substance use disorder will be screened as such and less youths with the presence of any substance use disorder will be screened as not having any substance use disorder. While, on the other hand it would decrease the specificity rate (from .82 to .71). Meaning that less youths with the absence of any substance use disorder will be screened as such and more youths with the absence of any substance use disorder will be screened as having any substance use disorder. Thus, lowering the ‘caution’ cut-off score on the MAYSI-2 scale alcohol/drug use from 4 to 3 for the French/Italian subsample would imply that more youths will receive the correct special attention (i.e., provide additional psychological assessment, increased staff attention, close monitoring within the facility in order to prevent harm to the youth or others, or emergency mental health services) for their substance disorder, however more youths who are not in need of this special attention would also receive this attention.
In addition, raising the ‘caution’ cut-off score on the MAYSI-2 scale suicide ideation from 2 to 3 for the German subsample would decrease the sensitivity rate (from .80 to .67). Consequently, less youths with the presence of suicide ideation/suicide attempts will be screened as such and more youths with the presence of suicide ideation/suicide attempts will be screened as not having suicide ideation/suicide attempts. While, on the other hand it would increase the specificity rate (from .67 to .79). Meaning that more youths with the absence of suicide ideation/suicide attempts will be screened as such and less youths with the absence of suicide ideation/suicide attempts will be screened as having suicide ideation/suicide attempts. Thus, raising the ‘caution’ cut-off score on the MAYSI-2 scale suicide ideation from 2 to 3 for the German subsample would imply that less youths will receive the correct special attention (i.e., provide additional psychological assessment, increased staff attention, close monitoring within the facility in order to prevent harm to the youth or others, or emergency mental health services) for their suicide ideation/suicide attempts, however less youths who are not in need of this special attention would also receive this attention.
Limitations
There are a few limitations of our study to mention. First, in the current study, the MAYSI-2 scales thought disturbance (a reliable scale only for boys) and traumatic experiences (no current ‘caution’ cut-off score determined [17]; were not included. Second, in the current study we did not test the diagnostic performance and optimal ‘caution’ cut-off scores of the MAYSI-2 across diverse ethnic subgroups. Because welfare and juvenile justice institutions in Switzerland are dealing with youths from ethnic subgroups that differ from those typically seen in the USA (e.g., Turkish versus African-American youths), future studies are critical to test whether the MAYSI-2 can be used within these subgroups. Third, we related the MAYSI-2 scale angry-irritable to any disruptive disorder. However, it should be emphasized that many youths may show angry or irritable behavior without being disruptively disordered. For example, irritable behavior may also be a symptom of a generalized anxiety disorder [31]. Fourth, in the original Massachusetts Study on which the USA National Norms Study for the MAYSI-2 was based [17], scales of the CBCL, YSR [Child Behavior Checklist, Youth Self-Report; 13, 14] and the Millon Adolescent Clinical Inventory (MACI) [32] were used to determine the diagnostic performance and the ‘caution’ cut-off scores, whereas in the current study (cluster of) psychiatric disorders were used. Relating the MAYSI-2 to (cluster of) psychiatric disorders may have been a strict way to estimate the diagnostic performance and the ‘caution’ cut-off scores of the instrument. Originally, the MAYSI-2 was not developed to diagnose specific psychiatric disorders; however its aim to screen for youths who may have severe psychiatric complaints indicates that MAYSI-2 scale scores are at least related to psychiatric disorders. Fifth, due to the study design and due to conflicting schedules of youths; the time that passed between facility intake and the MAYSI-2 screening, and the time that passed between the MAYSI-2 screening and the K-SADS-PL interview was different for all youths and could have influenced the results. Lastly, we should note that several findings of the current study should be interpreted with caution as low power may have influenced the results.