Abstract
Suicidality in the child and adolescent population is a major public health concern. There is, however, a lack of developmentally sensitive valid and reliable instruments that can capture data on risk, and clinical and psychosocial mediators of suicidality in young people. In this study, we aimed to develop and assess the validity of instruments evaluating the psychosocial risk and protective factors for suicidal behaviours in the adolescent population. In Phase 1, based on a systematic literature review of suicidality, focus groups, and expert panel advice, the risk factors and protective factors (resilience factors) were identified and the adolescent, parent, and clinician versions of the STOP-Suicidality Risk Factors Scale (STOP-SRiFS) and the Resilience Factors Scale (STOP-SReFS) were developed. Phase 2 involved instrument validation and comprised of two samples (Sample 1 and 2). Sample 1 consisted of 87 adolescents, their parents/carers, and clinicians from the various participating centres, and Sample 2 consisted of three sub-samples: adolescents (n = 259) who completed STOP-SRiFS and/or the STOP-SReFS scales, parents (n = 213) who completed one or both of the scales, and the clinicians who completed the scales (n = 254). The STOP-SRiFS demonstrated a good construct validity—the Cronbach Alpha for the adolescent (α = 0.864), parent (α = 0.842), and clinician (α = 0.722) versions of the scale. Test–retest reliability, inter-rater reliability, and content validity were good for all three versions of the STOP-SRiFS. The sub-scales generated using Exploratory Factor Analysis (EFA) were the (1) anxiety and depression risk, (2) substance misuse risk, (3) interpersonal risk, (4) chronic risk, and (5) risk due to life events. For the STOP-SRiFS, statistically significant correlations were found between the Columbia-Suicide Severity Rating Scale (C-SSRS) total score and the adolescent, parent, and clinical versions of the STOP-SRiFS sub-scale scores. The STOP-SRiFS showed good psychometric properties. This study demonstrated a good construct validity for the STOP-SReFS—the Cronbach Alpha for the three versions were good (adolescent: α = 0.775; parent: α = 0.808; α = clinician: 0.808). EFA for the adolescent version of the STOP-SReFS, which consists of 9 resilience factors domains, generated two factors (1) interpersonal resilience and (2) cognitive resilience. The STOP-SReFS Cognitive Resilience sub-scale for the adolescent was negatively correlated (r = − 0.275) with the C-SSRS total score, showing that there was lower suicidality in those with greater Cognitive Resilience. The STOP-SReFS Interpersonal resilience sub-scale correlations were all negative, but none of them were significantly different to the C-SSRS total scores for either the adolescent, parent, or clinician versions of the scales. This is not surprising, because the items in this sub-scale capture a much larger time-scale, compared to the C-SSRS rating period. The STOP-SReFS showed good psychometric properties. The STOP-SRiFS and STOP-SReFS are instruments that can be used in future studies about suicidality in children and adolescents.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Suicide is one of the major causes of death worldwide, with figures suggesting that approximately 1 million people commit suicide each year [1]. Although completed suicide is rare before the age of 10, suicidal behaviour increases sharply during adolescence and is a leading cause of death among young people [2]. Several biological, social, and psychological risk factors for suicidality seem to be shared by children, adolescents, and adults. Suicide risk follows a multifactorial trajectory and is increased in many psychiatric disorders varying by diagnosis, gender, and age [3]. The previous evidence suggests, however, that some risk factors for suicide might be different in adolescents compared with adults [4]. In adolescents, it is frequent that negative life events precede suicidal behaviour, most commonly family conflicts [5,6,7], changes of residence [8], romantic breakup [9], conflict with peers, including bullying [10, 11], and/or academic failure [12]. These differences in adolescence indicate a pressing need for the development of instruments aimed at specifically assessing protective and risk factors in young people. Considering that many of the adolescents committing suicide have never received any mental health support [13] and that several interventions have shown efficacy in preventing suicidal behaviour [14,15,16], it is essential to develop mechanisms that enable identification of subjects at risk and promote early intervention. Since suicide is a sensitive topic, which can be associated with stigma, web-based health monitoring platforms could be especially useful tools, as they provide a space for privacy.
To date, there are few valid and reliable, and developmentally sensitive instruments for collecting comprehensive data on risk, clinical and psychosocial mediators of suicidality in paediatric populations available for use by clinicians [17, 18]. One of the most widely accepted screening instruments, the Columbia-Suicide Severity Rating Scale (C-SSRS), has been shown to identify accurately individuals at risk of suicide, both in adult and paediatric populations, and has been used as the gold standard for the assessment of suicidal ideation and behaviours in clinical trials [19]; however, its clinical utility has been questioned [20]. One reason for this is that the C-SSRS has been deemed not to be sensitive enough to be able to capture the full range of suicidal ideation or behaviour [20]. The development of the STOP (Suicidality: Treatment Occurring in Paediatrics) Risk and Resilience Factors Scales has the potential to overcome this limitation by addressing the full range of suicidal ideation or behaviour whether singly or in combination.
The STOP project (Suicidality: Treatment Occurring in Paediatrics http://cordis.europa.eu/project/rcn/97369_en.html) was predominantly dedicated to the development of a comprehensive web-based assessment of suicidality and its mediators in children and adolescents. The aim of this specific study, which was embedded within the overall project, was to develop and assess the validity of the multi-informant STOP-Risk Factors Scale (STOP-SRiFS) and the multi-informant STOP Resilience Factors Scale (STOP-SReFS) as instruments for the collection of comprehensive data on psychosocial risk and protective factors for suicidal behaviours in the adolescent population.
Methods
Figure 1 shows a general overview of the development and validation of the STOP-SRiFS and the STOP-SReFS. Phase 1 focused on the development of the scales and Phase 2 focused on their validation. For Phase 2—sample 1 (n = 87), the scales were administered to a sample of adolescents, their parents/carers, and clinicians (Fig. 1). This sample served to explore the psychometric properties of the scales (test–retest reliability). Sample 2 consisted of adolescents (n = 259) that completed the STOP-SRiFS and/or the STOP-SReFS scales, parents (n = 213) who completed one or both scales, and the young persons’ clinicians (n = 254). The samples partially overlapped with one another. Sample 2 was used for the Exploratory Factor Analyses (EFA) and the other psychometric analyses of the scales.
Informed assent/consent was obtained from participants and/or their legal representatives, according to the ethical and legal standards in the participating countries. The study had approval from the Institutional Review Boards of all participating sites. Patients were recruited from the secondary/tertiary clinics from the various participatory Departments which were part of the project across the EU. The respiratory clinics from King’s College Hospital, London, and Evelina London Children’s Hospital contributed to the recruitment of the subjects with bronchial asthma and respiratory allergies. Those from the general population were identified through advertising on websites, schools, and libraries in the UK.
Development of the scales
Systematic literature review A comprehensive and systematic literature review was performed at the outset of the study to identify the common and frequently reported risk and protective factors for suicidality in the paediatric population [21]. It also considered the aspects of suicidality that were covered by the C-SSRS [20] and other features relating to the revised nomenclature for the study of suicidal behaviours [22].
Selection of items and Scale development For both the STOP-SRiFS and STOP-SReFS, three versions (Adolescent, Parent and Clinician versions) were designed based on the list of domains extracted from the systematic literature review, input from focus groups, and expert feedback. The authors followed the U.S. Food and Drug Administration (FDA) recommendations for patient outcome measure development [23].
Consumer feedback: focus groups To explore patient’s views on risk and resilience factors of suicidality, identify new items, and to verify the understanding of the items, six meetings were carried out with children, adolescents, and parents (see Fig. 1). Each group session was conducted by two clinicians and was recorded with a video camera. Notes were taken during each focus group and reviewed by the experts. Based on the focus groups, some items were simplified, re-worded using age-appropriate vocabulary, or dropped; and answer options were reduced and converted to a 4-point scoring scale.
Expert feedback: STOP scientific advisory board The various experts in the study and the STOP scientific advisory board reviewed the draft versions of each scale and suggested minor modifications, which were incorporated into the final versions. The final versions of the scales in English and Spanish were reviewed by a professional translator. Following this, the English versions were then translated into German, Dutch, French, and Italian, and then back-translated into English. Clinicians from each participating country in the consortium ensured that the meaning of each statement remained culturally appropriate and meaningful.
Upload of the scales to HealthTrackerTM Once developed, the STOP-SRiFS and STOP-SReFS scales were uploaded onto the web-based HealthTracker™ platform, an e-health platform that includes a range of different scales for monitoring physical or emotional problems [21]. It was decided that the risk factors and protective factors would be presented sequentially as two scales, the STOP-Suicidality Risk Factors Scale (STOP-SRiFS) and the STOP-Suicidality Resilience Factors Scale (STOP-SReFS). At the end of this process, there are six scales; two for each different role that the scale can be assigned to/completed by. The scales are the STOP-ReFS for adolescent, parent, and clinicians, and the STOP-RiFS for adolescents, parents and clinicians.
Scoring of the scales
The focus groups and the expert panel assisted in deciding the response options to the questions and also how to score the questions.
Scoring of the STOP-SRiFS
The majority of the items in the STOP-SRiFS were single questions, except for two items (“suicide on internet”, and “history of attempt”) that had two sub-questions each. The item on “suicide on Internet” dealt with (1) the number of times the adolescent had looked up information about suicidal behaviours or acts described in the item; and (2) when was the last time that they had searched the internet about this. The item on “history of attempt” dealt with (1) the number of times that they had attempted suicide in the past, and (2) when was the last attempt. For these two items, the score was obtained by the sum of the two scores divided by two. The score for each STOP-SRiFS questions ranged from 0 to 4. The undefined answers (“I don’t know”) were coded as 888 and then substituted with an empty cell.
Scoring of the STOP-SReFS
In the STOP-SReFS, each item was composed of two sub-questions: the first one dealt with the importance the adolescent gave to the item and the second one dealt with how useful the same item was in relation to protecting them against suicidality. The score of each item is obtained by the sum of the scores of the two sub-questions divided by two. The score for the STOP-SReFS items ranged from 0 (not at all) to 4 (a great deal). The undefined answers (“I don’t know”) were coded as 888 and then substituted with an empty cell. Each item score was given by the sum of the two questions which composed an item divided by two.
The scales scoring method allows the presence of undefined answers when a subject who filled the scales chose “I don’t know” as an answer. This was done to address the case in which the subject was unable to decide the answer, as forced answers are difficult when dealing with sensitive clinical issues such as suicidality. Little’s Missing at Random Tests for all the versions of the scales which were run before the estimation of the composite scores and for single question items. The results of those analyses showed that it was not possible to impute a value for the undefined answers, and therefore, those were left as empty (not given) answers.
Phase 2: data analyses (validation of the instruments)
Subjects completed the questionnaires online using the web-based HealthTracker™ platform. SPSS version 23 [24] was used for the analyses.
Sample 1 Consisted of 87 adolescents, their parents/carers, and clinicians from the various participating centres, who were re-administrated the scales within a maximum time of 3 weeks. This sample was used to test the time stability (test–retest reliability) of all versions of the STOP-SRiFS and STOP-SReFS.
Sample 2 Consisted of 259 adolescents, 213 parents of adolescents, and 254 clinicians. Completion rates varied, because an adolescent might have completed the scale but not the parent of the adolescent or the clinician (see Table 2).
Construct validity using Cronbach’s alpha, test–retest reliability using correlations between repeat completions within 3 weeks, inter-rater reliability through correlations between the three versions of the scales, content and concurrent validity, through comparing the scores with that of the C-SSRS, and the sub-scales were generated using the Exploratory Factor Analysis (EFA) on the Adolescent, Parent, and Clinician versions of the scales. The sample sizes for all versions of both scales were above 200 and were considered adequate for these analyses [25]. The extraction method used was principal axis factoring, and Promax rotation was undertaken.
To assess the concurrent validity, the adolescent, parent, and clinician versions of the scales were correlated with the C-SSRS using Pearson’s correlations. The previous studies using the C-SSRS have shown convergent and divergent validity with other multi-informant suicidal ideation and behaviour scales and high sensitivity and specificity for suicidal behaviour [26].
Results
Sample 1 comprised of 87 adolescents (mean age of 15.66 ± 1.66; 41.4% males and 58.6% females) (see Table 1 for the characteristics of Sample 1). Sample 2 was primarily composed of adolescents who had been screened as having some suicidality on the STOP 4-item Suicidality Screening questionnaire [21] and their parents and clinicians. The sample consisted of 259 adolescents (patient age at first assignment was 15.03 ± 1.599) who completed STOP-SRiFS and/or the STOP-SReFS scales; 213 parents (patient age at first assignment was 14.92 ± 1.797) who filled one or both of the scales; and 254 clinicians (patient age at first assignment was 15.17 ± 1.552) (see Table 2 for demographics of Sample 2).
STOP-SRiFS
-
1.
Construct validity The STOP-SRiFS Adolescent version demonstrated a good reliability (Cronbach’s α = 0.864) (Cronbach’s threshold was set at α > 0.700 [27]). For the Parent version of the scale, two items were excluded, because their Corrected Item-Total Correlation (CITC) was below the acceptance threshold [“Sexual Identity” (CITC = 0.056), and “Change of residence” (CITC = − 0.158)]. After excluding these two items, the STOP-SRiFS Parent version had good Cronbach’s alpha value (α = 0.842). Similarly, the STOP-SRiFS Clinician version showed a good Cronbach’s alpha (α = 0.722) when four items were excluded [chronic physical illness (CITC = − 0.066), being bullied (CITC = − 0.072), use of drugs (CITC = 0.122), and change of residence (CITC = 0.045)] (Table 3).
-
2.
Test–retest reliability The results showed that there was good temporal stability (test–retest reliability), through the intra-class correlation coefficients between the STOP-SRiFS sub-scales scores at the first and second administration (within 3 weeks ~ 19 Days). All the intra-class correlations were good (> 0.600) (see Table 4).
-
3.
Inter-rater reliability Table 5 presents the inter-version correlations between the different STOP-SRiFS sub-scales for the adolescent, parent, and clinician versions, and shows that they were good (Pearson’s correlation coefficient threshold was set at r > 0.200 [27]).
-
4.
Exploratory factor analysis The items that showed poor-corrected item-total correlation (CITC) for the Parent and Clinician versions of the scale were also excluded by the EFA and, therefore, from any subsequent psychometric analysis. The experts in childhood suicidality who reviewed these results recommended that the aforementioned items, given their clinical relevance, should continue to be administered at the end of the scale as extra items, which will not be used in the scoring of the scales. As shown in Table 6, EFA for the Adolescent version of the STOP-SRiFS (consisting 21 risk factor domains), a 5-factor model was determined to best fit the data based on the screen plot. The Kaiser–Meyer–Olikin (KMO) was 0.816 (X2 = 1787.257; Bartlett’s test of sphericity p ≤ 0.001; df = 210). The Parent version of the scale without the two items excluded based on the Corrected Item-Total Correlation showed again that the best model to explain the structure of the scale is again a 5-factor model. The KMO was 0.720 (X2 = 727.698; Bartlett’s test of sphericity p ≤ 0.001, df = 171). The results of the EFA for the parent version of the STOP-SRiFS showed that the item about the ‘misuse of other drugs’ had the highest loading on the factor which assessed the sub-domains concerning risk due to life events (0.216). The second highest loading for this item (0.179) was on the factor which assessed the sub-domains concerning substance misuse risk. Based on the clinical judgment of experts in child and adolescent mental health, it was decided that it was clinically relevant for this item to be part of the factor about substance misuse risk (see Table 6 for more details about the STOP-SRiFS factor structure). The Clinician version of the scale (5 factors) also presented with a KMO of 0.772 (X2 = 895.861; Bartlett’s test of sphericity p ≤ 0.001, df = 136). Based on the pattern of risk factors domain loading, the five factors were named as: (1) anxiety and depression risk, (2) substance misuse risk, (3) interpersonal risk, (4) chronic risk, and (5) risk due to life events. These factors capture the clinical risk clusters in adolescents with suicidal ideations or behaviours.
-
5.
Content validity As predicted, the correlations between the sub-scales of STOP-SRiFS adolescent version and the C-SSRS total score were significant, indicating that increased risk was associated with increased C-SSRS total score (Table 7). Broadly speaking, the STOP-SRiFS sub-scale scores in the parent and clinician versions were similarly correlated with the C-SSRS total score. The sub-scale scores that did not reach significance were those which would be rated differently by the parents and clinicians in comparison to the adolescent (Table 7).
STOP-SReFS
-
1.
Construct validity The Cronbach’s alpha values for all the versions of the STOP-SReFS (Adolescent: 0.775; Parent: 0.808; Clinician: 0.808) (Table 3) indicate good internal consistency of the scale (Table 3).
-
2.
Test–retest reliability The results showed a good temporal stability (test–retest reliability). This was assessed in Sample 1 through the intra-class correlation coefficients between the STOP-SReFS sub-scales scores at the first and second administration (within 3 weeks ~ 19 days). These results showed that all intra-class correlations were above the acceptance threshold (> 0.600), except for the parent cognitive resilience scale which was below the threshold (0.547) (see Table 4). This is understandable, because suicidality risk factors can change even in the short period used for the test–retest.
-
3.
Inter-rater reliability Table 5 presents the inter-version correlations between the different STOP-SRiFS and STOP-SReFS sub-scales for the Adolescent, Parent, and Clinician versions, which were all acceptable.
-
4.
Exploratory factor analysis As shown in Table 6, EFA for the Adolescent version of the STOP-SReFS identified that a two-factor model was the best fit (the KMO was 0.769 (X2 = 511.748; Bartlett’s test of sphericity p ≤ 0.001, df = 36). The EFA of the STOP-SReFS Parent version also showed that the best model to explain the structure of the scale was a two-factor model (the KMO was 0.819 (X2 = 446.362; Bartlett’s test of sphericity p ≤ 0.001, df = 36). The EFA of the STOP-SReFS Clinician version was similar and had a KMO of 0.813 (X2 = 572.156; Bartlett’s test of sphericity p ≤ 0.001, df = 36). Based on the pattern of resilience factors domain loading, the two factors were named: (1) interpersonal resilience and (2) cognitive resilience. These resilience factors are in keeping with known protective factors. The EFA revealed that the scales were not unidimensional and, therefore, precludes the use of a total score. In view of this, correlations between the STOP-SReFS sub-scales and their correlations with the C-SSRS total score were performed.
-
5.
Content validity Correlations between the STOP-SRiFS and STOP-SReFS sub-scales, and the C-SSRS total score are presented in Table 7. As expected, the C-SSRS negatively correlated with the STOP-SReFS (captures protective factors) cognitive resilience sub-scale for the adolescent (r = − 0.275). However, the clinician (r = − 0.143) versions of the scale did not meet the threshold of r > 0.200 [26] (Table 7). The STOP-SReFS Interpersonal resilience sub-scale correlations were all negative, but none of them were significantly different to the C-SSRS total scores for either the adolescent, parent, or clinician versions of the scales.
Discussion
Despite progress made in suicidality research, the risk and resilience factors involved in suicidal behaviour and ideation remain poorly understood. The present study describes the development and the subsequent psychometric validation of two scales: the STOP-Suicidality Risk Factors Scale (STOP-SRiFS) and the STOP-Suicidality Resilience Factors Scale (STOP-SReFS)—two web-based instruments that measure elements of suicidality on the web-based HealthTracker™ system. The measurement properties of the two instruments were assessed using the consensus-based standards for the selection of health status Measurement instruments (COSMIN) [28]. The COSMIN checklist was used to structure the layout of the manuscript when reporting a study describing psychometric instruments. Using this approach, the psychometric analyses revealed that the STOP-SReFS and the STOP-SRiFS were reliable and valid instruments for assessing suicidality risk and resilience factors in adolescents. The fact that the STOP-SRiFS and the STOP-SReFS are more age-specific scales, which have been designed and worded specifically for the adolescent population, and that they can be completed online, decreasing completion time and ensuring accessibility at all times, increases their potential applicability in an adolescent population [29]. As suicidal behaviour depends on diverse clinical, psychological, sociological, and biological factors, the consensus is that a multi-informant evaluation is strongly recommended [21]. Furthermore, adolescents who are a particularly high-risk group for suicidality differ from the adult population and need a deeper, wider, and multi-dimensional approach [30]. The study of cross-informant agreement has been shown to be useful in obtaining a more detailed understanding of the adolescent population [31] as they usually tend to not report the same information as their parents, teachers, or clinicians. In this study, results in adolescents and parents showed a good correlation, contrary to some studies that report low agreement between parents and adolescents [32].
The threshold for the minimum loading for EFA was set at > 0.200. Thresholds in the region of > 0.200 have been cited in the literature [33, 34] and we have previously used a similar threshold for factor loading to validate and assess the psychometric properties of a parent version of a neuropsychiatric scale [35]. In the context of the present study, we set a threshold of > 0.200 so that the factor loading would best reflect the phenomenon of interest in accordance with our sample size, clinical judgement, and exploratory nature of the study.
Suicide risk factors have been widely studied, whilst the study of protective factors has been usually neglected. However, in the past years, there has been an increasing interest in incorporating the concept of resilience into the suicidality paradigm [36]. The identification of specific risk and resilience factors in young people could help to develop personalized therapeutic strategies, in which treatment is tailored to the personal needs of each patient. In addition, this could lead to the development of targeted interventions for some of these risk and/or resilience factors, for example, intervention programs aimed at improving the family connectedness. This knowledge may lead to actions and changes which can have an impact on the suicide rates as shown in the Youth Aware of Mental Health Programme (YAM), a manualized, universal school-based intervention which has shown efficacy in reducing the number of suicide attempts and severe suicidal ideation in adolescents [16]. The SEYLE trial, which has been recruiting a large number of European adolescents, has also addressed these issues, concluding that screening is an efficient method to refer subjects in need of treatment [16].
The Internet has become a public and accessible information exchange forum for individuals. The use of new technologies could innovate healthcare, i.e., a web-based version of a questionnaire may enhance perceptions of privacy and confidentiality, which may improve honesty of responses, particularly when less socially desirable, especially to those items related to emotions [37].
As far as we are aware of, this is the first attempt to assess risk and resilience factors related to suicidality in the adolescent population using web-based measures, and accounting for different sources of information. The thorough methodology employed, the sample size, the focus groups in which all interested parties were involved in co-designing the scales, the external scientific supervision by experts in the field, and its applicability to multiple pathologies and settings offered added value to this study.
Limitations
There are limitations to this study that need to be considered. To identify subjects at risk, a positive and undefined answer to the screening questionnaire of the STOP-Suicidality Assessment Scale (STOP-SAS) [21] was necessary for patients to be allocated the full STOP-SRiFS and STOP-SReFS. Since the aim of the study was to develop a universal instrument, we did not account for the effect of diagnosis and sex on these risk and protective factors. Moreover, not being able to substitute the missing values in the database with estimations led to a reduced sample size.
Conclusion
The current study suggests that the STOP-SRiFS and the STOP-SReFS scales are viable instruments to assess risk and resilience factors in young people. They can be used to identify subgroups in the adolescent population who may need targeted intervention. In this vein, the STOP-SRiFS and the STOP-SReFS could be used as effective risk stratification tools to provide a multi-informant view on adolescent risk for suicidality and maybe of value for the assessment of suicidality in clinical trials. This sentiment has been echoed by others, who have highlighted the need for improving the detection and assessment of suicidality in clinical trials [38]. Moreover, the identification of subpopulations with a personalized level of specific risk and protective factors could guide personalized interventions, which ultimately may help to reduce suicide rates and improve prognosis in paediatric populations.
References
World Health Organization (1989) World report on violence and health, P 2002
Bridge JA, Goldstein TR (2006) Brent DA Adolescent suicide and suicidal ehaviour. J Child Psychol Psychiatry 47:372–394
Qin P (2011) The impact of psychiatric illness on suicide: differences by diagnosis of disorders and by sex and age of subjects. J Psychiatr Res 45:1445–1452
Parellada M, Saiz P, Moreno D et al (2008) Is attempted suicide different in adolescent and adults? Psychiatry Res 157:131–137
Brent DA, Emslie GJ, Clarke GN et al (2009) Predictors of spontaneous and systematically assessed suicidal adverse events in the treatment of SSRI-resistant depression in adolescents (TORDIA) study. Am J Psychiatry 166:418–426
Agerbo E, Nordentoft M, Mortensen PB (2002) Familial, psychiatric, and socioeconomic risk factors for suicide in young people: nested case-control study. BMJ 325(7355):74
Vitiello B, Silva SG, Rohde P et al (2009) Suicidal events in the treatment for adolescents with depression study (TADS). J Clin Psychiatry 70:741–747
Qin P, Mortensen PB, Pedersen CB (2009) Frequent change of residence and risk of attempted and completed suicide among children and adolescents. Arch Gen Psychiatry 66:628–632
Asarnow JR, Baraff LJ, Berk M et al (2008) Pediatric emergency department suicidal patients: two-site evaluation of suicide ideators, single attempters, and repeat attempters. J Am Acad Child Adolesc Psychiatry 47:958–966
Klomek AB, Sourander A, Niemelä S et al (2009) Childhood bullying behaviors as a risk for suicide attempts and completed suicides: a population-based birth cohort study. J Am Acad Child Adolesc Psychiatry 2009(48):254–261
Pan SW, Spittal PM (2013) Health effects of perceived racial and religious bullying among urban adolescents in China: a cross-sectional national study. Global Public Health. 8:685–697
Martin G, Richardson AS, Bergen HA et al (2005) Perceived academic performance, self-esteem and locus of control as indicators of need for assessment of adolescent suicide risk: implications for teachers. J Adolesc. 28:75–87
Marttunen MJ, Aro HM, Lönnqvist JK (1992) Adolescent suicide: endpoint of long-term difficulties. J Am Acad Child Adolesc Psychiatry 31:649–654
Meerwijk EL, Parekh A, Oquendo MA et al (2016) Direct versus indirect psychosocial and behavioural interventions to prevent suicide and suicide attempts: a systematic review and meta-analysis. Lancet Psychiatry 3:544–554
Herbeck Belnap B, Schulberg HC, He F et al (2015) Electronic protocol for suicide risk management in research participants. J Psychosom Res 78:340–345
Wasserman D, Hoven CW, Wasserman C et al (2015) School-based suicide prevention programmes: the SEYLE cluster-randomised, controlled trial. Lancet 385:1536–1544
Range LM, Knott EC (1997) Twenty suicide assessment instruments: evaluation and recommendations. Death Stud 21:25–58
Pfeffer CR, Jiang H, Kakuma T (2000) Child-Adolescent Suicidal Potential Index (CASPI): a screen for risk for early onset suicidal behaviour. Psychol Assess 12:304–318
United States Food and Drug Administration (2014) Guidance for industry: suicidality: prospective assessment of occurrence in clinical trials. Draft Guidance
Giddens JM, Sheehan KH, Sheehan DV (2014) The Columbia-Suicide Severity Rating Scale (C-SSRS): has the “Gold Standard” become a liability? Innov Clin Neurosci 11:66–80
Flamarique I, Santosh P, Zuddas A et al (2016) Development and psychometric properties of the Suicidality: treatment Occurring in Paediatrics (STOP) Suicidality Assessment Scale (STOP-SAS) in children and adolescents. BMC Pediatr 16:213
Silverman MM, Berman AL, Sanddal ND, et al (2007) Rebuilding the tower of Babel: a revised nomenclature for the study of suicide and suicidal behaviors. Part 2: Suicide-related ideations, communications, and behaviors. Suicide Life Threat Behav 37:264–277
United States Food and Drug Administration (2014) Guidance for industry: Suicidality: prospective assessment of occurrence in clinical trials. Draft Guidance
IBM Corp (2015) Released 2015. IBM SPSS statistics for windows, Version 23.0. IBM Corp, Armonk
Williams B, Onsman A, Brown T (1996) Exploratory factor analysis: a five-step guide for novices. J Emerg Prim Heal Care 19:42–50
Posner K, Brown GK, Stanley B et al (2011) The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry 2011(168):1266–1277
Nunnally JC (1978) Psychometric theory, 2nd edn. McGraw-Hill, New York
Mokkink LB, Terwee CB, Patrick DL, Stratford PW, Knol DL, Bouter LM, de Vet HCW (2010) The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 19:539–549
Lloyd-Richardson EE, Lewis SP, Whitlock JL et al (2015) Research with adolescents who engage in non-suicidal self-injury: ethical considerations and challenges. Child Adolesc Psychiatry Mental Health 9:37
De Leo D (2002) Struggling against suicide: the need for an integrative approach. Crisis 23:23–31
Renk K, Phares V (2004) Cross-informant ratings of social competence in children and adolescents. Clin Psychol Rev 24:239–254
Shoval G, Mansbach-Kleinfeld I, Farbstein I et al (2013) Self-versus maternal reports of emotional and behavioral difficulties in suicidal and non-suicidal adolescents: an Israeli nationwide survey. Eur Psychiatry 28:235–239
Child, D. (2006). The Essentials of Factor Analysis. 3rd edn. New York: Continuum
Peterson RA (2000) A meta-analysis of variance accounted for and factor loadings in exploratory factor analysis. Market Lett 11:261–275
Santosh P, Gringras P, Baird G et al (2015) Development and psychometric properties of the parent version of the Profile of Neuropsychiatric Symptoms (PONS) in children and adolescents. BMC Pediatr 15:62
Rutter PA, Freedenthal S, Osman A (2008) Assessing protection from suicidal risk: psychometric properties of the suicide resilience inventory. Death Stud 32:142–153
Vereecken CA, Maes L (2006) Comparison of a computer-administered and paper-and-pencil-administered questionnaire on health and lifestyle behaviors. J Adolesc Health 38:426–432
Chappell BP, Stewart M, Alphs L et al (2017) Assessment of suicidal ideation and behaviour: report of the international society for CNS clinical trials and methodology consensus meeting. J Clin Psychiatry 78:e638–e647
Acknowledgements
The members of the STOP Consortium: Alastair Sutcliffe, University College London, Institute of Child Health, London, United Kingdom; Sarah Curran, St George’s University Hospital, London, UK. Laura Selema, Institute of Psychiatry, Psychology and Neurosciences (IoPPN), King’s College London, London, UK; Robert Flanagan, Institute of Psychiatry, Psychology and Neurosciences (IoPPN), King’s College London, London, UK; Ian Craig, Institute of Psychiatry, Psychology and Neurosciences (IoPPN), King’s College London, London, UK; Nathan Parnell, Institute of Psychiatry, Psychology and Neurosciences (IoPPN), King’s College London, London, UK; Keren Yeboah, Institute of Psychiatry, Psychology and Neurosciences (IoPPN), King’s College London, London, UK; Gideon Lack, Peter Gorer Department of Immunobiology, School of Life Sciences and Medicine, Faculty of Life Sciences & Medicine, King’s College London, UK; Florence Pupier. CHRU Montpellier; Hôpital Saint Eloi, Médecine Psychologique de l’Enfant et de l’Adolescent, Montpellier, France; Loes Vinkenvleugel, Radboud University Medical Centre, Nijmegen, The Netherlands; Jeffrey Glennon, Radboud University Medical Centre., Nijmegen, The Netherlands; Mireille Bakker, Radboud University Medical Centre, Nijmegen, The Netherlands; Cora Drent, University of Groningen, University Medical Center Groningen, Department of Child and Adolescent Psychiatry, The Netherlands; Elly Bloem, University of Groningen, University Medical Center Groningen, Department of Child and Adolescent Psychiatry, The Netherlands; Mark-Peter Steenhuis, University of Groningen, University Medical Center Groningen, Department of Child and Adolescent Psychiatry, The Netherlands; Ruth Berg, Central Institute of Mental Health, Mannheim, Germany; Alexander Häge. Central Institute of Mental Health, Mannheim, Germany; Mahmud Ben Dau, Central Institute of Mental Health, Mannheim, Germany; Konstantin Mechler, Central Institute of Mental Health, Mannheim, Germany; Sylke Rauscher, Central Institute of Mental Health, Mannheim, Germany; Sonja Aslan, University of Ulm, Ulm, Germany; Simon Schlanser, University of Ulm, Ulm, Germany; Ferdinand Keller, University of Ulm, Ulm, Germany; Alexander Schneider, University of Ulm, Ulm, Germany; Paul Plener, University of Ulm, Ulm, Germany; Jörg M. Fegert, University of Ulm, Ulm, Germany; Jacqui Paton, University of Dundee, UK; Murray, Macey, University College London, UK; Noha Iessa, World Health Organization, London, UK; Alfred Kolozsvari, HealthTracker Ltd, Gillingham, UK; Furse, Helen. HealthTracker Ltd, Gillingham, UK; Nick Penkov, HealthTracker Ltd, Gillingham, UK; Claire Baillon, Assistance Publique—Hopitaux de Paris: Robert Debré Hospital, Paris, France; Hugo Peyre. Assistance Publique—Hopitaux de Paris:Robert Debré Hospital, Paris, France; David Cohen, Assistance Publique—Hopitaux de Paris: Groupe Hospitalier Pitié-Salpêtrière, Paris, France; Olivier Bonnot, Assistance Publique—Hopitaux de Paris: Groupe Hospitalier Pitié-Salpêtrière, Paris, France; Julie Brunelle, Assistance Publique – Hopitaux de Paris: Groupe Hospitalier Pitié-Salpêtrière, Paris, France; Nathalie Franc, CHRU Montpellier; Hôpital Saint Eloi, Médecine Psychologique de l’Enfant et de l’Adolescent, France; Pierre Raysse, CHRU Montpellier; Hôpital Saint Eloi, Médecine Psychologique de l’Enfant et de l’Adolescent, France; Véronique Humbertclaude, CHRU Montpellier; Hôpital Saint Eloi, Médecine Psychologique de l’Enfant et de l’Adolescent, France; Ana Espliego, CIBERSAM, Madrid, Spain; Jessica Merchán, CIBERSAM, Madrid, Spain; Cecilia Tapia, CIBERSAM, Madrid, Spain; Immaculada Baeza, Fundació Clínic per la Recerca Biomèdica, Barcelona, Spain; Soledad Romero, Fundació Clínic per la Recerca Biomèdica, Barcelona, Spain; Amalia La Fuente. University of Barcelona, Spain; Ana Ortiz, Fundació Clínic per la Recerca Biomèdica, Barcelona, Spain; Manuela Pintor, Cagliari University Hospital, Cagliari, Italy; Franca Ligas, University of Cagliari, Cagliari, Italy; Francesca Micol Cera, University of Cagliari, Cagliari, Italy; Roberta Frongia, Cagliari University Hospital, Cagliari, Italy; Bruno Falissard, Univ. Paris-Sud, INSERM U669, AP-HP, Paris, France; Ameli Schwalber, Concentris, Germany; Juliane Dittrich, Concentris, Germany; Andrea Wohner, Concentris, Germany; Katrin Zimmermann, Concentris, Germany; Andrea Schwalber, Concentris, Germany; Katherine Aitchison, University of Alberta, Calgary, Canada.
Funding
This research was funded by the European Community’s Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 261411. The research was also supported by the Spanish Ministry of Economy and Competitiveness, Instituto de Salud Carlos III, co-financed by ERDF Funds from the European Commission, “A way of making Europe”, CIBERSAM, Madrid Regional Government (S2010/BMD-2422 AGES) and European Union Structural Funds, Fundación Alicia Koplowitz and Fundación Mutua Madrileña.
Author information
Authors and Affiliations
Consortia
Contributions
AQR prepared the first draft of the manuscript, IF and JFC recruited subjects and carried out the focus groups. LK, CMDC, AZ, DPO, PJH, DC, and CL were involved in the literature review and provided expert feedback on the items for the scales and assisted with patient recruitment. AG and RS assisted with patient recruitment. AZ, DPO, and PJH approved the final version of the scale in the designated language. FF, JS, and PS helped in the preparation of the manuscript and provided intellectual oversight into the literature review. US, RWD, and JKB provided expert review on the scales. KL enrolled patients and helped in preparing the first draft of the manuscript. FF was responsible for the data management and analysed the data with input from PS. PS and CA provided critical oversight for the study and provided guidance and structure to the manuscript. PS was the Coordinator of the entire STOP project. All authors have seen and approved the final version of the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
Professor Paramala Santosh is Director & CEO and stockholder in HealthTracker Ltd. Dr. F. Fiori is Chief Technology Officer employed by HealthTracker Ltd. Dr. K. Lievesley is a Programme Manager employed at HealthTracker Ltd. Dr. Rodríguez-Quiroga has previously held a Río Hortega grant, Instituto de Salud Carlos III, Spanish Ministry of Economy and Competitiveness. Dr. Díaz-Caneja has previously held a Río Hortega grant, Instituto de Salud Carlos III, Spanish Ministry of Economy and Competitiveness, and a Grant from Fundación Alicia Koplowitz. Dr. Dittmann has received compensation for serving as consultant or speaker, or he or the institution he works for have received research support or royalties from the organizations or companies indicated: EU (FP7 Programme), US National Institute of Mental Health (NIMH), German Federal Ministry of Health/Regulatory Agency (BMG/BfArM), German Federal Ministry of Education and Research (BMBF), German Research Foundation (DFG), Volkswagen Foundation; Boehringer Ingelheim, Ferring, Janssen-Cilag, Lilly, Lundbeck, Otsuka, Servier, Shire, Sunovion/Takeda, and Theravance. Dr. Dittmann owns Eli Lilly stock. Dr. Zuddas has been a consultant to or has received honoraria or grants from EU (FP7 Programme), Angelini, Lundbeck, Janssen, Roche, Servier, Shire, Takeda, and Vifor. Prof. Coghill reports grants and personal fees from Shire, personal fees from Janssen-Cilag, personal fees from Lilly, grants and personal fees from Vifor, personal fees from Novartis, personal fees from Flynn Pharma, personal fees from Medice, and personal fees from Oxford University Press, outside the submitted work. Dr. Arango has been a consultant to or has received honoraria or grants from Acadia, Abbot, AMGEN, AstraZeneca, Bristol-Myers Squibb, Janssen-Cilag, Lundbeck, Merck, Otsuka, Pfizer, Roche, Servier, Sumitomo-Dainippon Pharma, Shire, Takeda, Teva, and Schering Plough. Dr Purper-Ouakil has been consultant for Shire, Boiron, and Mensia, and has received honoraria or travel grants from Shire, Otsuka, Medice, Jannssen-Cilag, and Ardix. None of the other authors have any conflicts of interest or disclosures to declare. Part of these data have been included in an FP7 STOP Report to the European Union.
Ethical approval
The study was approved by the Research Ethic Committees (RECs)/Institutional Review Boards (IRBs) of all participating centres.
Informed consent
Informed consent was obtained from all study participants
Additional information
The members of the STOP Consortium are listed in acknowledgements section.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Rodríguez-Quiroga, A., Flamarique, I., Castro-Fornieles, J. et al. Development and psychometric properties of the “Suicidality: Treatment Occurring in Paediatrics (STOP) Risk and Resilience Factors Scales” in adolescents. Eur Child Adolesc Psychiatry 29, 153–165 (2020). https://doi.org/10.1007/s00787-019-01328-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00787-019-01328-2