Introduction

Suicide is a major global health challenge and is the leading cause of death among young people aged 20–34 years in the UK [69]. A variety of sociodemographic, biological and psychological risk factors have been proposed for completed suicide (for review, see [89]). Among the most predictive, and potentially amenable to clinical intervention, are (1) history of self-harm, which is associated with 37.2 times increased risk of completed suicide within the first year following an act of self-harm [70], and (2) suicidal ideation, which in a recent meta-analysis is associated with increased risk ratios for competed suicide of 2.35–8.00 [48].

Self-harm is a common and debilitating behaviour characterised by self-injury or self-poisoning, irrespective of the apparent purpose of the act [63]. Estimated lifetime prevalence of self-harm is 1–6%, with the UK reportedly having the highest self-harm rate in Europe [46]. Incidence is estimated at 400/100,000 population per year [90]. However, many people who self-harm do not attend clinical services, and thus true prevalence may be considerably greater [43].

Self-harm is aetiologically associated with childhood maltreatment [32, 84] and physical illness [23]. In addition, a number of demographic factors are predictive of self-harm, including being female [76]; young adulthood [76]; being unmarried [76]; or separated/divorced [72]; being socioeconomically disadvantaged [88]; unemployed [74]; or low educational attainment [75].

Psychiatric illness also has well-known associations with self-harm [79]. One systematic review of non-fatal self-injury presenting to hospital reported a pooled prevalence for psychiatric disorder of 83.9%, with mood disorders the most common category (58.5%) [44]. The association between depressive disorder and self-harm has been found in numerous other studies [6, 16].

Self-harm is performed with a variety of motivations, including attempted suicide, self-mutilation, seeking psychological relief, and the communication of distress. Often, there is not a single readily definable motivation, but multiple factors occurring simultaneously [20]. In the majority of cases, the intention is not to die [79]. Given the difficulties encountered clinically in ascertaining intent and motivation, it has been argued that the terms ‘deliberate self-harm’, ‘self-harm’, ‘attempted suicide’ and ‘suicidality’ are imprecise for research purposes [67]. Recently, the Fifth Edition of the Diagnostic and Statistical Manual for Mental Disorders [2] has proposed a distinction between ‘nonsuicidal self-injury’ (NSSI) and ‘suicidal behaviour disorder’ as ‘Conditions For Further Study’. However, it remains controversial whether such discrete categorizations can be confidently made in clinical practise, or demonstrate differentiable suicidal outcomes, given the biases inherent in self-report, and the close association of NSSI with suicidal behaviour [19, 52]. Broadly defined ‘self-harm’, therefore, retains an important clinical outcome in current suicidology literature [42, 52].

Another approach to subcategorising self-harm is on the basis of whether it has received hospital treatment. Hospital-treated self-harm is recognised as an important intervention point in suicide prevention [12]. Approximately one-seventh to one-fifth of those with hospital-treated self-harm will repeat their self-harm within 1 year [71]. Self-harm that requires medical attention significantly increases the future risk of suicide [19], particularly if admission to hospital is required [37]. Within the UK, up to one-fifth of those who die by suicide have attended hospital for self-harm in the preceding year [34].

Suicidal ideation, additionally, is an important antecedent to progression to significant self-harm and suicide attempts [32, 56]. Individuals who express suicidal ideation have significantly greater 12-month prevalence of self-harm and completed suicide, especially if there is associated planning [89]. Nevertheless, the relationship between self-harm and suicidal ideation is complex, with suicidal ideation having reportedly more than three times greater prevalence than suicide attempts [68].

Both self-harm and suicidal ideation are associated with personality, including personality disorders [41] and normally distributed personality traits. In particular, neuroticism is associated with suicidal ideation [22, 75], suicide attempts [73, 78], and suicide [24, 87]. A systematic review of personality traits and suicidality [9] found that neuroticism (and hopelessness) was the most predictive traits in risk screening. Neuroticism is a partially heritable personality trait which incorporates negative affectivity [29, 62] and increased sensitivity to stress (for review see [57]). An important aspect of neuroticism is that individual differences in the trait are moderately to highly stable over many years [17, 35] and thus might be useful as a patient level predictor for future self-harm risk. However, the link between neuroticism and self-harm is not wholly consistent and one large study did not find an association between neuroticism and lifetime history of prior suicide attempts [22].

Neuroticism is also highly correlated with affective disorder and both conditions show evidence of substantially overlapping genetic architecture [49, 54, 64]. There is uncertainty about whether neuroticism is a significant predictor of self-harm irrespective of depressive disorder history [31, 75] or whether it is insignificant when comorbid depression is controlled for [4, 8]. A recent study [75] in Chinese females concluded that neuroticism was significantly associated with suicide attempts even after controlling for comorbid depression and also stressful life events. Stressful life events are an additional posited factor in suicidal behaviour and it is hypothesised that neuroticism may serve to increase negative perceptions of these events [53, 73].

While considerable work has been undertaken at elucidating risk factors for self-harm and suicidal ideation, less is known about protective factors, which are not merely the absence of risk [79]. One component of managing adversity is coping styles, the behavioural and cognitive strategies adopted in response to stressful life events. These are not only situational, but may be environmentally and genetically conditioned [33]. They are of particular interest because they are potentially modifiable and might be impacted by treatment [14, 25].

Coping strategies are elicited by questionnaires like the Coping Inventory for Stressful Situations [27] which yields three main groups of coping strategies. The first is a “task-“or problem-oriented coping style (ToC), which is characterised by purposeful efforts aimed at problem solving. “Avoidance-orientated” (AoC) coping, in contrast, is defined by behaviours aimed at avoiding difficult circumstances [21]. Finally, “emotion-orientated” coping (EoC) is characterised by attempts to regulate difficult emotions as a means of coping. While ToC is generally seen as positively related to health and psychological adaptation, AoC and EoC are generally seen as less psychologically adaptive, and have been associated with negative mental health outcomes [45]. Task-oriented coping is thought to be negatively correlated with neuroticism [18] while emotion-oriented coping is positively correlated [28]. Moreover, emotion- and avoidance-oriented coping are thought to be associated with greater risk of suicidal ideation, while task-oriented coping is associated with lower risk [14].

In the first part of study, we aimed to investigate the relationship between neuroticism and hospital-treated self-harm. We employed two large UK population-based cohorts with neuroticism quantified by the same Eysenck Personality Questionnaire-Revised Short Form (EPQ-SF) scale. In one cohort, Generation Scotland (GS:SFHS), we used record-linkage to administrative health data to identify individuals with previous hospital-treated self-harm (generally defined and including all types of intentional self-injury requiring admission to medical or psychiatric hospital, N = 15,798; self-harm cases = 339). In the second cohort, UK Biobank (UKB), we used self-reported intentional self-harm (whether or not with intention to end life) requiring hospital treatment (including emergency department) and/or review by psychiatric services (N = 35,227; self-harm cases = 772). We hypothesised that neuroticism would be positively associated with self-harm, even after adjustment for depressive disorder and other significant sociodemographic factors.

In the second part of the study, we employed a follow-up sample of GS:SFHS with contemporaneous self-reported measures of suicidal ideation (n = 3356, suicidal ideation cases = 161). This follow-up group also had self-reported questionnaire data on significant life events and coping styles in response to stress. We hypothesised that neuroticism would also be independently predictive of suicidal ideation in this group, when adjusted for depressive disorder, significant life events and other significant demographic factors. We also aimed to ascertain the relationships on suicidal ideation of coping styles, particularly those correlated with neuroticism.

Methods

Cohorts

Generation Scotland:Scottish Family Health Study (GS:SFHS) is a population- and family based epidemiological adult (age 18+) cohort recruited February 2006–March 2011, which has been described elsewhere [80, 81]. GS:SFHS had a higher proportion of females (59%) and was of older age (mean 49 males, 49 females) compared to the Scottish population (mean 37 males, 39 females, 2001 census) [80]. GS:SFHS participants were typically healthier and more affluent that the general Scottish population, nevertheless 32.9% of individuals lived in areas with worse than average socioeconomic deprivation [80]. 99% of the study group was of white ethnicity (Scottish population 98%). Sociodemographic information on age, sex, educational attainment and relationship status were collected by questionnaire on enrolment.

Neuroticism was measured using the Eysenck Personality Questionnaire-Revised Short Form (EPQ-SF) [30]. The neuroticism subsection of the EPQ-SF consists of 12 ‘Yes/No’ questions (e.g., ‘Are you a worrier?’). Scores range from 0 to 12, with higher scores indicating greater neuroticism. This scale has been concurrently validated with other quantitative measures of neuroticism [39] and has high-reported reliability (α-coefficients 0.85–0.88) [30].

Trained researchers elicited lifetime history of major depressive disorder (MDD) using the screening questions from the Structured Clinical Interview for DSM-IV Disorders [80] and, if either screening question was positive, going on to administer the mood sections of the SCID. The screening questions were: “Have you ever seen anyone for emotional or psychiatric problems?” and “Was there ever a time when you, or someone else, thought you should see someone because of the way you were feeling or acting?”. A diagnosis of MDD was made according to DSM-IV criteria and all interviews were conducted by a trained researcher (2011 cases identified, 12.7% of cohort). Individuals with a history of bipolar disorder were excluded.

Cognitive testing included the digit symbol substitution test from the Wechsler Adult Intelligence Scale III [91], logical memory from the Wechsler Memory Scale III [92] and verbal fluency [58]. From these tests, a measure of cognitive function (g) was derived as the first unrotated principal component, explaining 44% of the variance in scores [59]. Socioeconomic deprivation was determined using the Scottish Index of Multiple Deprivation 2009 (SIMD) [77]. This measure employs 6976 geographical area-based data-zones across Scotland which are then ranked in order of deprivation, ascertained through weighted scores in seven domains including employment, education, health, housing and crime, with data-zone 1 the most deprived and 6976 the least deprived.

Identification of self-harm

All Scottish citizens registered with a general practitioner are assigned a unique identifier, the Community Health Index (CHI). This was used to deterministically record-link GS:SFHS participants to the Scottish Morbidity Records to obtain information about hospital admissions (SMR01) and psychiatric hospital admissions (SMR04) associated with self-harm. Written informed consent was obtained from 98% of GS:SFHS and only those who consented were linked. Self-harm cases were identified by matching to admissions codes with E950–E959 (ICD-9) or X60–X84, Z915, E98 and Y1–Y3 (ICD-10) [5]. Scottish NHS data on mortality was also linked, to exclude any GS:SFHS participants who died during follow-up.

Recontact group and identification of suicidal ideation in GS:SFHS

In 2014, GS:SFHS participants were recontacted for a follow-up assessment of mental health [66]. Suicidal Ideation was elicited using two questions from the General Health Questionnaire-28 [38]. Participants were asked “During the past few weeks…Have you thought of the possibility you might make away with yourself?” and “Have you found the idea of taking your own life kept coming into your mind?”. Participants who answered ‘Definitely have’ or ‘Has crossed my mind’ to either question were defined as suicidal ideation cases [n = 3503, cases = 158 (4.7%)].

Stressful life events were ascertained using the List of Threatening Experiences (LTE), whereby respondents self-reported their experiences from a list of 12 common threatening life events, occurring in the preceding 6 months [10, 11]. Examples of LTE include “Serious injury or assault to yourself”, “Made redundant or sacked from job” and “marital difficulties or break off of a steady relationship” (for full list see Supplementary Materials). For each event endorsed, contextual threat was rated on a scale from 3 (“very bad”) to 1 (“not too bad”). The LTE has demonstrated high test–retest reliability and good agreement with informant information (Cohen’s κ 0.63–0.90) [11].

Coping styles were elicited using the Coping Inventory for Stressful Situations (CISS) [28]; Cosway, [21], a 48 item self-report questionnaire enabling responders to rate on a 5-point scale their engagement in coping styles in response to stress, including task-, avoidance- and emotion-oriented coping. The CISS shows robust validity and reliability (alpha reliability coefficients (Cronbach’s alpha) of 0.82–0.90 for the main factors) [21]. History of MDD was re-ascertained using the Composite International Diagnostic Interview-Short Form (CIDI-SF) self-report questionnaire [55], with 605 cases identified (18.1% of sample). Bipolar disorder cases were excluded. Unlike the main GS:SFHS cohort, only one member from each family was analysed (i.e., unrelated sample).

UKB

UK Biobank is a population-based cohort of adults aged 40–69 years recruited across the UK from 2006 to 2010, which has been described elsewhere [85]. During baseline assessment [82] participants provided sociodemographic information via a touch-screen questionnaire, including educational attainment and whether they lived as a singleton or couple. This study included a subset of 35,227 (7.0%) of UKB with complete case information for the variables of interest. Individuals in UKB who were also present in GS:SFHS (n = 201) were excluded.

Self-harm was ascertained through the touch-screen questionnaire. Participants were asked “Have you deliberately harmed yourself, whether or not you meant to end your life?”. A follow-up question enquired “Following any time when you took an overdose or deliberately tried to harm yourself did you (tick all that apply)”. Participants who ticked “see anyone from psychiatric or mental health services, including liaison services” and/or “need hospital treatment (e.g., A&E)” were included as cases in this study (772 cases, 2.2% of sample). The other answers, which were not included as cases, were “use a helpline”, “see own GP”, “receive help from friends/family” and “prefer not to answer”.

Neuroticism was assessed using the EPQ-SF [30], administered via the touch-screen questionnaire. Lifetime history of depression was ascertained by touch-screen questionnaire relating to lifetime experience of depressive symptoms and contact with mental health services [82].

Cognitive testing was administered via three touch-screen tests: (1) a symbol matching task over 12 trials (reaction time); (2) 13 logic/reasoning questions over 2 min (verbal-numerical reasoning); (3) card pair matching task (visuo-spatial memory). From these tests a single measure of cognitive ability (g) was extracted as the first unrotated principal component, explaining 42% of the variance.

Socioeconomic deprivation was measured via the Townsend Deprivation Index, a census-based measure incorporating unemployment, non-home ownership, household overcrowding and non-car ownership [50]. Each small postcode-based geographical area is assigned a Townsend Score, with zero indicating mean deprivation, negative scores indicating relative affluence, and positive scores indicating relative deprivation.

Statistical analysis

All analyses were carried out using R version 3.2.3. Complete case analysis was employed in both cohorts (see Supplementary Table S4 for analysis of complete case versus whole-cohort variables). Generalised linear models with logit-link function (logistic regression) were used to identify predictors of self-harm in UK Biobank. In the GS:SFHS self-harm study, additional adjustment for inter-relatedness of the family-based cohort was performed using a Bayesian mixed model approach, with pedigree fitted as a random effect, using an inverse pedigree matrix within the R package MCMCglmm. This implements a Markov Chain Monto Carlo estimator, with a “threshold” family probit link function which produces similar results to a logit function, optimised to pedigree based mixed effects models. In the GS:SFHS and UKB multivariable analyses of hospital-treated self-harm, predictor variables are reported unstandardized.

In the GS:SFHS suicidal ideation follow-up study, an unrelated sample was used and multivariable logistic regression was employed. In this analysis, continuous variables were scaled to have a mean of zero and standard deviation of one, to facilitate interpretation of the CISS and LTE predictor variables. During fitting of models, interaction terms for neuroticism and depression, and neuroticism and coping styles, were tested to investigate potential moderation on neuroticism.

Coefficients were expressed as odds ratios with 95% credible intervals and 95% confidence intervals as applicable. p values were reported after False Discovery Rate adjustment [7]. Group differences between numeric variables were ascertained using Cohen’s t test and Cohen’s d measure of effect size, and differences between proportions were assessed using z test and Cohen’s h. For all experiments, we have reported all measures, conditions, data exclusions and the determination of sample sizes and further information is available in Supplementary Table S4.

Results

GS:SFHS

As presented in Table 1, there were 339 (2.1%) GS:SFHS individuals identified with previous self-harm requiring hospital admission. Self-harm cases were slightly younger (mean age 44.7 versus 47.1, p = 0.0005, Cohen’s d = 0.16), predominantly female (66.7% versus 58.4%, p 0.002, Cohen’s h = 0.17), with lower mean cognitive ability scores, greater prevalence of depression history (47.5% versus 12%, p < 2.2 × 10−16, h = 0.81) and with higher mean neuroticism (mean 6.4 versus 3.7, p < 2.2 × 10−16, d = 0.89). Self-harm cases were more likely to be from more deprived areas as measured by SIMD (mean 1964 versus 1823, p < 2.2 × 10−16, d = 0.58). The proportion of graduates was lower in self-harm cases (17.1% versus 33.9%, p < 10.0 × 10−11, h = 0.39). A greater proportion of self-harm cases reporting being single (51.9% versus 31.7%, p < 3.6 × 10−15, h = 0.31).

Table 1 Socio-demographic, clinical and cognitive characteristics of GS:SFHS (N = 15798) and UK Biobank (N = 35227) cohorts used in this study
Full size table

The most predictive factor for previous self-harm (Table 2) was history of major depressive disorder [OR 5.6 95% credible interval (CI) 3.5–8.9, pFDR 0.0004]. Neuroticism was positively associated with self-harm risk by an odds ratio of 1.2 (95% CI 1.1–1.2, pFDR = 0.0003) per EPQ-SF unit. No significant interaction terms were found during model fitting. The significant effects of neuroticism were found in both male-only and female-only combined models (see Supplementary Table S1). Figure 1 displays the increased risk of self-harm per unit of EPQ-SF neuroticism score predicted by our model for both cohorts.

Table 2 Multivariable analysis of predictors of history of self-harm involving hospital/psychiatric treatment in GS:SFHS and UKB [comparison made to any reported history of self-harm in UKB (*)]
Full size table
Fig. 1
figure 1

Predicted risk of self-harm from the multivariable models in GS:SFHS and UKB for different EPQ-SF neuroticism scores

Full size image

The age groups 25–34, 35–44, and 45–54 were positively associated with self-harm whereas age groups 64–74 and 75+ were negatively associated, compared to the reference category of 55–64. Gender did not show a significant association in the combined model. Having a higher SIMD score (less deprived) was associated with decreased risk of self-harm (per quintile unit OR 0.8 95% CI 0.7–0.9, pFDR = 0.0004). Having no qualifications and being single increased risk. Cognitive ability showed an inverse association with self-harm (per unit OR 0.8; 95% CI 0.7–0.9, pFDR = 0.0005).

UKB

There were 772 (2.2%) individuals self-reporting self-harm requiring hospital or psychiatrist review in UKB (Table 1). Self-harm cases were slightly younger (UKB’s minimum age is 40), predominantly female (70.5% versus 54.0%, p < 2.2 × 10−16, h = 0.34), and with higher mean neuroticism (mean 5.6 versus 3.3, p < 2.2 × 10−16,d = 0.83) and higher prevalence of history of depression (90.5% versus 33.3%, p < 2.2 × 10−18, h = 1.3). Cognitive ability scores were not significantly different for any of the tests. Self-harm cases were more likely to be from deprived areas (more positive scores) as measured by the Townsend index (mean − 0.5 versus − 1.7, p < 2.2 × 10−16, d = 0.44). Educational attainment was not significantly different between the two groups (χ = 7.43, p value 0.28). The proportion of the self-harm group living as single was 39.6% versus 22.5% for those reporting no history of self-harm.

The most predictive factor in the multivariable logistic model was self-reported history of depression (Table 2, OR 12.7 95% confidence interval (CI) 9.9–16.4, pFDR 5.4 × 10−86). The odds of self-harm were significantly positively associated with increasing neuroticism scores, OR 1.1 95% CI 1.1–1.2, pFDR 9.8 × 10−17 per EPQ-SF unit. No significant interactions were found during model fitting. The significant effects of neuroticism were found in both the male-only and female-only models (Supplementary Table S1).

Being female was also associated with somewhat higher risk (OR 1.3 95% CI 1.1–1.5, pFDR 0.005), as was being in the 35–44 and 45–54 age groups, whereas the 65–74 age group was protective. No educational factors were significant in the multivariable analysis. Being single and higher Townsend scores (more deprived) were associated with higher odds of self-harm. Cognitive ability (g) was not significantly associated (pFDR = 0.051, OR 1.1, 95% CI 1.0–1.2).

In Table 2, comparison is also made to UKB participants who self-reported any self-harm, irrespective of whether hospital attention was sought (1578 cases, 4.4%). In this group self-harm was also positively associated with neuroticism scores (OR 1.1, 95% CI 1.1–1.2, pFDR 3.4 × 10−41 per EPQ-SF unit). Positive association was also found for history of depression, being female, younger age group, increasing Townsend deprivation score, and being single. However, in this group increasing cognitive function score increased odds of self-harm (OR 1.1, 95% CI 1.0–1.1, pFDR 0.004 per unit g).

GS:SFHS suicidal ideation recontact study

In the GS:SFHS recontact study (N = 3342) there were 158 individuals with self-reported suicidal ideation(4.7%) (Table 3). Of these 21 (13.3%) had a record-linkage based history of self-harm compared to 1.9% in the control group. History of self-harm was the most predictive factor for suicidal ideation in the multivariable model (OR 3.5, 95% CI 1.9–6.2, pFDR 7.5 × 10−5) followed by history of depression (OR 3.2, 2.3–4.7, pFDR = 7.8 × 10−10). Scores in the List of Threatening Experiences increased odds of suicidal ideation (1.3, 1.2–1.5, pFDR = 2.4 × 10−5 per standard deviation unit).

Table 3 Multivariable analysis of predictors of history of suicidal ideation in GS:SFHS Re-Contact Study (N = 3342)
Full size table

Neuroticism was positively associated with suicidal ideation in the multivariable model (OR 1.6, 1.3–1.8, pFDR = 5.8 × 10−8 per standard deviation unit). However, this association attenuated to non-significant OR 1.1 (0.9–1.4, p = 0.44) when coping styles were added to the model (Table 3). In the full multivariable model including coping styles, EoC was positively associated with suicidal ideation (OR 2.4, 1.9–3.0, pFDR = 1.7 × 10−12) and ToC was negatively associated (OR 0.8, 0.8–0.9, pFDR = 0.03), while AoC was not significantly associated. The correlation matrix revealed that EoC and neuroticism were significantly correlated, r = 0.50 p < 2.2 × 10−16 and task-oriented coping were moderately negatively correlated [r = − 0.18 p < 2.2 × 10−16, Table S2 (Supplementary)]. In moderation analysis no significant (p ≤ 0.05) interaction terms were found for neuroticism*ToC, neuroticism*AoC or neuroticism*EoC on suicidal ideation, controlled for age, sex and depression status.

Discussion

Here we report a significant independent association between neuroticism and history of self-harm requiring medical attention in two large population-based cohorts, using both self-reported and record-linkage derived measures of self-harm. This finding remained significant when controlling for history of depression, socioeconomic deprivation, educational attainment and relationship status.

In both UKB and GS:SFHS we found that history of depression was the predictor with largest effect size on hospital-treated self-harm risk. In our multivariable models, predicted self-harm risk (Fig. 1) was relatively low in UKB in non-depressed individuals even at higher neuroticism scores, whereas in GS:SFHS more neurotic non-depressed cases also had significant overall risk. This disparity may be explained by the use of self-reported depression in UKB, with broader inclusion criteria than GS:SFHS (which employed the objectively assessed SCID). Thus, 90.5% of self-harm cases reported history of depression in UKB, versus 47.5% in GS:SFHS (Table 1).

We found a significant protective relationship for higher cognitive scores against self-harm in GS:SFHS, but not in UKB. Previous studies have found that cognitive impairment is associated with suicide and self-harm [1, 5, 40, 51, 83]. However, other studies have found increased cognitive scores may increase self-harm risk [3, 13]. One explanation for the discrepancy in our results is that different measures of cognitive ability were used in the two cohorts (Table 1). Moreover, previous research on depression and cognitive ability in GS:SFHS and UKB [65] has been similarly inconclusive, with an association between g and depression being identified in GS:SFHS but not UKB. In terms of education attainment, we found fewer graduates and more individuals without qualifications in self-harm cases in GS:SFHS, but this difference was not significant in UKB. This might be accounted for in population sampling differences between GS:SFHS and UKB, with the latter having more graduates among controls also (Table 1).

We found socioeconomic deprivation was significantly associated with self-harm history in both cohorts, as was living as a singleton. Female gender was not predictive of self-harm in GS:SFHS but was significantly associated in UKB, albeit with modest effect size (Table 2). Previous multi-centre studies have shown female rates of self-harm to be significantly higher than male [76]. However, our GS:SFHS analysis was for hospital inpatient admitted self-harm and it may be that in this subgroup female gender is less predictive of risk, given that hospital-treated self-harm arguably lies on a spectrum between non-serious self-harm and suicide, the latter of which is four times more common in males [60].

In our follow-up analysis of suicidal ideation, we found an independent association between neuroticism and self-reported suicidal ideation, which remained significant when controlled for history of depression, socioeconomic deprivation and significant life events. When coping styles were added to the model, the association with suicidal ideation was no longer significant, implying that neuroticism’s effect is not independent of coping style. We showed that emotion-orientated coping is highly positively correlated with neuroticism (r = 0.50) and task-orientated coping negatively correlated (r = − 0.18). In addition, we found that emotion-oriented coping was positively associated with suicidal ideation whereas task-oriented coping was negatively associated. This relationship was also found in a study of suicidal ideation in middle-aged workers in Japan, albeit without employing a validated coping style instrument [86]. A further study found emotion-focused coping, but not problem-focused coping, was associated with suicidal ideation in adolescents [47]. “Active” (task-oriented) coping and positive reinterpretation were also associated with lower suicidality, adjusted for depression, in a study of 500 college students [14].

Study strengths and limitations

This study had a number of strengths for establishing the association of neuroticism to hospital-treated self-harm. We have employed two large, population-based cohorts which both have phenotypic information for major covariates of self-harm, allowing comparison between the groups while both using the same EPQ-SF measure of neuroticism. By utilising self-report in one cohort, and health data record-linkage in the other, our study design obviates some of the biases which can arise from utilising either method alone. GS:SFHS encompasses the range of adult age groups, and UKB focuses on middle-age to older adults, thus our findings are a significant contribution to self-harm research where many of the available studies are for teenagers or young adults. By extending our analysis to suicidal ideation, we were also able to demonstrate an association with neuroticism and correlated coping styles (emotion- and task-oriented coping), the latter of which are potentially modifiable by clinical intervention.

There are also some important limitations to our work. The cohorts we use are population-based, but are not fully representative, as UKB includes adults of ages 40–69 and GS:SFHS has an older mean age that the Scottish population. Additionally, the use of GP registration as an inclusion criteria for our GS:SFHS study (by enabling record-linkage via CHI number) leads to potential selection bias in our identification of self-harming individuals, although in the UK 96% of individuals are registered with a GP [80] indicating that such biases are likely to be small. The prevalence of self-harm we record should thus be used with caution and should not be taken as a reliable population estimate. Nevertheless, it is sobering that prevalence of hospital-treated self-harm was relatively high (2.1% for GS:SFHS and 2.2% for UKB). Since self-harm is more common in younger people, the true population prevalence is likely to be greater still. We have also adopted a cross-sectional design and thus causality between factors such as neuroticism and self-harm; and neuroticism, coping style and suicidal ideation; is suggested rather than conclusively demonstrated by our models.

The type of self-harm we have studied is self-harm involving hospital care. We used a general definition of self-harm as the data available to us did not allow distinction between nonsuicidal self-injury and suicide attempts, as this information is not available in the routinely collected administrative hospital data linked to in GS:SFHS (and was not part of the self-report question in UKB). This could limit the transferability of our results to other studies, although as discussed, the extent to which such distinctions of suicidal intent can be accurately made in practise is controversial.

In GS:SFHS we defined self-harm cases via admission to medical or psychiatric hospital, as ascertained by record-linkage. We therefore have not included a number of self-harm cases that were managed in the Emergency Department, where available data is incomplete [61]. This represents approximately 50% of self-harm cases presenting to hospital, although there are wide variations between hospitals [20]. A recent study has found that routine hospital data underestimates rates of self-harm by approximately 60% compared to combined survey-hospital database methods [15], as—for example—self-harm which is assessed in the Emergency Department, but which does not lead to hospital admission, may not be included. However, hospital admission self-harm is itself an important variable, as cases that are admitted are likely to be more serious and can therefore be expected to be of greater risk of further self-harm and completed suicide [37]. The UKB self-report variable was for self-harm requiring any hospital or psychiatric management (including Emergency Department) and therefore, while highly correlated with the GS:SFHS variable, was more general in its scope. The overall prevalence of self-harm in GS:SFHS and UKB was similar (2.1% and 2.2%, respectively). This might seem surprising as one might expect the more general self-harm definition in UKB to return a higher prevalence. This could be explained by the fact that the UKB cohort had no individuals younger than 40 and this has decreased the overall self-harm prevalence, since younger age groups are at relatively higher risk.

We employed a complete case design in our multivariable analyses in GS:SFHS and UKB. Potentially, this could have biased our results compared to the whole samples, although comparison (Supplementary Table S4) indicated that there were no significant and large-effect differences in major variables studied through the complete case approach. Nevertheless, this method could have introduced biases in ways we did not measure.

In summary, our findings must be seen in the context of self-harm with a high propensity to cause physical harm warranting medical attention. However, the UKB cohort did include a variable for any self-harm regardless of hospital attendance and we also included this multivariable analysis (Table 2). Neuroticism was found to be associated in this group also, with similar effect size (OR 1.1, 95 CI 1.1–1.2, p 3.4 × 10−41 per EPQ-SF unit).

With regard to our analysis of suicidal ideation and coping-style, neuroticism as a trait was measured during GS:SFHS enrolment, which was some years before the recontact when coping style and suicidal ideation were measured. However, as discussed, neuroticism is considered to be a relatively stable trait and would not be expected to change significantly over this time period. We also controlled neuroticism by age at enrolment rather than age at recontact within the models. Another important consideration is the extent to which neuroticism and emotion-oriented coping are separate constructs or both emanant from innate responses to stress. While we found the correlation of neuroticism and EoC to be significant (0.5), it was evidently not complete. There is also evidence that coping style is amenable to clinical treatment in prevention of suicide [36], whereas personality traits are understood as more therapeutically static.

Conclusion and implications for practice

We have found that a questionnaire which is relatively quick to administer in a clinical setting, the EPQ-SF, is significantly independently predictive of self-harm and suicidal ideation when adjusted for multiple other significant factors, including history of depression. Neuroticism is, therefore, an important factor which should be included in future studies of self-harm and suicidality risk.

Our research also implies a potential role for cognitive-behavioural therapies focused on decreasing emotion-oriented coping and increasing adaptive task-oriented coping in individuals with suicidal ideation. There is current limited research in this area, although previous studies are encouraging [25, 26]. The coping styles questionnaires are also relatively straightforward to administer clinically and our study suggests that greater attention to reducing emotion-orientated coping, and reinforcing task-oriented coping strategies, in individuals presenting with suicidal ideation is likely to have a beneficial effect in protecting against self-harm.

We also demonstrate the utility of record-linkage to health data for examining research variables such as self-harm, where there may be an unwillingness to self-report caseness, but a willingness to provide consent for anonymised data linkage. Such record-linked cohort studies provide an important new avenue for future research on self-harm and psychiatric illness.