Adolescent Research Review

, Volume 3, Issue 2, pp 193–217 | Cite as

Bullying and Suicidality in Children and Adolescents Without Predisposing Factors: A Systematic Review and Meta-analysis

  • George N. Katsaras
  • Evridiki K. Vouloumanou
  • Georgia Kourlaba
  • Eleni Kyritsi
  • Eleni Evagelou
  • Chryssa Bakoula
Quantitative Review
  • 367 Downloads

Abstract

Published evidence has suggested that engaging in school or cyber bullying may potentially be associated with a suicidal ideation and suicide attempts. The aim of our review/meta-analysis was to evaluate the potential association between school and cyber bullying and suicidality (including suicidal ideation, planning and/or committing a suicide attempt) in children and adolescents (< 19 years old) who are considered as a “healthy” population, without predispositions for suicidality factors (not subpopulations with characteristics that may constitute proneness to bullying and its consequences, including sexual minorities, drug users and youth with psychiatric comorbidity). Regarding school bullying, victims and bullies independently, and victims and bullies together, were significantly more likely to present suicidal ideation and commit a suicide attempt, compared to non-involved participants. Victims of school bullying were found to be significantly more likely to commit a suicide attempt that required medical treatment. Victims of cyber bullying were significantly more likely to present suicidal ideation and commit a suicide attempt. A positive relationship between involvement in both school bullying and cyber-bullying with suicidal ideation and suicidal behavior was observed. This review/meta-analysis contributes to further understanding bullying and suicidality as it includes results of participants without any predisposing factors for suicidality, thus providing more clear results with regard to the magnitude of the effects of both school and cyber bullying on suicidality.

Keywords

School bullying Cyber bullying Suicidal ideation Suicide attempt 

Introduction

Existing literature provides a plethora of evidence regarding increasing rates of school bullying as well as the emerging type of cyber bullying. Specifically, Indicators of School Crime and Safety of 2011 suggested that more than 28% of adolescents 12–18 years of age (girls 31% and boys 25%) have been a victim of school bullying (Finkelhor et al. 2013). Yet, the prevalence of cyber bullying seems to be more difficult to be estimated, as evidence from relevant studies ranges between 4 and 72% (Juvonen and Gross 2008; Kowalski and Limber 2007). The above alarming evidence suggests the need to evaluate the adverse consequences of both school and cyber bullying on adolescents’ physical and mental health (American Academy of and Adolescent 2001; van Heeringen 2012).

In particular, the interest of both society and research increasingly is focusing on the potential association between school and cyber bullying and suicidality (including suicidal ideation, planning and/or committing a suicide attempt) in adolescents (Hertz et al. 2013; Kim and Leventhal 2008). Studies (including relevant reviews) have suggested that bullies, as well as victims and bullies as a whole, are at an increased risk of presenting suicidal ideation and suicidal behavior (planning and/or committing a suicide attempt) (Winsper et al. 2012). Yet, other studies have suggested that victims independently are also at increased risk to present the above-mentioned outcomes (Hepburn et al. 2012). In victims, the association of bullying and suicidality seems to be intensified by parental internalizing disorders (Herba et al. 2008), while it does not seem to differentiate when other types of abuse are present (Meltzer et al. 2011).

On the other hand, other factors including drug abuse, violent behavior (Litwiller and Brausch 2013) and depression (Bauman et al. 2013) have been reported to intensify the association. An exposure of long duration to school bullying also has been associated with increased risk of self-destructive behaviors (Fisher et al. 2012). With regard to sex differences, published evidence is rather conflicting, as the findings of some studies suggest that girl victims have an increased risk to commit a suicide attempt (Klomek et al. 2009) while others have reported an increased risk for boy victims (Laukkanen et al. 2005). As far as cyber bullying is concerned, evidence from relevant reviews suggests that it is associated with suicidal ideation and suicide attempts (Hay and Meldrum 2010; Litwiller and Brausch 2013). Of note, it has been suggested that this association is stronger compared to the observed association between traditional school bullying and suicidal ideation/suicide attempts (Hinduja and Patchin 2010).

The Current Study

Taking into consideration the clinical, social and psychosocial importance of the above-mentioned findings, along with the observed variability of the currently available published evidence, the need to systematically assess this particular sensitive association seems warranted. In this regard, this study sought to investigate, evaluate, as well as quantify possible associations between school and cyber bullying with the presence of suicidal ideation and/or suicidal attempts in children and adolescents that do not belong in subpopulations with specific characteristics that may be considered as possible predisposing factors for suicidality (including psychiatric patients, sexual minorities). The study does so by performing a systematic review and meta-analysis of the currently available published evidence.

Methods

Design

The current study was performed according to the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements (Moher et al. 2009). Two independent reviewers performed the literature search as well as the evaluation of the identified studies to determine the eligibility for inclusion. A third reviewer was included i to resolve any possible problems that derived from the above-mentioned process.

Data Sources

The studies included in the current systematic review/meta-analysis were retrieved from an independent literature search performed in the PubMed and Scopus databases. Independent keywords along with their combinations were applied to both databases. The specific keywords were the following: “bullying”, “harassment”, “teasing”, “mobbing”, “intimidation”, “aggressive behavior”, and “suic*”. The above-mentioned keywords were generated through evaluation of the MeSH (Medical Subject Headings) database. Specifically, the search string applied to PubMed was: (bullying[Title/Abstract]) OR (harassment[Title/Abstract]) OR (teasing[Title/Abstract]) OR (mobbing[Title/Abstract]) OR (intimidation[Title/Abstract]) OR (aggressive behavior[Title/Abstract]) AND suic*[Title/Abstract]. The search string applied to the Scopus database was: (bullying AND suic* OR harassment AND suic* OR teasing AND suic* OR mobbing AND suic* OR intimidation AND suic* OR aggressive behavior AND suic*).

Study Selection Criteria

In order to be included in the current review/meta-analysis, a study needed to adhere to specific inclusion and exclusion criteria. Particularly, a study should have provided data regarding any association between school bullying and/or cyber bullying and the presence of suicidal ideation and/or suicide attempts in either children or adolescents (≤ 19 years old) without predisposing factors for suicidality. Specifically, study populations defined as “healthy subjects” included students who were not involved in subpopulations with characteristics that may constitute their being prone to bullying and its consequences, including sexual minorities, drug users and youth with psychiatric comorbidity. Prior reviews and meta-analyses that focused on a similar subject were not considered as eligible for inclusion. Moreover, studies involving subpopulations with specific characteristics that could be regarded as possible confounding factors, including psychiatric patients, prisoners or sexual minorities were also excluded. Time or country of origin restrictions were not applied during the identification of eligible studies, whereas studies that were published in languages other than English were not considered as eligible for inclusion. For the purpose of this systematic review and meta-analysis, “suicidality” was defined as suicidal ideation (defined as seriously considering suicide), planning a suicide attempt, and committing a suicide attempt that may or may not have required medical treatment.

Data Extraction

Two authors independently reviewed the full texts of all studies that were considered eligible for inclusion and extracted the individual study data. Any discrepancies were resolved by discussion with a third author to reach a final consensus. Specifically, the data extracted included the following: study characteristics (first author, year of publication, country of origin, study design), study population characteristics (age distribution), type of statistical analysis performed, individual study outcomes (suicidal ideation, planning of suicide attempts, suicide attempts, suicide attempts that required medical treatment, time of outcomes’ assessment), potential confounding factors (age, sex, sexual life, self-perception, drug abuse, any kind of abuse, domestic violence, problematic parenting, any kind of misbehavior, psychiatric co-morbidity, academic performance, academic degree, parents’ occupation, financial difficulties in the family).

Data Analysis

A systematic review of the studies that were regarded as eligible for inclusion was performed. Additionally, the studies that provided adequate data that enabled statistical comparisons were also included in a meta-analysis for specific outcomes.

All statistical analyses were performed with the use of the STATA software.(Jonathan AC Sterne et al. 2001) The I2 test was used to assess statistical heterogeneity among the analyzed studies. In cases of I2 > 75%, a random effects model was used (Higgins et al. 2002). Pooled odds ratios (ORs) along with 95% confidence intervals were calculated. The weight of the effect of each of the analyzed studies was assessed in the total effect size. By excluding each one of the analyzed studies at a time, we managed to identify sources of statistical heterogeneity. The publication bias was assessed with the use of the Egger test (Sterne et al. 2001). A p value of < 0.05 denoted statistically significant publication bias. Funnel plots were performed in order to depict publication bias (Sterne and Egger 2001).

Results

Our search procedure in PubMed and Scopus databases generated a total of 1.573 studies. After detailed screening, 22 studies were regarded as eligible for inclusion in the current systematic review/meta-analysis (Abell et al. 2012; Bannink et al. 2014; Bhatta et al. 2014; Brunstein Klomek et al. 2007; Cui et al. 2011; Elgar et al. 2014; Heikkila et al. 2013; Hepburn et al. 2012; Kaltiala-Heino et al. 1999; Kelly et al. 2015; Kim et al. 2005; Mark et al. 2013; Messias et al. 2014; Owusu et al. 2011; Rudatsikira et al. 2007; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012; Skapinakis et al. 2011; Turner et al. 2012; Wang et al. 2011; Winsper et al. 2012; Yen et al. 2014). The detailed process of study selection is depicted in Fig. 1, which presents the Flow Diagram.

Fig. 1

PRISMA results

Study Characteristics

A total of 22 studies were included in our review/meta-analysis. Seventeen of the 22 included studies referred to school bullying, whereas the remaining 5 studies referred to both school and cyber bullying. With regard to the country of origin, most of the included studies (7/22) were from the USA (Bhatta et al. 2014; Brunstein Klomek et al. 2007; Elgar et al. 2014; Hepburn et al. 2012; Messias et al. 2014; Schneider et al. 2012; Turner et al. 2012) and Europe (6/22) (Bannink et al. 2014; Heikkila et al. 2013; Kaltiala-Heino et al. 1999; Mark et al. 2013; Skapinakis et al. 2011; Winsper et al. 2012). The remaining studies were performed in various Asian countries, African countries, countries of Oceania and others (Abell et al. 2012; Cui et al. 2011; Kelly et al. 2015; Kim et al. 2005; Owusu et al. 2011; Rudatsikira et al. 2007; Sampasa-Kanyinga et al. 2014; Wang et al. 2011; Yen et al. 2014). Detailed data are presented in Table 1. Regarding the study design, the majority (18/22) were cross-sectional surveys (Abell et al. 2012; Bhatta et al. 2014; Brunstein Klomek et al. 2007; Cui et al. 2011; Elgar et al. 2014; Hepburn et al. 2012; Kaltiala-Heino et al. 1999; Kelly et al. 2015; Kim et al. 2005; Mark et al. 2013; Messias et al. 2014; Owusu et al. 2011; Rudatsikira et al. 2007; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012; Skapinakis et al. 2011; Wang et al. 2011; Yen et al. 2014); 3/22 were longitudinal studies (Bannink et al. 2014; Heikkila et al. 2013; Turner et al. 2012) and the 1 remaining was a cohort study (Winsper et al. 2012). Fifteen of the 22 included studies took under consideration potential confounding factors (mainly age, sex and nationality; see Bannink et al. 2014; Bhatta et al. 2014; Brunstein Klomek et al. 2007; Heikkila et al. 2013; Hepburn et al. 2012; Kaltiala-Heino et al. 1999; Kelly et al. 2015; Kim et al. 2005; Messias et al. 2014; Owusu et al. 2011; Rudatsikira et al. 2007; Schneider et al. 2012; Skapinakis et al. 2011; Winsper et al. 2012; Yen et al. 2014). Six of these studies evaluated additional confounding factors including family conditions, psychiatric comorbidity, drug abuse and prior history of bullying or abuse (Bannink et al. 2014; Bhatta et al. 2014; Heikkila et al. 2013; Hepburn et al. 2012; Skapinakis et al. 2011; Winsper et al. 2012). Detailed data are presented in Table 1.

Table 1

Characteristics of the 22 included studies

References

Clinical setting

Tool of suicidality assessment

Evaluated potential confounding factors

Country

Study design

Suicidality assessment—non-specified time period

 Bhatta et al. (2014)

USA

Cross-sectional survey

CDC’s Middle School YRBS

Year of survey response, age, School class, nationality, physical fighting, self-perception, sexual life, smoking, alcohol, glue, marihuana, cocaine

 Yen et al. (2014)a

Taiwan

Cross-sectional survey

K-SADS

Age, Sex

 Hepburn et al. (2012)

USA

Cross-sectional survey

2008 Boston Youth Survey

Sex, Race, School class

Suicidality assessment—the last year

 Sampasa-Kanyinga et al. (2014)

Canada

Cross-sectional survey

2009 CDCYRBS

NR

 Messias et al. (2014)

USA

Cross-sectional survey

2011 CDC YRBS

Age, sex, race

 Bannink et al. (2014)

Holland

Longitudinal study

NR

Model 1: socioeconomical status, vicitimization

Model 2: Model 1 + suicidal ideation on 1st assessment

Model 3: Model 2 + sex, traditional bullying

Model 4: Model 2 + sex, cyber bullying

 Mark et al. (2013)

Estonia

Lithuania

Luxembourg

Cross-sectional survey

NR

NR

 Schneider et al. (2012)

USA

Cross-sectional survey

2007 CDCYRBS

Massachusetts YRBS

Sex, grade, race/ethnicity, and sexual orientation, self-reported school performance, and school attachment and psychological distress indicators

 Wang et al. (2011)

Taiwan

Cross-sectional survey

CDC YRBS

NR

 Owusu et al. (2011)

Ghana

Cross-sectional survey

2008 GhanaGSHS

Age, sex, school class

 Cui et al. (2011)

China

Cross-sectional survey

GSHS

NR

 Rudatsikira et al. (2007)

Uganda

Cross-sectional survey

Uganda GSHS

Age, sex, loneliness, anxiety, smoking

Suicidality assessment—last 6 months

 Kelly et al. (2015)

Australia

Cross-sectional survey

BSI

Sex

Suicidality assessment—last month

 Elgar et al. (2014)

USA

Cross-sectional survey

NR

NR

 Abel et al. (2012)

Jamaica

Cross-sectional survey

NR

NR

 Turner et al. (2012)

USA

Longitudinal study

TSCC

NR

 Brunstein-Klomek et al. (2007)

USA

Cross-sectional survey

SIQ-JR

Type of School, school class, sex

Suicidality assessment—last week

 Skapinakis et al. (2011)

Greece

Cross-sectional survey

Epirus School Project CIS-R

Model 1: age, sex

Model 2: Model 1 + academic performance, academic status and parents’ occupation, relationship with parents, financial difficulties in the family

Model 3: Model 2 + psychiatric comorbidity

Model 4: Model 3 + concurrent bullying and victimization

Suicidality assessment—at the same time of the assessment

 Heikkila et al. (2013)

Finland

Longitudinal study

BDI

Model 1: age, sex

Model 2: Model 1 + depression

Model 3: Model 1 + signs of symptoms of externalization

Model 4: all the above-mentioned factors

 Winsper et al. (2012)

United Kingdom

Cohort study

ALSPAC

Model 1: age, sex

Model 2: Model 1 + abuse, domestic violence, maladaptive parenting

Model 3: Model 2 + disorder of negative feelings and behavior

 Kim et al. (2005)

South Korea

Cross-sectional survey

K-YSR

Signs of stress/depression, sex, family status, socio economical status, residency

 Kaltiala-Heino et al. (1999)

Finland

Cross-sectional survey

BDI

Age, sex

NR non-reported

aThis study evaluates both suicidal ideation and suicide attempts together as suicidality

Study Outcomes

The main outcomes evaluated in the involved studies included the assessment of suicidal ideation and suicidality (including suicidal ideation, planning, as well as committing suicide attempts). However, heterogeneity regarding the time of assessment of the above-mentioned outcomes was observed among the included studies. Specifically, the majority of the included studies (9/22) evaluated the study outcomes during a 12-month period prior to the study question (Bannink et al. 2014; Cui et al. 2011; Mark et al. 2013; Messias et al. 2014; Owusu et al. 2011; Rudatsikira et al. 2007; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012; Wang et al. 2011). The question derived from questionnaires including the 2009 and 2011 Center for Disease Control and Prevention (CDC) Youth Risk Behavior Surveillance (YRBS), the 2008 Ghana Global School-based Student Health Survey (GSHS) and the Uganda GSHS (The Clinical Interview Schedule-Revised). One of the 22 included studies assessed the study outcome during a 6-month period prior to the study question using the Brief Symptom Inventory (BSI) questionnaire (Kelly et al. 2015), whereas 1 study assessed outcomes during a 1-week period, using the Epirus School Project The Clinical Interview Schedule-Revised (CIS-R) (Skapinakis et al. 2011). Notably, all of the included studies used self-completion questionnaires and the most commonly used questionnaire was the CDC YRBS (in 5 of the 22 studies) (Bhatta et al. 2014; Messias et al. 2014; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012; Wang et al. 2011). Detailed data are provided in Table 1.

Systematic Review of the Included Studies

Association of School Bullying and Suicidal Ideation

All of the 22 included studies evaluated the potential association between school bullying and suicidal ideation. These studies suggested that victims of school bullying were 1.10–5.41 times more likely to present suicidal ideation compared to non-involved subjects. This finding was consistent with regard to bullies (0.56–4.7 times more frequent) (Hepburn et al. 2012; Kelly et al. 2015) and both bullies/victims (1.9–9.3 more frequent) (Heikkila et al. 2013; Kelly et al. 2015). In cases that additional cyber bullying was involved, victims were found to be 5.5 times more likely to present suicidal ideation compared to non-involved subjects (Messias et al. 2014). Three of the 22 included studies provided data evaluating the potential association between the duration of exposure to bullying and suicidal ideation. Specifically, these studies suggested that the victims with exposure of shorter duration (< 1 week) were 1.40–2.79 times more likely to present suicidal ideation compared to non-involved subjects, whereas the victims with exposure of long duration (frequent or all-time exposure) were 2.75–5.41 times more likely to present suicidal ideation compared to those non-involved subjects. Moreover, occasional bullies (< 1 week or sometimes) were 1.40–2.62 more likely to present suicidal ideation compared to non-involved subjects. This finding was more evident in frequent bullies (often or all-time): 3.44–4.00 times more likely (Brunstein Klomek et al. 2007; Cui et al. 2011; Kaltiala-Heino et al. 1999). Detailed data regarding the remaining evaluated studies are presented in Table 2.

Table 2

Characteristics and outcomes of the studies that evaluated the association of School Bullying and Suicidal Ideation

References

Study population characteristics

Follow up

Study main outcomes: Odds Ratios (ORs) vs non-involved subjects

Sample size

Age (years)

Kelly et al. (2015)

1.588

12–15

NR

Bullies (4.7, 95% CI 3.1–7.0)c

Victims (2.4, 95% CI 0.8–7.0)c

Bullies/ Victims (9.3, 95% CI 5.4–16.2)c

Bhatta et al. (2014)

1.082

11–15

NR

Victims (2.41, 95% CI 1.68–3.47)c

Victims boys (2.51, 95% CI 1.47–4.46)c

Victims girls (2.22.95% CI 1.34–3.68)c

Elgar et al. (2014)

18.834

12–18

NR

Bullies (1.05, 95% CI 1.02–1.08)b

Victims (1.10, 95% CI 1.06–1.14)b

Sampasa-Kanyinga et al. (2014)

2.999

12,5–16,1

NR

Victims (3.48, 95% CI 2.48–4.89)b

Messias et al. (2014)

15.425

9th–12th Graded

NR

Victims (2.6,95% CI 2.1–3.1)b (2.6,95% CI 2.1–3.1)c

Victims of school and cyber bullying (5.5.95% CI 4.5–6.8)b (5.3.95% CI 4.2–6.5)c

Bannink et al. (2014)

8.272

Mean: 12.5

In 2 years

N: 3.181

Mean age: 14.5

Victims

Model 1 (1.95,95% CI 1.53–2.48)c

Model 2 (1.56,95% CI 1.21–2.02)c

Model 3 (1.77,95% CI 1.29–2.44)c

Model 4 (1.57,95% CI 1.21–2.03)c

Yen et al. (2014)a

6.406

Highschoold

NR

Victims

Active vs passive (0.882, 95% CI 0.659–1.181)c

Both vs passive (1.080, 95% CI 0.947–1.231)c

Both vs active (1.251, 95% CI 0.872–1.794)c

Bullies

Active vs Passive (0.595, 95% CI 0.462–0.766)c

Both vs passive (1.033, 95% CI 0.891–1.198)c

Both vs active (1.617, 95% CI 1.014–2.578)c

Bullies/Victims vs Victims (1.147,95% CI 1.035–1.272)c

Bullies/Victims vs Bullies (1.498,95% CI 1.191–1.886)c

Bullies vs Victims (0.840,95% CI 0.680–1.038)c

Mark et al. (2013)

4.954

15

NR

Estonia

Victims (2.4,95% CI 1.66–3.36)b

Bullies (2.0,95% CI 1.41–2.79)b

Lithuania

Victims (2.1,95% CI 1.58–2.72)b

Bullies (1.15,95% CI 0.88–1.52)b

Luxemburg

Victims (1.9,95% CI 1.32–2.75)b

Bullies (2.5,95% CI 1.78–3.47)b

Abel et al. (2012)

2.997

10–15

NR

Victims (1.69,95% CI 1.17–2.43)b

Schneider et al. (2012)

20.406

9th–12th Graded

NR

Victims (2.49, 95% CI 2.13–2.92) b

Victims (2.20, 95% CI 1.86–2.62) c

Turner et al. (2012)

1.186

10–17

In 2 years

Victims (2.35, 95% CI 1.19–4.47)b

Heikkila et al. (2013)

2.070

15

In 2 years

Victims

Model 1 (3.8,95% CI 1.8–8.0)c

Model 2 (2.4,95% CI 1.1–5.3)c

Model 3 (3.3,95% CI 1.5–7.0)c

Model 4 (2.3,95% CI 1.0–5.2)c

Bullies

Model 1 (4.1,95% CI 1.7–9.5)c

Model 2 (2.7,95% CI 1.1–6.6)c

Model 3 (2.4,95% CI 0.9–5.9)c

Model 4 (2.5,95% CI 1.0–6.5)c

Bullies/ Victims

Model 1 (5.4,95% CI 1.5–20.0)c

Model 2 (2.0,95% CI 0.4–11.1)c

Model 3 (3.6,95% CI 0.9–13.8)c

Model 4 (1.9.95% CI 0.4–10.7)c

Winsper et al. (2012)

6.043

10.4–13.6

At the age of 8 years

Bullies/ Victims

Model 1 (3.50,95% CI 2.34–5.25)c

Model 2 (3.41,95% CI 2.24–5.18)c

Model 3 (2.84,95% CI 1.81–4.45)c

Victims

Model 1 (1.70,95% CI 1.26–2.29)c

Model 2 (1.70,95% CI 1.25–2.31)c

Model 3 (1.57,95% CI 1.15–2.16)c

Bullies

Model 1 (3.74,95% CI 1.56–8.97)c

Model 2 (3.74,95% CI 1.55–9.07)c

Model 3 (3.60,95% CI 1.46–8.84)c

At the age of 10 years

Bullies/ Victims

Model 1 (4.23,95% CI 2.88–6.20)c

Model 2 (3.84,95% CI 2.57–5.74)c

Model 3 (3.20,95% CI 2.07–4.95)c

Victims

Model 1 (2.40,95% CI 1.80–3.19)c

Model 2 (2.20,95% CI 1.64–2.96)c

Model 3 (1.95,95% CI 1.42–2.66)c

Bullies

Model 1 (1.16,95% CI 0.28–4.83)c

Model 2 (1.13,95% CI 0.27–4.75)c

Model 3 (0.56,95% CI 0.08–4.13)c

Hepburn et al. (2012)

1.838

9th–12th Graded

NR

Bullies (1.49,95% CI 1.07–2.09)c

Victims (1.69,95% CI 1.11–2.58)c

Bullies/ Victims (3.78,95% CI 2.86–4.99)c

Wang et al. (2011)

577

15–19

NR

Victims (1.76,95% CI 1.16–2.68)b,e (1.80,95% CI 1.16–2.81)b,f

Owusu et al. (2011)

7.137

Highschoold

NR

Victims (1.72,95% CI 1.45–2.05)c

Victims of mainly physical bullying (1.18. 95% CI 0.93–1.50)c

Skapinakis et al. (2011)

5.614 (Phase 1) 2.431 (Phase 2)

16–18

NR

Victims

Model 1 (3.72,95% CI 2.40–5.74)c

Model 2 (3.43,95% CI 2.15–5.49)c

Model 3 (2.03,95% CI 1.20–3.44)c

Model 4 (1.94,95% CI 1.12–3.34)c

Bullies

Model 1 (2.11,95% CI 1.18–3.20)c

Model 2 (1.72,95% CI 1.03–2.87)c

Model 3 (1.50,95% CI 0.89–2.54)c

Model 4 (1.35,95% CI 0.77–2.35)c

Cui et al. (2011)

8.778

11–17

NR

Victims sometimes (1.56,95% CI 1.23–1.75)b

Victims always (2.75,95% CI 1.41–2.89)b

Rudatsikira et al. (2007)

1.506

11–17

NR

Victims (1.58,95% CI 1.18–2.11)c

Victims boys (1.62,95% CI 1.06–2.47)c

Victims girls (1.63,95% CI 1.08–2.46)c

Klomek et al. (2009)

2.341

9th–12th Grade d

NR

Victims < 1 week (2.79,95% CI 1.64–4.75)c

Victims often (5.41,95% CI 2.94–9.96)c

Bullies < 1 week (2.62,95% CI 1.56–4.42)c

Bullies often (3.44,95% CI 1.78–6.65)c

Kim et al. (2005)

1.718

12–14

NR

Victims (1.04,95% CI 0.76–1.44)b (1.29,95% CI 0.91–1.84)c

Bullies (1.02,95% CI 0.76–1.37)b (1.11,95% CI 0.80–1.54)c

Bullies/ Victims (1.62,95% CI 1.13–2.32)b (1.90,95% CI 1.26–2.87)c

Kaltiala-Heino et al. (1999)

16.410

14–16

NR

Involved < 1 week (1.4, 95% CI 1.0–1.9)c

Bullies often (4.0, 95% CI 2.6–6.4)c

Victims often (2.1,95% CI 1.3–3.4)c

Bullies/ Victims often (2.5,95% CI 1.0–6.2)c

NR not reported, OR odds ratio, CI confidence interval

aAssesses suicidal ideation and suicide attempt together, as suicidality

bCrude OR

cAdjusted OR

dNot-specified age

eWithout protective factors

fWith protective factors

With regard to the sex of the included subjects, 2 studies provided data regarding the potential association of being a victim and suicidal ideation. These studies suggested that boy-victims were 1.62–2.51 times more likely compared to non-involved boys to present suicidal ideation. Consistently, girl-victims were 1.63–2.22 times more likely to present suicidal ideation compared to non-involved girls (Bhatta et al. 2014; Rudatsikira et al. 2007).

In 8 studies, adjustment for potential confounding factors (age, sex and nationality) did not differentiate the observed associations (Brunstein Klomek et al. 2007; Hepburn et al. 2012; Kaltiala-Heino et al. 1999; Kelly et al. 2015; Messias et al. 2014; Owusu et al. 2011; Rudatsikira et al. 2007; Yen et al. 2014), whereas in 7 studies the adjustment for confounding factors including family conditions, psychiatric comorbidity and prior drug use or abuse seemed to reduce the strength of the observed associations (Bannink et al. 2014; Bhatta et al. 2014; Heikkila et al. 2013; Kim et al. 2005; Schneider et al. 2012; Skapinakis et al. 2011; Winsper et al. 2012). Yet, even in these studies, the involved subjects were more likely to present suicidal ideation compared to non-involved subjects.

The 3 included longitudinal studies suggested that subjects who were involved in bullying a had higher possibility to present suicidal ideation compared to those non-involved for 2 years following the involvement (ORs: bullies: 2.50–4.10, victims: 1.56–3.80, bullies/victims: 1.90–5.40) (Bannink et al. 2014; Heikkila et al. 2013; Turner et al. 2012). Finally, according to the findings of the 1 included cohort study, bullies at the age of 8 years were significantly more likely to present suicidal ideation compared to non-involved subjects (ORs 3.60–3.74), whereas at the age of 10 years both bullies and victims were more likely to present suicidal ideation (ORs 3.20–4.23) (Winsper et al. 2012).

Association of School Bullying and Suicide Attempts

Three of the 22 included studies provided data regarding the potential association between school bullying and planning of suicide attempts (Bhatta et al. 2014; Messias et al. 2014; Sampasa-Kanyinga et al. 2014). These studies suggested that victims were 2.45–2.76 times more likely to plan a suicide attempt compared to non-involved subjects. This finding was more prominent in cases that cyber bullying was also present (5.3 times more likely) (Messias et al. 2014). Notably, Bhatta et al. (2014) indicated that boys were more likely (3.54 times) to plan a suicide attempt compared to non-involved patients, than girls (1.97 times).

According to the 8 studies that provided data regarding school bullying and suicide attempts (Brunstein Klomek et al. 2007; Cui et al. 2011; Elgar et al. 2014; Hepburn et al. 2012; Kim et al. 2005; Messias et al. 2014; Schneider et al. 2012; Winsper et al. 2012), victims were 1.10–4.49 times more likely (Brunstein Klomek et al. 2007; Elgar et al. 2014), bullies 99%–3.90 more likely (Hepburn et al. 2012; Kim et al. 2005), and both victims-bullies 1.85–9.32 times more likely to commit a suicide attempt compared to non-involved subjects (Messias et al. 2014). In cases of additional cyber bullying, victims were even more likely (5.6 times) to have a suicide attempt compared to non-involved subjects (Messias et al. 2014). Two studies investigated the time of exposure to school bullying and making a suicide attempt. In these studies, victims with exposure of shorter duration (< 1 week) were 1.25–2.66 more likely to commit a suicide attempt, and those with longer duration (often or all-time) 2.53–4.49 times, compared to non-involved subjects. Occasional bullies (< 1 week or sometimes) were 2.45 times more likely, whereas those with longer exposure (often or all-time) were 3.64 times more likely to commit a suicide attempt, compared to non-involved subjects (Brunstein Klomek et al. 2007; Cui et al. 2011).

Three of the included studies evaluated the potential association between school bullying and suicide attempts that required medical treatment (Messias et al. 2014; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012). These studies suggested that victims were 1.60–2.16 more likely to commit a suicide attempt that required medical treatment (Messias et al. 2014; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012). This finding was more prominent in cases of additional cyber bullying (Messias et al. 2014). Detailed data regarding the potential effect of possible confounding factors on the evaluated associations are depicted in Table 3.

Table 3

Characteristics and outcomes of the studies that evaluated the association of school bullying and suicide attempts (planning a suicide attempt, commit a suicide attempt, suicide attempts that required medical treatment)

References

Study population characteristics

Follow up

Study main outcomes: Odds Ratios (ORs) vs non-involved subjects

Sample size

Age (years)

Planning of a suicide attempt

 Bhatta et al. (2014)

1.082

11–15

NR

Victims (2.45, 95% CI 1.56–3.84)b

Victims boys (3.54, 95% CI 1.69–7.42)b

Victims girls (1.97, 95% CI 1.08–3.56)b

 Sampasa-Kanyinga et al. (2014)

2.999

12.5–16.1

NR

Victims (2.76, 95% CI 2.20–3.45)a

 Messias et al. (2014)

15.425

9th–12th Gradec

NR

Victims (2.6, 95% CI 2.2–3.2)a (2.7, 95% CI 2.2–3.2)b

Victims of school and cyber bullying (5.3, 95% CI 4.3–6.7)a (5.2, 95% CI 4.1–6.6)b

Commit a suicide attempt

 Elgar et al. (2014)

18.834

12–18

NR

Bullies (1.10, 95% CI 1.06–1.14)a

Victims (1.10, 95% CI: 1.06–1.14)a

 Messias et al. (2014)

15.425

9th–12th Gradec

NR

Victims (2.2, 95% CI: 1.7–2.8)a (2.3, 95% CI: 1.8–2.9)b

Victims of school and cyber bullying (5.5, 95% CI: 4.5–6.8)a (5.6, 95% CI: 4.4–7.0)b

 Schneider et al. (2012)

20.406

9th–12th Gradec

NR

Victims (2.11, 95% CI 1.60–2.77)a

Victims (1.63, 95% CI 1.20–2.20)b

 Winsper et al. (2012)

6.043

10.4–13.6

At the age of 8 years

Bullies/ Victims

Model 1 (2.92, 95% CI 1.88–4.53)b

Model 2 (2.60, 95% CI 1.64–4.13)b

Model 3 (2.67, 95% CI 1.66–4.29)b

Victims

Model 1 (2.36, 95% CI 1.75–3.18)b

Model 2 (2.28, 95% CI 1.67–3.09)b

Model 3 (2.05, 95% CI 1.48–2.83)b

Bullies

Model 1 (3.90 95% CI 1.61–9.42)b

Model 2 (2.93, 95% CI 1.12–7.69)b

Model 3 (3.02, 95% CI 1.14–8.02)b

At the age of 10 years

Bullies/ Victims

Model 1 (3.87, 95% CI 2.63–5.69)b

Model 2 (4.07, 95% CI 2.74–6.03)b

Model 3 (3.34, 95% CI 2.17–5.15)b

Victims

Model 1 (2.53, 95% CI 1.89–3.38)b

Model 2 (2.45, 95% CI 1.81–3.32)b

Model 3 (2.25, 95% CI 1.63–3.09)b

Bullies

Model 1 (1.57, 95% CI 0.48–5.13)b

Model 2 (1.61, 95% CI 0.49–5.29)b

Model 3 (1.12, 95% CI 0.26–4.74)b

 Hepburn et al. (2012)

1.838

9th–12th Gradec

NR

Bullies (0.99, 95% CI 0.20–4.88)b

Victims (2.90, 95% CI 1.58–5.36)b

Bullies/ Victims (9.32, 95% CI 4.91–17.73)b

 Cui et al. (2011)

8.778

11–17

NR

Victims sometimes (1.25, 95% CI 1.10–1.56)a

Victims always (2.53, 95% CI 1.54–2.67)a

 Klomek et al. (2009)

2.341

9th–12th Gradec

NR

Victims < 1 week (2.66, 95% CI 1.58–4.47)b

Victims often (4.49, 95% CI 2.40–8.38)b

Bullies < 1 week (2.45, 95% CI 1.46–4.12)b

Bullies often (3.64, 95% CI 1.91–6.94)b

 Kim et al. (2005)

1.718

12–14

NR

Victims (1.40, 95% CI 0.86–2.26)a (1.69, 95% CI 1.00–2.85)b

Bullies (1.26, 95% CI 0.79–2.00)a (1.16, 95% CI 0.70–1.94)b

Bullies/ Victims (2.01, 95% CI 1.20–3.37)a (1.85, 95% CI 1.01–3.40)b

Suicide attempts that required medical treatment

 Sampasa-Kanyinga et al. (2014)

2.999

12.5–16.1

NR

Victims (1.64, 95% CI 1.18–2.27)a

 Messias et al. (2014)

15.425

9th–12th Gradec

NR

Victims (1.6, 95% CI 1.0–2.6)a (1.6, 95% CI 1.0–2.6)b

Victims of school and cyber bullying (4.4, 95% CI 2.9–6.5)a (4.2, 95% CI 2.7–6.5)b

 Schneider et al. (2012)

20.406

9th–12th Gradec

NR

Victims (2.16, 95% CI 1.34–3.48)a

Victims (1.51, 95% CI 0.89–2.55)b

aCrude OR

bAdjusted OR

cAge not specified

Association of Cyber Bullying and Suicidality

Five studies provided data regarding the potential association between cyber bullying and suicidality. All 5 evaluated cyber bullying and suicidal ideation, 2 cyber bullying and planning of a suicide attempt and 4 the association of cyber bullying and performing a suicide attempt and 2 cyber bullying and suicide attempts requiring medical treatment. The 2009 and 2011 CDC YRBS questionnaires were used in the above-mentioned studies (Bannink et al. 2014; Elgar et al. 2014; Messias et al. 2014; Sampasa-Kanyinga et al. 2014). These studies suggested that victims were 1.36–3.35 times more likely to present suicidal ideation compared to non-involved subjects (Bannink et al. 2014; Messias et al. 2014; Schneider et al. 2012), while the risk ratio was increased (5.5 times) when school bullying was also present (Messias et al. 2014). Of note, Elgar et al. (2014) suggested that the observed association was getting stronger with the increase of exposure to cyber bullying [Odds Ratios (ORs): 1.84 and 2.97 in rare and often exposure of victimization, respectively].

Victims of cyber bulling were 2.79–3.10 times more likely to plan a suicide attempt 5.3 times in cases of additional school bullying, compared to non-involved patients (Messias et al. 2014; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012); They also were 1.81–5.00 (5.6 times in cases of additional school bullying) times more likely to commit a suicide attempt and 3.7–5.36 (4.4 times in cases of additional school bullying) more likely to commit a suicide attempt requiring medical treatment compared to non-involved subjects. In the study by Elgar et al. (2014), the observed association was more profound in cases of long exposure to cyber bullying (ORs 1.81 and 3.47 in rare and often exposure to victimization, respectively). The potential effect of confounding factors was evaluated in two studies. Detailed data are provided in Table 4.

Table 4

Characteristics and outcomes of the studies that evaluated the potential association of Cyber Bullying and Suicidality (Suicidal Ideation, Planning a Suicide Attempt, Commit a Suicide Attempt, Suicide Attempts that required Medical Treatment)

References

Study population characteristics

Follow up

Study main outcomes: Odds Ratios (ORs) vs non-involved subjects

Sample size

Age (years)

Suicidal ideation

 Elgar et al. (2014)

18.834

12–18

NR

Victims rarely (1.84. CI 1.60–2.13)a

Victims sometimes (2.43. CI 1.93–3.07)a

Victims often (2.97. CI 2.03–4.35)a

 Sampasa-Kanyinga et al. (2014)

2.999

12.5–16.1

NR

Victims (3.31. CI 2.16–5.07)a

 Messias et al. (2014)

15.425

9th–12th Gradec

NR

Victims (3.4. CI 2.8–4.2)a (3.3. CI 2.7–4.0)b

Victims of school and cyber bullying(5.5. CI 4.5–6.8)a (5.3. CI 4.2–6.5)b

 Bannink et al. (2014)

8.272

Mean: 12.5

In 2 years

Ν: 3.181

Mean age: 14.5 years

Victims

Model 1 (1.74, 95% CI 1.17–2.61)b

Model 2 (1.22, 95% CI 0.80–1.87)b

Model 3 (1.23, 95% CI 0.80–1.89)b

Model 4 (1.36, 95% CI 0.81–2.28)b

 Schneider et al. (2012)

20.406

9th–12th Gradec

NR

Victims unadjusted (3.35, 95% CI 2.71–4.13)

Victims adjusted (2.59, 95% CI 2.06–3.25)

Planning of a suicide attempt

 Sampasa-Kanyinga et al. (2014)

2.999

12.5–16.1

NR

Victims (2.79, 95% CI 1.63–4.77)a

 Messias et al. (2014)

15.425

9th–12th Gradec

NR

Victims (3.1, 95% CI 2.6–3.8)a (3.1, 95% CI 2.7–3.7)b

Victims of school and cyber bullying (5.3, 95% CI 4.3–6.7)a (5.2, 95% CI 4.1–6.6)b

Commit a suicide attempt

 Elgar et al. (2014)

18.834

12–18

NR

Victims rarely (1.81, 95% CI 1.43–2.28)a

Victims sometimes (3.01, 95% CI 2.30–3.93)a

Victims often (3.47, 95% CI 2.25–5.36)a

 Sampasa-Kanyinga et al. (2014)

2.999

12.5–16.1

NR

Victims (1.73, 95% CI 1.26–2.38)a

 Messias et al. (2014)

15.425

9th–12th Gradec

NR

Victims (3.6, 95% CI 2.7–4.7)a (3.5, 95% CI 2.6–4.7)b

Victims of school and cyber bullying (5.5, 95% CI 4.5–6.8)a (5.6, 95% CI 4.4–7.0)b

 Schneider et al. (2012)

20.406

9th–12th Gradec

NR

Victims unadjusted (5.00, 95% CI 3.73–6.71)

Victims adjusted (3.44, 95% CI 2.48–4.76)

Suicide attempts that required medical treatment

 Messias et al. (2014)

15.425

9th–12th Gradec

NR

Victims (4.0, 95% CI 2.4–6.6)a (3.7, 95% CI 2.1–6.5)b

Victims of school and cyber bullying (4.4, 95% CI 2.9–6.5)a (4.2, 95% CI 2.7–6.5)b

 Schneider et al. (2012)

20.406

9th–12th Gradec

NR

Victims unadjusted (5.36, 95% CI 3.28–8.75)

Victims adjusted (3.39, 95% CI 1.99–5.77)

aCrude OR

bAdjusted OR

cAge not specified

Meta-analysis

In cases that a study provided adequate data for a specific outcome, it was involved in a meta-analysis. The included studies were involved in respective sub-analyses with regard to the type of data provided (victims only, bullies only, victims and bullies). Subsequent sensitivity analyses including sub-groups of studies that adjusted or not for potential confounding factors were also performed when appropriate.

School Bullying

Studies reviewed in this section focused on bullies and victims of school bullying.

Victims of School Bullying and Suicidal Ideation

Ten studies that did not adjust for potential confounders (Abell et al. 2012; Cui et al. 2011; Elgar et al. 2014; Kim et al. 2005; Mark et al. 2013; Messias et al. 2014; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012; Turner et al. 2012; Wang et al. 2011) and 13 studies that adjusted for potential confounders (Bannink et al. 2014; Bhatta et al. 2014; Brunstein Klomek et al. 2007; Heikkila et al. 2013; Hepburn et al. 2012; Kaltiala-Heino et al. 1999; Kelly et al. 2015; Kim et al. 2005; Owusu et al. 2011; Rudatsikira et al. 2007; Schneider et al. 2012; Skapinakis et al. 2011; Winsper et al. 2012) provided relevant data and were included in the respective analyses. According to the findings of both analyses, victims of school bullying were significantly more likely to present suicidal ideation compared to non-involved subjects: (2.021 times, 95% CIs 1.570–2.432) and (1.853 times, 95/5 CIs 1.692–2.001), when adjusting and not adjusting for confounders, respectively. See Figs. 2 and 3. Notably, in the analysis of studies that did not adjust for confounders, a statistically significant publication bias was observed (Egger Test = 0.004).

Fig. 2

Forest plot: victims of school bullying-suicidal ideation (Crude ORs—random effects model)

Fig. 3

Forest plot victims of school bullying-suicidal ideation (adjusted ORs fixed effects model)

Victims of School Bullying and Suicide Attempts

Four studies (Cui et al. 2011; Elgar et al. 2014; Kim et al. 2005; Messias et al. 2014) that did not adjust for confounders and 6 studies that adjusted for confounders (Brunstein Klomek et al. 2007; Hepburn et al. 2012; Kim et al. 2005; Messias et al. 2014; Schneider et al. 2012; Winsper et al. 2012) were included in the respective analyses. According to the findings of the analysis of the 4 studies, victims were more likely to commit suicide attempts compared to non-involved subjects (1.720, 95% CIs 0.966–3.062); yet, the observed association did not reach statistical significance. On the other hand, the difference observed in the analysis of studies adjusting for confounders was statistical significant (2.113 times, 95% CIs 2.830–2.388). See Fig. 4. A statistically significant publication bias was not detected in any of the above analyses (Egger Test = 0.690 and 0.478, respectively).

Fig. 4

Forest plot: victims of school bullying-suicide attempts (adjusted ORs—fixed effects model)

Victims of Suicide Attempts that Required Medical Treatment

Three studies were included in this analysis (Messias et al. 2014; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012). According to these findings, victims of school bullying with medical treatment were significantly more likely to commit suicide attempts compared to non-involved subjects (1.711 times, 95% CIs 1.288–2.122). See Fig. 5. A statistically significant publication bias was not detected (Egger Test = 0.657).

Fig. 5

Victims of school bullying-suicide attempt that required medical treatment (Crude ORs—random effects model)

Bullies and Suicidal Ideation

Three studies that did not adjust for confounders (Elgar et al. 2014; Kim et al. 2005; Mark et al. 2013) and 8 studies that adjusted for confounders were included in the respective analyses (Brunstein Klomek et al. 2007; Heikkila et al. 2013; Hepburn et al. 2012; Kaltiala-Heino et al. 1999; Kelly et al. 2015; Kim et al. 2005; Skapinakis et al. 2011; Winsper et al. 2012). In both analyses, bullies who performed school bullying were significantly more likely to present suicidal ideation compared to non-involved subjects (1.404 times, 95% CIs 1.028–1.917) and (2.533 times, 95% CIs 1.663–3.857). See Figs. 6 and 7, respectively. In both analyses a statistical significant publication bias was not detected (Egger Test = 0.158 and 0.320, respectively).

Fig. 6

Forest plot: bullies (school bullying)—suicidal ideation (Crude ORs—random effects model)

Fig. 7

Bullies (school bullying)-suicidal ideation (adjusted ORs—fixed effects model)

Bullies and Suicide Attempts

Four studies that adjusted for confounders (Brunstein Klomek et al. 2007; Hepburn et al. 2012; Kim et al. 2005; Winsper et al. 2012) were included in this analysis, the findings of which suggested that bullies were significantly more likely to commit a suicide attempt (1951 times, 95% CIs 1.387–2.745) compared to non-involved subjects. See Fig. 8. No statistical significant publication bias was observed (Egger Test = 0.877).

Fig. 8

Forest plot bullies (school bullying)-suicide attempts (adjusted ORs—fixed effects model)

Victims and Bullies and Suicidal Ideation

Six studies that adjusted for potential confounders were included (Heikkila et al. 2013; Hepburn et al. 2012; Kaltiala-Heino et al. 1999; Kelly et al. 2015; Kim et al. 2005; Winsper et al. 2012). The findings of the respective analysis suggested that victims and bullies together were significantly more likely to present suicidal ideation (3.67 times, 95% CIs 3.110–4.325) compared to non-involved subjects. See Fig. 9. No statistically significant publication bias was detected (Egger Test = 0.782).

Fig. 9

Forest plot: (bullies and victims, school bullying)—suicidal ideation (adjusted ORs—fixed effects model)

Victims and Bullies and Suicide Attempts

Three studies that had not adjusted for confounders were included (Hepburn et al. 2012; Kim et al. 2005; Winsper et al. 2012). The findings suggested that the combination of victims and bullies of school bullying were significantly more likely to commit suicide attempts (3.68 times, 95% CIs 2.137–6.35) See Fig. 10. No statistically significant publication bias was observed in this analysis (Egger Test = 0.82).

Fig. 10

Forest plot: (bullies and victims, school bullying)—suicide attempts (Crude ORs—random effects model)

Cyber Bullying

Studies in this area focused on victims, suicidal ideation and suicide attempts.

Victims of Cyber Bullying and Suicidal Ideation

Three studies were included in the respective analysis (Elgar et al. 2014; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012). According to the respective findings, victims of cyber bullying were significantly more likely to present suicidal ideation compared to non-involved subjects (3.261 times, 95% CIs 2.698–3.810). See Fig. 11. A statistically significant publication bias was not detected (Egger Test = 0.569).

Fig. 11

Forest plot: victims of cyber bullying-suicidal ideation (Crude ORs—random effects model)

Victims of Cyber Bullying and Suicide Attempts

Three studies were included in the respective analyses (Elgar et al. 2014; Sampasa-Kanyinga et al. 2014; Schneider et al. 2012). According to the findings of the analyses, victims of cyber bullying were significantly more likely to commit suicide attempt compared to non-involved subjects (3.311 times, 95% CIs 1.220–5.411). See Fig. 12. A statistically significant publication bias was not detected (Egger Test = 0.933).

Fig. 12

Forest plot: victims of cyber bullying-suicide attempts (Crude ORs—random effects model)

Discussion

Existing research provides plethora of evidence regarding the association between school and cyber bullying and suicidality (Hertz et al. 2013; Kim and Leventhal 2008). More specifically, it has been suggested that bullies and victims as a whole, as well as bullies and victims independently, are at an increased risk of presenting suicidal ideation and suicidal behavior (planning and/or committing a suicide attempt) (Winsper et al. 2012; Hepburn et al. 2012). In many of the currently available studies, the evaluated populations had specific characteristics including drug abuse, violent behavior, depression, or were a part of a sexual minority (Litwiller and Brausch 2013). These factors have been reported to intensify the association between the two types of bullying and suicidality. On the other hand, due to increasing rates of both school and cyber bullying and their consequences, we performed a systematic review and meta-analysis of the currently available published evidence to evaluate the potential association between both school and cyber bullying and suicidality in children and adolescents without any of the above-mentioned predisposing factors. We did so to provide more substantial evidence regarding the magnitude of school and cyber bullying independently of suicidality in children and adolescents.

In this regard, the findings of our meta-analysis suggest that, with regard to school bullying, victims and bullies independently, as well as victims and bullies as a whole, were significantly more likely to present suicidal ideation and also commit a suicide attempt, compared to non-involved subjects. Moreover, victims of school bullying were found to be significantly more likely to commit a suicide attempt that required medical treatment. Consistently, with regard to cyber bullying, victims were found to be significantly more likely to present suicidal ideation and also to commit a suicide attempt, compared to non-involved subjects.

Until recently, the available relevant published evidence consisted of two systematic reviews and three meta-analyses (Brunstein Klomek et al. 2010; Holt et al. 2015; Kim and Leventhal 2008; Kowalski et al. 2014; van Geel et al. 2014). Specifically, Kim Y.S. et al. performed a systematic review of a total of 37 studies, 27 which involved young adults without specific characteristics, whereas the remaining 10 involved specific subpopulations that may be considered as more “prone to bullying” (Kim and Leventhal 2008). According to the findings of this systematic review, the subjects who were involved with school bullying were 1.4–10 times more likely to present suicidal ideation, compared to non-involved evaluated subjects. While pointing out the methodological and other types of heterogeneity among the included studies, Kim Y.S. et al. suggested that the risk of suicidality was higher for bullies/victims of school bullying in both groups without specific characteristics and also those that were included in subpopulations with specific characteristics.

Brunstein Klomek et al. (2010) also performed a systematic review of 31 studies that focused on the evaluation of bullying and suicidality (suicidal ideation and suicide attempts). Their findings are in accordance with the above-mentioned systematic review, with ORs that ranged between 1.4 and 10.0 among the included cross-sectional surveys and 1.7–11.8 in the included longitudinal studies. The evaluated cross-sectional studies suggested that there is an increased risk for suicidal ideation and suicide attempts in subjects involved to both school and cyber bullying. In addition, the limited number of the included longitudinal studies suggested that both peer bullying and peer victimization result in suicidality. Notably, a difference was observed with regard to sex (Brunstein Klomek et al. 2010).

Additionally, Kowalski et al. (2014) also performed a relevant systematic review and meta-analysis that focused on cyber bullying, its causes, different types, as well as its potential consequences. In particularly, this meta-analysis of 131 relevant studies aimed to evaluate the association of cyber bullying with traditional school bullying, as well as the potential association of cyber bullying with different psychological and behavioral variables. According to the study findings, the factor that was found to be strongly associated with cyber-bullying perpetration was normative beliefs about aggression and moral disengagement, while the strongest associations with cyberbullying victimization were observed for stress and suicidal ideation (Kowalski et al. 2014).

Moreover, in their meta-analysis of 34 relevant studies, van Geel et al. (2014) focused on the association of peer victimization and suicidal ideation, as well as on the association between peer victimization and suicide attempts by meta-analyzing 9 relevant studies. According to the findings of van Geel et al., peer victimization was found to be associated with both suicidal ideation (pooled OR: 2.23) and suicide attempts (pooled OR: 2.55). Of note, this meta-analysis also suggested a stronger association between cyber bullying and suicide attempts, compared to traditional bullying and suicide attempts (van Geel et al. 2014). Finally, in their latest meta-analysis, Holt et al. (2015) included a total of 47 studies and performed 6 individual analyses to evaluate the potential association of bullying with suicidal ideation, as well as other types of behavior. According to their findings, bullying perpetration, bullying victimization, as well as their co-occurrence were strongly associated with suicidal ideation (ORs range: 2.12–4.02).

As detailed previously, the above-mentioned systematic reviews/meta-analyses aimed to investigate the association of school and/or cyber bullying and suicidality. With regard to school bullying in particular, the above-mentioned studies (Brunstein Klomek et al. 2010; Holt et al. 2015; Kim and Leventhal 2008; Kowalski et al. 2014; van Geel et al. 2014) involved healthy populations, along with subpopulations with specific characteristics, including young participants with psychiatric comorbidity and sexual minorities. Notably, one of the above-mentioned systematic reviews/meta-analyses that focused on cyber bullying involved also age sub-groups other than children and adolescents (Kowalski et al. 2014).

Taking into consideration the available evidence provided by the above-mentioned relevant systematic reviews and meta-analyses, we also performed a systematic review and meta-analysis to assess the potential association of different types of involvement in school and cyber bullying (victims, bullies, bullies and victims together) with suicidality (including suicidal ideation, suicide attempts and suicide attempts that required medical treatment) in a populations of children and adolescents who did not have any types of comorbidity or any other characteristics that may potentially make them prone to involvement in bullying. Specifically, the findings of the systematic review of 17 studies that provided data regarding school bullying suggested a strong positive association between different types of bullying involvement with suicidal ideation. This finding was consistent with regard to suicide attempts. Notably, the observed associations were stronger for victims and bullies as a whole. This finding is consistent with a study that suggested that victims and bullies as a whole are more likely to report adverse health events including depression, anxiety, internalization disorders, compared to children/adolescents who are exclusive bullies or exclusive victims (Duval and Tweedie 2000).

Additionally, according to our findings, a considerably strong positive association was detected between victims of school bullying and planning a suicide attempt, as well as with committing a suicide attempt that required medical treatment. Moreover, no difference was identified in the observed associations with regard to sex. Adjusting for potential confounding factors tends to decrease the strength of the observed association but did not alter them. Finally, in the subgroup of studies that provided relevant data, the observed strongly positive associations tended to be identified even for 2 years following exposure to bullying.

With regard to cyber bullying, the findings of a meta-analysis of three studies that provided relevant data suggested that victims of cyber bullying were significantly more likely to present suicidal ideation and also to commit a suicide attempt, compared to non-involved subjects. Notably, in cases that school bullying was also present, the observed associations between victims of both cyber and school bullying and suicidality was twice as strong as the associations of exclusive victims of school bullying and exclusive victims of cyber bullying with suicidality. It is worth to note that, consistent with the findings reported in the meta-analysis by van Geel et al. (2014), the observed associations between victims of cyber bullying and suicidal ideation and also suicide attempts were stronger compared to the association of victims of school bullying and suicidal ideation/suicide attempts. Yet, to the best of our knowledge, this review/meta-analysis is the first to involve studies providing data for cyber bullying and suicidality in children and adolescents that do not have specific characteristics in a meta-analytic procedure. Further studies focusing on cyber bullying in particular may enable even more solid conclusions.

The currently available literature has limitations that may potentially interfere with the strength of the observed associations. Specifically, the considerable heterogeneity regarding the definitions of cyber bullying, school bullying and suicidality need to be considered. Additionally, the inclusion of subjects that have predisposing factors for suicidality is also a limitation.

This review/meta-analysis also has limitations that should be taken into consideration in the interpretation of its findings. First, it did not include non-published evidence. However, the potential presence of publication bias was assessed in all respective statistical analyses of the review/meta-analysis. Notably, a statistical significant publication bias was observed in only one of our analyses. Second, we chose to exclude studies that involved suicidality aspects relevant to deliberate self-harming. Moreover, we chose not to include subpopulations of children and adolescents with specific characteristics, as that may have potentially underestimated the observed associations. Finally, the language restrictions of this review, as well as the inherent limitations of the analysis of non-randomized comparative studies that involve questionnaires, also should be taken into consideration.

Conclusion

This study’s meta-analyses and systematic review revealed important results. The meta-analyses suggest that all kinds of involvement in school bullying (victims only, bullies only, victims and bullies as a whole) were significantly more likely to present suicidal ideation and also commit a suicide attempt, compared to non-involved subjects. Consistently, with regard to cyber bullying, victims were found to be significantly more likely to present suicidal ideation and also to commit a suicide attempt. The systematic review revealed the co-existence of school and cyber bullying; the results showed stronger associations between victims and bullies as a whole and suicidality. Of note, all studies included in the systematic review and meta-analyses involved children and adolescents who did not have any predisposing factors for suicidality. In this regard, the findings of our review/meta-analysis intensify the magnitude of the association of both school and cyber bullying with the evaluated forms of suicidality and contribute to further understanding this increasingly alarming health/public issue. The inherent limitations of the available relevant studies (use of self-reporting questionnaires, different time frames, as well as a wide variety of definitions of school and cyber bullying), intensify the need for further studies focusing on this important issue in order to reach more solid conclusions and enable effective prevention and management of both school and cyber bullying.

Notes

Author contributions

GNK conceived study, participated in its design and coordination, collection and interpretation of the data; he also performed the data collection and extraction, as well as the statistical analyses, and helped to draft the manuscript; EKV contributed to the data collection and also participated in drafting the manuscript; GK participated in the manuscript’s design and coordination and also performed the statistical analyses; EK participated in the manuscript’s design and coordination; EE participated in the manuscript’s design and coordination; CB participated in the manuscript’s design and coordination. All authors have read and approved the final version of the manuscript.

Compliance with ethical standards

Data sharing declaration

All data generated or analyzed during this study are included in this published article (and its supplementary information files).

Conflict of interest

The authors report no conflict of interests.

Ethical approval

Ethical approval for this study was not needed.

Informed consent

This study included only already published data.

References

  1. Abell, W. D., Sewell, C., Martin, J. S., Bailey-Davidson, Y., & Fox, K. (2012). Suicide ideation in Jamaican youth: sociodemographic prevalence, protective and risk factors. West Indian Medical Journal, 61(5), 521–525.CrossRefPubMedGoogle Scholar
  2. American Academy of Child & Adolescent Psychiatry. (2001). Practice parameter for the assessment and treatment of children and adolescents with suicidal behavior. American Academy of Child and Adolescent Psychiatry. Journal of American Academy of Child Adolescent Psychiatry, 40(7 Suppl), 24S–51S.Google Scholar
  3. Bannink, R., Broeren, S., van de Looij-Jansen, P. M., de Waart, F. G., & Raat, H. (2014). Cyber and traditional bullying victimization as a risk factor for mental health problems and suicidal ideation in adolescents. PLoS One, 9(4), e94026.  https://doi.org/10.1371/journal.pone.0094026.CrossRefPubMedPubMedCentralGoogle Scholar
  4. Bauman, S., Toomey, R. B., & Walker, J. L. (2013). Associations among bullying, cyberbullying, and suicide in high school students. Journal of Adolescence, 36(2), 341–350.  https://doi.org/10.1016/j.adolescence.2012.12.001.CrossRefPubMedGoogle Scholar
  5. Bhatta, M. P., Shakya, S., & Jefferis, E. (2014). Association of being bullied in school with suicide ideation and planning among rural middle school adolescents. Journal of School Health, 84(11), 731–738.  https://doi.org/10.1111/josh.12205.CrossRefPubMedGoogle Scholar
  6. Brunstein Klomek, A., Marrocco, F., Kleinman, M., Schonfeld, I. S., & Gould, M. S. (2007). Bullying, depression, and suicidality in adolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 46(1), 40–49.  https://doi.org/10.1097/01.chi.0000242237.84925.18.CrossRefPubMedGoogle Scholar
  7. Brunstein Klomek, A., Sourander, A., & Gould, M. (2010). The association of suicide and bullying in childhood to young adulthood: A review of cross-sectional and longitudinal research findings. The Canadian Journal of Psychiatry, 55(5), 282–288.CrossRefPubMedGoogle Scholar
  8. Cui, S., Cheng, Y., Xu, Z., Chen, D., & Wang, Y. (2011). Peer relationships and suicide ideation and attempts among Chinese adolescents. Child: Care, Health and Development, 37(5), 692–702.  https://doi.org/10.1111/j.1365-2214.2010.01181.x.Google Scholar
  9. Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.CrossRefPubMedGoogle Scholar
  10. Elgar, F. J., Napoletano, A., Saul, G., Dirks, M. A., Craig, W., Poteat, V. P., Koenig, B. W. (2014). Cyberbullying victimization and mental health in adolescents and the moderating role of family dinners. JAMA Pediatrics, 168(11), 1015–1022.  https://doi.org/10.1001/jamapediatrics.2014.1223.CrossRefPubMedGoogle Scholar
  11. Finkelhor, D., Turner, H. A., Shattuck, A., & Hamby, S. L. (2013). Violence, crime, and abuse exposure in a national sample of children and youth: An update. JAMA Pediatrics, 167(7), 614–621.  https://doi.org/10.1001/jamapediatrics.2013.42.CrossRefPubMedGoogle Scholar
  12. Fisher, H. L., Moffitt, T. E., Houts, R. M., Belsky, D. W., Arseneault, L., & Caspi, A. (2012). Bullying victimisation and risk of self harm in early adolescence: Longitudinal cohort study. BMJ, 344, e2683.  https://doi.org/10.1136/bmj.e2683.CrossRefPubMedPubMedCentralGoogle Scholar
  13. Hay, C., & Meldrum, R. (2010). Bullying victimization and adolescent self-harm: Testing hypotheses from general strain theory. Journal of Youth and Adolescence, 39(5), 446–459.  https://doi.org/10.1007/s10964-009-9502-0.CrossRefPubMedGoogle Scholar
  14. Heikkila, H. K., Vaananen, J., Helminen, M., Frojd, S., Marttunen, M., & Kaltiala-Heino, R. (2013). Involvement in bullying and suicidal ideation in middle adolescence: A 2-year follow-up study. European Child & Adolescent Psychiatry, 22(2), 95–102.  https://doi.org/10.1007/s00787-012-0327-0.CrossRefGoogle Scholar
  15. Hepburn, L., Azrael, D., Molnar, B., & Miller, M. (2012). Bullying and suicidal behaviors among urban high school youth. Journal of Adolescent Health, 51(1), 93–95.  https://doi.org/10.1016/j.jadohealth.2011.12.014.CrossRefPubMedGoogle Scholar
  16. Herba, C. M., Ferdinand, R. F., Stijnen, T., Veenstra, R., Oldehinkel, A. J., Ormel, J., & Verhulst, F. C. (2008). Victimization and suicide ideation in the TRAILS study: Specific vulnerabilities of victims. Journal of Child Psychology and Psychiatry, 49(8), 867–876.  https://doi.org/10.1111/j.1469-7610.2008.01900.x.CrossRefPubMedGoogle Scholar
  17. Hertz, M. F., Donato, I., & Wright, J. (2013). Bullying and suicide: A public health approach. Journal of Adolescent Health, 53(1 Suppl), 1–3.  https://doi.org/10.1016/j.jadohealth.2013.05.002.CrossRefGoogle Scholar
  18. Higgins, J., Thompson, S., Deeks, J., & Altman, D. (2002). Statistical heterogeneity in systematic reviews of clinical trials: A critical appraisal of guidelines and practice. Journal of Health Services Research & Policy, 7(1), 51–61.  https://doi.org/10.1258/1355819021927674.CrossRefGoogle Scholar
  19. Hinduja, S., & Patchin, J. W. (2010). Bullying, cyberbullying, and suicide. Archives of Suicide Research, 14(3), 206–221.  https://doi.org/10.1080/13811118.2010.494133.CrossRefPubMedGoogle Scholar
  20. Holt, M. K., Vivolo-Kantor, A. M., Polanin, J. R., Holland, K. M., DeGue, S., Matjasko, J. L., Reid, G. (2015). Bullying and suicidal ideation and behaviors: A meta-analysis. Pediatrics, 135(2), e496–e509.  https://doi.org/10.1542/peds.2014-1864.CrossRefGoogle Scholar
  21. Juvonen, J., & Gross, E. (2008). Extending the school grounds? Bullying experiences in cyberspace. Journal of School Health, 78, 496e505.CrossRefGoogle Scholar
  22. Kaltiala-Heino, R., Rimpela, M., Marttunen, M., Rimpela, A., & Rantanen, P. (1999). Bullying, depression, and suicidal ideation in Finnish adolescents: School survey. BMJ, 319(7206), 348–351.CrossRefPubMedPubMedCentralGoogle Scholar
  23. Kelly, E. V., Newton, N. C., Stapinski, L. A., Slade, T., Barrett, E. L., Conrod, P. J., & Teesson, M. (2015). Suicidality, internalizing problems and externalizing problems among adolescent bullies, victims and bully-victims. Preventive Medicine, 73, 100–105.  https://doi.org/10.1016/j.ypmed.2015.01.020.CrossRefPubMedGoogle Scholar
  24. Kim, Y. S., Koh, Y. J., & Leventhal, B. (2005). School bullying and suicidal risk in Korean middle school students. Pediatrics, 115(2), 357–363.  https://doi.org/10.1542/peds.2004-0902.CrossRefPubMedGoogle Scholar
  25. Kim, Y. S., & Leventhal, B. (2008). Bullying and suicide. A review. International Journal of Adolescent Medicine and Health, 20(2), 133–154.CrossRefPubMedGoogle Scholar
  26. Klomek, A. B., Sourander, A., Niemela, S., Kumpulainen, K., Piha, J., Tamminen, T., Gould, M. S. (2009). Childhood bullying behaviors as a risk for suicide attempts and completed suicides: A population-based birth cohort study. Journal of the American Academy of Child and Adolescent Psychiatry, 48(3), 254–261.  https://doi.org/10.1097/CHI.0b013e318196b91f.CrossRefPubMedGoogle Scholar
  27. Kowalski, R., Giumetti, G., Schroeder, A., & Lattanner, M. (2014). Bullying in the digital age: A critical review and meta-analysis of cyberbullying research among youth. Psychological Bulletin, 140, 1073e1137.CrossRefGoogle Scholar
  28. Kowalski, R., & Limber, S. (2007). Electronic bullying among middle school students. Journal of Adolescent Health, 41, S22e30.CrossRefGoogle Scholar
  29. Laukkanen, E., Honkalampi, K., Hintikka, J., Hintikka, U., & Lehtonen, J. (2005). Suicidal ideation among help-seeking adolescents: Association with a negative self-image. Archives of Suicide Research, 9(1), 45–55.  https://doi.org/10.1080/13811110590512930.CrossRefPubMedGoogle Scholar
  30. Litwiller, B. J., & Brausch, A. M. (2013). Cyber bullying and physical bullying in adolescent suicide: The role of violent behavior and substance use. Journal of Youth and Adolescence, 42(5), 675–684.  https://doi.org/10.1007/s10964-013-9925-5.CrossRefPubMedGoogle Scholar
  31. Mark, L., Samm, A., Tooding, L. M., Sisask, M., Aasvee, K., Zaborskis, A., Varnik, A. (2013). Suicidal ideation, risk factors, and communication with parents. An HBSC study on school children in Estonia, Lithuania, and Luxembourg. Crisis, 34(1), 3–12.  https://doi.org/10.1027/0227-5910/a000153.CrossRefPubMedGoogle Scholar
  32. Meltzer, H., Vostanis, P., Ford, T., Bebbington, P., & Dennis, M. S. (2011). Victims of bullying in childhood and suicide attempts in adulthood. European Psychiatry, 26(8), 498–503.  https://doi.org/10.1016/j.eurpsy.2010.11.006.CrossRefPubMedGoogle Scholar
  33. Messias, E., Kindrick, K., & Castro, J. (2014). School bullying, cyberbullying, or both: Correlates of teen suicidality in the 2011 CDC Youth Risk Behavior Survey. Comprehensive Psychiatry, 55(5), 1063–1068.  https://doi.org/10.1016/j.comppsych.2014.02.005.CrossRefPubMedPubMedCentralGoogle Scholar
  34. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Journal of Clinical Epidemiology, 62(10), 1006–1012.  https://doi.org/10.1016/j.jclinepi.2009.06.005.CrossRefPubMedGoogle Scholar
  35. Owusu, A., Hart, P., Oliver, B., & Kang, M. (2011). The association between bullying and psychological health among senior high school students in Ghana, West Africa. Journal of School Health, 81(5), 231–238.  https://doi.org/10.1111/j.1746-1561.2011.00590.x.CrossRefPubMedGoogle Scholar
  36. Preferred reporting items for systematic reviews and meta-analyses. http://www.prisma-statement.org/. Accessed 15 Oct 2015
  37. Rudatsikira, E., Muula, A. S., Siziya, S., & Twa-Twa, J. (2007). Suicidal ideation and associated factors among school-going adolescents in rural Uganda. BMC Psychiatry, 7, 67.  https://doi.org/10.1186/1471-244X-7-67.CrossRefPubMedPubMedCentralGoogle Scholar
  38. Sampasa-Kanyinga, H., Roumeliotis, P., & Xu, H. (2014). Associations between cyberbullying and school bullying victimization and suicidal ideation, plans and attempts among Canadian schoolchildren. PLoS One, 9(7), e102145.  https://doi.org/10.1371/journal.pone.0102145.CrossRefPubMedPubMedCentralGoogle Scholar
  39. Schneider, S. K., O’Donnell, L., Stueve, A., & Coulter, R. W. (2012). Cyberbullying, school bullying, and psychological distress: A regional census of high school students. American Journal of Public Health, 102(1), 171–177.  https://doi.org/10.2105/AJPH.2011.300308.CrossRefPubMedPubMedCentralGoogle Scholar
  40. Skapinakis, P., Bellos, S., Gkatsa, T., Magklara, K., Lewis, G., Araya, R., Mavreas, V. (2011). The association between bullying and early stages of suicidal ideation in late adolescents in Greece. BMC Psychiatry, 11, 22.  https://doi.org/10.1186/1471-244X-11-22.CrossRefPubMedPubMedCentralGoogle Scholar
  41. Sterne, J. A., Bradburn, M. J., & Egger, M. (2001). 18 Meta-analysis in StataTM. http://www.biostat.jhsph.edu/~fdominic/teaching/bio656/software/meta.analysis.pdf.
  42. Sterne, J. A., & Egger, M. (2001). Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical Epidemiology, 54(10), 1046–1055.CrossRefPubMedGoogle Scholar
  43. Sterne, J. A., Egger, M., & Smith, G. D. (2001). Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. BMJ, 323(7304), 101–105.CrossRefPubMedPubMedCentralGoogle Scholar
  44. Turner, H. A., Finkelhor, D., Shattuck, A., & Hamby, S. (2012). Recent victimization exposure and suicidal ideation in adolescents. Archives of Pediatrics & Adolescent Medicine, 166(12), 1149–1154.  https://doi.org/10.1001/archpediatrics.2012.1549.CrossRefGoogle Scholar
  45. van Geel, M., Vedder, P., & Tanilon, J. (2014). Relationship between peer victimization, cyberbullying, and suicide in children and adolescents: A meta-analysis. JAMA Pediatrics, 168(5), 435–442.  https://doi.org/10.1001/jamapediatrics.2013.4143.CrossRefPubMedGoogle Scholar
  46. van Heeringen, K. (2012). The neurobiological basis of suicide. Chapter 6. Stress–diathesis model of suicidal behavior. Boca Raton, FL: CRC Press.Google Scholar
  47. Wang, R. H., Lai, H. J., Hsu, H. Y., & Hsu, M. T. (2011). Risk and protective factors for suicidal ideation among Taiwanese adolescents. Nursing Research, 60(6), 413–421.  https://doi.org/10.1097/NNR.0b013e3182337d83.CrossRefPubMedGoogle Scholar
  48. Winsper, C., Lereya, T., Zanarini, M., & Wolke, D. (2012). Involvement in bullying and suicide-related behavior at 11 years: A prospective birth cohort study. Journal of the American Academy Child and Adolescent Psychiatry, 51(3), 271–282 e273.  https://doi.org/10.1016/j.jaac.2012.01.001.CrossRefGoogle Scholar
  49. Yen, C. F., Yang, P., Wang, P. W., Lin, H. C., Liu, T. L., Wu, Y. Y., & Tang, T. C. (2014). Association between school bullying levels/types and mental health problems among Taiwanese adolescents. Comprehensive Psychiatry, 55(3), 405–413.  https://doi.org/10.1016/j.comppsych.2013.06.001.CrossRefPubMedGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • George N. Katsaras
    • 1
  • Evridiki K. Vouloumanou
    • 2
    • 3
  • Georgia Kourlaba
    • 4
  • Eleni Kyritsi
    • 5
  • Eleni Evagelou
    • 5
  • Chryssa Bakoula
    • 6
  1. 1.NICUGeneral Hospital of Nikaia-Piraeus “Ag. Panteleimon”Nikaia-PiraeusGreece
  2. 2.Department of PaediatricsTzaneion General HospitalPiraeusGreece
  3. 3.Alfa Institute of Biomedical Sciences (AIBS)AthensGreece
  4. 4.The Stavros Niarchos Foundation-Collaborative Center for Clinical Epidemiology and Outcomes Research (CLEO)National and Kapodistrian University of Athens, School of MedicineAthensGreece
  5. 5.Nursing Department ATechnological Educational Institute of AthensAthensGreece
  6. 6.1st Department of Paediatrics, Children’s Hospital “Agia Sophia”, Medical SchoolUniversity of AthensAthensGreece

Personalised recommendations