Abstract
Purpose of Review
Suicide risk assessment is a challenge in clinical practice. Implicit measures may present with advantages with respect to explicit methods, and therefore may be useful for the assessment of suicide risk. We conducted a systematic review of 2 databases (PubMed and EMBASE) about implicit tests that measure suicide risk to explore their validity and reliability.
Recent Findings
Initial research revealed 321 articles. After the selection process, 31 articles were included in the review. The most death-related implicit cognition test used was the Death/Suicide Implicit association test (D/S IAT), followed by the Suicide Stroop Task. The Suicide Affect Misattribution Procedure (S-AMP) and the Death version of the Implicit Relational Assessment Procedure (D-IRAP) were also used.
Summary
We found that the measures reviewed were generally valid for the assessment of past and future suicidal thoughts and behaviors, with statistically significant results regarding retrospective and prospective associations.
Similar content being viewed by others
Introduction
A person dies by suicide every 40 s [1]. Suicide represents 1.5% of all deaths worldwide [2]. Suicide attempts (SAs) are estimated to be twenty times more prevalent than death by suicide. Suicide ideation (SI) is not only a recognized risk factor for SAs and death by suicide but also represents a public health problem in its own right [3]. Due to the social, economic, and psychological stresses of the Covid-19 crisis last year, rates of SI, SAs and death by suicide are expected to rise, especially among populations at risk, thus becoming an urgent concern for public health worldwide [4].
Suicide risk assessment is a challenge in clinical practice. Traditionally, it relies on patients’ self-report about their suicidal intentions [5, 6]. The results of a suicide risk assessment will lead to a specific intervention that addresses the patient’s suicidal intentions to prevent fatal outcomes [7]. Nonetheless, self-report measures for suicide risk assessment/estimation may present some limitations.
On the one hand, suicidal patients may be ambivalent about expressing their suicidal thoughts and hide crucial information during a structured clinical assessment. In a prospective study, 78% of the people who had died by suicide explicitly denied such intentions during the previous clinical evaluation [8]. Reasons why patients deny their suicidal intentions are broad; patients may not feel comfortable disclosing their intention to end their lives [9]. Alternatively, they may not be fully aware of their intention or even not be capable of verbalizing it [10]. Likewise, patients may underestimate the severity of their thoughts and their needs for clinical services [11]. Moreover, others may conceal certain information to avoid hospitalizations that would frustrate their suicidal plans [12].
On the other hand, suicide risk assessments and suicide screening are usually conducted in crowded psychiatric emergency departments with limited time and resources to perform clinical evaluations [7]. These conditions make it particularly challenging for the clinician to estimate and predict suicide risk [13]. In a retrospective study of people who died by suicide, results showed that clinicians classified suicide risk as low or absent in 80–90% of the cases [14].
In the last decades, literature has sought to investigate indirect markers of suicidal behavior that are not based on self-report [15•, 16]. In this context, the use of tests based on implicit cognition arises. Implicit cognition is the ensemble of judgments, assumptions, and associations made automatically and unconsciously [17]. Thus, tests based on implicit cognition are not subjected to introspection, so that individuals can hardly control or manipulate their responses [18]. Implicit cognition tests have a large background that started in 1935 with the traditional Stroop task, which have been used by cognitive psychologists to study attentional processes [19]. Nowadays implicit cognition tests have been extended to different measures, and there are specifically some related to suicide, such as the Suicide Stroop Task version [20]. In the context of suicide risk assessment, implicit measure tasks allow to assess implicit biases for suicide-related content in real time without having to ask participants directly. Tasks are usually associative and constitute a behavioral marker of suicide risk. Among the different implicit cognition tests, the most frequently used is the Implicit Association Test (IAT) [21, 22]. The IAT is based on the response time given by a person when performing an association task [23].
The first IATs were used to explore implicit attitudes on sensitive issues, such as racism [24]. The assumption behind the test is that it should be easier to associate two concepts when they are already related to each other in our minds [24]. For example, IAT studies show that many participants who claim not to have racist attitudes nevertheless responded more quickly and accurately when they were asked to associate White with Positive and Black with Negative than the opposite [21].
In 2010, Nock et al. [12] adapted the original IAT for suicide risk assessment, thus creating the Death/Suicide IAT (D/S IAT), which aimed to identify people who quickly associated suicide with the self.
Other tests based on implicit cognition in suicide risk assessment have emerged in later years, such as the Suicide Stroop task [20, 25,26,27,28,29], which evaluates the time it takes a person to identify the font color of death/suicide related words. Also, the Death version of the Implicit Relational Assessment Procedure (D-IRAP) [30], which is similar to the IAT in that participants have to pair stimulus under time and accuracy pressure. And the Suicide Affect Misattribution Procedure (S-AMP) [16, 31] that evaluates how participants misattribute a stimulus after being presented with a suicide/death-related prime.
The ease of implementing these implicit measures would allow clinicians to quickly assess real-time suicide risk like in emergency departments [13].
A recent meta-analysis explored the validity—discriminative and predictive value—of the D/S IAT, finding that the test is accurate when predicting both past and future suicide behaviors. However, they recommend that suicide risk be determined based on multiple sources of assessment and not solely based on the D/S IAT [32]. Additionally, another review tested the reliability and concurrent validity of the Suicide Stroop task and found poor psychometric properties [33]. Given the mixed results, it is important to systematically explore the value of implicit cognition tests for the discrimination and prediction of suicide behavior in order to clarify and standardize the relevance of their use in clinical and research settings. However, there are no systematic reviews about the use of implicit cognition tests for the assessment of suicide risk. Here, we perform a systematic review of the retrospective and prospective validity of implicit cognition tests for the assessment of suicide. We discuss the implications of our findings for clinical practice and future research.
Methods
This review followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [34]. The review protocol was registered in the PROSPERO database (registration number CRD42020165368).
Inclusion/Exclusion Criteria
Inclusion criteria were as follows:
-
i)
Original studies published in peer-reviewed journals that report measurable outcomes (e.g., score in a test).
-
ii)
Studies that use an implicit cognition test of death-related and/or self-injury-related variables (e.g., D/S IAT).
-
iii)
Studies that measure the past or future occurrence of any kind of suicidal behaviour (e.g., number of past suicide attempts).
-
iv)
Studies that explore the correlation between the results of the test and the occurrence of suicidal behaviour (e.g., statistical analysis of correlation such as Pearson correlation test).
Exclusion criteria were as follows:
-
i)
Proofs of concepts, protocols for randomized clinical trials.
-
ii)
Studies that do not provide measurable outcomes.
There were no restrictions regarding language or publication date.
Search Strategy
We conducted a systematic literature search in two of the most accessible databases: PubMed and EMBASE. The last search date was 20 January 2021.
The following search terms were used: Implicit AND Suicide (suicide OR suicidal OR self-harm OR self-injury). The references of included studies were also screened.
Study Selection Process
The articles were selected if they were of relevance to the research question (i.e., the validity and reliability of implicit cognition tests for the assessment of suicide risk), met the inclusion criteria, and were of sufficient methodological quality. Eligible studies were critically appraised [35].
Studies were independently reviewed for inclusion by two authors (MM and APS). Any inconsistencies were resolved with the involvement of a third author (LGR). Agreement between reviewers was measured by intraclass correlation coefficient (ICC).
Study Selection and Data Extraction
The quality of all eligible studies was assessed independently by two reviewers (MM and APS). Discussion between reviewers resolved all discrepancies. Aspects assessed included methodological design, risk of bias, and quality of reporting. Data were identified, checked, and mined by two independent authors (MM and APS). Using pre-made tables, the following variables were collected: author; year of study publication; country; aims of study; sample size; sample characteristics; mean age of the sample; gender distribution of the sample; test used in the study; outcome; timeframe (retrospective association or prospective association); results and main findings.
Results
The initial search revealed 321 results. After screening, full-text review and study selection, 31 articles were finally included in the review (see Fig. 1). ICC among reviewers was 0.79 (95% CI 0.67–0.89).
Characteristics of the Reviewed Studies
Table 1 summarizes the characteristics of the reviewed studies [3, 7, 12, 16, 20, 25,26,27,28,29,30,31, 36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54].
The sample size of the reviewed studies ranged between 32 [51] and 7015 [43]. All studies explored the relationship with past or future suicide behavior. The most common implicit test used was the D/S IAT [6, 12, 36,37,38,39,40,41,42,43,44,45,46, 48,49,50,51, 53, 54] followed by the Suicide Stroop Task [20, 25,26,27,28,29, 38]. Most studies employed a control group composed of participants or patients without presenting suicidal behaviors [3, 12, 20, 25,26,27, 30, 31, 36, 38, 40, 44, 51, 53, 54]. Additionally, four studies used undergraduate students [16, 27, 31, 45]. Three studies used community-based adults [38, 43, 48], one used military service members [39] and one used a high demographic suicide risk sample [6]. The majority of our studies reviewed used adult’s samples, but six studies used a sample with adolescents [3, 29, 40, 42, 44, 48]. Mean age across studies ranged from 14.8 to 49.5.
All but one of the tests reviewed were based on a computerized behavioral task and built their score out of participants’ reaction times employing different algorithms. The exception was the S-AMP, in which time was not registered.
Death and Suicide Implicit Association Test (D/S IAT)
The D/S IAT (Death/Suicide Implicit Association Test [12, 19] is a computer-based test that measures people reaction times when doing a categorization task. The task consists in classifying stimuli of the construct of “death/suicide” (i.e., die, dead, deceased, lifeless, and suicide) and “life” (i.e., alive, survive, live, thrive, and breathing) and the attributes of “me” (i.e. I, myself, my, mine, and self) and “not me” (i.e., they, them, their, theirs, and other). Participants are asked to sort stimuli as quickly as possible to their attribute/construct. Reaction times in correctly classifying the stimuli result in the D-score. Higher D-scores are supposed to indicate greater suicide risk. Although, originally the Death and Suicide IAT was abbreviated as D/S IAT [12], some authors [6, 42, 43, 48] use the abbreviation D-IAT (Death IAT).
Concurrent Validity of the D/S IAT
Four studies have reported the concurrent validity of the D/S IAT [6, 50, 51, 54]. Three studies employed the Beck Scale for Suicide ideation (BSS) as gold standard and found a positive correlation between the D/S IAT and the scale [6, 51, 54]. For their part, the study by Moreno et al. [50] used the Columbia Suicide Scale for Risk Assessment (CSSRS) as gold standard and found a positive correlation.
Retrospective Validity of the D/S IAT
Twenty-four studies explored the retrospective validity of the D/S IAT (see Table 2). Three studies explored the association between D/S score and SI. From this, two found a positive correlation [41, 42], while Chiurliza et al. [39] found no significant association. Regarding the association between the D/S IAT and a history of SA, there are also mixed results. Four studies found a positive correlation with higher D/S IAT score [6, 12, 45, 54]. However, Harrison et al. [46] only found this positive correlation when controlling for the number of past SAs (multiple past SAs, t = 2.12, p < 0.05). In contrast, two studies did not find a significant correlation between the D/S IAT and a history of SA [36, 43]. Moreover, the study by Rath et al. [52] found an inversed association between number past SAs and D/S IAT score (r = − 0.27, p < 0.01).
Prospective Validity of the D/S IAT
Regarding the prospective validity of the D/S IAT, eight out of eleven studies found a positive correlation between the D/S IAT score and the occurrence of suicidal thoughts and behaviours [12, 36–38, 41, 43, 44, 50] (see Table 2). For instance, in the study by Nock et al. [12], the likelihood of committing a SA at 6-month follow-up was higher when the D/S IAT score at baseline was above 0. Two studies [41, 42] explored how changes in D/S IAT scores could be associated with treatment received. Ellis et al. [41] found that the D/S IAT changed throughout the course of the treatment and that it significantly predicted SI at discharge (Cohen’s d = 0.27, p < 0.01). In the study by Glenn et al. [42], D/S IAT score significantly predicted SI at discharge among participants with a long stay in a psychiatric inpatient facility (b = 5.50, p < 0.001), but not in participants with a short stay. In the study by Rath et al. [52], they found a positive correlation when predicting SAs at 3-month follow-up (r = 0.18, p < 0.05) but not at 6 nor 12-month follow-up. Finally, two studies did not find any positive correlation between the D/S IAT score and the occurrence of future suicidal thoughts and behaviors [46, 48].
Test Reliability of the D/S IAT
Four studies explored the D/S IAT test reliability. Rath et al. [52] found an r of 0.22, and Millner et al. [48] found a Cohen’s d of 0.76 (0.74–0.78). Glenn et al. [43] reported good internal replication, without specifying numeric results. The test also showed good consistency in the study of Harrison et al. [49] with an r of 0.85.
D/S IAT Variants
Some studies used variants of the classic D/S IAT. For instance, Millner et al. [48] used the Suicide-IAT, which includes specific suicide methods stimuli (i.e., gunshot, hanging, overdose, cutting). They also included other minor variations, such as the Death Single Category–IAT (DSC-IAT), the Suicide Single Category–IAT (SSC-IAT), and the SSC-IAT with pictures. In this study, the found no significant association between the scores of the different tests used and past or future SAs [48].
There is also a brief version of the D/S IAT created by Millner et al. [49] and called the D/S B-IAT which have shown a good accuracy, though smaller than the original D/S IAT [49]. Tello et al. [53] and Glenn et al. [43] also used the Suicide-IAT. Tello et al. [53] found a high internal consistency of the test and a positive correlation with SAs at 6-month follow-up. In contrast, scores were not significantly associated with past SAs. Whereas, Glenn et al. [43] found a positive correlation with a history of suicidal thoughts and behavior (see Table 2).
D-IRAP
The IRAP is a computer-based test that measures the reaction time when participants are asked to paired stimulus under speed and accuracy pressure. The IRAP is based on the Relational Frame Theory (RFT) [55]. The premise of RFT is that the foundation components of cognition are relational rather than associative [30, 56].
In the study by Hussey et al. [30] they defined two separate death-evaluation IRAPS, the “personal IRAP” and the “abstract IRAP.” The “personal IRAP” is based on a theoretical supposition that suicide differs from homicide by the presence of a desire to die [57]. The “personal IRAP” included a reference to self (i.e., “my death” or “my life) whereas the “abstract IRAP” did not include that reference (i.e., “death” or “life”). In both tasks, the label stimuli (“my death” or “death”) are presented with either a positive (i.e., pleasant, enjoyable, lovely) or a negative target stimulus (i.e., horrible, upsetting, painful). Participants are asked to pair a label and a target stimulus on each trial as fast as possible. Time is registered in millisecond generating a D-IRAP algorithm, which is considered a variant of the D algorithm for the IAT [23].
Only one study (see Table 2) tested the D-IRAP and its association with suicidal thoughts and behaviors [30]. Authors found that recent SI was positively associated with higher scores in “my death-negative” trial from the personal version (self-focused) and could differentiate participants from the SI or control group (OR = 10.50, 95% CI: 2.34–47.03). However, the “death-negative” trial from the abstract version did not significantly distinguish between people with and without SI.
SI-IAT
The Self-Injury IAT (SI-IAT) follows the same theory that the D/S IAT, but constructs are oriented to self-injury, such as cutting. Participants are asked to categorize stimuli of the construct of “escape versus stay” (i.e., leave, quit versus hold on, remain, etc.), “cutting versus no cutting” (i.e., images of cut versus intact skin), “suicide versus life” (overdose, hanging versus live, survive, etc.), and “death versus life” (die, funeral versus live, survive, etc.) and the attributes of “me” (i.e., I, myself, my, mine, and self) and “not me” (i.e., they, them, their, theirs, and other) [3].
In the original study (see Table 2) by Nock and Banaji [3] higher SI-IAT score were positively associated with both past occurrence of suicidal thoughts and behaviors (OR = 5.59, 95% CI: 2.09–26.98) and likelihood of SAs at 6-months (OR = 6.42, 95% CI: 1.19–34.58) and 12-month follow-up (OR = 10.91, 95% CI: 2.66–45.86). Subsequent studies found a positive association between SI-IAT score and a history of non-suicidal self-injury [40, 43]. Suicide behavior, whether past or future, was not associated with the SI-IAT in the reviewed studies [47, 48].
S-AMP
The Suicide Affect Misattribution Procedure (S-AMP) is a computerized-based task that relies on participants’ misattributions based on their emotions, cognitions, and perceptions of themselves. The test assesses how people misattribute the relevance of a target stimulus after being presented with a prime stimulus. Participants are presented with four types of prime images: related to suicide (i.e., dead bodies provoked by suicide via overdose, firearm or overdose), negative but non-related to suicide (i.e., insects crawling on half-eaten food items), neutral (i.e., umbrella or barstool), and positive (i.e., children laughing and playing, flowers and blue skies). After the prime stimuli, participants are asked to quickly rate in a 4-point scale how the target stimulus (i.e., a Chinese pictograph) fits their self-concept (i.e. 1 = doesn’t fit me well, 2 = fits me a little, 3 = moderately fits me, 4 = fits me well). Higher scores represent greater self-identification with the prime stimuli. Thus, the test assesses how the prime influences the affective categorization of the target stimuli.
Two studies (see Table 2) used the S-AMP test [16, 31]. In the study by Tucker et al. [16], they found higher self-identification with suicide prime stimulus in participants with recent SA (OR = 1.19, 95% CI: 1.05–1.34). Similarly, Wells et al. [31] found a positive correlation between the S-AMP score and a history of SI (Cohen’s d = 0.48, p < 0.05).
Suicide Stroop Task
The classic Stroop task is a computer-based test that analyses the time it takes a person to identify the font color of words that indicate colors. Sometimes the word matches the font color and sometimes it does not, which increases the need to pay attention. The test assumes that the participant’s ability is interfered by the emotional salience of each word, creating an attentional bias [25]. In the Suicide Stroop test, colored words are relevant to the construct of death and suicide (i.e., death, suicide, funeral) and are compared to negative words (i.e., rejected, stupid, alone) and neutral words (i.e., museum, paper, engine). Larger response latencies are interpreted as an indicator of greater interference due to the semantic content of the words [26].
Seven studies (see Table 2) explored the validity of the Suicide Stroop Task [20, 25,26,27,28,29, 38]. In four studies, participants with a history of suicide behaviors showed greater interferences with suicide-related words stimulus [20, 25, 27, 38]. In the study by Stewart et al. [29], they found greater interferences in suicide and positive word in adolescents with a history of SAs compared with those with SI. In the study by Cha et al. [26], they found that the suicide words stimulus score was associated with both history of SAs (OR = 1.01, 95% CI: 1.00–1.01) and the occurrence of future SAs at 6-month follow-up (OR = 1.02, 95% CI: 1.00–1.03). However, two studies did not found any significance between the score in the Suicide Stroop Task and SI at 1 and 6-month follow-up [38], nor with a history of lifetime SAs [28].
Discussion
In this systematic review, we explored the evidence about the validity of death-related implicit cognition tests. We found that death-related implicit cognition tests were generally aimed on assessing past suicide thoughts and behaviors and showed great past prediction. However, only a few studies have tried its validity for future suicide thoughts and behaviors, and the most used test for this purpose was the D/S IAT showing adequate predictive power. Has shown in a recent meta-analysis of the discriminative and prospective utility of the D/S IAT, the authors stated that, although this test has sufficient predictive value, clinical decisions should not be based exclusively on it [32].
Stimuli and Performance
The death-related implicit cognition tests reviewed share the same aim to assess automatic attitudes and cognitions associated with suicidal thoughts and behaviors. However, they do so through different tasks and stimuli. On the one hand, the Suicide Stroop Task and the S-AMP aim to assess the implicitness by measuring the degree of interference that stimuli can generate, either by presenting a prime (a stimulus presented just before another) or by presenting two distinctive cues in a single stimulus (the color/meaning of a word). Therefore, the outcome of these tests will depend on a person’s cognitive ability to perceive and process these stimuli/cues to respond to the assigned task. On the other hand, the D/S IAT, the SI-IAT and the D-IRAP try to access the implicitness through the individual’s ability to assign words to the semantic field to which they belong, which requires knowing the meaning of each word. In addition, the D-IRAP, unlike the IATs, includes a referential component on the task (a reference to the self in association to words, such as “My death – positive/negative”), which could add complexity to the task because it requires an additional component. In this line, it seems convenient to evaluate at the neurobiological level whether the simplicity or complexity of elements included in death-related implicit cognition tests influence on individual performance and thus may affect the predictive capacity of the test.
Neurobiological and Cognitive Correlates
The mechanisms underlying the accuracy of implicit cognition tests have not yet been fully elucidated. Recent neuroimaging studies have found neuronal correlates with the IAT [58•, 59]. In one study, they used an adapted functional magnetic resonance imaging (fMRI) version of the S-IAT. Imaging results detected an increased activation in the bilateral insula, medial prefrontal cortex, middle occipital cortex, and parahippocampal gyri during the self-death condition compared with the self-life condition in the test [58•]. In another study, connectivity changes between the early visual cortex, amygdala, and anterior insula were observed during the S-IAT, and increased connectivity discriminated between participants with and without recent SAs [59].
Additionally, a study discovered that adults with mood disorders and a history of SAs had changes in default mode and basal ganglia activity during the emotional face-word Stroop task [60]. Furthermore, the original Stroop task has been related to the attentional bias, which is implicated in activating the specific brain region associated with affective disorders and plays a role in predicting future suicidal behavior [61, 62]. In the previous study by Thompson et al. [63], participants with a history of SA showed reduced activity in leftward frontal areas during the performance of the emotional Stroop task, which would suggest difficulties in the ability to regulate emotional processing. Moreover, participants with a history of SAs have, on average, worse memory, more impulsivity, and less cognitive flexibility [64•, 65]. According to these results, using the affective go/no-go performance, such dysfunctions were found in participants with current or past suicide thoughts and behaviors but were greater in participants with current SI [66•]. How exactly these findings affect the prediction of suicide behavior is still unknown, and future research is needed to explore brain regions and neural networks associated with suicide risk, which will aid in mapping the suicidal mind [67].
Clinical Implications
Death-related implicit cognition tests may be useful in assessing suicidal behavior by overcoming certain limitations of explicit methods. As argued in the study by Baucom et al. [68•] implicit cognition tests do not only not rely on the direct disclosure of suicide intention by patients, but also they do not require full face-to-face interaction with a clinician. Thus, the test is suitable for people who may be reluctant to express their suicidal thoughts or resistant to take a traditional suicide risk assessment with another person. Moreover, the tests we analyzed could be used in clinical settings when there are concerns that patients may be withholding crucial information that affects clinical decisions. In addition, implicit measures of suicide have the potential to be highly portable and are easy to administer. Accordingly, they could be applied by different kind of professionals in a large range of settings, as they do not require specific training, which makes these measures suitable not only for clinical, but also for research purposes.
Ease of implementation would allow clinicians and researchers to quickly assess suicide risk in real-time, particularly when time is restrained (i.e., in an emergency department). Findings indicate that screening methods did not identify many people who died by suicide, nor declared explicitly their suicide intentions [8, 69]. Hence, using an implicit cognition assessment method may allow to capture a more accurate information regarding suicide risk.
Although the tasks of these tests are relatively simple and do not require complex cognitive processing, some people with attentional or behavioral difficulties, such as high states of agitation in episodes of anxiety or aggressive non-cooperative behaviours, may impede the correct performance of these tests and therefore bias their results. Moreover, people with basic language difficulties or diminished cognitive capabilities should not be assessed through these tests. Professionals administering these measures must first consider the participants’ condition and ability to perform an automatic task in the required time.
This, despite their advantages, implicit measures should be administered combined with other well-known explicit measures to detect suicide risk accurately.
Future Lines of Research
All implicit cognition tests were computerized and consisted of relatively simple software. One of the fronts to be explored in the future could be integrating this software into smartphones in the form of mobile applications. This could increase the portability of the tests, thus facilitating their use outside the hospital environment—for example, for following-up patients after discharge. Mobile technology is increasingly being used for suicide prevention [70]. Implicit cognition tests could be a valuable addition to this field. Additionally, implicit cognition tests could be combined with other mobile health assessment tools, such as Ecological Momentary Assessment (EMA). EMA is based on self-report by users in their usual environment, without the direct supervision of a clinician, and is being increasingly employed in suicide research [71].
Strengths and Limitations
This is the first study to review all the available implicit cognition tests for the assessment of suicide risk. Among the limitations of the review, the heterogeneity of the studies—with different study designs, population, and implicit tests used—precluded performing a quantitative synthesis of the results. One of such limitations is that results regarding their psychometric properties of each test, such as their reliability and validity, are scarcely reported by the reviewed studies. Accordingly, future research shall address this gap.
Conclusions
Our review shows that the implicit cognition tests generally have good concurrent, retrospective, and prospective validity. However, an important caveat on the available research reviewed is that results on psychometric properties are scarce, limiting our capacity for a comprehensive analysis of all these tests. Nonetheless, overall, we can conclude that implicit cognition tests represent a potentially helpful tool for assessing suicide risk and could be an important complement to traditional measures used in clinical practice. Among their advantages are the ease of administration and interpretation, the fact that they do not depend on the explicit suicide intentions self-reported by the patient, and the fact that they yield an objective score that is not influenced by the subjectivity of the evaluator. Despite these benefits, there are still limitations, such as the lack of knowledge of their underlying mechanisms and the lack of familiarity of clinicians with this type of test, which may delay their implementation in routine clinical practice.
Availability of Data and Material
Under request.
Code Availability
Not applicable.
Change history
05 March 2022
A Correction to this paper has been published: https://doi.org/10.1007/s11920-022-01332-5
References
Papers of particular interest, published recently, have been highlighted as: • Of importance
World Health Organization. WHO | Suicide. 2020. https://www.who.int/news-room/fact-sheets/detail/suicide.
Naghavi M. Global regional and national burden of suicide mortality 1990 to 2016: systematic analysis for the Global Burden of Disease Study 2016. BMJ. 2019;364.
Nock MK, Banaji MR. Prediction of suicide ideation and attempts among adolescents using a brief performance-based test. J Consult Clin Psychol. 2007;75(5):707.
Zalsman G, Stanley B, Szanto K, Clarke DE, Carli V, Mehlum L. Suicide in the time of COVID-19: review and recommendations. Arch Suicide Res. 2020;24(4):477–82.
Gunnell D, Appleby L, Arensman E, Hawton K, John A, Kapur N, Khan M, O’Connor RC, Pirkis J, Caine ED, Chan LF. Suicide risk and prevention during the COVID-19 pandemic. Lancet Psychiatry. 2020;7(6):468–71.
Wortzel HS, Nazem S, Bahraini NH, Matarazzo BB. Why suicide risk assessment still matters. J Psychiatric Pract. 2017;23(6):436–40.
Podlogar MC, Gutierrez PM, Joiner TE. Improving our understanding of the death/life implicit association test. J Pers Assess. 2020;102(6):845–57.
Bjureberg J, Dahlin M, Carlborg A, Edberg H, Haglund A, Runeson B. Columbia-Suicide Severity Rating Scale Screen Version: initial screening for suicide risk in a psychiatric emergency department. Psychol Med. 2021;1–9.
Busch KA, Fawcett J, Jacobs DG. Clinical correlates of inpatient suicide. J Clin Psychiatry. 2003;64(1):2717.
Wiklander M, Samuelsson M, Åsberg M. Shame reactions after suicide attempt. Scand J Caring Sci. 2003;17(3):293–300.
Hom MA, Stanley IH, Joiner TE Jr. Evaluating factors and interventions that influence help-seeking and mental health service utilization among suicidal individuals: A review of the literature. Clin Psychol Rev. 2015;40:28–39.
Nock MK, Park JM, Finn CT, Deliberto TL, Dour HJ, Banaji MR. Measuring the suicidal mind: Implicit cognition predicts suicidal behavior. Psychol Sci. 2010;21(4):511–7.
Menon V. Suicide risk assessment and formulation: an update. Asian J Psychiatry. 2013;6(5):430–5.
Appleby L, Kapur N, Shaw J, Hunt I, Ibrahim S, Turnbull P, Bojanić L, Rodway C, Tham SG, Richards N, Burns J. National Confidential Inquiry into Suicide and Safety in Mental Health: Annual Report: England, Northern Ireland, Scotland and Wales. Manchester: University of Manchester; 2019.
• Glashouwer KA, de Jong PJ, Penninx BW, Kerkhof AJ, van Dyck R, Ormel J. Do automatic self-associations relate to suicidal ideation?. J Psychopathol Behav Assess. 2010;32(3):428–37. This study is the first to use the depression IAT and the anxiety IAT and found that self-depressive associations were significantly related to suicidal behavior.
Tucker RP, Wingate LR, Burkley M, Wells TT. Implicit association with suicide as measured by the suicide affect misattribution procedure (s-amp) predicts suicide ideation. Suicide Life Threat Behav. 2018;48(6):720–31.
Greenwald AG, Banaji MR. Implicit social cognition: attitudes self-esteem and stereotypes. Psychol Rev. 1995;102(1):4.
Tosi JD, Ledesma RD, Poó FM, Montes SA, López SS. El test de asociaciones implícitas (implicit association test): Una revisión metodológica; Associação Iberoamericana de Diagnóstico e Avaliação Psicológica. Rev Iberoam de Diagnostico y Evaluacion Psicol. 2018;1(46):175–87.
Stroop JR. Studies of interference in serial verbal reactions. J Exp Psychol Gen. 1992;121(1):15.
Williams JM, Broadbent K. Distraction by emotional stimuli: use of a Stroop task with suicide attempters. Br J Clin Psychol. 1986;25(Pt 2):101–10.
Greenwald AG, McGhee DE, Schwartz JL. Measuring individual differences in implicit cognition: the implicit association test. J Pers Soc Psychol. 1998;74(6):1464.
Bar-Anan Y, Nosek BA. A comparative investigation of seven indirect attitude measures. Behav Res Methods. 2014;46(3):668–88.
Greenwald AG, Nosek BA, Banaji MR. Understanding and using the implicit association test: I An improved scoring algorithm. J Pers Soc Psychol. 2003;85(2):197.
McConnell AR, Leibold JM. Relations among the Implicit Association Test, discriminatory behavior, and explicit measures of racial attitudes. J Exp Soc Psychol. 2001;37(5):435–42.
Becker ES, Strohbach D, Rinck M. A specific attentional bias in suicide attempters. J Nerv Ment Dis. 1999;187:730–5.
Cha CB, Najmi S, Park JM, Finn CT, Nock MK. Attentional bias toward suicide-related stimuli predicts suicidal behavior. J Abnorm Psychol. 2010;119(3):616.
Chung Y, Jeglic EL. Use of the modified emotional Stroop task to detect suicidality in college population. Suicide Life Threat Behav. 2016;46(1):55–66.
Richard-Devantoy S, Ding Y, Turecki G, Jollant F. Attentional bias toward suicide relevant information in suicide attempters: A cross-sectional study and a meta-analysis. J Affect Disord. 2016;196:101–8.
Stewart JG, Glenn CR, Esposito EC, Cha CB, Nock MK, Auerbach RP. Cognitive control deficits differentiate adolescent suicide ideators from attempters. J Clin Psychiatry. 2017;78(6):614–21.
Hussey I, Barnes-Holmes D, Booth R. Individuals with current suicidal ideation demonstrate implicit “fearlessness of death.” J Behav Ther Exp Psychiatry. 2016;51:1–9.
Wells TT, Tucker RP, Kraines MA, Smith LM, Unruh-Dawes E. Implicit bias for suicide persists after ideation resolves. Psychiatry Res. 2020;285:112784.
Sohn MN, McMorris CA, Bray S, McGirr A. The death-implicit association test and suicide attempts: a systematic review and meta-analysis of discriminative and prospective utility. Psychol Med. 2021;51(11):1789–98.
Wilson KM, Millner AJ, Auerbach RP, Glenn CR, Kearns JC, Kirtley OJ, Najmi S, O’Connor RC, Stewart JG, Cha CB. Investigating the psychometric properties of the Suicide Stroop Task. Psychol Assess. 2019;31(8):1052.
Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Savović J, Schulz KF, Weeks L, Sterne JA. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343.
Barnes SM, Bahraini NH, Forster JE, Stearns-Yoder KA, Hostetter TA, Smith G, Nagamoto HT, Nock MK. Moving beyond self-report: implicit associations about death/life prospectively predict suicidal behavior among veterans. Suicide Life Threat Behav. 2017;47(1):67–77.
Barnes SM, Monteith LL, Forster JE, Nazem S, Borges LM, Stearns-Yoder KA, Bahraini NH. Developing predictive models to enhance clinician prediction of suicide attempts among veterans with and without PTSD. Suicide Life Threat Behav. 2019;49(4):1094–104.
Cha CB, O’Connor RC, Kirtley O, Cleare S, Wetherall K, Eschle S, Tezanos KM, Nock MK. Testing mood-activated psychological markers for suicidal ideation. J Abnorm Psychol. 2018;127(5):448.
Chiurliza B, Hagan CR, Rogers ML, Podlogar MC, Hom MA, Stanley IH, Joiner TE. Implicit measures of suicide risk in a military sample. Assessment. 2018;25(5):667–76.
Dickstein DP, Puzia ME, Cushman GK, Weissman AB, Wegbreit E, Kim KL, Nock MK, Spirito A. Self-injurious implicit attitudes among adolescent suicide attempters versus those engaged in nonsuicidal self-injury. J Child Psychol Psychiatry. 2015;56(10):1127–36.
Ellis TE, Rufino KA, Green KL. Implicit measure of life/death orientation predicts response of suicidal ideation to treatment in psychiatric inpatients. Arch Suicide Res. 2016;20(1):59–68.
Glenn CR, Kleiman EM, Coppersmith DD, Santee AC, Esposito EC, Cha CB, Nock MK, Auerbach RP. Implicit identification with death predicts change in suicide ideation during psychiatric treatment in adolescents. J Child Psychol Psychiatry. 2017;58(12):1319–29.
Glenn JJ, Werntz AJ, Slama SJ, Steinman SA, Teachman BA, Nock MK. Suicide and self-injury-related implicit cognition: a large-scale examination and replication. J Abnorm Psychol. 2017;126(2):199.
Glenn CR, Millner AJ, Esposito EC, Porter AC, Nock MK. Implicit identification with death predicts suicidal thoughts and behaviors in adolescents. J Clin Child Adolesc Psychol. 2019;48(2):263–72.
Harrison DP, Stritzke WG, Fay N, Ellison TM, Hudaib AR. Probing the implicit suicidal mind: does the Death/Suicide Implicit Association Test reveal a desire to ie, or a diminished desire to live? Psychol Assess. 2014;26(3):831.
Harrison DP, Stritzke WG, Fay N, Hudaib AR. Suicide risk assessment: Trust an implicit probe or listen to the patient? Psychol Assess. 2018;30(10):1317.
Kene P. Self-Injury Implicit Association Test: comparison of suicide attempters and non-attempters. Psychiatr Q. 2017;88(1):155–65.
Millner AJ, Coppersmith DD, Teachman BA, Nock MK. The Brief Death Implicit Association Test: scoring recommendations reliability validity and comparisons with the Death Implicit Association Test. Psychol Assess. 2018;30(10):1356.
Millner AJ, Augenstein TM, Visser KH, Gallagher K, Vergara GA, D’Angelo EJ, Nock MK. Implicit cognitions as a behavioral marker of suicide attempts in adolescents. Arch Suicide Res. 2019;23(1):47–63.
Moreno M, Porras-Segovia A, Lopez-Castroman J, Peñuelas-Calvo I, Díaz-Oliván I, Barrigón ML, Baca-García E. Validation of the Spanish version of the Death/Suicide Implicit Association Test for the assessment of suicidal behavior. J Affect Disord Rep. 2020;1:100012.
Rath D, Hallensleben N, Glaesmer H, Spangenberg L, Strauss M, Kersting A, Teismann T, Forkmann T. Implizite Assoziationen mit dem Tod: Erste Validierung einer deutschen Version des Impliziten Assoziationstests für Suizidalität (Suizid-IAT). Psychother Psychosom Med Psychol. 2018;68(03/04):109–17.
Rath D, Teismann T, Schmitz F, Glaesmer H, Hallensleben N, Paashaus L, Spangenberg L, Schönfelder A, Juckel G, Forkmann T. Predicting suicidal behavior by implicit associations with death? Examination of the death IAT in two inpatient samples of differing suicide risk. Psychol Assess. 2021;33(4):287–99.
Tello N, Harika-Germaneau G, Serra W, Jaafari N, Chatard A. Forecasting a fatal decision: direct replication of the predictive validity of the Suicide-Implicit Association Test. Psychol Sci. 2020;31(1):65–74.
Wang X, Lei W, Liu K, Liang X, Wang Y, Huang C, Zhang T, Chen J. Implicit measure of suicidal ideation in patients with depression. Death Stud. 2020;1–7.
Barnes-Holmes SC, Roche B. Relational frame theory: a post-Skinnerian account of human language and cognition. New York: Kluwer Academic Publishers; 2001.
Hughes S, Barnes-Holmes D, De Houwer J. The dominance of associative theorizing in implicit attitude research: propositional and behavioral alternatives. Psychol Rec. 2011;61(3):465–96.
Silverman MM, Berman AL, Sanddal ND, O’Carroll PW, Joiner TE. Rebuilding the tower of Babel: a revised nomenclature for the study of suicide and suicidal behaviors part 2: suicide-related ideations communications and behaviors. Suicide Life Threat Behav. 2007;37(3):264–77.
• Ballard ED, Reed JL, Szczepanik J, Evans JW, Yarrington JS, Dickstein DP, et al. Functional imaging of the implicit association of the self with life and death. Suicide Life Threat Behav. 2019;49:1600–8. https://doi.org/10.1111/sltb.12543. This study is the first in using functional imaging while performing the S-IAT and found specific patterns of neural activation that are associated with suicide-related words.
Ballard ED, Gilbert JR, Fields JS, Nugent AC, Zarate CA. Network changes in insula and amygdala connectivity accompany implicit suicidal associations. Front Psychiatry. 2020;11: 577628. https://doi.org/10.3389/fpsyt.2020.577628.
Malhi GS, Das P, Outhred T, Gessler D, John Mann J, Bryant R. Cognitive and emotional impairments underpinning suicidal activity in patients with mood disorders: an fMRI study. Acta Psychiatr Scand. 2019;139:454–63. https://doi.org/10.1111/acps.13022.
Keilp JG, Gorlyn M, Oquendo MA, Burke AK, Mann JJ. Attention deficit in depressed suicide attempters. Psychiatry Res. 2008;159:7–17.
Malloy-Diniz LF, Neves FS, Abrantes SS, Fuentes D, Corrêa H. Suicide behavior and neuropsychological assessment of type I bipolar patients. J Affect Disord. 2009;112(1–3):231–6.
Thompson C, Ong EL. The association between suicidal behavior attentional control and frontal asymmetry. Front Psychiatry. 2018;9:79.
• Cha CB, Wilson KM, Tezanos KM, Divasto KA, Tolchin GK. Cognition and self-injurious thoughts and behaviors: a systematic review of longitudinal studies. Clin Psychol Rev. 2019;69:97–111. https://doi.org/10.1016/j.cpr.2018.07.002. This review discovered that cognitive processes constitute a risk factor for self-injury and behavior, and outlined the importance of future research to develop cognitive profiles of self-injurious and suicidal behavior.
Keilp JG, Beers SR, Burke AK, Melhem NM, Oquendo MA, Brent DA, et al. Neuropsychological deficits in past suicide attempters with varying levels of depression severity. Psychol Med. 2014;44:2965–74. https://doi.org/10.1017/S0033291714000786.
• Harfmann EJ, Rhyner KT, Ingram RE. Cognitive inhibition and attentional biases in the affective go/no-go performance of depressed suicidal populations. J Affect Disord. 2019;256:228–33. This study demonstrated that cognitive dysfunction assessed by the go/no-go task was visible in participants with risk of suicide behavior, but more pronounced in participants with suicide ideation and history of suicide attempts.
Ballard ED, Gilbert JR, Wusinich C, Zarate CA Jr. New methods for assessing rapid changes in suicide risk. Front Psychiatry. 2021;12:31.
• Baucom BR, Georgiou P, Bryan CJ, Garland EL, Leifker F, May A, Wong A, Narayanan SS. The promise and the challenge of technology-facilitated methods for assessing behavioral and cognitive markers of risk for suicide among US Army National Guard Personnel. Int J Environ Res Public Health. 2017;14(4):361. This review exposed the usefulness of assessment tools for measuring suicide risk, and argued that computer-based task are useful in the evaluation of the suicidal risk in both military and other sensitive setting.
Hall RC, Platt DE, Hall RC. Suicide risk assessment: a review of risk factors for suicide in 100 patients who made severe suicide attempts: Evaluation of suicide risk in a time of managed care. Psychosomatics. 1999;40:18–27.
Meliá R, Francis K, Hickey E, Bogue J, Duggan J, O’Sullivan M, Young K. Mobile Health Technology Interventions for Suicide Prevention: Systematic Review. JMIR mHealth uHealth. 2020;8(1): e12516. https://doi.org/10.2196/12516.
Sedano-Capdevila A, Porras-Segovia A, Bello HJ, Baca-García E, Barrigon ML. Use of ecological momentary assessment to study suicidal thoughts and behavior: a systematic review PMID: 34003405. Curr Psychiatry Rep. 2021;23(7):41. https://doi.org/10.1007/s11920-021-01255-7.
Funding
This study received grant support from Instituto de Salud Carlos III (ISCIII PI13/02200; PI16/01852; CM19/00026), the American Foundation for Suicide Prevention (LSRG-1–005-16), the Spanish Ministry of Science, Innovation, and Universities (RTI2018099655-B-I00; TEC2017-92552-EXP), the regional government of Madrid (Y2018/TCS-4705, PRACTICO-CM, PEJD-2018-PRE/SAL8417), and the program providing financial support for the hiring of predoctoral and postdoctoral researchers, co-funded by the European Social Fund through the Operational Program on Youth Employment and the Youth Employment Initiative (YEI) (PEJD-2018-PRE/SAL8417).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics Approval
Not applicable.
Consent to Participate
Not applicable.
Conflict of Interest
The authors declare no competing interests.
Human and Animal Rights and Informed Consent
This article does not contain any studies with human or animal subjects performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original online version of this article was revised: The name of our co-author should be corrected to ‘Luis Gutiérrez-Rojas ’.
This article is part of the Topical Collection on Mood Disorders
Rights and permissions
About this article
Cite this article
Moreno, M., Gutiérrez-Rojas, L. & Porras-Segovia, A. Implicit Cognition Tests for the Assessment of Suicide Risk: a Systematic Review. Curr Psychiatry Rep 24, 141–159 (2022). https://doi.org/10.1007/s11920-022-01316-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11920-022-01316-5