Abstract
Internal bodily signals provide an essential function for human survival. Accurate recognition of such signals in the self, known as interoception, supports the maintenance of homeostasis, and is closely related to emotional processing, learning and decision-making, and mental health. While numerous studies have investigated interoception in the self, the recognition of these states in others has not been examined despite its crucial importance for successful social relationships. This paper presents the development and validation of the Interoceptive States Static Images (ISSI), introducing a validated database of 423 visual stimuli for the study of non-affective internal state recognition in others, freely available to other researchers. Actors were photographed expressing various exemplars of both interoceptive states and control actions. The images went through a two-stage validation procedure, the first involving free-labelling and the second using multiple choice labelling and quality rating scales. Five scores were calculated for each stimulus, providing information about the quality and specificity of the depiction, as well as the extent to which labels matched the intended state/action. Results demonstrated that control action stimuli were more recognisable than internal state stimuli. Inter-category variability was found for the internal states, with some states being more recognisable than others. Recommendations for the utilisation of ISSI stimuli are discussed. The stimulus set is freely available to researchers, alongside data concerning recognisability.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Internal bodily signals, such as hunger, thirst, fatigue, nausea, pain, temperature, and cardiac and respiratory signals, are essential for human survival, indicating the physiological state and functioning of the body (e.g. the sensation of thirst signalling the level of dehydration in the body). The ability to perceive and identify these internal sensations, known as interoception (Craig, 2003a), is fundamental to multiple psychological processes, such as emotional processing (e.g., Critchley & Garfinkel, 2017; Garfinkel & Critchley, 2013; Schachter & Singer, 1962; Seth, 2013), and learning and decision-making (Bechara & Damasio, 2002; Dunn et al., 2010; Werner et al., 2009). Furthermore, a growing body of research has linked interoception to mental health and subjective wellbeing; atypical perception of interoceptive states has been found in several mental health conditions and neurodevelopmental disorders, such as Eating Disorders (Klabunde et al., 2013; Pollatos et al., 2008), autism (Garfinkel et al., 2016; Hatfield et al., 2019; Mul et al., 2018; Nicholson et al., 2019), anxiety and Panic Disorder (Ehlers, 1993; Paulus & Stein, 2006; see Khalsa et al., 2018 for a review), depression (Dunn et al., 2007; Furman et al., 2013; Forrest et al., 2015; Harshaw, 2015; see Eggart et al., 2019 for a review), and schizophrenia (Ardizzi et al., 2016). Given the vital role of interoception in understanding typical emotion processing and learning and decision-making, as well as its atypicality in several mental health conditions, research on interoception and emotion has grown significantly in recent years.
While numerous studies have focused on the perception of interoceptive states in the self, very few (e.g., Kaulard et al., 2012) have researched the recognition of these states in others, beyond the domain of affective emotion (e.g., happiness, anger, sadness). Recognition of others’ affective emotional states (which feature an interoceptive component; Schachter & Singer, 1962) has been studied in detail, in typical adulthood, clinical samples, and across development; indeed a PubMed search using the term “emotion recognition” generated 15,009 results. Recognition of others’ emotional states is crucial for successful social interactions, as well as building and maintaining relationships, making it an important area for psychological research. Recognition of interoceptive states (beyond the affective domain) in others, including identifying others’ hunger, nausea, pain, and breathlessness, for example, is presumably equally important for social interaction, and arguably more important from an evolutionary perspective (as identifying perturbations in these states is necessary in order to offer care and assistance to others). Similarly, studying the mechanisms behind the ability to recognise other people’s bodily sensations is crucial to improve our understanding of empathy for these states in others, with important theoretical and clinical implications. It is somewhat surprising, therefore, that research has, thus far, neglected to investigate this ability.
One reason for the dearth of research investigating the recognition of others’ non-affective internal states is presumably the lack of available stimuli. Compared to affective emotion recognition research, the lack of stimuli for the investigation of non-affective state recognition is striking. Since the publication of the “Pictures of Facial Affect” (Ekman & Friesen, 1976), the first standardised battery of facial emotion stimuli, several databases of visual stimuli depicting facial and bodily affective expressions have been developed (e.g., Beaupré et al., 2000; Langner et al., 2010; Lundqvist et al., 1998; Matsumoto & Ekman, 1988; Volkova et al., 2014; Wingenbach et al., 2016). Visual stimuli depicting facial and bodily expressions of affective states have been a key component of emotion research, and substantially contributed to our knowledge of affective and cognitive neuroscience, and social and clinical psychology. A purpose-built battery of stimuli depicting non-affective interoceptive states in others will enable research on social cognition to investigate the ability to perceive and recognise these signals in others. This will lead to an expansion of our theoretical understanding of the constructs of interoception and social perception in typical adult populations, developmental samples and clinical groups, both at the behavioural and neurological levels.
This report presents the development and validation of the Interoceptive States Static Images (ISSI), a database of full body static images of actors expressing either a non-affective interoceptive state or a control action, which is freely available to other researchers. The battery consists of 423 stimuli, which depict eight actors expressing various exemplars of nine internal states and nine control actions. All photos were taken from a frontal view in a controlled environment, and underwent a standardised image processing procedure to control for lighting conditions, size, position, and background. Stimuli were validated in two stages, one utilising free labelling and the other utilising visual analogue rating scales. Recognition data for each individual stimulus and for the state and control actions overall, including the extent to which they are confused with each other, are provided.
Methods
Stimulus development
Actors
Eight trained actors (four female) aged 22 to 48 were recruited through online and campus advertisement. Neither ethnicity nor first language was specified as a recruitment criterion, but they were recorded. No specific ethnic group was targeted for recruitment and no actors were excluded based on their ethnicity. The recruitment was interrupted once the necessary number of actors was reached. All the actors who responded to our recruitment call reported being of Caucasian ethnicity. Actors were either drama students or had previously completed acting training. Actors were informed about the procedure and the purpose of the stimulus set, and gave their consent to take part in the recording session and for their images to be used in scientific research, presented at conferences, published in academic journal articles, and shared with other researchers. Actors received financial remuneration for their time.
Procedure
Prior to the recording session, actors were provided with the list of the internal states and actions they would be required to perform, and were asked to practice depicting each state or action prior to the recording session. During the recording session, they were required to wear black trousers, black socks, and a black t-shirt. Female actors were asked not to wear make-up and to tie their hair to ensure their face was completely visible at all times. Photos were taken in a purpose-built photography studio. Actors stood in a specified position in the centre of a white background, facing a camera placed on a tripod. Softbox LED lighting was used to control lighting conditions across different shooting sessions, and to reduce shadows.
Actors first produced ten control actions (jumping, clapping, lifting, running, washing hands, spinning/twirling, stumbling, walking, waving, beckoning) and then expressed ten non-emotional internal states (cold, fatigue, nausea, pain, breathlessness, hunger, thirst, hot, satiety, itch). For each stimulus category, the actor was asked to practice before posing the state or control action five separate times, which were used as different exemplars of the same stimulus. Between each attempt, the actor was asked to re-set to a neutral body position and to re-position in the middle of the background. Between each stimulus type, a longer break was given to allow actors to rest and prepare for the next stimulus category. The order of stimulus production was fixed and did not vary across actors.
Image processing
Raw photos were edited in Adobe Photoshop 2019. The backdrop was replaced with an artificial white matte background. Image artefacts and distracting visual information (e.g. tattoos) were removed. Brightness and contrast were adjusted and standardised across different images. Sharpness was increased with the function Smart Sharpness by 309%, with 0.6-pixel radius, 100% noise reduction, and by removing lens blur.
Image size and actor position were matched across stimuli using a 3456 × 5184 pixels white template. The first stimulus was positioned in the centre of the template to reach the desired size, which was sized to subtend 12° of visual angle vertically when viewed at 60 cm. Guidelines were drawn to delimit the boundaries of the actor in this position (extremes of head and feet in the vertical axis and extremes of right and left shoulders in the horizontal axis) and to provide a frame of reference for all the subsequent stimuli. For each actor, images were layered onto the original template and the size and position of the actor was adjusted to fit these guidelines. Each layered image was saved as a new file.
Stimuli validation
All stimuli went through a pre-selection process based on basic visual properties by the researchers. Photos where parts of the actor’s body were missing (e.g. the head being outside the picture top edge in some ‘jumping’ exemplars), or where motion blur could not be resolved through editing, were removed from the database. This resulted in all stimuli depicting the control action ‘stumbling’ being removed, due to a high proportion of images including several of these issues. To retain an equal number of control actions and internal states, stimuli depicting ‘thirst’ were also removed, owing to actors reporting that this state was difficult to portray because of the lack of a visible behavioural response to feeling thirst, and the authors agreeing that stimuli were not recognisable as depicting thirst. For each actor, and for each stimulus category, the four exemplars with the highest visual quality, judged by the researchers, were selected to be included in the first validation task, yielding a total of 560 stimuli.
Stimulus selection: Free-labelling task
Forty participants (four male) aged 18–30 years (M = 19.05, SD = 2.68) were recruited through Royal Holloway, University of London (RHUL) SONA System to take part in a free-rating task. Participants were all students at RHUL and received course credits for their participation. There were no exclusion criteria for this task, although any diagnosis of a mental health condition was recorded. A general description of the task procedure was provided but participants were not informed of the aim of the study until the end of the session, to avoid influencing their responses. Stimuli were divided into two sets of 280 images. Participants viewed one of the two sets, in order to reduce fatigue. Instructions were standardised across participants and the experimenter provided them verbatim as follows: You will see a series of body postures, one by one. For each one, you need to provide a very brief description of what you think the body posture represents (for example what the person is doing, thinking or feeling). There will be many stimuli, so it’s very important that you keep your answers as brief as possible. Ideally, you will use a single word or a short phrase. For example, if you see an image depicting a person sneezing, you can simply answer ‘sneezing’. If you think that the person could be doing, thinking or feeling more than one thing, you can give multiple answers, but please try to keep the description of each one brief. If I need more details, I will ask for them. There are not right or wrong answers, so I will not provide any feedback during or after the session. I will simply record your answers and occasionally intervene if I think something is not clear or if I need more details. Following the instructions, participants were invited to ask any questions they may have about the procedure. Then, the experimenter sat a few meters behind the participant and typed their responses verbatim. When additional information was required, the experimenter used standardised phrases to prompt the participant. If the answer required more details, the experimenter would say “Can you tell me more about that?”. If the answer was ambiguous/unclear, the experimenter would say “Can you be more specific?” or “Can you tell me what you mean by that?”. Finally, if participants’ responses were too verbose, the experimenter would say “Try to use single words or short phrases”. This task took approximately 20 min to complete.
Stimulus validation: Label selection and rating task
Based on the results of the free-rating task, 423 stimuli were selected to be used in the second step of validation (details on the selection procedure can be found in the Results section). Of these, 202 stimuli depicted nine internal states (breathlessness, cold, fatigue, hot, hunger, itch, nausea, pain, satiety) (Fig. 1a) and 221 stimuli depicted nine control actions (beckoning, clapping, jumping, lifting, running, twirling, walking, washing hands, waving) (Fig. 1b). Participants were recruited from the RHUL Sona System, Testable Minds database (www.testable.org), and through advertisements on social media. A total of 412 participants (169 female) aged 18–71 years (M = 30.08, SD = 10.56) with no diagnosis of any mental health condition took part in an online labelling task. The task was programmed in Testable and presented participants with a random sample of 100 stimuli. On each trial, a single image was presented in the centre of the computer screen and remained visible until participants had finished responding. Participants were provided with a list of the nine internal state and nine action labels (presented in alphabetical order) and asked to select which label best described the image. If they were unsure, participants could select more than one label or skip to the next trial if they thought that no label applied. Following label selection, participants were prompted to rate how well each chosen label described the image, using a five-point Likert scale (Very Poorly; Poorly; Moderately; Well; Very Well). This task took approximately 30 min to complete.
Results
Free-labelling task
Participants’ responses in this stage were analysed qualitatively. First, two coders independently coded the responses for accuracy (referring to identification of the intended state or action). A score of 1 was given to those responses which either correctly identified the state or action or, for state stimuli, correctly described the action portrayed, associated with the state (e.g. for the ‘fatigue’ stimuli, both ‘tired’ and ‘yawning’ were considered correct responses). A score of 0 was given to inaccurate responses (e.g. ‘hot’ or ‘shocked’ to describe a ‘breathlessness’ stimulus). In instances where coders disagreed, responses were discussed by all authors until an agreement was found. Inter-coder agreement was near perfect (k = 0.81).
Each stimulus was given a recognisability index(RI) which corresponded to the mean accuracy score. Overall, internal state and action stimuli were recognised correctly 65% and 75% of the time, respectively. Of the internal states, itch (M = 88%, SD = 13%, range 55–100%) and cold (M = 88%, SD = 15%, range 55–100%) were the best recognised states, while hunger was the least well recognised state (M = 22%, SD = 8%, range 10–40%). Among the control actions stimulus set, walking was the best recognised state (M = 91%, SD = 12%, range 55–100%), whereas beckoning stimuli were the least well recognised (M = 49%, SD = 11%, range 30–75%). See Table 1 for a full summary of RIs.
Based on the RI, each stimulus was categorised according to recognisability, into five categories: Very poor (RI scores 0.0–0.2), Poor (RI scores 0.21–0.4), Average (RI scores 0.41–0.6), Good (RI scores 0.61–0.8), and Very good (RI scores 0.81–1). All stimuli categorised as Very good, Good, and Average were kept in the final database. In addition, we retained a minimum of two exemplars per actor for each stimulus category. For stimulus categories where fewer than two stimuli for a given actor were categorised as Very good, Good, or Average, the two stimuli with the highest RI were retained. See Appendix Table 3 for RI scores for every retained stimulus. A final set of 423 stimuli was retained and used in the second stage of validationFootnote 1. In this final stimulus set, 209 stimuli depicted male actors, whilst 214 depicted female actors. Each of the eight actors was present in at least 50 stimuli, and the most depicted actor appeared in 56 stimuli.
Label selection and rating task
Quality and Accuracy Scores
Each stimulus was rated by a mean of 97 participants (Min = 74, Max = 123). There are multiple ways in which the validity and quality of stimuli can be defined, so to allow researchers to select stimuli based on their own requirements, a comprehensive range of stimulus measures has been created and is provided below. For each stimulus, five separate scores were calculated: the quality index(QI); the specificity index(SI); the maximum-distractor specificity index (SI+), the choice rate(CR); and the high-quality choice rate (CR+) (Table 2). The scores were calculated based on the ratings of the whole sample (both female and male observers), as well as on ratings of female and male observers separately.
The QI is a score ranging between 0 and 5, and was computed by taking the mean (across all stimulus ratings) of all quality judgements given to the target (intended) label. A score of 0 was assigned whenever the target label was not selected. The QI therefore reflects the extent to which the target label is perceived as describing the image well. High QI scores indicate that the target label describes the image very well. Conversely, lower QI scores indicate that the target label does not describe the stimulus well. The SI reflects the extent to which the target label is perceived as a good description of the image, over and above distractor states or action labels. SI was computed by subtracting the mean rating given to selected distractor labels from the rating given to the target label, and taking the mean of these values across all stimulus ratings. SI values range between – 5 and 5. Negative values indicate that the target label received a lower score than the distractor labels taken together. Conversely, positive values signify that the target label received a higher rating compared to distractor labels taken together. The SI+ was obtained by subtracting the highest distractor rating from the rating given to the target label, and taking the mean of these values across all stimulus ratings. SI+ is a score ranging between – 5 and 5, whereby negative values indicate that distractor labels were given higher ratings than the target label, whilst positive values indicate that the target label received a higher rating than the distractor with the highest rating. The SI and SI+ are more conservative scores than the QI, as they take into account the discrepancy between ratings of intended and unintended labels. Values of SI and SI+ close to 0 indicate that the target label is not perceived to be a better description of the stimulus than the distractor labels. The CR consists of the proportion of participants who selected the target label to describe the stimulus, regardless of the quality rating given. CR scores range between 0% to 100%, whereby 0% indicates that the target label was never selected to describe the image, whilst 100% indicates that the target label was always selected to describe the image. The CR+ is the proportion of participants who gave the target label the highest quality rating of all labels. CR+ was calculated by assigning a score of 1 to those stimuli whose target label received the highest quality rating. Whenever a distractor obtained a quality rating equal to or higher than the target, a score of 0 was assigned. CR+ scores of 0% indicate that the target label was never rated higher than distractor labels when describing the image. CR+ scores of 100% indicate that the target label always received the highest rating, compared to distractor labels, when describing the image. All five scores are presented for each stimulus in Table 3 in the Appendix.
Whole-sample analyses revealed that QI scores were higher for action stimuli (M = 3.62, SD = .55) than for internal state stimuli (M = 3.22, SD = 1.02) [t(421) = – 5.09, p < .001]. Separate ANOVAs were conducted for QI of internal states and QI of control actions, with Stimulus Category (all internal state/action stimulus categories) and Actor Sex (male, female) as IVs. For the internal states, a significant main effect of Stimulus Category [F (17, 184) = 62.97, p < .001, η2 = .73] was found. Cold received the highest QI (M = 4.2, SD = .35). Conversely, the lowest QI was attributed to hunger (M = 1.67, SD = .44) (Fig. 2a). Post hoc t tests were conducted across all pairs of states with Bonferroni corrections and are shown in Fig. 2a. Similarly, the ANOVA for the action stimuli resulted in a significant main effect of Stimulus Category [F (17, 203) = 2.63, p = .009, η2 = .09]. Clapping stimuli had the highest QI (M = 3.8, SD = .59), while the mean QI for twirling was the lowest of the action stimulus set (M = 3.18, SD = .60) (Fig. 2b). Post hoc t tests comparing all pairs of actions, with Bonferroni corrections, and are shown in Fig. 2b). Actor Sex did not contribute to variations in QI for either internal states [F (17, 184) = .06, p = .81] or control actions [F (17, 203) = .49, p = .48], and did not interact with Stimulus Category in either internal states [F(17, 184) = 1.75, p = .09] or control actions [F(17, 203) = 1.38, p = .21].
SI scores were higher for action stimuli (M = 3.01, SD = .86) than internal state stimuli (M = 1.90, SD = 1.84) [t(421) = – 8.08, p < .001)]. Again, separate ANOVAs were conducted for the action and the internal states stimulus sets, with SI as the DV and Stimulus Category and Actor Sex as IVs.
For the internal states, the main effect of Stimulus Category was significant [F (17, 184) = 63.197, p < .001, η2 = .73]. Cold had the highest SI (M = 3.71, SD = .50), while SI was lowest for satiety (M = – 1.07, SD = .91) (Fig. 3a). Post hoc t tests for Stimulus Category using Bonferroni corrections are shown in Fig. 3a. There was a significant main effect of Actor Sex [F (17, 184) = 5.04, p = .02, η2 = .03], whereby SI scores were higher for stimuli depicted by female actors (M = 1.99, SD = 1.77) than those portraying male actors (M = 1.81, SD = 1.90). Actor Sex did not interact significantly with Stimulus Category [F (17, 184) = 1.96, p = .054]. For the action stimulus set, a significant main effect of Stimulus Category [F (17, 203) = 4.76, p < .001, η2 = .16] was observed. Beckoning and twirling had the highest (M = 3.36, SD = .58) and lowest (M = 2.21, SD = 1.14) SIs, respectively (Fig. 3b). Post hoc t tests using Bonferroni corrections were conducted on Stimulus Category and are reported in Fig. 3b. The main effect of Actor Sex [F (17, 203) = .41, p = .52] and the interaction between Actor Sex and Stimulus Category [F (17, 203) = 1.67, p = .11] were non-significant.
SI+ was significantly higher for action stimuli (M = 2.99, SD = .87) than for internal state stimuli (M = 1.82, SD = 1.89) [t(421) = – 8.21, p < .001]. Separate ANOVAs were conducted for the action and internal state stimulus sets, with Stimulus Category and Actor Sex as IVs and SI+ scores as the DV. A significant main effect of Stimulus Category was found for internal state stimuli [F (17, 184) = 63.797, p < .001, η2 = .735]. Cold and Satiety had the highest (M = 3.68, SD = .51) and lowest (M = – 1.25, SD = .95) SI+, respectively (Fig. 4a). Bonferroni-corrected post hoc t tests were conducted on all the levels of Stimulus Category and are reported in Fig. 4a. A significant main effect of Actor Sex was found [F (17, 184) = 5.35, p < .05, η2 = .03] whereby stimuli depicting female actors (M = 1.93, SD = 1.82) received slightly higher SI+ score than those depicting male actors (M = 1.73, SD = 1.96). Actor Sex did not interact with Stimulus Category [F (17, 184) = 1.93, p = .06]. The ANOVA for SI+ scores of action stimuli resulted in a significant main effect of Stimulus Category [F (17, 203) = 4.87, p < .001, η2 = .16]. Beckoning was the category to receive the highest SI+ scores (M = 3.34, SD = .58), whilst Twirling received the lowest SI+ scores (M = 2.17, SD = 1.16) (Fig. 4b). Post hoc t tests for each pair of action categories were conducted, using Bonferroni corrections (Fig. 4b). Actor sex did not contribute to variation in total SI+ scores [F (17, 203) = .38, p = .54] and did not interact with Stimulus Category [F (17, 203) = 1.65, p = .11].
CR scores showed that participants selected the target label to describe action stimuli (M = 86%, SD = 8%) significantly more often than they did to describe internal states (M = 77%, SD = 20%) [t(421) = -6.10, p < .001]. ANOVAs were computed for CR scores of internal states and control actions separately, with Stimulus Category and Actor Sex as IVs. For the internal states, a significant main effect of Stimulus Category [F (17, 184) = 73.89, p < .001, η2 = .76] was observed. Cold was the state with the highest CR (96%), whilst satiety had the lowest CR (40%) (Fig. 5a). Post hoc t tests using Bonferroni corrections are displayed in Fig. 5a. There was no main effect of Actor Sex [F (17, 184) = .09, p = .93] or interaction between Actor Sex and Stimulus Category [F (17, 184) = 1.08, p = .38]. The ANOVA for the action stimuli resulted in a significant main effect of Stimulus Category [F (17, 203) = 4.03, p < .001, η2 = .14]. Clapping had the highest CR (89%), whereas twirling was the action with the lowest CR (78%) (Fig. 5b). Post hoc t tests with Bonferroni correction across all categories are shown in Fig. 5b. Actor Sex did not contribute to variations of CR [F (17, 203) = 1.25, p = .26] or interact with Stimulus Category [F (17, 203) = .83, p = .58].
Finally, CR+ scores for action stimuli (M = 84%, SD = 10%) were significantly higher than CR+ scores for internal state stimuli (M = 69%, SD = 25%) [t(421) = – 8.56, p < .001]. Once again, separate ANOVAs were conducted for the CR+ scores of action and internal states stimuli, with Stimulus Category and Actor Sex as IVs. For the internal states, a main effect of Stimulus Category was found [F (17, 184) = 61.80, p < .001, η2 = .73]. Cold stimuli received the highest (M = 92%, SD = 6%) CR+ scores, whilst Satiety stimuli had the lowest CR+ scores (M = 30%, SD = 12%) (Fig. 6a). Bonferroni corrected post-hoc t-tests across all pairs of states are shown in Fig. 6a. A significant main effect of Actor Sex was found [F (17, 184) = 5.77, p < .05, η2 = .03] whereby internal state stimuli depicting female actors (M = 70%, SD = 24%) received slightly higher CR+ scores than those depicting male actors (M = 67%, SD = 25%). Finally, Actor Sex did not interact significantly with Stimulus Category [F (17, 184) = 1.62, p = .12]. The ANOVA for the action stimuli returned a significant main effect of Stimulus Category [F (17, 203) = 6.27, p < .001, η2 = .198], whereby Walking and Twirling received the highest (M = 89%, SD = 8%) and lowest (M = 75%, SD = 14%) CR+ scores, respectively (Fig. 6b). Post-hoc t-tests with Bonferroni corrections across all pairs of actions are shown in Fig. 6b. The effect of Actor Sex on variations of CR+ scores did not reach statistical significance [F (17, 203) = .41, p = .52]. Likewise, Actor Sex did not interact with Stimulus Category [F (17, 203) = 1.71, p = .10].
To investigate the effect of observer gender on the evaluation of internal state stimuli, separate ANOVAs were conducted for the five recognition indices with Stimulus Category (all the internal states) and Observer Gender (female, male) as factors. The ANOVA with QI as DV did not reveal a main effect of Observer Gender [F(17, 386) = .57, p = .45, η2 = .001]. Moreover, Observer Gender did not interact significantly with Stimulus Category [F(17, 386) = .77, p = .63, η2 = .02]. The ANOVA for SI scores returned a main effect of Stimulus Category [F(17, 386) = 111.41, p < .001, η2 = .698], and a main effect of Observer Gender [F(17, 386) = 10.74, p = .001, η2 = .03], whereby female observers (M = 2.09, SD = 1.83) had higher SI indices than male observers (M = 1.76, SD = 1.90). Observer Gender did not interact significantly with Stimulus Category [F(17, 386) = .57, p = .80, η2 = .01]. The ANOVA for SI+ scores revealed a main effect of Stimulus Category [F(17, 386) = 112.55, p < .001, η2 = .70], and a main effect of Observer Gender [F(17, 386) = 11.25, p = .001, η2 = .03], with female observers (M = 2.03, SD = 1.88) having higher SI+ indices than male observers (M = 1.67, SD = 1.95), but no interaction between Observer Gender and Stimulus Category [F(17, 386) = .54, p = .82, η2 = .01]. The ANOVA for CR scores resulted in a main effect of Stimulus Category [F(17, 386) = 123.91, p < .001, η2 = .72], and a main effect of Observer Gender [F(17, 386) = 7.13, p = .008, η2 = .02], where female observers (M = 95.53, SD = 4.65) had slightly higher CR scores than males (M = 94.87, SD = 4.87). The interaction between the two factors was non-significant [F(17, 386) = .68, p = .708, η2 = .01]. Finally, the ANOVA for CR+ scores returned a main effect of Stimulus Category [F(17, 386) = 108.03, p < .001, η2 = .691], and a main effect of Observer Gender [F(17, 386) = 5.66, p = .02, η2 = .01], with higher CR+ scores in female observers (M = 72.59, SD = 23.70) than male observers (M = 69.34, SD = 24.50), but no interaction between Observer Gender and Stimulus Category [F(17, 386) = .56, p = .812, η2 = .01].
Confusion across stimulus categories
In order to determine which states/actions were confused with each other, confusion scores were created based on CR and CR+ scores. Confusion matrices were created whereby each row corresponds to the intended state or action portrayed by the actor, and each column represents the proportion of times each state or action label was selected regardless of quality rating in the CR matrix, and the proportion of times each state or action label was given the highest quality rating in the CR+ matrix. Among the internal states, some categories were particularly confused with others; Hunger stimuli were often rated as depicting Pain (CR = 46%; CR+ = 20%) and Nausea (CR = 40%; CR+ = 16%), Satiety stimuli were also rated as depicting Hunger (CR = 39%; CR+ = 25%) and Nausea (CR = 36%; CR+ = 17%), and Nausea stimuli were often rated as depicting Pain (CR = 30%; CR+ = 7%) (Fig. 7a). On the other hand, the confusion matrix for action stimuli revealed lower levels of confusion (i.e. target actions were less often labelled as non-target actions). Clapping stimuli were sometimes labelled as depicting Washing Hands (CR = 22%; CR+ = 10%), Running stimuli were also rated as depicting Walking (CR = 23%; CR+ = 11%), Twirling stimuli were sometimes rated as depicting Jumping (CR = 16%; CR+ = 6%), Waving stimuli were also rated as depicting Beckoning (CR = 14%; CR+ = 6%), and Beckoning stimuli were occasionally labelled as depicting Waving (CR = 10%; CR+ = 4%) (Fig. 7b).
Discussion
The current report presents the creation and validation of the ISSI database, a novel stimulus set of 423 static images representing non-affective internal bodily states and control actions. Each stimulus is presented alongside a range of indices from the second stage of validation, representing the quality and specificity of depiction, and the extent to which each stimulus was recognised as the intended state or action. Confusion matrices of internal states and control actions are also included to provide an indication of which states and which actions tend to be confused with each other. The stimuli are freely available to researchers for their use in scientific research and can be downloaded from the Insulab website (https://www.insulab.uk).
Overall, 77% (Ra: 40–96%) of participants selected the intended label to describe the internal state stimuli, and 86% (Ra: 78–89%) of participants selected the intended label to describe action stimuli. When observer gender was considered, female observers gave higher ratings and were more likely to select the intended label for the stimuli compared to male observers. Within the internal state stimulus set, there was high variability between stimulus categories in terms of quality and specificity of depiction, and proportion of participants selecting the target state, with the pattern of results across different indices being relatively consistent. Satiety was the most difficult state to recognise and discriminate from other states, followed by hunger. Hunger and satiety stimuli were given fairly low quality (QI) scores, with the majority being given a mean score below 2 (‘Poor’ on the rating scale), and negative specificity (SI and SI+) scores, indicating that distractor labels were often judged to be better descriptors of the stimulus than the target label. Similarly, CR and CR+ scores were often under 50%, indicating that the target label was selected to describe the stimulus (CR), or as the best descriptor of the stimulus (CR+), less than half of the time. Other internal state categories, however, were given high quality and specificity ratings, and the intended label was selected frequently. The vast majority of cold, itch, pain, fatigue and nausea stimuli, for example, were given QI scores above 3, positive SI/SI+ scores, and CR/CR+ scores above 70%. While there is therefore variability across internal state categories and individual stimuli, all stimuli rated in the second validation stage have been retained in the final stimulus set, in order for researchers to select stimuli according to their own research requirements. While we would recommend using stimuli with high quality, specificity and choice rate scores where studies require stimuli that have been validated and are recognisable by typical participants as their intended state, the range of recognition scores also allows for the study of ambiguous stimuli, or internal states that are easily confused. Notably, action stimuli were consistently recognised better than internal state stimuli, and there was less variability among different action stimulus categories in terms of quality and specificity ratings, and the extent to which the intended label was selected to describe the stimuli. Variability was also observed across actors, both in terms of quality of depiction and recognisability of the stimuli produced. Individual differences in the ability to produce recognisable non-affective internal states are expected, and elucidating the predictors of such differences should be investigated in future research. Previous works on facial expressions of emotion indicates, for example, that autistic individuals produce less typical emotional expressions compared to neurotypical individuals (e.g., Brewer et al., 2016; Langdell, 1981). Further research is needed to elucidate whether a similar pattern is observed for the expression of interoceptive states.
It is likely that internal states were recognised less well than action stimuli due to the associations and similarities between internal states giving rise to greater confusability. In particular, there is an over-representation of gastric internal signals in the current stimulus set (i.e. nausea, hunger, satiety), which could be responsible for lower specificity scores and choice rates for these stimulus categories. Actors frequently expressed these internal states by placing their hands on or around the abdomen, likely making these stimuli difficult to differentiate. Crucially, despite variability in the low-level visual features of the stimuli within state categories, there was consistency across actors’ depictions of states, and visual cues were often in line with those that would be expected based on the location at which states are perceived within the body (e.g., the abdomen). Notably, recognition scores are likely to be dramatically increased if fewer gastric response options are available to participants (e.g., researchers could include nausea, hunger, and satiety under the same umbrella term ‘gastric discomfort’); in the current validation task, the availability of all target labels may have led to more conservative recognition estimates, while in a two-alternative forced choice task where stimuli must be labelled as either cold or satiety, for example, it is likely that participants would perform near ceiling, as the visual cues associated with these states are highly distinct. Indeed, in tasks assessing affective emotion recognition, recognition accuracy is improved by having fewer available response options, or less confusable response options in alternative forced choice tasks. For example, angry facial expressions were less likely to be labelled as depicting anger in a task where response options included “anger”, “frustration”, and “contempt” than when fewer closely related response options were available (Russell, 1993). Similarly, recognition of happiness expressions (which often shows ceiling effects even in those with difficulties recognising other facial expressions) has been found to be impaired in those with emotion processing impairments (alexithymia) when stimuli depicting pain are included in the recognition task, likely due to painful expressions sharing perceptual characteristics with happy expressions (Brewer et al., 2015). In contrast, action stimulus categories were more distinct from each other in their associated behavioural cues, and therefore less confusable with each other.
Naturalness of expression may have also played a role in the disparity of recognition scores among stimulus categories. Although visual behavioural expressions of states such as hunger and satiety, such as rubbing the abdomen (hunger and satiety) or exhaling heavily (satiety), do occur, they may be less spontaneous than behavioural expressions of other states such as feeling cold (e.g. rubbing one’s arms) or feeling itchy (e.g. scratching one’s skin). This may be due to the behavioural responses to cold and itch serving a purpose to reduce the internal state, and thus being performed more often, rather than serving a more communicative purpose and therefore only being used in social situations, and potentially less frequently. Similarly, actions that are performed with a more communicative purpose may be more frequently accompanied by a verbal description (e.g. stating ‘I’m so hungry’ while rubbing one’s abdomen), reducing the requirement for an observer to recognise the visual signals. It is worth noting that facial expressions of affective states can be either spontaneous or posed for communicative purposes, and these tend to differ in their visual features, such as onset time, duration, and amplitude of physical facial movement (Schmidt et al., 2006; Valstar et al., 2006). It is likely that spontaneous and posed/communicative expressions of non-affective internal states also differ, and the extent to which they differ may vary across internal states. It is possible that actors’ depictions of internal states were therefore more recognisable for states where spontaneous and posed expressions of the state are more similar, making the actors’ depiction more ecologically valid. For states which either are infrequently expressed, or for which spontaneous expressions differ greatly from posed expressions, actors’ depictions may have been less recognisable. Notably, the communicative value of individual internal states may also vary across different cultures. Future research is needed to examine cross-cultural influences on the expression and recognition of internal states.
Moreover, the expression of certain internal states is likely to be multidimensional, with expressions including a combination of visual (e.g. kinematic), auditory (e.g. vocal) and contextual cues. Recognition of internal states in others may, therefore, be greatly improved by the addition of vocal cues, body movement, or contextual information. When observing an individual rubbing their abdomen, for example, contextual information might be necessary in order to interpret the action accurately as a sign of hunger (e.g. it is lunch time and we are in a queue to buy food), rather than a sign of satiety (e.g. we just ate a large meal). Future research is needed to elucidate whether some states rely more than others on visual cues for their expression, and what type of cues are necessary for their recognition. The availability of stimuli depicting states that are easily confused with each other in this stimulus set will make it possible to address these research questions. Notably, research into the perception and recognition of non-affective internal states in others will pose new methodological challenges, in part complementary to those faced when studying the perception of internal states in the self. On one hand, some internal states (e.g. itch, fatigue) are associated with visual cues but are difficult to measure objectively, potentially making study of these states easier in relation to others than to the self. Conversely, some internal states, such as cardiac signal changes, are easy to objectively assess in the individual, but are not accompanied by visual cues, making them difficult to observe in others.
Another crucial aspect to consider is the relative role of facial and bodily information in participants’ recognition of the current stimulus set. Facial cues were not obscured from the stimuli in either validation stage, as both facial expressions and postural cues are likely to be important for conveying internal states, and full body postures were deemed to be the most ecologically valid. It is possible, however, that facial and body information are recognisable in isolation, or that the relative contribution of facial and body cues to state recognition varies across internal states. As emotional cues are particularly expressed by the face (Adolphs, 1999; Frith, 2009), and it may be possible to experience affective and non-affective states simultaneously, interference effects from emotional cues may be especially evident when facial cues are present. While we note that stimuli have only been validated with integrated facial and body cues, it is of course possible for future work to investigate recognition from distinct regions of the stimuli, for example by separating or manipulating facial and body cues.
Crucially, the theoretical distinction between affective (emotional) and interoceptive states is not clear-cut. Here we refer to interoceptive states as internal bodily sensations beyond the affective domain. With this, we do not imply that emotions and interoceptive states are necessarily separate entities. On the contrary, according to the leading model of emotion perception, interoception is a fundamental component of emotional experience, which derives from sensory and affective experiences in combination with contextual cues (Schachter & Singer, 1962). However, it is common in the literature to find emotion processing and interoception treated as separate components. Similarly, some states, such as pain, seem to be considered as both emotional and interoceptive, with Craig describing pain as a ‘homeostatic emotion’ due to its sensory component alongside a motivational drive to re-establish the body’s homeostasis (Craig, 2003b). This definition could arguably be applied to a number of interoceptive states. Future work is needed to assess whether individuals process affective and interoceptive states in others differently. To this end, the call for stimuli depicting internal sensations beyond the affective domain is even more critical. Going forward, it is important that categories of internal states are clearly defined, both theoretically and operationally.
In conclusion, the ISSI stimulus set will allow, for the first time, the investigation of humans’ ability to recognise non-affective internal states in others. There are opportunities for investigating this basic process, for example the role of contextual cues and the contribution of facial and body postural cues to recognition, as well as for investigating correlates of individual differences in this ability, the genetic and neural basis of recognition, developmental trajectories, and the relationship between psychopathology and recognition abilities. Less recognisable stimuli have not been eliminated from the database, as researchers are encouraged to select stimuli based on their specific needs and research questions. If the aim of the study is that of assessing the accuracy of internal state recognition, then we advise researchers to select stimuli with higher quality, specificity, and choice rates, as these offer greater validity. The availability of more ambiguous stimuli, however, will allow investigation of individual differences in interpretation, and the biasing role of additional cues, for example. Researchers using the ISSI stimuli are encouraged to report their stimulus selection process transparently, and may utilise the validation statistics in the ISSI database to do this.
Notes
It should be noted that these results may reflect the disproportionate presence of female raters in the sample. To account for gender differences in stimulus ratings, quality and accuracy scores from the validation data were analysed separately for female and male observers, as well as in the combined sample.
References
Adolphs R. (1999). Social cognition and the human brain. Trends in Cognitive Sciences, 3(12), 469–479.
Ardizzi, M., Ambrosecchia, M., Buratta, L., Ferri, F., Peciccia, M., Donnari, S., Mazzeschi, C., & Gallese, V. (2016). Interoception and Positive Symptoms in Schizophrenia. Frontiers in Human Neuroscience, 10, 379.
Beaupré, M. G., Cheung, N., & Hess, U. (2000). The Montreal Set of Facial Displays of Emotion. Available from Ursula Hess, Department of Psychology, University of Quebec at Montreal, Canada.
Bechara, A., & Damasio, H. (2002). Decision-making and addiction (part I): impaired activation of somatic states in substance dependent individuals when pondering decisions with negative future consequences. Neuropsychologia, 40(10), 1675–1689.
Brewer, R., Cook, R., Cardi, V., Treasure, J., & Bird, G. (2015). Emotion recognition deficits in eating disorders are explained by co-occurring alexithymia. Royal Society Open Science, 2(1), 140382.
Brewer, R., Biotti, F., Catmur, C., Press, C., Happé, F., Cook, R., & Bird, G. (2016). Can neurotypical individuals read autistic facial expressions? Atypical production of emotional facial expressions in autism spectrum disorders. Autism Research, 9(2), 262–271.
Craig, A.D. (2003a). Interoception: the sense of the physiological condition of the body. Current Opinion in Neurobiology, 13 (4), 500–505.
Craig, A.D. (2003b). A new view of pain as a homeostatic emotion. Trends in Neurosciences, 26(6), 303–307.
Critchley, H. D., & Garfinkel, S. N. (2017). Interoception and emotion. Current Opinion in Psychology, 17, 7–14.
Dunn, B.D., Dalgleish, T., Ogilvie, A.D., & Lawrence, A.D. (2007). Heartbeat perception in depression. Behaviour Research and Therapy45(8), 1921–1930.
Dunn, B. D., Galton, H. C., Morgan, R., Evans, D., Oliver, C., Meyer, M., ... & Dalgleish, T. (2010). Listening to your heart: How interoception shapes emotion experience and intuitive decision making. Psychological Science, 21(12), 1835–1844.
Eggart, M., Lange, A., Binser, M. J., Queri, S., & Müller-Oerlinghausen, B. (2019). Major depressive disorder is associated with impaired interoceptive accuracy: A systematic review. Brain Sciences, 9(6), 131.
Ehlers, A. (1993). Somatic symptoms and panic attacks: A retrospective study of learning experiences. Behaviour Research and Therapy, 31(3), 269–278.
Ekman, P., & Friesen, W. (1976). Pictures of facial affect. Consulting Psychologists Press.
Forrest, L.N., Smith, A.R., White, R.D., & Joiner, T.E. (2015). (Dis)connected: An examination of interoception in individuals with suicidality. Journal of Abnormal Psychology, 124(3), 754–763.
Frith C. (2009). Role of facial expressions in social interactions. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 364(1535), 3453–3458.
Furman, D. J., Waugh, C. E., Bhattacharjee, K., Thompson, R. J., & Gotlib, I. H. (2013). Interoceptive awareness, positive affect, and decision making in major depressive disorder. Journal of Affective Disorders, 151(2), 780–785.
Garfinkel, S.N., & Critchley, H.D. (2013). Interoception, emotion and brain: new insights link internal physiology to social behaviour. Commentary on: "Anterior insular cortex mediates bodily sensibility and social anxiety" by Terasawa et al. (2012). Social Cognitive and Affective Neuroscience. 8: 231–4.
Garfinkel, S. N., Tiley, C., O'Keeffe, S., Harrison, N. A., Seth, A. K., & Critchley, H. D. (2016). Discrepancies between dimensions of interoception in autism: Implications for emotion and anxiety. Biological Psychology, 114, 117–126.
Harshaw, C. (2015). Interoceptive dysfunction: Toward an integrated framework for understanding somatic and affective disturbance in depression. Psychological Bulletin, 141(2), 311–363.
Hatfield, T. R., Brown, R. F., Giummarra, M. J., & Lenggenhager, B. (2019). Autism spectrum disorder and interoception: Abnormalities in global integration. Autism, 23(1), 212–222.
Kaulard, K., Cunningham, D.W., Bülthoff, H.H., Wallraven, C. (2012). The MPI Facial Expression Database – A Validated Database of Emotional and Conversational Facial Expressions. PLoS One, 7(3), e32321.
Khalsa, S. S., Adolphs, R., Cameron, O. G., Critchley, H. D., Davenport, P. W., Feinstein, J. S., Feusner, J. D., Garfinkel, S. N., Lane, R. D., Mehling, W. E., Meuret, A. E., Nemeroff, C. B., Oppenheimer, S., Petzschner, F. H., Pollatos, O., Rhudy, J. L., Schramm, L. P., Simmons, W. K., Stein, M. B., Stephan, K. E., … Interoception Summit 2016 participants (2018). Interoception and Mental Health: A Roadmap. Biological psychiatry. Cognitive Neuroscience and Neuroimaging, 3(6), 501–513.
Klabunde, M., Acheson, D.T., Boutelle, K., Matthews, S.C., & Kaye, W.H. (2013). Interoceptive sensitivity deficits in women recovered from bulimia nervosa. Eating Behaviors, 14 (4), 488–492.
Langdell, T. (1981). Face perception: an approach to the study of autism. PhD thesis, London.
Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D. H. J., Hawk, S. T., & van Knippenberg, A. (2010). Presentation and validation of the Radboud faces database. Cognition & Emotion, 24(8), 1377–1388.
Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska Directed Emotional Faces—KDEF (CD ROM). Karolinska Institute, Department of Clinical Neuroscience, Psychology Section.
Matsumoto, D., & Ekman, P. (1988). Japanese and Caucasian facial expressions of emotion (JACFEE). Intercultural and Emotion Research Laboratory, Department of Psychology, San Francisco State University.
Mul, C. L., Stagg, S. D., Herbelin, B., & Aspell, J. E. (2018). The feeling of me feeling for you: Interoception, alexithymia and empathy in autism. Journal of Autism and Developmental Disorders, 48 (9), 2953–2967.
Nicholson, T., Williams, D., Carpenter, K., & Kallitsounaki, A. (2019). Interoception is impaired in children, but not adults, with autism spectrum disorder. Journal of autism and developmental disorders, 49 (9), 3625–3637.
Paulus, M. P., & Stein, M. B. (2006). An insular view of anxiety. Biological Psychiatry, 60(4), 383–387.
Pollatos, O., Kurz, A.L., Albrecht, J., Schreder, T., Kleemann, A.M., Schöpf, V., Kopietz, R., Wiesmann, M., & Schandry, R. (2008). Reduced perception of bodily signals in anorexia nervosa. Eating Behaviors, 9, 381–388.
Russell, J.A. (1993). Forced-choice response format in the study of facial expression. Motiv Emot 17, 41–51.
Schachter, S.; Singer, J. (1962). "Cognitive, Social, and Physiological Determinants of Emotional State". Psychological Review. 69 (5), 379–399.
Schmidt, K. L., Ambadar, Z., Cohn, J. F., & Reed, L. I. (2006). Movement differences between deliberate and spontaneous facial expressions: Zygomaticus major action in smiling. Journal of Nonverbal Behavior, 30(1), 37–52.
Seth, A.K. (2013). Interoceptive inference, emotion, and the embodied self. Trends in Cognitive Sciences, 17 (11), 565–573.
Valstar, M., Pantic, M., Ambadar, Z., & Cohn, J. (2006). Spontaneous vs. posed facial behavior: Automatic analysis of brow actions. Proc. ACM Int’l Conf. Multimodal Interfaces, 162–170.
Volkova, E., de la Rosa, S., Bülthoff, H.H., & Mohler, B. (2014). The MPI Emotional Body Expressions Database for Narrative Scenarios. PLoS ONE, 9(12), e113647.
Werner, N. S., Jung, K., Duschek, S., & Schandry, R. (2009). Enhanced cardiac perception is associated with benefits in decision-making. Psychophysiology, 46(6), 1123–1129.
Wingenbach, T. S. H., Ashwin, C., & Brosnan, M. (2016). Validation of the Amsterdam Dynamic Facial Expression Set - Bath Intensity Variations (ADFES-BIV): A Set of Videos Expressing Low, Intermediate, and High Intensity Emotions. PLoS One, 11(1), e0147112.
Funding
The Principal Investigator R.B. is funded by a Medical Research Council New Investigator Research Grant (MR/S003509/1).
Author information
Authors and Affiliations
Contributions
F.B. was responsible for actors’ recruitment, recording sessions and stimulus development. F.B. and R.B. conceived and designed Stage 1 of validation. F.B. programmed the task. F.B., S.A., and R.Q. ran the free-labelling task and were responsible for data entry and for the qualitative analysis. F.B. and R.B. conceived and designed Stage 2 of validation. F.B., S.A., and R.Q. were responsible for data collection and data entry. F.B. ran the analyses with inputs from R.B. F.B. and R.B. contributed to the interpretation of the results and wrote the manuscript.
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest and have no competing financial interests.
Ethics approval
All the different stages of this study were reviewed and approved by the College Ethics Committee at Royal Holloway, University of London.
Consent to participate and for publication
The actors gave their informed consent to take part in the recording session and for their images to be used in scientific research, presented at conferences, published in academic journal articles, and shared with other researchers. Participants in the first and second stages of validation gave their informed consent to take part in the study and for their responses to be recorded anonymously on the Open Science Framework in accordance with the GDPR and Data Protection Act 2018. The participants also gave their consent for their responses to be published in academic journals and/or disseminated at academic conferences.
Additional information
Open Practices Statement
The ISSI database is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. The ISSI database will be published and freely available to researchers for non-commercial use. The database will be accessible via the Figshare platform. A link will be created upon publication of this manuscript and accessible via www.insulab.uk/issi.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Biotti, F., Ahmad, S., Quinn, R. et al. Development and validation of the Interoceptive States Static Images (ISSI) database. Behav Res 54, 1744–1765 (2022). https://doi.org/10.3758/s13428-021-01706-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13428-021-01706-2