Introduction

Proponents of the theoretical perspective of compositionality argue that the meaning of a message is determined by the meaning of the constituent parts of that message (e.g., Szabó, 2019). However, there is debate as to whether visual representations lack the necessary “syntactic structure” required for compositional semantics (i.e., Fodor, 2007; Greenberg, 2011). The proposed semantic theories for visual representations look very different from those offered in formal semantics for language. Indeed, it has been argued that pictures represent information in a way that is fundamentally from declarative sentences (e.g., Heck, 2007). Emoji—pictograms, ranging from different faces with different expressions, to animals, symbols, and objects—have enabled individuals to better-express themselves in computer-mediated communication (CMC) contexts (Provine et al., 2007). Emoji have significant advantages over plain text, as they can convey emotions more vividly, facilitating CMC (Riordan, 2017). Emoji are increasingly used in CMC, and they have both unique semantic and emotional features (Skovholt et al., 2014). The current study investigated the effect of emoji on the perception of otherwise affectively neutral statements. We explored whether there were any between-group differences in any such effects, dependent on participants’ national identity (US American, British, Danish). These groups represent native vs. non-native (but highly proficient) readers of English.

Emoji and CMC

Emoji are rapidly becoming the dominant form of emotional expression in CMC (Chairunnisa & Benedictus, 2017; Pavalanathan & Eisenstein, 2015). Emoticons were used to portray emotion in environments where only basic text was available; emoji are extensions of the character set, available on most digital devices (Kralj Novak et al., 2015). Emoticons and their corresponding emoji are recognised/categorised equivalently—for example, :-) and —but emoji typically have a greater impact on individuals’ mood (Ganster et al., 2012). Kaye et al. (2016) indicated that the main motivations behind facial emoji usage were “aiding personal expression” and “establishing emotional tone” (p. 464). Sampietro (2019) found that emoji contribute to the expression of speech acts, as well as establishing interpersonal contact. Several studies have also found that emoji are used to manage and maintain interpersonal relationships (Albawardi, 2018; Riordan, 2017; Skovholt et al., 2014). Chairunnisa and Benedictus (2017) found that emoji were used to strengthen message meaning and to avoid misunderstandings.

Hu et al. (2017) investigated the sentiment effects of emoji. They discovered that a positive emoji made both neutral and negative verbal statements more-positive but had no effect on positive verbal statements. Furthermore, they found that a neutral emoji only influenced positive verbal statements; neutral emoji resulted in positive statements being rated more-negatively than positive statements alone. Similarly, negative emoji decreased the positivity of positive and neutral verbal statements yet had no effect on negative verbal statements. Boutet et al. (2021) found that neutral texts appeared to be vulnerable to the effects of emoji—that is, the presence of an emotionally-valent face emoji swayed participants perceptions of the message.

Robus et al. (2020) investigated the effects of emoji position and emotional expression on eye-movements behaviour during reading and participants’ perceptions of the emotional valence of the written materials. Pre-tested neutral sentence stimuli—third person narratives—were used. Two emoji were utilised—slightly smiling () and slightly frowning (). Emoji influence on perception of text valence was predominantly non-significant; this may have been influenced by the ‘weakness’ of the expressions of the emoji used, the unfamiliar third-person narrative structure of the written sentences, and/or the artificial laboratory eye-tracking set-up. Hand et al., (2022; Study 2) revisited this question, considering the written materials used by Robus et al., but with ‘more emotional’ emoji (, ). Hand et al. found a significant effect of emoji type on participant ratings of otherwise-neutral sentences: sentences + happy emoji were rated as emotionally positive, sentences + sad emoji were rated as emotionally negative. However, Hand et al. failed to include a ‘no emoji’ condition, failed to consider the influence of non-face emoji, and relied upon third-person narrative texts rather than ecologically-valid text-message type exchanges.

Kralj Novak et al. (2015) created the first emoji lexicon through sentiment analysis. Sentiment analysis is the computational treatment of opinions, sentiments, and subjectivity of text. It is the process of determining the emotional tone behind a series of words, used to gain an understanding of the emotions and attitudes expressed within a piece of written text. Kralj Novak et al. considered 751 popular emoji, and classified them as negative, neutral, or positive. They based the classifications on data from the microblogging service Twitter, where 64,000 tweets across 13 different European languages were analysed. Approximately 4% of the annotated tweets contained emoji and most emoji were positive. Sentiment distribution of tweets with and without emoji was significantly different (Kralj Novak et al., 2015).

Emoji in sentiment analysis was further developed by Ayvaz and Shiha (2017), who conducted a study investigating the effects of emoji in sentiment analysis. Their analysis revealed that the use of emoji in sentiment analysis appeared to improve the expressivity and overall sentiment scores. The sentiment analyses carried out on text with emoji had higher sentiment scores than the text without emoji. This suggests that emoji emphasise the sentiment (either positively or negatively) of text, or that users tend to include an emoji when generating emotional messages. Erle et. al (2022) conducted 11 experiments to test how effectively emojis convey emotionality in digital communication, and to determine if emojis have functional equivalence to facial expressions in face-to-face communication, in line with predictions of the Emotion as Social Information (EASI) model (van Kleef, 2009). At its core, the EASI model explains that observer behaviours are influenced emotional expressions; emotional expressions trigger inferential processes and/or affective responses (van Kleef, 2009). Erle et al.’s experimental setup was ecologically valid, as participants were asked to read and rate instant chat messages. Results showed that positive messages with an emoji were perceived as more positive than without an emoji, and negative messages with an emoji were perceived as more negative than without an emoji. Emojis were found to be functionally equivalent to facial expressions of emotions in face-to-face communication, in line with the EASI model. As Erle et. al (2022) found support for the notion that emojis are equivalent to facial expressions, the perceived valence of affectively neutral messages coupled with emoji should directly correspond to the emotional valence conveyed by the emoji.

Emoji and Cultural Effects

Shared culture is crucial in determining how one communicates one’s thoughts and feelings (Gudykunst, 1997; Gudykunst & Lee, 2005; Park et al., 2014), and this applies to both verbal and non-verbal interpersonal communication (Park et al., 2014). In this respect, it has been argued that communication styles vary cross-culturally (e.g., Gudykunst et al., 1996); for example, differences in cultural collectivism-individualism (Gudykunst et al., 1996; Hall, 1976; Hofstede et al., 2011) might be particularly influential in determining communication styles—including one’s use of language, presentation of self, and use of non-verbal cues (Park et al., 2014). Park et al. (2014) found evidence for cultural differences in terms of emoticon interpretation; for example, people from more individualistic cultural backgrounds favoured horizontal and mouth-oriented emoticons—(^ ◡ ^)—whereas people from more-collectivist backgrounds favoured vertical, eye-oriented emoticons—:-) (Park et al., 2014). Recent evidence is mixed regarding cultural differences in emoji interpretation and emoji usage. Evans (2017) described emoji as “the world’s first truly universal form of communication” (p. 38). Emoji usage research suggests that there are some cultural differences. For example, Cheng (2017) found that Chinese users were more likely to use emoji to express negative emotions than Spanish users. Chandra Guntuku et al. (2019) compared emoji usage in terms of context and frequency across three western countries (the United Kingdom, Canada, and the USA) and two eastern countries (China and Japan). Their findings were mixed. For instance, they learned that Western users tend to use more emoji than Eastern users.

Regarding emoji interpretation, Barbieri et al. (2016) found that the semantics of the 150 most popular emoji are generally preserved across different languages (American English, British English, Spanish, and Italian), suggesting that emoji interpretation may be consistent across cultures. However, they found slight differences in emoji interpretation between American English and British English samples. For example, British English participants interpreted the wrapped gift emoji () as a Christmas gift, comparable with other Christmas emoji, e.g., Father Christmas/Santa Claus (), Christmas Tree (). American English participants did not generally interpret the wrapped gift emoji specifically as a Christmas gift. Furthermore, American English participants associated the waving hand emoji () with similar emoji, such as woman raising hand () and victory hand (). Curisously, British English participants associated the waving hand emoji with emoji typically associated with travelling, like the high-speed train () and airplane () emoji (Barbieri et al., 2016).

The findings of Barbieri et al. (2016) signal the need for additional studies to understand any potential differences in emoji interpretation between English speakers. Previous research has explored inter-group differences in emoji interpretation and emoji usage. However, to our knowledge, no study has examined the differences between native and non-native (but highly proficient) English speakers with respect to the influence of emoji on perceptions of otherwise affectively neutral statements.

The Current Study

Most previous research has focused on the effect of emoticons or emoji on affectively positive or negative statements (e.g., Erle et al., 2021; Miller et al., 2017; Walther & D’Addario, 2001). Few studies (e.g., Boutet et al., 2021; Hand et al., 2022; Robus et al., 2020) have investigated the effects of emoticons or emoji in combination with affectively neutral statements. Across these studies, findings are mixed and a number of criticisms can be levied. For example, previous studies have been limited to third-person narrative written stimuli (e.g., Hand et al., 2022; Robus et al., 2020), or have been conducted in otherwise artificial/unfamiliar environments, such as eye-tracking laboratories (e.g., Boutet et al., 2021). The research that has explored potential cross-cultural group differences has typically focused more on emoji usage and ‘semantics’—that is, the labelling of the emoji, rather than the impact of emoji on the perceived emotionality of the messages; furthermore, prior research has often relied on the mining of data from social network sites (e.g., Twitter; Barbieri et al., 2016), without gathering data from actual human message recipients/‘perceivers’.

Based on previous research on emoji perception only (e.g., Barbieri et al., 2016), we explored possible differences in emoji + text perceptions between American English and British English speakers (with English as their native language), and Danish nationals. The Danish group enabled us to consider differences between native and non-native (but highly proficient) speakers/readers. Danish nationals are typically highly proficient in English, and ranked number 2 in the EF English Proficiency Index in 2020 (Education First, 2020). Given the composition of our research team and our planned recruitment strategy, we had an a priori expectation that these three groups would be heavily-represented in our dataset. However, we chose to recruit participants of all national/native language backrounds so as to potentialy identify other sufficiently-sampled sub-groups. Due to the limited amount of research on differences in emoji + text perceptions (and any concomitant inter-group effects), a two-tailed hypothesis regarding the interaction between the effects of emoji type and national group was determined.

Previous studies have been limited by their use of either third-person narratives as stimuli, ‘passive’ mined data with no human ‘recipient’ interpretation, or artificial laboratory contexts. Our stimuli and method eschewed these issues, using ecologically-valid screenshots of text-based social interactions. We gathered data via an online survey method that participants accessed through either desktop or portable devices in ‘natural’ settings; this again enhances ecological validity, as opposed to studies that were conducted in, for example, eye movement laboratories. We took a highly objective approach to stimulus creation; sentiment analyses ensured the affective neutrality of the statements prior to data collection. As previously mentioned, the goal of sentiment analysis is to determine the polarity/semantic orientation of a text in context, i.e., if it is positive, negative, or neutral. Most sentiment analysis tools are based on Machine Learning algorithms; algorithms that use computational linguistics and natural language processing to “learn” how to determine the polarity of text, based on large datasets (e.g., tweets, product reviews, headlines, etc.) (Taboada, 2016). A priori sentiment analyses determined that the stimuli used in this study were neutral.

We considered three distinct participant groups—native English speakers/readers (US American, British) and Danish citizens—highly-proficient (Education First, 2020) but non-native English readers/speakers. The challenge that we faced was in finding a group who would potentially be different from the American and British participants (culturally, linguistically) but still able to comprehend the written texts easily. Data exploration using the Hofstede Insights Cultural Compass (2021) shows that Denmark has a somewhat lower Individualism score than the UK or US. Within individualistic cultures, people are relatively free to express their feelings; thus, expressions of emotion are more direct and less constrained (Park et al., 2014). However, those living within collectivistic cultures are more attuned to decoding others’ feelings (Park et al., 2014). These three countries also demonstrate somewhat different Uncertainty Avoidance (Denmark < UK < US); this is potentially important given that emoji could serve to mediate the level of ambiguity/uncertainty in messages’ emotional tone/intended impact. Importantly, all three countries show similar scores in Indulgence; that is, the extent to which they enjoy life, have positive attitudes and tendencies towards optimism. As such, there is potential for individuals from Denmark, the USA, and the UK to interpret text + emoji somewhat differently, either through the native/non-native distinction and/or subtle inter-cultural differences (or a combination of these influences).

Furthermore, as we were using emoji that depicted typically facial/facial feature-based expressions of emotion, we wanted all groups to be ‘Western’, as previous research suggested that including, for example, an East Asian comparison group, might impact on how the emoji themselves were perceived (Jack et al., 2012; Park et al., 2013, 2014; Yuki et al., 2007), not just how the emoji shaped the perceptions of the overall messages.

Many studies presented have used general linear model-type analyses to test their hypotheses, despite the outcomes being measured by ordinal response categories and/or the data failing to satisfy assumptions of the techniques deployed. Likert-type scales are ordinal by nature, and although the scale points might appear equally-spaced, one cannot guarantee that each participant makes the same value-judgements as to what constitutes a response at each scale point, or that each participant’s evaluations of adjacent points are equal (Taylor et al., 2021). The relationship between participants’ responses and underlying latent dimension(s) are underspecified (Taylor et al., 2021). Our analytical approach—CLM modelling—mapped ordinal outcomes against ordered regions of a latent distribution (Bürkner & Vuorre, 2019; McCullagh, 1980). We extended this approach to include cumulative link mixed modelling (CLMM) to determine fixed effects (emoji type, nationality, emoji type × nationality), and how these fluctuate across the ‘randomness’ associated with individual participants’ differences (e.g., Jaeger et al., 2018) and individual stimulus items (Taylor et al., 2021). There is ample evidence of the problems in using general linear modelling/analysis of variance (ANOVA) to evaluate ordinal data sets, and the necessity for CLMM approaches within experimental studies has been powerfully argued by, for example, Liddell and Kruschke (2018).

Based on previous work, we established the following hypotheses:

  • H1a Messages paired with a negative emoji would be rated as more emotionally-negative than any other display condition (neutral emoji, positive emoji, no emoji).

  • H1b Messages paired with a positive emoji would be rated as more emotionally-positive than any other display condition (negative emoji, neutral emoji, no emoji).

  • H2 Participants of different nationalities would perceive the emotionality of the messages differently.

Method

Design and Participants

A 4 (emoji type: no emoji; negative emoji, neutral emoji, positive emoji) × 3 (national identity: American, British, Danish) mixed-factorial design was employed, utilising an online survey method. The outcome measure was perception of message emotionality (valence).

Emoji-using participants completed the online survey in full (N = 217; 135 females, 66 males, 8 trans/non-binary respondents, 8 preferred not to say). Participants ages ranged from 18 to 60 years (Mage = 24.38 years, SDage = 7.47; medianage = 22 years). Participants were recruited from the general population through convenience sampling, representing 30 different nationalities. The most-represented nationalities were indeed American (n = 42), British (n = 30), and Danish (n = 80). All participants were familiar with emoji prior to participating in this study. All participants reported normal or corrected-to-normal eyesight. All participants spoke/read English with high proficiency (self-reported); individuals with communication disorders (i.e., dyslexia) were excluded from this study (recruitment materials/information sheets made this clear). Recruitment took place predominantly online by posting the questionnaire link in different web-forums (www.thestudentroom.co.uk and www.reddit.com). Other participants were recruited via email, as the questionnaire link was distributed through different universities’ and a Danish high school’s networks.

Materials and Procedure

Five independent stimulus pairs were constructed, consisting of a question and a response (“How will you get there?” “By train”; “When is it?” “It’s on Wednesday”; “When did you receive it?” “Yesterday”; “Did you get my email?” “I haven’t checked my email yet”; “Did they respond?” “Not yet”). Responses were pre-tested without emoji to ensure their neutrality. A sentiment analysis (Feldman, 2013; Kiritchenko et al., 2014; Kumar & Prabhu, 2018; Liu, 2012; Lüdtke & Jacobs, 2015) was conducted on the stimuli, using a machine learning platform for text analysis (MonkeyLearn v3, Garreta & Pascual, 2020). All five responses were classified as “neutral”, with confidence levels between 83 and 91.5%. The sentiment analysis confirmed that the response stimuli were affectively neutral without emoji.

Responses were presented in one of four conditions—without emoji (control), with a negative emoji (), with a neutral emoji (), or with a positive emoji (). These emoji were selected from the Emoji Sentiment Scale (Emoji Sentiment Ranking v1.0; Kralj Novak et al., 2015), and cross-referenced across platform renderings to ensure that users of different platforms would be familiar with the emoji, minimising the risk of misinterpretations (Miller et al., 2017). Emoji were selected based on sentiment ratings and inspection of sentiment bars (Kralj Novak et al., 2015). The components of the distribution in the Emoji Sentiment Scale (p−, p0, p +) denote the negativity, neutrality, and positivity of the emoji, respectively. The sentiment bar captures all three sentiment properties, computed from the sentiment distribution of the emoji occurrences with a 95% confidence interval. The negative emoji chosen (“open eyes with a skewed frown” []) has strong negativity (p− = 0.601) and/or low positivity (p+  = 0.204); the neutral emoji chosen (“eyes” []) has clear neutrality (p0 = 0.482), low negativity (p− = 0.228) and low positivity (p+  = 0.291); the positive emoji chosen for the study (“smiling face with open mouth and smiling eyes” []) has a strong positivity rating (p+  = 0.558) and/or low negativity (p− = 0.137).

The study was designed in accordance with the principles of the British Psychological Society (BPS, 2014). Participant anonymity and confidentiality was protected during this study; all data collected was confidential and fully anonymised. Participants were asked to create a memorable identifier with which to contact the researchers if they wished to withdraw their data from the study. Ethical approval was obtained from the Psychology, Social Work, and Allied Health Sciences Ethics Committee at Glasgow Caledonian University. The online survey was constructed using Google Forms and was UK General Data Protection Regulation (GDPR) compliant.

Participants could complete the questionnaire from a device of their choosing. They were initially shown an information sheet containing details about the study, contact information for the lead researcher, and their rights, before indicating their consent to participate in the study and creating their unique personal identifier. They were then asked for their demographic information including gender-sex, age, and nationality, as well as their familiarity with and usage of emoji. Participants were then presented with 20 images of text-message interchanges (in a random order)—only one stimulus was visible per screen. Each image contained two messages: a question followed by a response consisting of an affectively neutral statement either without emoji (control) or with one of the three emoji described previously. Participants were instructed to rate the perceived positivity/negativity (valence) of the response message based on their first impression. Under each image there was a 10-point rating scale where they could rate their impression of the response message (1 = most-negative, 10 = most-positive). Figure 1 shows example stimuli across emoji conditions.

Fig. 1
figure 1

Example stimuli across all four display conditions. Note A = no emoji control; B = positive emoji; C = neutral emoji; D = negative emoji

Full size image

Once participants had rated all 20 stimuli, they were debriefed and presented with an optional feedback form, where they could leave feedback about the study. Participants were then thanked for their participation and given information on how to contact the research team if they had further queries or would like to have their data withdrawn.

Data Analysis

Reliability analyses were carried out by comparing the five responses to each of the stimulus types (no emoji, negative emoji, neutral emoji, positive emoji). Cronbach’s alpha (α) showed the responses to each demonstrate acceptable reliability: α = 0.85, 0.78, 0.82, 0.81, for the no emoji, negative emoji, neutral emoji, and positive emoji, respectively. Across all participants and trials, there were 4340 data points available for analysis. We used the ‘ordinal’ package (Christensen, 2019) in R (R Development Core Team, 2016) to generate CLMMs. Optimal random effect structures were identified using forward model selection (see Barr et al., 2013; Matuschek et al., 2017). Fixed effects were tested using likelihood-ratio tests comparing full and reduced models. Post-hoc tests were conducted using the ‘emmeans’ package (v1.4.8; Lenth et al., 2020); significance thresholds were adjusted using the Bonferroni method. Effect sizes (Cliff’s delta) were calculated using the ‘effsize’ package (Torchiano, 2020), and are reported for significant fixed effects only. Per Cohen (1988), Cliff’s delta ≈ 0.15 is considered ‘small’ (analogous to Cohen’s d effect size of 0.20), Cliff’s delta ≈ 0.33 is considered ‘medium’ (analogous to Cohen’s d effect size of 0.50), Cliff’s delta ≈ 0.47 is considered ‘large’ (analogous to Cohen’s d effect size of 0.80).

Results

Descriptive statistics are presented in Table 1.

Table 1 Descriptive statistics
Full size table

Our main model included random intercepts by-participants [χ2(1) = 3556, p < 0.001] and by-items [χ2(1) = 170.25, p < 0.001], and a random slope of emoji type by participants [χ2(9) = 1098, p < 0.001]; all other models failed to converge. A significant and large fixed effect of emoji type was found [χ2(4) = 64.26, p < 0.001; Cliff’s delta = 0.714]. Responses paired with negative emoji are perceived more-negatively than all other stimulus conditions (all ps < 0.001). There was no difference between the ratings of stimuli paired without an emoji and those presented with a neutral emoji (p = 0.501). Stimuli paired with a positive emoji were rated more-positively than all other stimulus conditions (all ps < 0.001).

We then extracted a dataset based on our three largest national identity groups—American (n = 42), British (n = 30), and Danish (n = 80). This dataset had 3040 data points and was analysed identically to the master data set. The random factor structure included random intercepts by-participants [χ2(1) = 2449, p < 0.001] and by-items [χ2(1) = 105.11, p < 0.001], and a random slope of emoji type by participants [χ2(9) = 713.5, p < 0.001]. This analysis revealed a significant and large fixed effect of emoji type [χ2(3) = 63.35, p < 0.001, Cliff’s delta = 0.688], with identical follow-up comparisons (in terms of significance outcomes). No evidence of a between-groups difference based on national identity [χ2(2) = 1.41, p = 0.494], nor any evidence of an emoji type × national identity interaction [χ2(6) = 11.632, p = 0.071], was found.

Discussion

We investigated the effect of different types of emoji on the perceived emotional valence of otherwise affectively neutral text in a CMC context. Previous research has yielded mixed findings as to the efficacy of emoji in shaping participants’ perceptions of sentiment. We recruited a large sample of participants highly familiar with emoji. We employed pre-tested affectively-neutral stimuli presented as part of a socially-interactive CMC. We predicted that messages paired with a negative emoji would be rated as more emotionally-negative than any other display condition, and that messages paired with a positive emoji would be rated as more emotionally-positive than any other display condition (H1a, H1b). Following cumulative link mixed modelling, both hypotheses were upheld. This contrasts with, for example, Robus et al. (2020), but is consistent with, for example, Hand et al. (2022). Our main model effect size (Cliff’s delta = 0.714) and sub-analysis effect size (Cliff’s delta = 0.688) are very large; indeed, they are larger than those obtained by, for example, Hand et al., (2022; Cliff’s delta = 0.594). It is likely that such differences are in part due to the robust neutrality of the base stimuli and the ecological validity of the stimuli in the current stimuli. Our findings are consistent with the compositionality perspective of the determination of meaning (Szabó, 2019).

Previous research (e.g., Barbieri et al., 2016) had suggested cross-cultural differences in the perception of the ‘semantics’ of emoji—we extrapolated this to explore whether emoji effects on participants’ perceptions of text-valence varied across national identities of English speakers/readers (US American, British, Danish). No evidence of an emoji type × national identity group interaction was found, leading us to reject H2. It may be the case that these groups are homogenous to the point of non-distinctiveness.

Integrating Our Findings with Previous Literature

As Pavalanathan and Eisenstein (2015) and Chairunnisa and Benedictus (2017) recognised, emoji have become increasingly popular, and have generally replaced emoticons in CMC. Emoji are often used similarly to how emoticons were used, and both emoticons and their ‘replacement’ emoji are typically interpreted in the same way (Ganster et al., 2012). Thus, we will compare the results of the current study with previous research on both emoticons and emoji.

The results of the current study are inconsistent with certain studies (e.g., Robus et al., 2020; Walther & D’Addario, 2001), but consistent with the findings of others (e.g., Boutet et al., 2021; Derks et al., 2007). The differences between these studies and the current findings may be due to differences in methodological approaches, as well as societal changes (e.g., familiarity with CMC in general/emoji in CMC). It is important to note that 14% of Walther and D’Addario’s (2001) participants had never used emoticons in CMC before. In the present study, we ensured that all participants were familiar with emoji in CMC. Our underlying written stimuli were determined to be affectively neutral via sentiment analyses based on machine learning, whereas previous studies have involved only emotionally polarised texts. Thus, it is reasonable that emoji would differentially influence the perceptions of such neutral texts when considered against an intricate relationship between an emoji and an emotionally complex written message. Furthermore, our stimuli were ecologically valid representations of social exchanges, as opposed to, for example, third-person neutral narratives. Our research paradigm added to this ecological validity—participants accessed an online survey platform to view the stimuli, as opposed to reading while constrained within an eye-tracking laboratory, or under otherwise ‘alien’ conditions. We additionally ensured that our emoji strongly represented their intended sentiments, using an objective sentiment measurement.

We observed no between-group differences in sentiment ratings across American, British, or Danish English speakers, nor an interaction between emoji type and national identity group. This suggests that the effects of emoji on perceptions of otherwise affectively neutral responses are universal across American English, British English, and Danish non-native but highly proficient English readers/speakers. This finding is novel. Although some previous research (e.g., Barbieri et al., 2016) has suggested cross-cultural differences in the use of/perception of emoji in American English and British English groups, methodological differences (as stated previously) may explain differences between our results and previous studies’ findings. It may be the case that differences between-groups are dependent on whether or not the stimuli are being presented in a participant’s native language or another language. However, a re-analysis of the full current data set (N = 217) based on whether participants were native English speakers (and therefore processing stimuli in their native language) or not suggested no main effect of language group and no interaction between emoji type and language group. It would be interesting to re-visit this issue with a group of less-proficient non-native speakers/readers, however, this is not without problems and potential confounds. What might be more appropriate is to revisit the matter with groups that are more diverse across multiple demographic markers (i.e., age, language/writing system, neurodivergence).

Applications of the Current Findings

Our findings have several research and practical applications. Compellingly demonstrating the effects of emoji on affectively neutral statements could aid in the research and creation of psychometric and psychological assessment tools (Phan et al., 2017). Marengo et al. (2019) investigated the utility of emoji in depression assessment. They found that their emoji-based screening tool had a remarkable accuracy in identifying depression in young adults. This research has led to a promising alternative to text-based depression assessment tools.

Another practical application of the findings of the current study relates to performance feedback in the workplace and in educational assessment. Wang et al. (2014) conducted a study investigating the effectiveness of emoticons as surrogates for facial expressions when delivering negative feedback in CMC. Their results suggested that using positive emoticons increased the perceived good intention of the feedback provider and decreased the perceived feedback negativity, when the feedback is specific. This in turn allowed for better feedback acceptance. Based on the findings of Ganster et al. (2012) that emoticons and their ‘related’ emoji share a common interpretation, it could be argued that emoji in workplace feedback may illicit a similar response to emoticons. Similarly, Riordan (2017) found that a negative message coupled with a neutral non-face emoji decreases perceived negativity. Extending this research, one could consider the use of emoji in education settings, as part of assessment feedback, or course/instructor evaluations. Padgett et al. (2021) investigated the effect of emoticons—in many ways, the precursor to emoji—on student perceptions of online assignment feedback. Their results showed that the inclusion of emoticons significantly increased perceptions of social presence and instructor qualities. Their results suggested that emoticons can positively influence the perceptions of the instructor, without impacting the perception of the feedback itself. These findings provide preliminary evidence of how incorporating emoticons in feedback may be advantageous to both students and instructors. Further studies are needed to establish whether emoji would elicit similar responses in feedback perception.

Limitations and Suggestions for Future Research

Some methodological caveats inhibit the generalisability of the current findings and require attention in future research studies. For instance, the current study did not engage the participants as real co-communicators in an authentic exchange of messages. The participants were told to rate the messages based on their first instinct, and that their ratings should only reflect how they perceived the answer to be, without thinking about the extended context, previous conversation, or relationship between communicators. This may have been a difficult task for the participants, as it is very likely distinctive interpretations exist within specific relationships, based on familiarity with senders’ style or habits. Wiseman and Gould (2018) investigated the idea that varying interpretations could lead to emoji being assigned different meanings. They studied how emoji were repurposed between two friends, family members, or partners, which led to emoji being given additional or alternative meanings. They concluded that emoji are naturally ambiguous, and that users can create a shared personal meaning for emoji amongst themselves. The current study utilised three emoji; it is unknown whether participants had different-than-intended interpretations of the emoji. Future studies should include more than one emoji per condition (positive, neutral, and negative), to increase generalisability. Furthermore, future studies might attempt to ensure that participants have no preconceived/alternative interpretations of the selected emoji, to ensure that all participants interpret them similarly. It must be noted that our negative and positive emoji stimuli were ‘digital faces’ and were unambiguous in terms of the position of their features and thus the likely sentiment that they displayed. Furthermore, it is likely that our participants, cross-nationally, would have perceived these expressions similarly (e.g., Ekman, 1992; Jack et al., 2012), thus ‘flattening out’ any possible differences between-groups.

Another limitation of the current study was the lack of specification of the sender. The fact that the sender of the messages was unspecified may have altered participant responses. Derks et al. (2007) found that individuals interpret messages variably, depending on their relationship with the sender. Their study found that positive messages sent from friends were interpreted to be more positive than positive messages sent from strangers. In future research, this sender-recipient relationship could be specified and/or expanded (e.g., teacher-learner, practitioner-client). It is also important to acknowledge that the question-and-response messages (and emoji) were presented out-of-context. The response message which the participant was asked to rate was not read in the context of a more-extensive conversation, which may have altered their interpretation of the message + emoji stimuli. Miller et al. (2016) found that emoji meaning may fluctuate, depending on context. Due to the lack of background information about the previous conversation, participants may have interpreted the message + emoji stimuli differently.

Conclusion

The current study expanded upon previous investigations of emoji effects on the perceived emotional valence of otherwise affectively neutral communications. Using well-controlled, ecologically valid stimuli, a mixed-factors design, and a large representative sample, we demonstrated that negative/positive emoji will enhance the perceived negativity/positivity of otherwise neutral messages. Neutral emoji result in suitably neutral perceptions, no different from no emoji conditions. There were no differences in emotional valence ratings across participant nationality groups (US American, British, Danish), nor an emoji type × nationality group interaction, suggesting that the effects of emoji on text-sentiment perception were universal.