1 Introduction

The Russian Federation’s special military operation in Ukraine began on February 24 2022 and, at the time of writing, shows no sign of abating. Social media has become essential to understanding the Russian invasion of Ukraine [4, 18]. Within social media, memes have become a popular way of conveying information about the conflict. For example, there exist pro-Ukraine memes portraying Ukraine President Volodymyr Zelenskyy as an Avenger, and pro-Russia memes claiming that Ukraine belongs to Russia. The Russian disinformation campaign uses pro-Russia memes to polarize Americans, particularly those at the extreme ends of the political spectrum [2], and increase support for the Russian invasion of Ukraine. Thus, it is critical for governments and similar stakeholders to identify pro-Russia memes, countering them with evidence-based information and corresponding regulation of social media platforms. To effectively counter such memes and prevent their further dissemination, identifying broad themes is crucial. Only by understanding the thematic elements and cultural ramifications of these memes can effective counter strategies be successfully developed. For example, if pro-Russia memes focus on the existence of secret US-funded biolabs in Ukraine, interventions can provide evidence-based information that demonstrates the non-existence of such facilities.

Similarly, there are a range of pro-Ukraine memes that bolster support for the Ukrainian cause and may increase global aid for Ukrainians. As such we need to identify pro-Ukraine memes and aid with their dissemination, augmenting global support for Ukraine, possibly reducing effects of the conflict. There currently exists no annotated dataset of pro-Russia and pro-Ukraine memes, central to developing classifiers for these cases. We address the indicated issues through the following contributions: 1) Creation of an annotated dataset of pro-Russia (N = 70) and pro-Ukraine (N = 121) memes regarding the Ukraine conflict; and 2) Identification of broad themes within pro-Russia and pro-Ukraine categories.

2 Related Work

2.1 Ukraine-Related Memes

A qualitative content analysis was conducted of memes posted to the RuNet Memes Twitter account in 2014 and indicated that most memes fell into one of two categories: pro-Russia or pro-Ukraine. While the memes reference a given news story or event, they continued to be consumed and reproduced along similar thematic categories [28]. Another study analyzed memes created during anti-government protests in Ukraine (2013–2014), indicating that pro-government memes usually rely on simple emotional messages for propaganda/polarization purposes, whereas anti-government memes produce more nuanced statements used as a form of creative protest or as a coping mechanism [14]. Researchers studied memes created during the 2019 Ukrainian presidential election, examining the influence of Ukrainian politicians on the development of such memes. The researchers also detailed the role memes play in societal polarization. While past work detailed and thematically categorized Ukraine-related memes, previous research did not explore Ukraine memes in relation to the current conflict, or assemble datasets for developing pro-Russia or pro-Ukraine meme classifiers.

2.2 Mis/Disinformation in Memes

Existing research on identifying mis- or disinformation in memes is limited. One study focused on two widely circulating memes in the anti-vaccination movement, namely lists of vaccine ingredients containing mercury, and quotes attributed to Mahatma Gandhi [3]. The article analyzed both memes, and illustrated how the repurposed, often ironic use of visual tropes can either undermine or strengthen the accompanying claims, exploring how memes can function as vehicles for the spread of controversial health-related information. Another study used an experimental design to examine the credibility and persuasiveness of COVID-19-related memes [27]. Results indicated that memes with expert source attribution are more credible than those with nonexpert source attribution. A positive correlation between the credibility of a meme and its persuasiveness was observed. Overall, previous work provides an overview of mis/disinformation in memes, but does not detail possible disinformation within memes around the Ukraine conflict.

Complementarily, there exist several classifiers for offensive [23] or hateful [7, 10, 29] meme content. However, there are limited classifiers for disinformation within memes and no classifiers for disinformation in memes for the Ukraine conflict context. We thus provide an annotated dataset covering the initial memes produced during the conflict, with an emphasis on memes possibly resulting from disinformation campaigns, simultaneously aiding the development of Ukraine conflict-centric meme classifiers.

3 Data

We first assembled a list of keywords derived from related literature reviews [11, 13, 15, 19] on the Ukraine conflict, such as Ukraine invasion, Ukraine-Russia Conflict, Russian special military operation. Two content experts then reviewed the keyword list independently. We selected only keywords which both experts approved. We then used these keywords to search for and collect all relevant memes in the following sites: memegine.com; knowyourmeme.com. We also used the keywords + meme e.g., Ukraine conflict meme, to obtain all memes within the first 20 pages in a Google image search.

4 Methods

Two content experts then independently coded (85% agreement) the resultant 1426 memes into three categories: pro-Russia; pro-Ukraine; irrelevant. A third content expert made the final decisions on coding disagreements. We selected content experts who had published at least five academic articles on mis/disinformation and/or Ukraine, defined broadly. Irrelevant memes were deleted to result in 70 pro-Russia and 121 pro-Ukraine memes. Examples of irrelevant memes included those which had too low resolution to determine the content, memes that were screenshots of tweets, and memes that were relating the conflict to anime content. We sought to develop multimodal classifiers to detect pro-Russia and pro-Ukraine memes, but were unable to do so due to limited data.

We then grouped memes within the pro-Russia and pro-Ukraine meme groups into broad themes, to determine if there were themes related to disinformation or pro-Ukraine sentiment. We first used DeepCluster. However, due to limited data, clusters were not meaningful and we categorized memes manually. Two content experts independently coded the pro-Russia and pro-Ukraine memes separately to identify broad themes. Coders first indicated if a meme was part of a broader theme, based on the overarching message of the meme. For example, if a meme was about the US preferring Ukrainian refugees to Syrian refugees, the coder would assign refugee as the theme. Memes were then organized based on their theme. Coders then reviewed thematic groups with two or fewer memes to see if these groups could be subsumed into larger thematic groups. Coders then compared thematic groups to develop themes common across coders (75% agreement). Any disagreement was resolved with a third content expert.

5 Results

5.1 Pro-Russia Memes

Coders identified five broad themes for pro-Russia memes: Russian Competence (n = 7); US and Allies Incompetence (n = 35); Azov Regiment (n = 6); Refugee (n = 12); Sexuality (n = 10). Overall, pro-Russia memes fall into specific thematic categories, e.g., targeting aspects of US and their allies’ culture and beliefs. We provide an overview of these categories next.

Fig. 1.
figure 1

Russian competence memes

Russian Competence Memes (Fig. 1) frame Putin or Russian soldiers as highly competent or benevolent. Such memes may be linked to existing Russian media framing of Putin as a strongman [12]. These memes may be part of broader disinformation campaigns where Putin and the Russian Army are seen as able to take on all challenges as a global force for good [25].

Fig. 2.
figure 2

US and Allies incompetence memes

US and Allies Incompetence memes (Fig. 2) are about how the US and its allies are incompetent. Examples of such memes suggest that NATO and the UN are unable to respond to the conflict, or NATO or Ukrainian soldiers are poorly trained. A significant portion of memes centered on how inmates in Ukraine with combat experience were released from jail to help defend against Russia, implying that the Ukrainian government is disingenuous. These memes may suggest that the US, its allies, and Ukraine are ineffectual, building support for Russia’s claim on Ukraine.

Fig. 3.
figure 3

Azov regiment meme examples

Azov Regiment Memes (Fig. 3) center on the Azov Regiment and their purported National Socialist leanings. The Azov Battalion, a regiment of the Ukrainian Army with roots in ultranationalist political groups, has been used by the Russian media since 2014 as an example of far-right support in Ukraine. The Russian media’s portrayal of the group exaggerates the extent to which its members hold neo-Nazi views. Multiple expert assessments conclude the modern Azov Regiment is a fairly typical fighting unit, with little, if any, political bent [21]. Examples of such memes detail National Socialist imagery within the Azov Regiment and the Western media’s supposed suppression of reports around the Azov Regiment’s National Socialist leanings. Such memes may be part of larger influence campaigns to bolster Russia’s claims of denazification and that the US is implicitly supporting a National Socialist state [8].

Fig. 4.
figure 4

Refugee meme examples

Memes in the Refugee theme (Fig. 4) focus on how the US and its allies supposedly prefer white Ukrainian refugees over Middle Eastern refugees [1, 9]. Examples of such memes indicate that Europeans are Islamophobic and do not want to resettle Syrian and Afghan refugees, but are supportive of Ukrainians displaced by the conflict. These memes bolster Russian influence campaigns that the US and its allies are hypocritical and racist, casting Russia as a benevolent state that has a rightful claim over Ukraine [24].

Fig. 5.
figure 5

Sexuality meme examples

Memes in the Sexuality theme (Fig. 5) center on how the US and its allies are too focused on progressive issues i.e., woke, to be effective supporters of Ukraine in the conflict [22]. Examples of such memes purport that the US army, despite not being deployed in the conflict, is comprised of numerous individuals who use they/them pronouns, which somehow renders the US army as ineffective. Such memes build on Russia’s anti-LGBT+ policies [17], which hold Russia to be a paragon of conservatism, hypermasculinity, and white culture, in opposition to the supposedly weak and ineffectual US. These memes may be part of Russian disinformation campaigns that reaffirm anti-LGBT+ rhetoric and build support for Russia’s position among right-leaning US individuals [20].

5.2 Pro-Ukraine Memes

Coders identified three broad themes for pro-Ukraine memes: Russian Deceit (n = 15); Russian Incompetence (n = 31); Ukrainian Fortitude (n = 75). Because of the diffuse nature of the Pro-Ukraine memes compared to the pro-Russia means, fewer broad themes emerged as durable patterns within the data. We provide an overview of these categories.

Fig. 6.
figure 6

Russian deceit meme examples

Memes in the Russian Deceit theme (Fig. 6) center on how Russia and/or Putin have engaged in underhanded tactics during the conflict. For example, some memes indicate that the Russian media obscures facts about Ukraine, and the Russian army is peacefully occupying parts of Ukraine. Such memes build on evidence of Russian disinformation campaigns, dispelling notions of Russia being a altruistic entity.

Fig. 7.
figure 7

Russian incompetence meme examples

Memes in the Russian Incompetence theme (Fig. 7) center on how Russia and/or Putin are inept and are unable to organize their attempted occupation of Ukraine. Such memes denote how Putin has endangered the Russian economy and is not providing leadership to Russian troops. These memes echo reports of Russian economic collapse and ineffectual Russian military movements, providing a concise version of news reports regarding certain aspects of the conflict. We suggest that news organizations or similar stakeholders use memes to present the latest news on the conflict in a simple format, suitable for younger people or those with limited English language proficiency.

Fig. 8.
figure 8

Ukrainian fortitude meme examples

Memes in the Ukrainian Fortitude theme (Fig. 8) are about the bravery and resourcefulness exhibited by Ukrainians and Zelenskyy. Such memes focus on sub-themes such as Ukrainian farmers appropriating Russian tanks, the Ghost of Kyiv (folk hero who destroyed numerous Russian aircraft), and Zelenskyy’s courage in the face of Russian occupation. Such memes may boost morale among Ukrainians, aiding Ukrainian mental health and wellbeing during the conflict.

6 Discussion

6.1 Implications of Findings

Our goal was to categorize memes involving the Ukraine conflict into pro-Russia and pro-Ukraine categories, and then group memes into broad themes within these categories. The dataset curated can be used to build pro-Russia and pro-Ukraine meme classifiers. A strength of our work is our systematic annotation strategy. The systematic strategy we employed suggests the veracity of our findings and we hope that results can add to research and policy around limiting disinformation around the Ukraine conflict and bolstering support for Ukrainians. Results detailed that pro-Russia memes tend to fall into specific thematic categories that seek to undermine specific elements of US and their allies’ policy and culture. Pro-Ukraine memes are far more diffuse thematically, highlighting admiration for Ukraine’s people and its leadership. Our findings are supported by previous research, around Russian disinformation campaigns [24] and attempts by Ukrainians to build support for their cause and improve morale among Ukrainians affected by the conflict. However, previous work does not explore the range of memes around the conflict, instead centering on other forms of media, and does not explore how memes are used to build support for Russia or Ukraine. We expand on previous research, providing an overview of memes around the Ukraine conflict.

6.2 Recommendations

Key to mitigating pro-Russia memes, which are likely part of Russian intelligence operations, are targeted efforts that focus on the underlying themes of such memes. Pro-Ukraine memes with specific themes may counter pro-Russia memes more effectively than more generic pro-Ukraine memes detailed in our findings. For example, stakeholders can develop memes around how support for Ukrainian refugees is leading to debate around refugee policy [26], or memes indicating the National Socialist leanings of the Azov Regiment are exaggerated. In line with recent work [5, 6], we suggest that where possible, Ukrainians affected by the conflict should be consulted around development of media-based efforts to counter pro-Russia influence campaigns, co-creating work [16]. For example, stakeholders developing memes to counter influence operations can discuss strategies with Ukrainians active in meme development, to ensure memes are effective, culturally responsive, and trauma informed. Such strategies may not only mitigate effects of influence operations, but also improve Ukrainian mental health and bolster public support for Ukrainians.

6.3 Limitations

Our findings relied on the validity of data collected with our search terms, and there may be memes which did not include our search terms. Our data may not be generalizable to non-English language memes around the Ukrainian conflict. We will include non-English terms in future work. We were unable to develop multimodal classifiers for pro-Russia or pro-Ukraine memes due to limited data and the limitations of computational tools. Given recent advancements in multimodal zero-shot and few-shot classification, we hope that future techniques will allow us to build meme classifiers with limited data, allowing us to respond swiftly to influence operations at the start of crises.