Keywords

Introduction

Despite the huge efforts put by Member States (MS) to suppress terrorist content online (TCO), terrorist groups have proven versatile and adaptable in finding new ways to distribute such content. Terrorist organisations and extremist groups leverage a large ecosystem of Internet-based platforms, varying from social media to hosting/cloud services and file-sharing systems, cutting across the Surface and the Dark Web, in order to accomplish their subversive intentions and support their goals and operations [1]. A huge amount of terrorism-related multimedia content is spread every day in several languages through such channels to reach a large audience for violent radicalisation and recruitment purposes, as well as for training and financing terrorist acts, which are not restricted by the different locations of the parties involved. Furthermore, terrorist and extremist groups often use fake digital identities to post links of relevant sources on darknets, to sell illegal goods and services, and organise terrorist attacks and criminal activities.

To address this challenge, Directive 2017/541 [2] urged MS to collaborate among themselves and also with third countries to remove such content or block it from the EU territory. This legal provision has also invoked the Internet industry to support MS by preventing such misuse of their services, albeit on a voluntary basis [2]. A step further has been taken by the EU by adopting Regulation (EU) 2021/784 [3] (also known as the TCO Regulation) urging the implementation of specific protective measures, requiring online hosting service providers (HSPs) to remove any illegal terrorist content within one hour after receiving an official removal order issued by a European competent authority (e.g. EUROPOL and MS LEAs). The obligations stemming from the Regulation are enforced to any HSP, irrespective of their size or popularity, creating significant operational, financial, and technical overhead, mainly impacting micro and small HSPs, given their limited capacity and human resources to apply effective response measures in a timely manner.

In this context, the ALLIESFootnote 1 project (https://www.alliesproject.com/) aims to assist micro and small HSPs in adhering to the TCO Regulation and protecting their platforms from terrorist abuse in an efficient and cost-effective way, as well as to enhance their capability to respond in a timely manner to removal orders issued by MS competent authorities. In particular, ALLIES proposes a framework that equips HSPs with a powerful suite of AI tools permitting the automated detection and the proactive removal of TCO on their end in a short period after the content upload and the prevention of its re-upload. In addition, the framework encompasses a distributed infrastructure allowing HSPs to leverage federated learning techniques for (re-)training, validating, and testing the underlying AI models utilising an ecosystem of annotated datasets hosted in private HSP storage spaces, ensuring that the raw data are inaccessible by any third party. Furthermore, the framework comprises a secure centralised online environment where multimodal hash representations of TCO are shared among HSPs to further facilitate the automated removal of subversive content. Finally, the ALLIES framework integrates a unified reporting mechanism to support the submission of removal orders issued by competent authorities, as well as the reporting of TCO removed by HSPs when necessary. The framework is being developed in a user-driven manner based on the requirements of HSP and law enforcement agency (LEA) personnel, while its development is also guided by the outputs of desktop and empirical research on online radicalisation, extremism, and terrorism, providing up-to-date information on these criminal activities.

TCO Regulation

The Regulation (EU) 2021/784 [3] of the European Parliament and of the Council of 29 April 2021 on addressing the dissemination of TCO also known as the TCO Regulation is the fruit of a four-year work based upon EU-led efforts of cooperating with HSPs, so that the dissemination of TCO is eradicated [4]. The TCO Regulation became applicable as of 7 June 2022, as per its Art. 24.

The main idea behind the TCO Regulation is to make sure that the digital single market of the EU is functioning seamlessly, and that public security is upheld by providing guarantees that hosting services available in the EU are not abused for terrorism-related purposes [3]. To this end, the TCO Regulation contains provisions requiring EU MS to introduce measures for the prompt removal of TCO, while also safeguarding fundamental rights, namely freedom of expression and freedom of information—two central EU values embedded in its core, being promulgated by Art. 12 of the Charter of Fundamental Rights of the EU. Additionally, as with other EU legal instruments in the security domain, a second requirement to the measures to be put into place by MS concerns the cooperation within the EU tackling this threat.

The main set of obligations addresses two types of entities: (i) HSPs—any entity storing information provided by and at the request of a content provider (the regulation applies regardless of the HSPs’ location, as long as they provide their services in the EU) [3], and (ii) competent authorities—authorities specially designated by an EU MS to issue removal orders, scrutinise removal orders issued by competent authorities of other MS, oversee the implementation of specific measures, and be competent to impose penalties for the non-compliance with the TCO Regulation [3].

The main instrument established in Art. 3 of the TCO Regulation is the ‘removal order’. The removal order prescribes either the complete removal or deactivation of access to the piece of content which has been identified as terrorism-related. The latter is understood as ‘material that incites or solicits an individual or a group of people to commit a terrorist act, or that provides instruction on making weapons or on other methods or techniques for use in a terrorist attack’ [3]. The removal order should be predeceased by information related to the applicable procedures and deadlines at least 12 hours prior its issue, in case this is the very first removal order issued in view of a respective HSP, and provided the case at hand does not constitute an emergency. Once the removal order is placed, the HSP has only one hour to remove the identified content. The removal order should be issued using a template provided by the TCO Regulation itself in its Annex 1. As outlined above, the TCO Regulation requires that a balance is stuck between fundamental rights and public security. This is quite visible in the said removal order template, which prescribes that information on the available redress mechanism is likewise noted. Being an EU instrument, the TCO Regulation foresees how cross-border cases should be handled. In case the HSP, whose content is identified as TCO, is based in another MS, a copy of the removal order should be sent to the competent authority of the said MS.

Last but not least, the TCO Regulation imposes obligations to HSPs which have been exposed to terrorist content—in case the competent authority has issued a relevant decision based on objective facts, such as the receipt of two or more removal orders in the last 12 months. In such a case, the HSP must take measures to protect itself and the provided services from further dissemination of TCO. The TCO Regulation does not prescribe such measures but leaves them to the discretion of the respective HSP; it only enumerates several examples including technical and organisational measures, such as allocating sufficient human resources to monitor the uploaded content or enabling users to report suspicious cases. The measures taken should meet the following requirements imposed by the TCO Regulation: (i) they must be effective in mitigating the level of exposure of the HSP services to TCO; (ii) they should be targeted and proportionate considering the TCO exposure level of the HSP services, as well as the technical and operational capabilities, financial strength, the number of users, and the amount of content they provide; (iii) they should be applied in a manner that takes full account of the rights and legitimate interest of the users, in particular the users’ fundamental rights on freedom of expression and information, respect for private life and protection of personal data; and (iv) they should be applied in a diligent and non-discriminatory manner [3].

Online Radicalisation, Extremism, and Terrorism Landscape

Traditionally, radicalisation has relied on direct social interactions in physical settings, like prisons, places of worship, and other places of community. In the last decades, the advent of the Internet has revolutionised how individuals communicate and spend their time, providing new opportunities for the spread of violent extremist ideologies [5]. Indeed, in recent years the expansion towards the online environment has facilitated the rapid dissemination of extremist ideologies, allowing to reach larger audiences with greater efficiency and guaranteeing more anonymity to the recruiters. Despite not being the only factor, the presence of TCO has been demonstrated to be a catalyst for radicalisation, which can lead individuals to commit acts of violence and terrorist attacks [6, 7]. Tackling TCO has therefore become a matter of urgency in recent years, within and beyond the EU.

Drawing on both desktop and empirical research (semi-structured interviews with 11 experts, namely scholars and representatives of LEAs), ALLIES has conducted a comprehensive overview of the strategies employed by jihadists, specifically the Islamic State (IS), and far-right supporters to disseminate content online.

The analysis highlights that the actors involved in the dissemination of TCO adapt their narratives to current events and regional dynamics to engage and keep the audience engaged, yet often drawing from traditional themes and ideologies. In particular, for what concerns the IS the main narratives are focused on: double-salvation, victimhood, and oppression [8, 9], while for far-right groups, movements, and supporters are: white supremacy, antisemitism, and male supremacy [10]. The theme of martyrdom and glorification of attackers is spreading in the far-right environment, showing similarities to Salafi Jihadism [11]. In various online platforms, far-right supporters, especially within the white supremacy movements, are increasingly sanctifying perpetrators for the purpose of propaganda and proselytism [1].

The strategic objectives pursued by the IS and far-right actors through the dissemination of these narratives partially overlap: radicalisation, recruitment, fundraising, and mobilisation are primary aims shared by both movements [9]. The target audience varies, but experts highlight that is increasingly younger, especially within the far-right environment.

The channels employed to disseminate TCO are multiple, and the strategy varies comparing IS and far-right supporters. The IS and its supporters have actively strived to establish a presence across various platforms, mainly through the massive dissemination in mainstream platforms of hyperlinks to file-sharing repositories and terrorist-operated websites, which guarantee a null or weaker content moderation and removal, frequently using bots [12]. The focus is therefore on the swiftness of dissemination rather than on the content or specific channels. Online violent far-right actors also adopt a multiplatform strategy for spreading TCO, but they differ from IS supporters in their dissemination methods. Instead of drawing on extensive lists of out-links, far-right actors primarily rely on loosely connected content creators to spread their messages. This reflects the nature of the extreme right environment, which, unlike the IS one, consists of a dynamic and fragmented network of individuals, groups, and movements, embracing various ideologies [1].

Both the IS and far-right extremists disseminate a variety of media outputs and content online, adapting them to the target and channel used, and adopting several visual strategies to make them more appealing [13, 14]. Aesthetics play a central role in conveying extremist narratives, making ideologies more accessible and attractive allowing users to familiarise with violence [14]. In this respect, humour and memes are strategic means used within the far-right extremist movements to disguise hate speech and make it more accessible for non-radicalised individuals [15]. The use of gameplay characteristics, known as gamification, is another visual strategy employed by both far-right and IS supporters: the gaming aesthetic makes the content instantly recognisable by a large audience, being familiar to millions of people, mainly young male, who are the primary target for both IS and far-right extremists [16, 17].

The activity of IS and far-right supporters shows a complex and rapidly mutating nature, driven by current events and new opportunities. As the Internet continues to play a pivotal role in the process of radicalisation and recruitment, it is necessary to further investigate and understand how these strategies adapt to new challenges and opportunities.

The ALLIES Framework

The technological solutions being developed as part of the ALLIES framework will assist micro and small HSPs towards implementing the TCO Regulation and protecting their online spaces from terrorist abuse, while facilitating the seamless communication of competent authorities and HSPs in the context of the submission and monitoring of removal orders. ALLIES aspires to build an ecosystem of modular interconnected AI solutions providing HSPs with the necessary means towards detecting and removing TCO in an efficient and cost-effective manner, as well as to promote their collaboration through a federated learning infrastructure and the sharing of existing knowledge in terms of TCO removal through a centralised hash repository. ALLIES will also support the speedy and trustworthy reporting between competent authorities and HSPs through a secure unified reporting mechanism.

By enhancing their arsenal with advanced AI tools supporting the continuous automated monitoring, processing, and analysis of large amounts of data uploaded by numerous users of their platforms, the HSPs will be in a position to proactively detect and remove TCO in a more efficient manner. The federated infrastructure will contribute towards improving the performance of the AI tools and the underlying models, leveraging multilingual and multimodal annotated datasets hosted in the private HSP spaces, ensuring that the raw data are not accessible by any third party. Additionally, any TCO detected and validated by an HSP will be fingerprinted based on multimodal hashing techniques and will be stored in a shared hash repository, enabling the prevention of re-uploading identical or highly similar content to any HSP platform. Finally, a unified reporting mechanism will facilitate the submission of removal orders by competent authorities to HSPs, as well as the notification of the authorities for TCO content that has been proactively removed by an HSP.

Figure 20.1 depicts an overview of the ALLIES framework. At its core, the powerful multimodal AI analysis tools enable HSPs to continuously monitor any content uploaded to their end by their users, and auto-detect any potentially violating material. The analysis tools process all modalities of interest including text, audio, and video, as well as their multimodal combinations thereof.

Fig. 20.1
A framework of the ALLIES. It includes A I analysis tools interconnected to federation with 3 H S Ps, leading to shared hash repository through hash generation and comparison service and also T C O reports to reporting mechanism along with competent authorities removal orders.

The ALLIES framework

Full size image

In particular, the framework is equipped with AI-based natural language processing (NLP) tools providing useful insights by analysing the multilingual online textual content and extracting, disambiguating, linking, and semantically enhancing concepts and named entities [18] associated with terrorism- and extremism-related content. These tools are further enhanced by machine translation and transliteration capabilities for multilingual support, targeting languages of interest for the ALLIES stakeholders. An automatic speech recognition (ASR) tool processes audio and visual files to produce accurate transcriptions for languages of interest that can be subsequently analysed by NLP technologies [19]. To this end, ALLIES is assessing and fine-tuning pretrained state-of-the-art open-source neural sequence-to-sequence ASR models. In addition, a multimedia visual understanding tool based on deep learning approaches and algorithms analyses videos and images to detect and recognise objects, concepts, human behaviour, and activities [20] relevant to the domain of terrorism. In particular, state-of-the-art deep neural network methods for robust object detection and recognition on multimedia content (i.e. images and videos), as well as for event and activity detection on videos, will be deployed based on supervised and semi-supervised techniques.

The indicators produced by the separate analysis of textual, audio, and visual modalities are leveraged by the classifiers that automatically categorise the online data into predefined categories relevant to the terrorism domain based on deep learning multimodal fusion techniques combined with neural representation methods [21].

The multimodal AI analysis tools serve as a first step towards the automated detection of TCO on the online spaces of the HSPs. Their outputs are introduced into a knowledge graph which meaningfully integrates knowledge extracted from multiple and heterogeneous data silos and different HSPs. The relevant content and semantics are leveraged by a powerful AI-boosted explanation generator engine [22] supporting human interpretation and inference, providing HSP moderators with reasoning and explanation on the results delivered by the AI solutions. The engine is accompanied by a set of analytics capabilities and visualisation dashboards, enabling an informative and user-friendly overview of the associated processes. The explanation engine together with the threat assessment tool, providing insights regarding the severity of threats posted online [23], helps the HSP moderators making informed decisions related to the removal of abusive content from their online spaces in an efficient manner, in the sense that this process eliminates the need for human inspection on all data uploaded to a hosting platform, and requires validating only the pieces of content flagged as TCO by the AI analysis tools and the explanation engine.

The framework is built upon a distributed infrastructure that enables the application of federated machine/deep learning techniques [24] for generating and training the underlying AI models of the analysis tools. In this context, each HSP platform is considered as a private space, inaccessible by third parties. The federated infrastructure allows (re-)training, validating, and testing AI models in a distributed way, leveraging annotated datasets included in the separate private spaces, prohibiting the access on the actual data and permitting the exchange of the relevant federated analysis results. The federated approach is respectful of social, ethical, legal, and privacy rules and constraints, adopting a privacy-by-design architecture that ensures personal data protection, and integrity and robustness of the produced models.

To further enhance the capability of micro and small HSPs in terms of preventing the terrorist abuse of their spaces, ALLIES introduces a centralised shared repository with multimodal hashes of TCO. In particular, the hash repository will be populated by HSPs with hashes of TCO detected on their end, whereas HSPs will also be equipped with hash-comparison services to quickly identify already flagged material on the space of other HSPs, and thus prevent its (re-)upload to their space. The shared hash repository permits the inexpensive cross-provider collaboration for the identification and removal of similar TCO that is potentially stored on multiple HSP platforms, without the need to share the actual content among multiple partners. To this end, ALLIES develops a multimodal data hashing service generating multimodal hash representations of the content hosted by the HSPs aiming to facilitate the inexpensive detection of identical or highly similar material posted on the same or different providers. Hashes will be generated for all modalities of interest, including text, audio, and image/video, as well as their multimodal combinations thereof, based on supervised and/or unsupervised AI approaches [25].

Furthermore, ALLIES will facilitate the submission of removal orders by competent authorities to micro and small HSPs according to the TCO Regulation, through a unified reporting mechanism towards the efficient and effective progress of the removal order, enhanced with real-time monitoring of the order status. The goal of the framework is to enhance the capacity of micro and small HSPs to respond to removal orders issued by competent authorities within the timeframe set by the one-hour rule. Finally, the reporting mechanism will also be used by HSPs when it is mandatory according to the TCO Regulation to notify competent authorities for TCO that has been proactively removed.

Conclusions

This work proposed a framework that integrates several AI tools aiming to assist micro and small HSPs in adhering to the TCO Regulation and protecting their platforms from terrorist abuse. It is envisaged that within such a framework, the tools developed not only will enhance the capacity of micro and small HSPs in terms of proactively removing TCO and implementing the TCO Regulation with the limited resources available but will also establish a model of communication among such enterprises so that they share best practices and experiences. The development of this framework is currently ongoing, and the integrated tools and technologies will be further adapted, improved, tested, and evaluated, in accordance with the end user and domain expert requirements in the context of the activities of the ALLIES project.