Algorithm substitution attacks against receivers

This work describes a class of Algorithm Substitution Attack (ASA) generically targeting the receiver of a communication between two parties. Our work provides a unified framework that applies to any scheme where a secret key is held by the receiver; in particular, message authentication schemes (MACs), authenticated encryption (AEAD) and public key encryption (PKE). Our unified framework brings together prior work targeting MAC schemes (FSE’19) and AEAD schemes (IMACC’19); we extend prior work by showing that public key encryption may also be targeted. ASAs were initially introduced by Bellare, Paterson and Rogaway in light of revelations concerning mass surveillance, as a novel attack class against the confidentiality of encryption schemes. Such an attack replaces one or more of the regular scheme algorithms with a subverted version that aims to reveal information to an adversary (engaged in mass surveillance), while remaining undetected by users. Previous work looking at ASAs against encryption schemes can be divided into two groups. ASAs against PKE schemes target key generation by creating subverted public keys that allow an adversary to recover the secret key. ASAs against symmetric encryption target the encryption algorithm and leak information through a subliminal channel in the ciphertexts. We present a new class of attack that targets the decryption algorithm of an encryption scheme for symmetric encryption and public key encryption, or the verification algorithm for an authentication scheme. We present a generic framework for subverting a cryptographic scheme between a sender and receiver, and show how a decryption oracle allows a subverter to create a subliminal channel which can be used to leak secret keys. We then show that the generic framework can be applied to authenticated encryption with associated data, message authentication schemes, public key encryption and KEM/DEM constructions. We consider practical considerations and specific conditions that apply for particular schemes, strengthening the generic approach. Furthermore, we show how the hybrid subversion of key generation and decryption algorithms can be used to amplify the effectiveness of our decryption attack. We argue that this attack represents an attractive opportunity for a mass surveillance adversary. Our work serves to refine the ASA model and contributes to a series of papers that raises awareness and understanding about what is possible with ASAs.


Introduction
Consider two parties communicating over an untrusted channel (in the presence of an adversary). Desired security properties for this scenario include confidentiality and integrity. Confidentiality means that the adversary is unable to learn anything about the messages sent between the parties. Integrity means that the parties can be sure that the messages have not been tampered with in transit. Both confidentiality and integrity are well-studied problems, and there are many reliable and provably secure cryptographic solutions. These solutions rely on the assumption that the software or hardware in which they are implemented behaves as expected. However, we know that in the real world, this assumption does not necessarily hold. Powerful adversaries have the means to insert unreliability into cryptography via external ("real-world") infrastructure: whether by influencing standards bodies to adopt "backdoored" parameters, inserting exploitable errors into software implementations, or compromising supply chains to interfere with hardware. The Snowden revelations showed that this is indeed the case, and that large and powerful adversaries (interested in mass surveillance) have sought to circumvent cryptography. The reader is referred to the survey by Schneier et al. [57], which provides a broad overview of subversion of cryptography, with some useful case studies detailing known subversion attempts.
The idea that an adversary may embed a backdoor or otherwise tamper with the implementation or specification of a cryptographic scheme or primitive predates the Snowden revelations, and was initiated in a line of work by Young and Yung that they named kleptography [60,61]. This area of study can be traced back to Simmons' work on subliminal channels, e.g. [58], undertaken in the context of nuclear non-proliferation during the Cold War. In the original conception [60], kleptography considered a saboteur who designs a cryptographic algorithm whose outputs are computationally indistinguishable from the outputs of an unmodified trusted algorithm. The saboteur's algorithm should leak private key data through the output of the system, which was achieved using the same principles as Simmons' earlier subliminal channels. Post-Snowden, work in this area was reignited by Bellare, Paterson and Rogaway (BPR) [14], who formalised the study of so-called algorithm substitution attacks (ASAs) through the example of symmetric encryption schemes. In abstract terms, the adversary's goal in an ASA is to create a subverted implementation of a scheme that breaks some aspect of security (such as IND-CPA in the case of encryption) while remaining undetected by the user(s).

Contributions
We provide formal definitions for subversion attacks against generic cryptographic primitives whose syntax allows for both the sending and receiving party to be subverted. Previous work in this area considered only subversion of the sender; our main contribution is to show that this assumption misses an important class of attack that targets the receiver. In this work, we describe how such an ASA against the receiver can be used to exfiltrate the (receiver's) key, which represents the most devastating attack from the point of view of an attacker. We show that this class of ASA can be applied to symmetric settings (authenticated encryption, message authentication codes, and data encapsulation mechanisms), as well as asymmetric settings (public key encryption and key encapsulation mechanisms). Our work brings together previous work targeting AEAD schemes [5] and MAC schemes [4] in a common framework, expanded to incorporate public key encryption.
Concretely, we alter the behaviour of the receiver's algorithm to leak information through (artificially induced) decryption error events-the subverted algorithm either rejects (particular, "trigger") valid ciphertexts or accepts (particular, "trigger") bogus ciphertexts. An adversary observing the receiver who is able to determine whether a ciphertext has been accepted or rejected learns some information; this subliminal channel can be used to exfiltrate the user's key. The assumption that a surveillance adversary is able to observe whether a receiver's algorithm implementation accepts or rejects a ciphertext is a mild one in many practical scenarios; for example, a decryption error may result in a network packet being dropped and automatically retransmitted. 1 A subverted algorithm could, furthermore, go beyond this by e.g. influencing timing information in future messages sent to the network. We conclude that this attack represents an attractive and easy to implement opportunity for a mass surveillance adversary.

AEAD
We first examine authenticated encryption with associated data (AEAD), a symmetric cryptographic primitive that offers the combined properties of confidentiality and message integrity. We show that our class of ASA applies to the decryption component of AEAD schemes, leaking the symmetric key. Our results stand in opposition to previous work [11,14,28] which proposed subversion resilience of a large class of AEAD schemes to which many if not all realworld constructions such as GCM, CCM and OCB belong, as long as their nonces are generated deterministically via a shared state maintained by both encryptor and decryptor. The crucial observation to resolve this apparent contradiction is that previous work has assumed, besides explicitly spelled out requirements like uniqueness of ciphertexts and perfect decryptability, implicit notions such as integrity of ciphertexts. In the ASA setting for AEAD where undermining the confidentiality of a scheme is the primary goal of an adversary, it seems just as natural to assume that the adversary is also willing to compromise the integrity guarantees as well.

MACs
We next show that our results apply equally in the setting of message authentication schemes (MACs). MACs provide a message authentication code or tag for a given message; conversely, given a message and a tag, the MAC provides verification that the tag was generated from the message (that is, that the tag is genuine). The security of a MAC is determined by the difficulty of forging tags. If no adversary can forge a tag, then a message with a correct tag must have been generated by the sender. An ASA against a MAC replaces either the tagging function (the generation of message authentication codes) or the verification function (checking that tags have been honestly generated) in such a way as to leak information to an adversary. Applying our attack to a MAC leaks the secret key to an adversary, allowing them to forge any tag. This is an attractive goal for an adversary in real-world settings, as once integrity has been compromised, this can often be leveraged to perform any number of other attacks, for example: enabling attacks against ("encrypt-then-MAC") confidentiality; getting users to accept compromised (authenticated) software updates; injecting malicious packets into (secured) communication streams to de-anonymise users.

PKE
Lastly, we show that public key encryption (PKE) is also vulnerable to our class of ASA. A public key (or asymmetric) encryption scheme allows secure communication between parties that have not shared a secret key with one another. PKE works by having two keys: the public key is used to encrypt messages, and the private key is used to decrypt. The security of a PKE scheme is determined by the difficulty of determining any information about underlying messages for a given ciphertext.
Our ASA attacks on PKE require a fairly large number of ciphertexts to be sent and observed to reject erroneously in order for the private key to be exfiltrated. In practice, this condition will be met: consider a server that hosts traffic for a large number of clients. The server will have a private/public key pair which is held static over long periods of time. Observing the server receive ciphertexts from many clients will allow an adversary to witness a large enough amount of traffic to recover the server's private key, rendering all communications between clients and server compromised.
Due to the high overheads associated with PKE, symmetric encryption is better suited to bulk communication. In most practical settings, PKE is used to establish a shared secret between the sender and receiver, so that the shared secret may be used as a key for communicating via symmetric encryption. This notion of sending keys for symmetric encryption via public key methods is formalised as a key encapsulation mechanism (KEM). We show how our notions of subversion apply also to KEMs in Appendix 1. KEMs are typically used together with a data encapsulation mechanism (DEM) in a so-called hybrid encryption scheme to PKE-encrypt messages. We give the definition of a DEM in Appendix 1 for completeness.

Structure of this document
We first describe related work in Sect. 2, focussing on ASAs that target symmetric encryption, PKE and MACs. Section 3 describes the notation used in this article. We give an abstract description of an ASA targeting generic cryptographic schemes consisting of a sender and receiver in Sect. 4, together with notions of undetectability (Sect. 4.1) and key recovery (Sect. 4.2). We also discuss hybrid subversion (Sect. 4.3), the idea that multiple algorithms (e.g. key generation and encryption) is subverted in tandem. In Sect. 5, we discuss authenticated encryption with associated data, giving syntax and security definitions: privacy in Sect. 5.1 and integrity in Sect. 5.2. We show that our notion of ASAs apply to AEAD schemes in Sect. 5.3. Section 6 discusses MACs, including the definition of integrity; Sect. 6.1 shows that our notion of ASAs apply to MAC schemes. Section 7 discusses PKE, giving syntax and security definitions; Sect. 7.1 shows that our notion of ASAs applies to PKE schemes. We describe our concrete subversion attack, targeting the receiver algorithm, in Sect. 8, together with an analysis of the undetectability and key recovery properties of our attack. We give two versions, a passive attack in Sect. 8.2 and an active attack in Sect. 8.3.

Symmetric encryption
BPR [14] demonstrate an attack against certain randomised encryption schemes that relies on influencing the randomness consumed in the course of encryption. Their attack, which they call the "biased-ciphertext attack", is a generic method that relies on rejection sampling. Randomness is resampled until ciphertexts satisfy a particular format (for example, implanting information in the least significant bits), resulting in a subliminal channel.
There is a tension for "Big Brother" between mounting a successful attack and being detected; clearly an attack that simply replaces the encryption algorithm with one that outputs the messages in plaintext would be devastating yet trivially detectable. BPR stipulate that ciphertexts generated with a subverted encryption algorithm should at the very least decrypt correctly with the unmodified decryption routine, in order to have some measure of resistance to detection. Furthermore, BPR define the success probability of a mass surveillance adversary in carrying out a successful attack, as well as the advantage of a user in detecting that a surveillance attack is taking place. The attack of BPR was later generalised by Bellare, Jaeger and Kane (BJK) [11] whose attack applies to all randomised schemes. Furthermore, whereas the attack of BPR is stateful and so vulnerable to detection through state reset, the BJK attack is stateless. BJK [11] later formalised the goal of key recovery as the desired outcome of an ASA from the point of view of a mass surveillance adversary. Lastly, BPR also establish a positive result that shows that under certain assumptions, it is possible for authenticated encryption schemes to provide resistance against subversion attacks.
Degabriele, Farshim and Poettering (DFP) [28] critiqued the definitions and underlying assumptions of BPR. Their main insight is that the perfect decryptability-a condition mandated by BPR-is a very strong requirement and artificially limits the adversary's set of available strategies. In practice, a subversion with negligible detection probability, say 2 −128 , should be considered undetectable. 2 As DFP note, decryption failures may happen for reasons other than a subverted encryption algorithm, and if they occur sporadically may easily go unnoticed. Thus, a subverted encryption scheme that exhibits decryption failure with a very low probability is a good candidate for a practical ASA that is hard to detect. DFP demonstrate how this can be achieved with an input-triggered subversion, where the trigger is some message input that is difficult to guess, making detection practically impossible. Our work complements the trigger message approach of DFP by limiting ciphertext integrity and establishing a covert channel through decryption error events.

PKE
Yung and Young (YY) in [60] examine subverting asymmetric protocols in so-called "SETUP" attacks. Their core idea is to encode some information within the public key that allows the private key to be reconstructed. As a simple example, let the public key encode the encryption of the user's private key under the adversary's key. Subverted keys should be indistinguishable from real keys and only the adversary should be able to recover a user's private key from the subverted public key. As well as showing how to subvert RSA keys, YY also give examples of attacks against ElGamal, DSA and Kerberos. Later, Crépeau and Slakmon [27] gave an improved subversion attack against RSA which works by hiding half of the bits of p in the representation of the RSA modulus N = pq. Using Coppersmith's partial information attack [24], it is then possible to recover p and q.
For the prior work on symmetric encryption discussed above, the techniques can be translated naturally into a PKE setting. Attacks against the encryption algorithm of a PKE scheme however do not present an attractive attack to a mass surveillance adversary, as there is limited scope to undermine confidentiality. The covert channel usually has a bandwidth of a small number of bits per (subverted) ciphertext: not enough to leak the underlying messages. Leaking the private key would allow confidentiality to be broken completely, but the encryption algorithm does not have access to the private key. Chen, Huang and Yung [23] overcome these limitations by considering hybrid PKE constructions consisting of a KEM to send encapsulated session keys which are used for symmetric encryption with a DEM. Their non-generic attack applies to a particular class of practical KEM constructions and leaks session keys, that in turn break the security of the DEM. In contrast, for a PKE primitive not consisting of a hybrid KEM/DEM construction, targeting the decryption algorithm remains the only way to subvert the encryption/ decryption facility of a PKE scheme.

MACs
The only prior work on MAC subversion that we are aware of is by Al Mansoori, Baek, and Salah [1] who explore how a MAC component in the EAP-PSK wireless protocol could be subverted. After first arguing [1, §II.D] that randomised MAC schemes offer better protection against a kind of birthday attack, they restrict attention to precisely one corresponding construction (two-key CBC-MAC with a random translation of the second key, a scheme that already turned out to be broken in [44]) and show that the rejection-sampling-based key-extraction techniques from [14] are applicable in this setting as well. We emphasise that our results reach far beyond this: our subversion attacks are generic (rather than being focused on one specific MAC) and we do not require exotic technical conditions like randomised tag generation. 3

Further work
Cryptographic reverse firewalls [21,33,47,48,59] represent an architecture to counter ASAs against asymmetric cryptography via trusted code in network perimeter filters. At a high level, the approach is for a trusted third party to re-randomise ciphertexts before transmission over a public network to destroy any subliminal messages. Fischlin and Mazaheri show how to construct ASA-resistant (asymmetric) encryption and signature algorithms given initial access to a trusted base scheme [37]. Their approach uses trusted samples to essentially perform re-randomisation of ciphertexts.
In a series of work, Russell, Tang, Yung and Zhou [53][54][55][56] study ASAs on one-way functions, trapdoor one-way functions and key generation as well as defending randomised algorithms against ASAs using so-called watchdogs. The watchdog model allows a trusted party to test the implementation of a primitive for subversion, in a variety of different assumptions (e.g. on-or offline, black-or white-box access). Combiners are often used to provide subversion resilience, particularly in the watchdog model. A combiner [38,50] essentially combines the output from different algorithms (or runs of the same algorithm) in such a way as to produce secure (in this case, unsubverted) combined output as long as any one of the underlying outputs is secure. Aviram et al. [7] consider combining (potentially maliciously chosen) keys for Post-Quantum protocols such as TLS. Bemman, Chen and Jager [16] show how to construct a subversion-resilient KEM, using a variant of a combiner and a subversion resilient randomness generator. Their construction considers Russell et al.'s watchdog from a practical perspective, meaning an offline watchdog that runs in linear time. Another line of work, [9,31,36], examined backdoored hash functions, showing how to immunise hash functions against subversion.
Bellare, Kane and Rogaway [12] explore how large keys can prevent key exfiltration in the symmetric encryption setting. Bellare and Hoang [10] give PKE schemes that defend against the subversion of random number generators. The use of state reset to detect ASAs is studied by Hodges and Stebila [41]. Berndt and Liśkiewicz [17] reunite the fields of cryptography and steganography. Goh, Boneh, Pinkas and Golle [39] show how to add key recovery to the SSH and SSL/TLS protocols. Ateniese, Magri and Venturi [6] study ASAs on signature schemes. Berndt et al. consider ASAs against protocols such as TLS, WireGuard and Signal [18]. Dodis, Ganesh, Golovnev, Juels and Ristenpart [32] provide a formal treatment of backdooring PRGs, another form of subversion. This work was extended by Degabriele, Paterson, Schuldt and Woodage [29] to look at robust PRNGs with input. Camenisch, Drijvers and Lehmann [22] consider Direct Anonymous Attestation in the presence of a subverted Trusted Platform Module.

Cryptographic versus non-cryptographic subversion
In the literature on cryptography, the notion of an ASA assumes the malicious replacement of one or more algorithms of a scheme by a backdoored version, with the goal to leak key material, or at least to weaken some crucial security property. Different types of substitution attack appear in other areas of computing and communication. We discuss some examples in the following. Program code in the domain of computer malware routinely modifies system functions to achieve its goals, where the latter comprises delivering some damaging payload, ensuring non-detection and thus survival of the malware on the host system, and in some cases even self-reproduction. Numerous techniques towards suitably modifying a host system have been developed and reported on by academic researchers and hackers. Standard examples include redirecting interrupt handlers, changing the program entry point of an executable file, and interfering with the OS kernel by overwriting its data structures [40].
Malicious modifications of implemented functionality are also a recognised threat in the hardware world. It is widely understood that circuit designers who do not possess the technical means to produce their own chips but instead outsource the production process to external foundries, risk that the chips produced might actually implement a maliciously modified version of what is expected. A vast number of independent options are known for when (within the production cycle) and how (functionally) subversions could be conducted. For instance, the survey provided in [19] reports that circuit design software (CAD) could be maliciously altered, that foundries could modify circuits before production, and that after production commercial suppliers could replace legitimate chips by modified ones. Further, [19] suggests that appealing types of functionality modification include deviating from specification when particular input trigger events are recognised, and/or to leak values of vital internal registers via explicitly implemented side channels. Any such technique (or combination thereof) has an individual profile regarding the associated costs and attack detectability. Which of the many options is most preferable depends on the specific attack scenario and target.
We refer to the software-and hardware-based subversion techniques discussed above as "technology driven". This is in contrast to the techniques considered in this paper which we refer to as "semantics driven". We consider the two approaches orthogonal: Our (semantics driven) proposed subversion can be implemented using techniques from e.g. [19,40] (but likewise also through standard methods), and technology-driven subversion proposals can be applied against cryptographic implementations (but likewise also against any other interesting target functionality). Our semantics-driven approach in fact aims to maximise technology independence. As a consequence, the line of attacks proposed in this paper can be implemented easily in software (e.g. in libraries or drop-in code), in hardware (e.g. in ASICs and FPGAs), and in mixed forms (e.g. firmware-programmed microcontrollers). The strategy to achieve this independence is to base the attacks and corresponding notions of (in)security on nothing but the abstract functionalities of the attacked scheme as they are determined by their definitions of syntax and correctness.
As the technology-driven and semantics-driven approaches are independent, they can in particular be combined. This promises particularly powerful subversions. For instance, consider that virtually all laptops and desktop PCs produced in the past decade are required to have an embedded trusted platform module (TPM) chip that supports software components (typically boot loaders and operating systems) with trusted cryptographic services. In detail, software can interact with a TPM chip through standardised API function calls and have cryptographic operations applied to provided inputs, with all key material being generated and held exclusively in the TPM. As TPMs are manufactured in hardware, it seems that the (technology driven) subversion options proposed in [19] would be particularly suitable. However, as most of the attacks from [19] require physical presence of the adversary (e.g. to provide input triggers via specific supply voltage jitters or for extracting side channel information by operating physical probes in proximity of the attacked chip), only those options seem feasible where all attack conditions and events can be controlled and measured via the software interface provided by the API. This is precisely what our semantics driven attacks provide. We thus conclude by observing that dedicated cryptographic hardware like TPMs can only be trusted if extreme care is taken during design and production. While our article lays open the most general and clean line of attack, other attacks might exist as well.

Discussion
As the discussion of cryptographic ("semantics driven") vs. non-cryptographic ("technology driven") subversion shows, achieving security against adversaries mounting ASAs is difficult, and essentially reduces to assuming trust in particular components or architectures. The three main theoretical approaches to preventing or mitigating against ASAs in the literature, discussed in Sect. 2.4, are reverse firewalls, self-guarding protocols and watchdogs. We note that these approaches apply in the main to asymmetric primitives, and so (appropriately adapted to target receiver algorithms) would be suitable to defend against our attack against asymmetric schemes in Sect. 7.
Defending against our attacks on AEAD and MACs is more difficult. We note that the watchdog model applies in theory, while reverse firewalls and self-guarding approaches are ineffective against symmetric primitives. The watchdog model considers splitting a primitive into constituent algorithms that are run as subroutines by a trusted "amalgamation" layer. This allows the constituent algorithms to be individually checked and sanitised. Considering the verification algorithm of a MAC scheme as an example, the canonical approach of recalculating and checking the tag is modelled by letting the verification algorithm be a trusted amalgamation of the tagging algorithm with an identity test. The tagging algorithm typically runs a hash function as a subroutine, and so applying results from [9,31,36] would allow for the claim that the verification algorithm can be made subversion-resilient in the watchdog model. The assumption of a trusted amalgamation is precisely what makes our attack infeasible, but this assumption is questionable in real-world settings. In particular, as we discussed above, the presence of non-cryptographic vectors makes this assumption unlikely to hold in practice.
Lastly, we note that none of the theoretical approaches are fully satisfying, requiring strong or impractical assumptions. Indeed, it is telling that there are no implementations of subversion-resilient primitives to date, although some recent work seems promising in this regard [16,21]. The best defence seems to be the unglamorous task of minimising risk by implementing a variety of control mechanisms across the whole infrastructure, in a process of security management. In particular: software implementations could be protected by measures including regular integrity tests and secure boot, hardware implementations could be protected by technical controls such as threshold implementations or testing amplification [35], and both cases can be strengthened by relying on open source implementations and verified supply chains. While such measures can go some way towards minimising risk, we emphasise that there are no security guarantees.

Notation
We refer to an element x ∈ {0, 1} * as a string, and denote its length by |x|. The set of strings of length l is denoted {0, 1} l . By ε, we denote the empty string. For We use code-based notation for probability and security experiments. We write ← for the assignment operator (that assigns a right-hand-side value to a left-hand-side variable). If S, S are sets, we write S ∪ ← S shorthand for S ← S ∪ S . If S is a finite set, then s ← $ S denotes choosing s uniformly at random from S. For a randomised algorithm A, we write y ← $ A(x 1 , x 2 , . . .) to denote the operation of running A with inputs x 1 , x 2 , . . . and assigning the output to variable y. We denote a γ -biased Bernoulli trial by B(γ ), i.e. a random experiment with possible outcomes 0 or 1 such that We use superscript notation to indicate when an algorithm (typically an adversary) is given access to specific oracles. An experiment terminates with a "stop with x" instruction, where value x is understood as the outcome of the experiment. We write "win" ("lose") as shorthand for "stop with 1" ("stop with 0"). We write "require C", for a Boolean condition C, shorthand for "if not C: lose". (We use require clauses typically to abort a game when the adversary performs some disallowed action, e.g. one that would lead to a trivial win.) The ":=" operator creates a symbolic definition; for instance, the code line "A:=E" does not assign the value of expression E to variable A but instead introduces symbol A as a new (in most cases abbreviating) name for E.

Notions of subversion attacks
We consider subversions of the algorithms of cryptographic schemes. Abstractly, we consider a cryptographic scheme = ( .gen, .S, .R) consisting of three components: a key generation algorithm together with an algorithm on the sender side and an algorithm on the receiver side, where the cryptographic scheme is an encryption scheme, .S represents encryption and .R decryption; when we consider message authentication schemes, the corresponding components represent tagging and verification.
We give a generic syntax to the scheme as follows: key generation .gen outputs a key pair (k S , k R ) ∈ K S × K R ; the sender algorithm has associated input and output spaces X , Y and takes as input a key k S ∈ K S and x ∈ X , outputting y ∈ Y; the receiver algorithm has associated input and output spaces Y, X (respectively). We note that X X ; in particular, ⊥ ∈ X \ X . The receiver algorithm takes as input a key k R ∈ K R and y ∈ Y, outputting x ∈ X ; the special symbol ⊥ is used to indicate failure. A shortcut notation for this syntax is where the probability is over all random coins involved. In the case that δ = 0, the scheme is said to be perfectly correct. We note that this generic syntax applies to symmetric encryption (Sect. 5) and message authentication (Sect. 6), as well as to public key encryption (Sect. 7), where for the symmetric case we require k S = k R .
In the following, we give formal definitions for subversion of key generation, sender and receiver algorithms, together with the notion of undetectability (UD). In a nutshell, a subversion is undetectable if distinguishers with black-box access to either the original scheme or to its subverted variant cannot tell the two apart. A subversion should exhibit a dedicated functionality for the subverting party, but simultaneously be undetectable for all others. This apparent contradiction is resolved by parameterising the subverted algorithm with a secret subversion key, knowledge of which enables the extra functionality. (The same technique is used in most prior work, starting with [14].) In what follows, we denote the corresponding subversion key spaces with I gen , I S and I R .
In this section, we also specify, by introducing notions of key recoverability, how we measure the quality of a subversion from the point of view of the subverting adversary (who is assumed to know the subversion keys).

Undetectable subversion
We first define undetectability notions for subverted key generation, sender and receiver algorithms separately. We then offer a joint definition. Subverted Key Generation A subversion of the key generation algorithm .gen of a cryptographic scheme consists of a finite index space I gen and a family of algorithms That is, for all i ∈ I gen the algorithm .gen i can syntactically replace the algorithm .gen.
As a security property, we require that also the observable behaviour of .gen and .gen i be effectively identical (for uniformly chosen i ∈ I gen ). This is formalised via the games UDG 0 , UDG 1 in Fig. 1(left). For any adversary A, we define the advantage and say that family Gen undetectably subverts algorithm . gen if Adv udg (A) is negligibly small for all realistic A.
Subverted Sender A subversion of the sender algorithm .S of a cryptographic scheme consists of a finite index space I S and a family S = {S i } i∈I S of algorithms That is, for all i ∈ I S the algorithm .S i can syntactically replace the algorithm .S.
As a security property, we also require that the observable behaviour of .S and .S i be effectively identical (for uniformly chosen i ∈ I S ). This is formalised via the games UDS 0 , UDS 1 in Fig. 1 (centre). Note that in contrast to prior work like [14,28], our distinguishers are given free choice over the keys to be used. 4 For any adversary A, we define the advantage Fig. 1 Games UDG, UDS and UDR modelling undetectability for the subversion of (respectively) key generation, sender and receiver algorithms for a cryptographic scheme . See Sect. 3 for the meaning of ":=". Note that in each game, the two unsubverted oracles are actually redundant and say that family S undetectably subverts algorithm .S if Adv uds PKE (A) is negligibly small for all realistic A. Subverted Receiver A subversion of the receiver algorithm .R of a cryptographic scheme consists of a finite index space That is, for all i ∈ I R , the algorithm .R i can syntactically replace the algorithm .R.
As a security property, we also require that the observable behaviour of .R and .R i be effectively identical (for uniformly chosen i ∈ I R ). This is formalised via the games UDR 0 , UDR 1 in Fig. 1 (right). For any adversary A, we define the advantage and say that family R undetectably subverts algorithm .R if Adv udr PKE (A) is negligibly small for all realistic A. The above undetectability notions demand that subversions do not change the observable behaviour of the key generation, sender and receiver algorithms. A consequence of this is that none of the correctness or security properties of the scheme are noticeably harmed by subversion.

Hybrid subversion of key generation, sender and receiver algorithms
We give a joint definition of undetectability, in the case where the key generation, sender and receiver algorithms are subverted. This is the most general definition; in particular contexts, it may not be appropriate to consider subversion of a particular algorithm-we discuss this in Sects. 4.3, 5.3, 6.1.1 and 7.1.1.

Subversion leading to key recovery
We observed above that if any of the components .gen, .S, .R of a cryptographic scheme is undetectably subverted, with uniformly chosen indices i gen , i S , i R that remain unknown to the participants, then all security guarantees are preserved from the original scheme. This may be different if (any of) i gen , i S , i R are known to an attacking party, and indeed, we assume that mass-surveillance attackers leverage such knowledge to conduct attacks. For any cryptographic scheme, the most devastating attack goal for an attacker is key recovery (KR): Users generate keys using their key generation algorithm (k S , k R ) ← $ .gen i gen . 5 Generated secret keys are kept hidden, and the adversary aims at recovering these keys through the subversion. Note that in the symmetric case, k S = k R , whereas in the asymmetric case the receiver's key k R represents the private key. In either case, the value k R is the target of a KR adversary. and KRA modelling key recoverability for passive and active attackers, respectively. Note that the adversary's aim is to recover the receiver's key k R , as in both symmetric and asymmetric settings this value is secret We formalise this attack goal in two versions. The KRP game in Fig. 2(centre) assumes a passive attack in which the adversary cannot manipulate inputs or outputs (typically representing messages or ciphertexts) to the sender or receiver, and the KRA game in Fig. 2(right) assumes an active attack in which the adversary can inject and test arbitrary receiver inputs (which potentially correspond to sender outputs). In both cases, with the aim of closely modelling real-world settings, we restrict the adversary's influence on the sender inputs x by assuming a stateful "message sampler" algorithm MS (reflecting the fact that, in the contexts we consider inputs to .S typically represent messages) that produces the inputs to .S used throughout the game. The syntax of this message sampler is where σ, σ ∈ are old and updated state, input α ∈ A models the influence that the adversary may have on message generation, and output β ∈ B models side-channel outputs. In Fig. 2, we write for the initial state. Note that while we formalise the inputs α and the outputs β for generality (so that our models cover most real-world applications), our subversion attacks are independent of them. 6 For any message sampler MS and adversary A, we define the advantages  6 … meaning that the reader may safely choose to ignore them.
We say that subversion family Gen, S, R is key recovering for passive attackers if for all practical MS there exists a realistic adversary A such that Adv krp MS(A) reaches a considerable value (e.g. 0.1). 7 The key recovery notion for active attackers is analogous.

Discussion
We note that the adversary need not necessarily exfiltrate each individual bit of the user's key, 8 in order to successfully recover it-this is implicit in our definitions of key recovery. To formalise this, we let the "leakage key" k ∈ {0, 1} λ be a string such that knowledge of k is sufficient for an adversary to break the security of the primitive. At worst, from the perspective of the adversary, the leakage key may simply be the bit representation of the user's key. We note that in practice a leakage key consisting of most of the user's key is sufficient for an adversary to recover the full key using brute force; the exact number of bits to be brute forced would depend on the context and would involve a trade-off for the adversary. Nevertheless, the notion is intuitively clear.
Furthermore, in some contexts, there may be some redundancy or structure that allows for a shorter leakage key. As an example, one may consider DES keys as being 64-bit strings with 8 bits of redundancy, so that an effective leakage key 7 Our informal notions ("realistic" and "practical") are easily reformulated in terms of probabilistic polynomial-time (PPT) algorithms for readers who prefer a treatment in the asymptotic framework. Given that asymptotic notions don't reflect practice particularly well, we prefer to use the informal terms. 8 Or a bit representation thereof, if it is not a bit string would be of size 56 bits. As another example, the private key in RSA encryption is knowledge of the factorisation of the public modulus N = pq. Supposing that the modulus N can be represented using n = log N -bits, one may consider RSA private keys as being n/2 = log p -bit strings. However, knowledge of around half the bits of p is sufficient to be able to factorise N using Coppersmith's partial information attack [24], so that an effective leakage key might have length λ = log p /2. 9 A different approach might be to leak, for example, the seed of a pseudo-random number generator. We discuss breaking security without extracting the full key further in Sect. 4.4.

Hybrid subversion
Previous work on subversion has looked at either subverted key generation 10 or subverted encryption/ decryption, but not considered the case where these are subverted in tandem. For key generation, this has meant that the subverted algorithm needs to leak the whole key in a single operation. This setting was studied by Young and Yung [60] under the name "kleptography", and they showed how it is possible to subvert key generation such that the adversary is able to recover the private key sk from the public key pk (together with any public parameters and knowledge of secret trapdoor information). They show how such attacks against key generation could look in the case of RSA and ElGamal cryptosystems. Such subversion imposes a large cost on the subverter: requiring that all key bits are leakable in one operation means that the subverted keys are given some structure (e.g. the public key is the encryption of the secret key under the attacker's key). This imposes significant overhead, and would likely lead to detection in a real world setting (using either timing information, code review or hardware inspection). Considering the subversion of key generation and sender/receiver algorithms in tandem, it is possible to reduce this overhead.
Generically, this tandem subversion can be achieved by subverting key generation to produce weaker keys and combining this with a subverted sender and/or receiver that provides a subliminal channel. Consider a subverted key generation algorithm .gen i that outputs (receiver keys) in some reduced key set K R ⊂ K R . The smaller this subverted key set K R , the less information needs to be leaked via the subliminal channel. One method to implement such a subverted key generation algorithm is to use the rejection sampling method 9 We note that in practice the security for an RSA modulus of size n is far less than n/2 bits; for example, an RSA modulus of size 1024 is believed to have security at most 80 bits [8], corresponding to the computational effort required to factorise an RSA modulus. 10 And potentially, the associated public parameters if those form part of the formalisation used. described in Sect. 2.1, so that .gen i runs the unsubverted algorithm .gen as a subroutine and resamples until keys are in K R . There are certainly more targeted attacks that take into account the specific structure of keys being generatedand that may leverage more specific attacks than the generic weakening of keys.

Breaking security without extracting the full key
The KRA and KRP notions introduced in Sect. 4.2 assume that key recovery is the ultimate goal in subversion. This suggests that longer keys make a scheme more resilient, an approach explored in big key cryptography [12]. In practice, it may be more efficient to exploit non-generic features of a particular scheme to minimise the information to be leaked. In this section, we will consider AEAD schemes as an illustrative example.
As we detail, many current AEAD schemes have inner building blocks that maintain their own secret values, and scaling up key sizes does not automatically also increase the sizes of these internal values. We note that proposed ASAs against AEAD schemes (including our attacks presented in Sect. 8) can easily be adapted to leak this internal information instead of the key. As the recovery of such values might not always directly lead to full message recovery, the assessment of whether the resulting overall attack is more or less effective than our generic attacks has to be made on a per scheme basis. We exemplify this on the basis of two of the currently best-performing AES-based AEAD schemes: GCM [34] and OCB3 [45]. In both cases, the size of the crucial internal value and the block size of the cipher have to coincide and the latter value is fixed to 128 bits for AES (independently of key size). AES-GCM We consider the following abstraction of GCM. The AEAD key k is used directly to create an instance E of the AES blockcipher. To encrypt a message m with respect to associated data d and nonce n, E is operated in counter mode, giving a pad E(n + 1) E(n + 2) . . . , where a specific nonce encoding ensures there are no collisions between counter values of different encryption operations. The first part c 1 of the ciphertext c = c 1 c 2 is obtained by XORing the pad into the message, and finally the authentication tag c 2 is derived by computing c 2 ← E(n) + H h (d, c 1 ). Here, H h is an instance of a universal hash function H indexed (that is, keyed) with the 128-bit value h = E(0 128 ). Concretely, . . , v l are such that a prefix v 1 . . . v j is a length-padded copy of the associated data d, the middle part v j+1 . . . v l−1 is a lengthpadded copy of ciphertext component c 1 , and the last item v l is an encoding of the lengths of d and c 1 . The addition and multiplication operations deployed in this computation are those of a specific representation of the Galois field GF(2 128 ).
In executing a practical ASA against AES-GCM, it might suffice to leak the value h (which has length 128-bits independently of the AES key length, and furthermore stays invariant across encryption operations). The insight is that if the key of a universal hash function is known, then it becomes trivial to compute collisions. Concretely, assume the adversary is provided with the AES-GCM encryption c = c 1 c 2 = enc(k, n, d, m) for unknown k, m but chosen d, n. Then by the above, we have where the coefficients v 1 . . . v j are an encoding of d and R is some residue. If, having been successfully leaked by the ASA, the internal value h is known, by solving a linear equation, it is easy to find an associated data string d = d, |d | = |d|, such that for its encoding v 1 n, d, m). In a CCA attack, the adversary can thus query for the decryption of c with associated data d and nonce n, and thus fully recover the target message m. We finally note that this attack can be directly generalised to one where also the c 1 and c 2 components are modified, resulting in the decryption of a message m = m for which the XOR difference between m and m is controlled by the adversary. OCB3 Multiple quite different versions of the OCB encryption scheme exist [42], but a common property is that the associated data input is incorporated via "ciphertext translation" [52]. To encrypt a message m under key k with associated data d and nonce n, in a first step, the message m is encrypted with a pure AE scheme 11 ) to an intermediate ciphertext c * ← enc * (k, n, m). Then to obtain the final ciphertext c, a pseudo-random function value F k (d) of the associated data string is XORed into the trailing bits of c * . Concretely, in OCB3, we have F k (d) = l i=1 E(v i + C i ) where all addition operations are XOR combinations of 128 bit values, E(·) stands for AES enciphering with key k, values v 1 , . . . , v l represent a length-padded copy of associated data d, and coefficients C 1 , . . . , C l are (secret) constants deterministically derived from the value L = E(0 128 ).
In the context of an ASA, we argue that it is sufficient to leak the 128 bit value L. The attack procedure is, roughly, as in the AES-GCM case. Assume the adversary is provided with the OCB3 encryption c = enc(k, n, d, m) for unknown k, m but chosen d, n, and assume the adversary knows L and thus C 1 , . . . , C l . Now let 1 ≤ s < t ≤ l be any two indices, let = C s + C t and let d = d, |d | = |d|, be the associated data string with encoding v 1 , . . . , v l such that we have and ultimately enc(k, n, d , m) = enc(k, n, d, m). In a CCA attack environment, this can immediately be leveraged 11 Assuming the above notation for AEAD schemes, we give a similar syntax to AE schemes: an AE scheme encrypts a message m under key k with nonce n to produce a ciphertext denoted enc * (k, n, m).
to the full recovery of m. As in the AES-GCM case, we note that many variants of our attack exist (against all versions of OCB), including some that manipulate message bits in a controlled way.

AEAD schemes
We recall standard notions of (deterministic) nonce-based AEAD, as per [52], and study how to adapt them to the ASA setting. Formally, a scheme AEAD providing authenticated encryption with associated data consists of algorithms AEAD.gen, AEAD.enc, AEAD.dec. The scheme has associated spaces K, N , D, M, C. The key generation algorithm AEAD.gen outputs a key k ∈ K. The encryption algorithm AEAD.enc takes key k ∈ K, nonce n ∈ N , associated data d ∈ D and message m ∈ M, to produce ciphertext c ∈ C. The decryption algorithm AEAD.dec takes key k, nonce n ∈ N , associated data d ∈ D and ciphertext c ∈ C to output either a message m ∈ M or the special symbol ⊥ / ∈ M to indicate rejection. A shortcut notation for this syntax is Scheme AEAD is said to be δ-correct if for k ← $ AEAD.gen and c ← AEAD.enc(k, n, d, m) for some (n, d, m) and m ← AEAD.dec(k, n, d, c) the probability that m = m is upper-bounded by δ, where the probability is over all coins involved.

IND-CCA
We formalise indistinguishability under chosen/ciphertext attack for an AEAD scheme via the game IND − CCA in Fig. 3 (left). For any adversary A, we define the advantage

Authenticity
We formalise the authenticity of an AEAD scheme via the game AUTH in Fig. 4(left). For any adversary A, we define the advantage

Subverting AEAD
We note that AEAD satisfies the generic syntax introduced above in Sect. 4, with key generation algorithm .gen = AEAD.gen, sender algorithm .S = AEAD.enc and receiver algorithm .R = AEAD.dec. We may thus apply the generic notions of subversion and undetectability introduced in Sect. 4.1. In Fig. 3(right) and Fig. 4(right), we specify the games subIND − CCA and subAUTH (respectively), modelling the adversary's ability to compromise the expected security properties of a scheme when that scheme has been subverted.
We note that for symmetric primitives, key generation is unlikely to be subverted in practice as symmetric keys are typically generated by some external means not connected with or influenced by the scheme itself-e.g. through key agreement protocols, or by a trusted platform module. Nevertheless, we retain a syntax that allows for the more general case.

Message authentication schemes
Cryptographic message authentication is typically realised with a message authentication code (MAC). Given a key k and a message m, a tag t is deterministically derived as per t ← tag(k, m). The (textbook) method to verify the authenticity of m given t is to recompute t ← tag(k, m) and to consider m authentic iff t = t. If this final tag comparison is implemented carelessly, a security issue might emerge: A natural yet naive way to perform the comparison is to check the tag bytes individually in left-to-right order until either a mismatch is spotted or the right-most bytes have successfully been found to match. Note that, if tags are not matching, such an implementation might easily give away, as timing side-channel information, the length of the matching prefix, allowing for practical forgery attacks via step-wise guessing.
This issue is understood by the authors of major cryptographic libraries, which thus contain carefully designed constant-time string comparison code. A consequence is that services for tag generation and verification are routinely split into two separate functions tag and vfy. 12 Our notion of a message authentication scheme follows this approach. It comprises MAC-based authentication as a special case, but it also comprises the more exotic randomised MACs as considered in [1].
Formally, a scheme MAC providing message authentication consists of algorithms MAC.gen, MAC.tag, MAC.vfy and associated spaces K, M, T . The key generation algorithm MAC.gen outputs a key k ∈ K. The tagging algorithm MAC.tag takes a key k ∈ K and a message m ∈ M, and returns a message, tag pair (m, t) ∈ M × T . The verification algorithm MAC.vfy takes a key k ∈ K, a message m ∈ M, and a tag t ∈ T , and returns either the message m (indicating that the tag is accepted) or the special symbol ⊥ to indicate rejection. 13 A shortcut notation for this syntax is We formalise the (strong) unforgeability of a message authentication scheme via the game UF in Fig. 5

Subverting MACs
We note that MACs satisfy the generic syntax introduced above in Sect. 4, with key generation algorithm .gen = MAC.gen, sender algorithm .S = MAC.tag, receiver algorithm .R = MAC.vfy. We may thus apply the generic notions of subversion introduced in Sect. 4.1. We obtain the notion of subverted unforgeability subUF, as in Fig. 5(left, second column).

Discussion
We note that for symmetric primitives, key generation is unlikely to be subverted (see the discussion at Sect. 5.3), leaving us with the possibility that either the tagging or the verification algorithm (or both) could be subverted. However, as tagging and verification are typically performed by distinct, remote parties, successfully conducting such attacks would require replacing implementations of two participants, which we think is considerably less feasible than replacing only one implementation.

Public key encryption schemes
In this section, we consider ASAs against PKE schemes. A treatment of key encapsulation mechanisms (KEMs) is given in Appendix 1.
A PKE scheme PKE = (PKE.gen, PKE.enc, PKE.dec) consists of a triple of algorithms together with key spaces K S , K R , a message space M and a ciphertext space C. The key-generation algorithm PKE.gen returns a pair (pk, sk) ∈ K S × K R consisting of a public key and a private key. The encryption algorithm PKE.enc takes a public key pk and a message m ∈ M to produce a ciphertext c ∈ C. Finally, the decryption algorithm PKE.dec takes a private key sk and a ciphertext c ∈ C, and outputs either a message m ∈ M or the special symbol ⊥ / ∈ M to indicate rejection. The correctness requirement is that for (pk, sk) ← $ gen and m ∈ M and c ← PKE.enc(pk, m) and m ← PKE.dec(sk, c), the probability that m = m is upper-bounded by δ, where the probability is over all coins involved.
We formalise the indistinguishability under chosen/ ciphertext attack of a PKE scheme via the game IND − CCA in Fig. 5(right, first column). For any adversary A, we define the advantage

Subverting PKE schemes
We note that PKE schemes satisfy the generic syntax introduced above in Sect. 4, with the key generation algorithm .gen = PKE.gen, sender algorithm .S = PKE.enc, and receiver algorithm .R = PKE.dec. We may thus apply the generic notions of subversion introduced in Sect. 4.1. See Fig. 5(right, second column) for the game subIND − CCA, modelling the adversary's ability to compromise IND − CCA when interacting with a subverted scheme.

Discussion
For PKE schemes, subverting the encryption algorithm is less interesting, as the sender has no secret information to leak. It would be possible to consider the subversion of encryption with the view of compromising confidentiality of ciphertexts, but we are targeting the stronger notion of key recovery (which will lead to a full compromise of the confidentiality of all ciphertexts). For PKE schemes, in contrast to symmetric encryption, subverting the key generation algorithm is a meaningful option, and we explain in Sect. 4.3 how subversion attacks can be amplified when applied together with a subverted key generation algorithm.

Concrete subversion attacks via acceptance versus rejection
We assume that the objective of a subverted receiver algorithm is to leak a bit string k ∈ {0, 1} λ representing either some leakage that will enable recovery of the secret (private) key k R , following the discussion at Sects. 4.2.1 and 4.3, or else a string that is sufficient to break security in the sense of Sect. 4.4. We refer to k as the leakage key in what follows. At worst, from the subverter's perspective, the leakage key will simply be a bit string representation of k R . We propose two key-recovering subversion attacks against a scheme = ( .gen, .S, .R) satisfying the syntax given in Sect. 4. While both attacks subvert the receiver algorithm only, they differ in that our first attack is passive (can be mounted by a mass surveillance adversary who eavesdrops), and our second attack is active (requires intercepting and modifying sender outputs in transmission-i.e. ciphertexts, in the case of AEAD or PKE, or message-tag pairs in the MAC case). The driving principles behind the two attacks are closely related: In both cases, the receiver algorithm of the attacked scheme is manipulated such that it marginally deviates from the regular accept/reject behaviour; by making these deviations depend on the leakage key, the bits of the latter are leaked one by one.
Our passive attack rejects a sparse subset of the receiver inputs that the unmodified algorithm would accept. Our active attack does the opposite by accepting certain receiver inputs that the unmodified algorithm would reject. A property of the former (passive) attack is that the scheme's probability of incorrect decryption is increased by a small amount (rendering it detectable with the same probability); we believe however that in settings where rejected messages are automatically retransmitted by the sender (for example, in low-level network encryption like IPSec), this attack is still practical and impactful. Our active attack does not influence correctness. However, as key bits are leaked only when the receiver algorithm is exposed to bogus inputs, successful adversaries are necessarily active. The active attack furthermore has the following attractive property: The underlying receiver outputs (i.e. messages) corresponding to the injected inauthentic receiver inputs are not arbitrary (and thus unexpected to the processing application), but identical with sender inputs previously sent by the sender algorithm. This allows attacks to be kept "under the radar": the receiver will not realise that an attack has been mounted, as all accepted messages it receives will be those sent by the sender.
We note that both of our subversions are stateless, which not only allows for much easier backdoor implementation from a technical perspective but also should decrease the likelihood that an implemented attack is detected through code review or observing memory usage. That said, our passive attack also has a stateful variant with an interesting additional practicality feature. We discuss this further below. We note that our subversion approach, for leaking at most one bit per operation, remains on the conservative side. Depending on the circumstances, in practice, more aggressive methods that leak more than one bit per operation, are expected to be easily derived from our subversion proposals.

Combinatorics: coupon collection
The passive and active attacks both exfiltrate secret key material one bit at a time. The following lemma recalls a standard coupon collector statement that will help analysing the efficiency of this approach, in particular how long it takes until all bits are extracted. For a proof of the lemma, see e.g. [43, §8.4]. Fig. 6 measures the number of iterations it takes to visit all elements of S ("collect all coupons") when picked uniformly at random and considered with probability η. The expected number of iterations is given by O(n log n), where n = |S|. More precisely, we have

Lemma 1 Fix a finite set S (of "coupons") and a probability
Note that parameter η is fully absorbed by the O(·) notation.

Passive attack
We first give an intuition of our passive attack. Our attack subverts the receiver algorithm so that an adversary who observes decryption error events in a "normal" run of communication between sender and receiver is able to learn bits of the leakage key. The subverted receiver monitors incoming ciphertexts. It applies a hash function to each of them to obtain a pointer to a bit of the leakage key. It then, with a configurable probability, artificially rejects the ciphertext if the indicated bit of the leakage key does not match some hard-coded reference value. The adversary is able to apply the same hash function to the ciphertext and thus learns whether the bit position deviates from the reference value. By bit-wise learning the difference between the leakage key and the reference value, eventually the adversary can put the complete leakage key together.
In the remaining part of this section, we describe the specification of our subversion and KRP adversary in detail, and analyse their effectiveness.

Description of our passive attack
We define our passive subversion of the receiver algorithm .R of a scheme in Fig. 7(left). It is parameterised by a probability 0 ≤ γ ≤ 1, a large index space I R , a PRF (F i ) i∈I R , and a family (G i ) i∈I R of random constants. For the PRF we require that it be a family of functions F i : Y → [0 .. λ − 1] (that is: a pseudo-random mapping from the ciphertext space to the set of bit positions of a leakage key k ), and for the constants we require that G i ∈ {0, 1} λ (that is: a random element of the set of leakage keys {0, 1} λ ). (That we use the same index space I R for two separate primitives is purely for notational convenience; our analyses will actually assume that (F i ) and (G i ) are independent. 14 ) We provide details on our attack. The idea is that k (line 00), which is shared by the subverted algorithm and the key recovering adversary through knowledge of G i , rep- resents an initial reference key 15 ; the key recovery adversary, throughout the attack, learns the bits that differ between k and k . This means that the subversion only needs to leak (on average) half as many bits compared to leaking the whole of k . The Bernoulli trial (line 03) controls the rate with which such differing bits are exfiltrated, and the PRF (line 04) controls which bit position ι is affected in each iteration. By PRF security, these bit positions can be assumed uniformly distributed (though knowledge of the subversion index i allows tracing which ciphertext is mapped to which position). Any bit difference is communicated to the adversary by artificially rejecting (line 06) the ciphertext, although it is actually valid.
We specify a corresponding KRP adversary in Fig. 7 (right). It starts with the same random string G i as the subversion and traces the bit updates of .R i until eventually the full key k is reconstructed. We assume that A can tell whether the full leakage key k has been recovered (line 10), e.g. by recovering the secret key k R from k and verifying one or more recorded authentic outputs with it.
Note that our adversary A does not need to know the sender inputs (equivalently, receiver outputs) x ∈ X , which typically represent plaintext messages in the settings we consider, emerging throughout the experiment: The core of the attack, in lines 13 to 16, is independent of the value of x. This considerably adds to the practicality of our attack: While messages are not always secret information, in practice they might be hard to obtain. Conducting mass-surveillance attacks is certainly easier if the attacks depend exclusively on the knowledge of ciphertexts (like in our case, line 15).
While we present our subversion as stateless (i.e. the reference key is kept static between invocations), it also works if the .R i algorithm maintains state between any two invocations and remembers which differing bit positions have already been communicated. Activate line 07, and execute line 00 only during the first invocation, to obtain the stateful attack. With respect to the detectability and key recovery notions from Sect. 4, the attack's performance is the same whether the subversion is stateful or not. The stateful version offers better correctness after a subversion is detected, in the sense that the algorithm will only behave unexpectedly at most |k | = λ occasions; once the leakage key k has been exfiltrated, the subverted scheme .R i behaves identically to the honest scheme .R. (This case is practically less relevant and not covered by our formal models.) We establish the following statements about the key recoverability and undetectability of our passive subversion attack.
Theorem 1 For a δ-correct scheme , let .R i be defined as in Fig. 7(left) and A as defined in Fig. 7(right). If F i behaves like a random function and constants G i are uniformly distributed, then for any message sampler MS, the key recovery advantage Adv • Through an early exit with ⊥ = .R(k R , y) at line 02, which has probability δ. • Else, continuing to line 03 with ⊥ = .R(k R , y) and triggering both line 03 (with probability γ ) and line 05 (with probability 1/2, as Pr Applying Lemma 1 gives the expected number of messages to be sent as O(λ log λ).

Theorem 2 Let
A be an adversary playing the UDR game (as in Fig. 1 right), such that A makes at most q queries to the receiver oracle Recv. The undetectability advantage of the subversion .R i , as defined in Fig. 7(left), is bounded by Proof Any adversary playing the UDR game against the subverted .R i must, in order to win, trigger x = ⊥ with a valid sender output (receiver input) y. More precisely, the adversary A must find y such that .R(k R , y) = ⊥ but .R i (k R , y) = ⊥. Figure 8(left) shows the (obviously) Fig. 8 Detection adversaries for Game UDR as in Fig. 1. Left: For the passive attack from Fig. 7. Right: For the active attack from Fig. 9 best adversarial strategy. Even if the adversary can submit y such that ι ← F i (y) would be assigned in line 04 (Fig. 7), this is contingent on B(γ ) succeeding in line 03; thus Pr[x = ⊥] ≤ γ in line 05 (Fig. 8). Clearly, detection adversary A (Fig. 8left) always returns 1 when interacting with the unsubverted receiver algorithm, as always x = ⊥. Thus,

Active attack
In this section, we describe our second subversion attack. In contrast to the previous attack, key recovery requires an active adversary, i.e. one who injects crafted ciphertexts into the regular transmission stream. Our ASA has the desirable property (from the point of view of the subverter) that correctness is maintained. We give an overview of our attack for the case of AEAD. (The generalisation to MAC and PKE is immediate; for the generic version following our abstract syntax see Fig. 9.) A prerequisite of the attack is a keyed random permutation P i of the AEAD ciphertext space. The key is known exclusively to the subversion adversary. The AEAD encryption algorithm S remains unmodified. For honestly generated ciphertexts, the (subverted) algorithm R i implements the unmodified AEAD decryption routine R. This ensures correctness.
To start a key recovery attack, the subversion adversary waits for an honest ciphertext c and replaces it with P i (c). That is, the adversary suppresses the delivery of c and instead injects a "randomised same-length version" of the ciphertext. By the authenticity property of the AEAD scheme, the The adversary needs to have no influence over messages (modelled by α; see the discussion at Sect. 4.2). As before, we let ! denote the inversion of a bit value unmodified R algorithm would reject this ciphertext. This is where the (subverted) R i deviates from R: If any incoming ciphertext is deemed invalid upon decryption with R, with the expectation that this could be the case due to a KRA attack being in operation, R i applies P −1 i to c and tries to decrypt the result (with R). If R rejects, R i rejects also (interpretation: c was simply a random ciphertext, not injected by the KRA adversary). If however R accepts, then R i concludes that a KRA attack is in operation, and that it (R i ) is supposed to leak key material. 16 Observing that R i just recovered the originally encrypted message m, we let R i either deliver that message, or we let it return ⊥, i.e. indicate decryption failure. That way, if a message is delivered by R i , it is always correct. As in Sect. 8.2, we modulate key bits into the decision of delivering vs rejecting.
The above should make clear how the attack works. Details, and the generic version, are in Sect. 8.3.2. We note that a technical prerequisite of the attack is that for valid ciphertexts c, P i (c) should not also be a valid ciphertext. As P is a random permutation, the standard AUTH and UF properties of AEAD and MAC ensure this. However, the situation is different for PKE where it is easy to define schemes that accept every ciphertext input, e.g. by outputting a valid but dummy message. We resolve this technicality by requiring the mild assumption of ciphertext sparseness: We say that a PKE scheme has a sparse ciphertext space if the decryption algorithm makes an internal decision about the validity of an incoming ciphertext, with the property that uniformly picked ciphertexts are deemed invalid with overwhelmin probability.
We studied a range of practically relevant PKE schemes and observe that all of them satisfy the ciphertext sparseness demand. For reference, we provide corresponding details for OAEP and Cramer-Shoup encryption in Appendix 1. We further observe that the general classes of plaintext-aware schemes [13] (see also Appendix 1) and of schemes with publicly verifiable ciphertexts [49] have this property as well. We confirm also for all four of the NIST post-quantum cryptography round 3 finalists 17 , that are specifically designed to mask decryption failures by accepting every ciphertext and outputting a random value rather than the rejection symbol, that they provide ciphertext sparseness: The mechanics of the decryption/decapsulation algorithms are such that first an internal yet explicit ciphertext validity decision is made and then either the correct or an independent, randomised value is output. Our subversions can easily adapt to such specifications and be directly based on the outcome of the validity check.
Lastly, we note that for the active attack there is no advantage to the subverted receiver keeping state. This is because the subverted receiver reveals key bits only when explicitly queried by the adversary-thus, the adversary is able to maintain all necessary state. Note this is in contrast to our passive attack where the adversary observes the receiver but does not interact with it, and ultimately thus the attack could benefit mildly from the subversion R i keeping state.
In the remaining part of this section, we define ciphertext sparseness, describe the specification of our subversion and KRA adversary, and analyse their effectiveness.

Ciphertext sparseness
We define ciphertext sparseness for a scheme as follows: We say that = ( .gen, .S, .R) is ciphertext ε-sparse if Pr [R(y) = ⊥] ≤ ε for y ← $ Y. If is negligibly small, we refer to the scheme as being ciphertext sparse. For AEAD and MAC schemes, ciphertext sparseness is a corollary of the unforgeability (authenticity) properties. In particular, an AEAD scheme that has Adv auth AEAD (A) ≤ ε is ciphertext εsparse, as if Pr [R(y) = ⊥] > ε for y ← $ Y, then an adversary who simply chooses an element of Y uniformly at random would win the AUTH game with probability greater than ε. Similarly, a MAC scheme with Adv uf MAC (A) ≤ ε is ciphertext ε-sparse. As discussed above, ciphertext sparseness is a reasonable assumption also for many practical PKE schemes. 17 Classic McEliece, CRYSTALS-KYBER, NTRU and SABER. Detailed information on the submissions, in particular their specifications, are available at the NIST PQC website https://csrc.nist.gov/ projects/post-quantum-cryptography/round-3-submissions.

Description of our active attack
We define our active subversion of the receiver algorithm of ciphertext-sparse scheme in Fig. 9(left). It is parameterised by a large index space I R , a PRF (F i ) i∈I R , a PRP (P i ) i∈I R , and a family (G i ) i∈I R of random constants. (As in Sect. 8.2, our analyses will assume that (F i ) and (P i ) and (G i ) are independent.) For the PRF, we require that it be a family of functions F i : Y → [0 .. λ − 1] (that is: a pseudo-random mapping from the output space to the set of bit positions of a leakage key k ∈ {0, 1} λ ), for the PRP, we require that it be a family of length-preserving permutations P i : Y → Y (that is: a pseudo-random bijection on the sender output space), and for the constants, we require that G i ∈ {0, 1} λ (that is: a random element of the set of leakage keys).
The idea of our attack is as follows. Lines 01-03 of .R i ensure that authentic receiver inputs are always accepted (no limitation on correctness). If however a receiver input (sender output) y is identified as not valid, i.e. is unauthentic, then a secret further check is performed: The original value y is mapped to an unrelated value y using the random permutation (line 04), and the result y is checked for validity (line 05). For standard (invalid) sender outputs y this second validity check should also fail, and in this case, algorithm .R i rejects as expected (lines 06 and 07). The normally not attainable case that the second validity check succeeds is used to leak key bits. The mechanism for this (lines 08-11) is as in our passive attack from Sect. 8.2, namely by communicating via accept/reject decisions the positions where the bits of a hard-coded random reference value k and the to-be-leaked key k differ.
The corresponding key recovery adversary crafts these required bogus receiver inputs by obtaining a valid sender output 18 y (line 15) and modifies it in line 16. The information thus leaked by the validity checking routine is used to reconstruct target leakage key k in the obvious way (lines 19 to 21). We establish the following statements about the key recoverability and undetectability of our active subversion attack.

Theorem 3
For an ε-sparse scheme , let .R i be defined as in Fig. 9(left) and A as in Fig. 9(right). If F i and P i behave like random functions, and constants G i are uniformly distributed, then for any message sampler MS, the key recovery advantage Adv kra MS (A) is expected to reach value 1 once the receive algorithm was invoked on O(λ log λ) different inputs. 18 Note that in the symmetric case, such authentic outputs are obtained by intercepting valid communications between the sender and receiver; in the public key case, an adversary can easily craft their own authentic outputs using the public key. We consider adversaries that have no influence on message choices for the most powerful attack (hence the arbitrary value of α in line 14); adversaries who are able to utilise α (and β) may be even more effective.
Proof By the ciphertext ε-sparseness of the scheme , each invocation of algorithm .R i in an execution of attack A(i) has x = ⊥ in line 02 ( Fig. 9) with probability ε and thus line 04 is reached with probability 1 − ε. We model algorithm A(i) by the experiment CC(S, η) from Fig. 6, with S = [0 .. λ − 1] and η = (1 − ε)/2. The (pseudo-)randomness of F i ensures that elements of s ∈ S, here representing the possible values of the index ι (line 20), are picked uniformly at random. The probability 1/2 arises through success of the CC experiment being equivalent to the condition x = ⊥ in line 19. This occurs precisely when .R i returns x = ⊥ in line 10, which is conditional on .R i reaching past line 07. The probability that x = ⊥ in line 19 is 1/2 as this is the probability that for any sender output y and ι ← F i (y ), k [ι] = G i [ι] (line 09). We now apply Lemma 1, which gives us that the expected number of messages to be sent is O(λ log λ).

Theorem 4 Let
A be an adversary playing the UDR game (as in Fig. 1right), such that A makes at most q queries to the verification oracle Recv. If P i behaves like a random function, and the scheme is ciphertext ε-sparse, then the undetectability advantage of the subversion .R i , as defined in Fig. 9(left), is given by Proof Any detection adversary A playing the UDR game against the subverted .R i must, in order to win, triggers .R i (y) = ⊥ with a bogus y. That is, a sender output y with .R(k R , y) = ⊥ but .R i (k R , y) = ⊥. This will occur if y = P i (y ), where .R(k R , y ) = ⊥. As i is chosen uniformly randomly from I R and P is a (pseudo-)random function, the optimal strategy is to sample values of y and test whether Recv(k R , y ) = ⊥. Algorithm A in Fig. 8(right) shows this strategy. When A interacts with the unsubverted receiver algorithm, we have that Pr[UDR 1 (A)] = 1 by construction. When interacting with the subverted receiver algorithm, A returns x = ⊥ by either triggering line 02 or line 10 of .R i . By the ciphertext sparseness of the scheme, line 02 is triggered with probability 1−ε. Triggering line 10 happens with probability ≤ ε. Thus, we have

Conclusion
Our work examines subversion attacks against the receiving party, a class of ASA that was missed by previous work. We give an abstract framework and show that ASAs targeting receivers apply equally to any primitive meeting the syntaxnamely AEAD, MAC and PKE schemes. The internal details of our attacks (described in Sects. 8.3.2 and 8.2.1) are such that we require a PRF (F i ) i∈I R to uniformly hash ciphertexts to bit positions. In the AEAD setting, this requirement can be dropped where the AEAD scheme meets the widespread design goal of IND$ security [52], i.e. ciphertexts indistinguishable from random bits. Combined with the fact that symmetric keys are typically 256 bits, the first 8 bits of the (uniformly distributed) ciphertexts are sufficient to point to the bit position. This allows for a reduced footprint (and thus significantly adds to the practicability of the attack for an adversary).
Acknowledgements The research of Armour was supported by the EPSRC and the UK government as part of the Centre for Doctoral Training in Cyber Security at Royal Holloway, University of London (EP/P009301/1).

Conflict of interest
There are no conflicts of interest for this research.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecomm ons.org/licenses/by/4.0/.

A.1 Key encapsulation mechanisms
For completeness, we give the corresponding definitions of subversion attacks against key encapsulation mechanisms, together with notions of undetectability and key recovery.

A.1.1 KEM definition
A KEM scheme KEM = (KEM.gen, KEM.enc, KEM.dec) for a finite session key space K is a triple of algorithms together with a key space K S ×K R and ciphertext space C. The key generation algorithm KEM.gen returns a pair (pk, sk) ∈ K S ×K R consisting of a public key and a secret key. The encapsulation algorithm KEM.enc takes a public key pk to produce a session key k ∈ K and a ciphertext c ∈ C. Finally, the decapsulation algorithm KEM.dec takes a secret key sk and a ciphertext c ∈ C, and outputs either a session key K ∈ K or the special symbol ⊥ / ∈ K to indicate rejection. The correct- ness requirement is that for all (pk, sk) ∈ K S × K R , we have Pr KEM.dec(sk, c) = k ≤ δ for (k, c) ← KEM.enc(pk).

A.1.2 IND-CCA
For a key encapsulation mechanism, we formalise the indistinguishability under chosen-ciphertext attack via the game IND − CCA in Fig. 10(left). For any adversary A, we define the advantage

A.1.3. Subverting KEM
We note that KEM schemes satisfy the generic syntax introduced above in Sect. 4, with key generation algorithm .gen = KEM.gen, sender algorithm .S = KEM.enc, receiver algorithm .R = KEM.dec. We may thus apply the generic notions of subversion introduced in Sect. 4.1, and observe that the passive attack in Sect. 8.2 applies. If the KEM scheme is in addition ciphertext sparse, according to the notion in Sect. 8.3.1, then the attacks in Sect. 8.3 will also apply. Figure 10(right) shows the game modelling subverted indistinguishability under chosen-ciphertext attacks. The key generation algorithm DEM.gen returns key k ∈ K. The encapsulation algorithm DEM.enc takes key k ∈ K and a message m ∈ M, and outputs a ciphertext c ∈ C. The decapsulation algorithm DEM.dec takes a key k ∈ K and a ciphertext c ∈ C, and outputs either a message m ∈ M or the special symbol ⊥ / ∈ M to indicate rejection. The correctness requirement is that for all keys k ∈ K, m ∈ M it holds that Pr DEM.dec(k, c) = m ≤ δ for c ← DEM.enc(k, m).

A.1.5. IND-CCA
We formalise the indistinguishability under one-time chosenciphertext attack of a data encapsulation mechanism via the game IND − CCA in Fig. 11(left). Note how lines 04 and 08 ensure that the adversary's first query is an encryption query, and that all further queries are decryption queries. (This precisely matches the typical situation as it emerges in a KEM/DEM hybrid.) For any adversary A, we define the advantage and say that scheme DEM is indistinguishable against chosenciphertext attacks if Adv ind−cca A is negligibly small for all realistic A.

A.1.6. Subversion of DEM
We note that Data Encapsulation Mechanism schemes satisfy the generic syntax introduced above in Sect. 4, with key generation algorithm .gen = DEM.gen, sender algorithm .S = DEM.enc, receiver algorithm .R = DEM.dec. We may thus apply the generic notions of subversion introduced in Sect. 4.1, and observe that the passive attack in Sect. 8.2 applies. If the DEM scheme is in addition ciphertext sparse, according to the notion in Sect. 8.3.1, then the attacks in Sect. 8.3 will also apply. Figure 11(right) shows the game modelling subverted indistinguishability under chosenciphertext attacks. Discussion Typically, a DEM is used together with a KEM in a so-called hybrid encryption scheme that uses the KEM to share (symmetric) session keys with which plaintext messages are encrypted under the DEM. In such a setting, subverting the KEM is sufficient to undermine the security of messages sent via the hybrid scheme. Following the discussion at Sect. 4.3, it is conceivable to subvert a KEM and DEM in tandem so that the KEM's secret key is leaked by the both together. This could allow the subversion to effectively be distributed between the two primitives, aiding undetectability in practice.

B. Example ciphertext sparse PKE schemes
We describe two widespread PKE schemes that satisfy the notion of ciphertext sparseness described in Sect. 8.3.1. OAEP Optimal Asymmetric Encryption Padding (OAEP) was introduced by Bellare and Rogaway [15] and is a widely deployed and standardised PKE scheme. The encryption algorithm of OAEP works on message space M = {0, 1} with fixed message length . Let k 0 and k 1 be integers, and G : {0, 1} k 0 → {0, 1} +k 1 and H : {0, 1} +k 1 → {0, 1} k 0 be two hash functions. Messages are padded before being encrypted using the trapdoor permutation (typically RSA): To pad a message m ∈ M, set m ← m 0 k 1 and choose r ← $ {0, 1} k 0 . Then set s ← m ⊕ G(r ), t ← r ⊕ H (s) and m ← s t. To decrypt a ciphertext, first decrypt (i.e. apply the trapdoor inverse) before unpadding the resulting padded messagem: Parsem as s t with s ∈ {0, 1} +k 1 and t ∈ {0, 1} k 0 . Now compute r ← t ⊕ H (s) and m ← s ⊕ G(r ). If m = m 0 k 1 for some m then reject, otherwise return m.
For a randomly chosen element c in the ciphertext space C, the redundancy introduced by padding will ensure that decrypting c results in a valid message with probability 2 −k 1 . This is because choosing c ← $ C is equivalent to choosing a random m ← $ {0, 1} +k 1 , assuming that the trapdoor permutation and hash functions all behave like random functions. Equivalently, the scheme is ciphertext 2 −k 1 -sparse, according to the definition in Sect. 8.3.1. Cramer--Shoup The Cramer-Shoup PKE scheme was introduced in [26]. The encryption scheme CS = (CS.gen, CS.enc, CS.dec) is defined in relation to a set of public parameters consisting of finite group G with |G| = q and a pair of generators g,ĝ for G, together with a hash key hk for a family of keyed collision resistant universal hash functions H hk : G 3 → Z q . The family of keyed hash functions is such that given a randomly chosen tuple of group elements and randomly chosen hash function key, it is computationally infeasible to find a different tuple of group elements that hashes to the same value using the given hash key. We give details of Cramer-Shoup in Fig. 12.
For a randomly chosen element c = (a,â, e, v) in the ciphertext space G 4 , the redundancy introduced by the hash function will ensure that decrypting c results in a valid message with probability q −1 . To see this, consider fixed a,â, e: this gives a fixed value of a x 1 +y 1 ρ ·â x 2 +y 2 ρ ∈ G and thus Pr v = a x 1 +y 1 ρ ·â x 2 +y 2 ρ = q −1 .

C. Plaintext awareness
We here give the definition of plaintext awareness, a property of PKE schemes that implies ciphertext sparseness (see Sect. 8.3.1). Plaintext awareness essentially means that an adversary is unable to create ciphertexts without knowing the underlying plaintext message. This means that ciphertexts which have not been generated from underlying plaintext messages should be rejected, implying that ciphertexts chosen uniformly at random from the ciphertext space are unlikely to be valid. Both the Cramer-Shoup cryptosystem [25,30] and RSA+OAEP [15,51], outlined in Sect. 8.3.1, satisfy plaintext awareness.
The formal definitions of plaintext awareness in the standard model were proposed by Bellare and Palacio [13], and were slightly extended by Dent and Birkett [20,30]. A scheme is plaintext aware if for all ciphertext creators (attackers) A, there exists a plaintext extractor K which takes as input the random coins of A and can answer the decryption queries of Fig. 13 Game modelling plaintext awareness (PA), for a public key encryption scheme PKE. Note that we retain β (modelling side-channel information) in the syntax of message sampler MS for consistency with previous sections, but here the adversary is not given output β A in a manner that A cannot distinguish from a real decryption oracle. In order to model the attacker's ability to obtain ciphertexts for which it does not know the underlying plaintext, the ciphertext creator is equipped with an oracle that will return the encryption of a randomly chosen message m ← $ MS, where MS is an arbitrary (stateful) message sampling algorithm that takes as input α allowing an adversary (ciphertext creator) to specify a distribution on messages. In Fig. 13, we write for the initial state of MS. After interacting with either the real decryption algorithm or the knowledge extractor simulating decryption, the ciphertext creator outputs a ciphertext c. A distinguisher D is now tasked with guessing which case we are in. Note that the knowledge extractor K does not have access to the distinguisher's randomness.
We formalise plaintext awareness of a public key encryption scheme via the game PA in Fig. 13. For any distinguisher D, we define the advantage We say that a scheme is plaintext aware if for all realistic ciphertext creators A, there exists a knowledge extractor K such that for all message samplers MS and distinguishers D, the advantage Adv pa K,MS,A (D) is negligibly small. We note that a PKE scheme that satisfies plaintext awareness and indistinguishability against chosen-ciphertext attacks (Sect. 7) is necessarily ciphertext sparse. To see this, suppose that the PKE scheme is not ciphertext sparse. For a randomly chosen-ciphertext c ← $ C, the real game PA 0 K,MS,A (D) will output a valid message m. However, in the random game PA 1 K,MS,A (D) the knowledge extractor will not be able to output m without contradicting the plaintext awareness and CCA security of the scheme. We thus conclude that the scheme is ciphertext sparse.