Control theory

This article studies a fundamental problem of security of cyber-physical systems (CPSs). We focus on a class of attacks where some of the actuators could be malicious while all the sensors are considered to be honest. We introduce a novel idea of separability of state trajectories that are generated by the honest and corrupt actuators, and establish its connection to the security of CPSs in the context of detecting the presence of malicious actuators (if any,) in the system. As a defense strategy to guard the CPS against malicious attacks, we focus on the mechanism of perturbing the pre-determined control action by injecting a certain class of random process by the honest actuators called private excitation, which is assumed to have a known distribution. As primary contributions we give sufficient conditions for the existence and non-existence of a separator for linear time-invariant stochastic systems, under the assumption that the policies are randomized-Markovian and randomized history dependent. Several technical aspects of the established results are discussed extensively.


§1. Introduction
Cyber-Physical systems (CPSs) monitor and regulate several critical large-scale infrastructures such as smart grids, transportation systems, and wearable medical systems. Some recent cyber-attacks such as the Stuxnet computer worm attack [Kus13], the cyber attack on Ukrainian's power grid [Zet16], and the Maroochy Shire water incident [SM17] demonstrate the security vulnerabilities of large-scale CPSs. With the increasing complexities of the system, the possibilities available to the attackers to launch sophisticated and intelligent attacks have increased, and hence there is an emergent need to give considerable attention to the issue of the security of CPSs.
This article examines the problem of detecting the presence of malicious components in CPSs. A malicious attack is performed by an adversarial agent with the intent of degrading the performance of the underlying system, thereby restricting the system from achieving its goal. In this article we focus on those attacks that affect the performance of the physical layer of the CPSs consisting of sensors, actuators, and controllers connected over a network. These attacks differ from the attacks on the cyber layer of the system that typically involve several issues including cryptography, communication protocol, etc., associated with the underlying network of the CPSs.
Throughout this article, we stipulate that all sensors are honest while malicious components may lurk in some of the actuators and arbitrarily tamper with the corresponding actions. 1 We further assume that the malicious components may collude with each other, devise strategies, and remain undetected by acquiring perfect and complete data about the states of the system at each instant of time by fully utilizing the underlying network of the system. We emphasize that on the one end of the spectrum, if all the actuators are malicious, then the task of detecting the presence of malicious components fails. If, on the other end of the spectrum, all the actuators are honest, then there is no malicious actuator to detect. However, it is difficult to know a priori whether malicious components lurk in the system, and therefore, the problem of detecting the presence of such malicious components in the system is of great practical importance and relevance. Several approaches to this problem have appeared in the literature, and we review a large fraction of them below while focusing on the promising technique of injecting carefully designed random processes into the system for such detection purposes.
Review of watermarking techniques to secure CPSs. Several methods and important technologies have been developed over the last two decades to defend and secure a control system against malicious attacks; see [SGJ22, HXCL14, GUC + 18, DHX + 18, SAJ15, HLLL17, MC14, TDS + 16, HY16] for surveys of such techniques and their applications. Among the most promising directions is one that involves injecting private excitation (also known as watermarks in the literature) by the honest actuators into the system. They have been studied and used as active defense mechanisms against malicious attacks on the sensors and/or actuators or both; see [LMJ21] for a general overview.
We start by reviewing the relevant works on watermarking techniques where the underlying system is considered to be linear. The term watermarking was first coined in the article [MS09] where it was used as an active defense mechanism against replay attacks in the context of a linear time-invariant (LTI) system with Gaussian process noise. Under this premise, the technique has attracted considerable attention over the past few years, and securing the control systems by employing Gaussian-based watermarking against replay attacks (see [HCA17,FQCZ17,ZVH21] and [GSNR22] for co-design of watermarking signals and robust controllers), integrity attacks [MCS13,WMS14,MWS15], stealthy attacks [MWS15,HTG16], and false data injection attacks in [MGCS10], have been explored. In most of the prior works, the authors assumed that the sensors were vulnerable, except in [WMS14,HTG16], where both the sensor networks and control inputs were considered to be attack prone. The authors in the articles [WMS14,LYMJ18] introduced a method where the optimal watermarking signal was synthesized by reformulating the design problem in the language of optimization to counter against replay attacks, and the theory was supported by numerical examples. Security of LTI SISO/MIMO systems over networks under Gaussian random noise with complete and partial observation were considered in [SK16a,SK16b,SK17], wherein the authors assumed that the attack strategies employed by the adversary could be arbitrary. A Gaussian-based dynamic watermarking scheme was proposed and the power distortion was used as a metric to detect the presence of malicious sensors. Based on the statistics of the output signals, several tests were introduced that ensure zero power distortion almost surely. We highlight two notable features of these works: Firstly, the sensor attacks were considered to be arbitrary, and secondly, the reported result is concerned with the detection of sensor attacks almost surely; these results are fundamentally stronger in the sense that it eliminates the possibility of error in detection. For SISO/MIMO LTI systems with partial observations, these results were generalized in [HPVA17] under a carefully designed attack model. The authors showed that persistent disturbances restrict the optimal design of the watermarking signal; to that end, they proposed an approach based on the internal model principle to compensate for persistent disturbance. Under an identical premise, the authors in [HPVA20] designed a sensor switching strategy based on attack detection by dynamic watermarking. An extension to the case of networked LTI systems was developed in [HPVA18] where the authors designed a watermarking signal based on null hypothesis testing. Further, the case of the statistics of the measurement noise being unknown was examined in [OSH + 20]; specifically, certain covariance matrices were assumed to be unknown or slowly varying, and under these assumptions, several tests were designed relying on Gaussian-based watermarking. Security of linear time-varying dynamics against replay attacks was addressed in [PHA + 20, PDJ + 20] where tests based on matrix normalization were designed to accommodate the time-varying dynamics. In [HCA17,FQCZ17,GSNR22] and [ZVH21, WMS14, LYMJ18] the synthesis of the watermarking signal was reformulated as a co-design problem of the watermarking signal, and as an optimization problem, respectively.
In the non-Gaussian regime the following investigations have paved the way. The article [SK19] suggests a statistical test for LTI systems with non-Gaussian process noise and non-Gaussian dynamic watermarks. In particular, the authors established necessary and sufficient conditions on the statistics of the watermarking signals that ensured zero power distortion by the malicious sensors. The design of non-Gaussian watermarks was also examined in [LYMJ18] where the synthesis of the watermarking signal was reformulated as an optimization problem. We refer the readers to [PJH + 19, PDJ + 20, HSKX18] for several applications of dynamic watermarking techniques for the detection of malicious components in the area of robotic mobility, autonomous vehicles, and automatic power generation.
While the literature on the security of linear systems is dense as one can clearly observe, to the best of our knowledge, the existing literature on the security of nonlinear systems is considerably sparse in contrast to their linear counterparts. The articles [KSK19,KSK16] and [PJD + 21] studied the applications of dynamic watermarking based secured control of transportation systems and platooning networked robotic systems, respectively. The application of learning-based dynamic watermarking for scalar nonlinear systems in certain reproducing kernel Hilbert space is examined in [KKFJ19,KKFJ20]. The authors studied the performance of nonlinear attacks based on machine learning algorithms. More recently, a dynamic-watermarking algorithm was proposed in [TSG21] for finite state-space, finite-action Markov decision processes where the authors obtained upper bounds on the mean time between false alarms, and the mean delay between the time an attack occurs and when it is detected. Various kinds of learning-based attacks were studied in [FS18,KKFJ19,KKFJ20]; see [ZPH + 21] for a survey on the applications of machine learning in attack detection in cyber-physical systems.
We refer to Table 1 for a summary of all the related works in the security of CPS where watermarking techniques have been employed. We have broadly classified the various works on the basis of types of systems, types of watermarking signal, and the attack model. 2 Salient features of this work. Before we state the contributions of our work, let us highlight its salient features: Firstly, our main focus lies on the class of attacks in which a subset of the actuators is hijacked by adversaries. Thus, in contrast to most of the works where sensors are assumed to be under attack, we study the case of actuator attacks. Types of well-known actuator attacks in the literature include denial-of-service, false data injection into the actuator channels, and eavesdropping. Secondly, our main results (stated in §3.1 and §3.2) remain agnostic to the specific functional nature of the control policies. In particular, we do not enforce any particular limitations on the class of control actions of the malicious actuators when the system is under attack; see Remark 3.2 for an elaborate discussion. Finally, the results established in this article are related to almost sure detection of the presence of malicious actuators; as mentioned earlier, such results are technically different from those that cater to assertions of the type 'on average' or 'with high probability', neither of which preclude the possibility of errors.
Our contributions. The key contributions of this article are as follows: (1) We introduce an idea of separability of state trajectories generated by the system when it is influenced by all honest actuators versus those generated by the same system influenced by at least one malicious actuator. 3 We define the relationship between separability and security of CPSs in the context of detecting the presence of malicious activities in the system as evidenced via its trajectories (see Definition 2.1 and Remark 2.2 for further technical details) and establish the first abstract result (Theorem 2.3) in the context of linear CPSs.
(2) For a linear time-invariant stochastic CPS with Gaussian process noise, we use the mechanism of injection of a random process called private excitation with suitable statistics, by the honest actuators into the linear system for separating the state trajectories. In this article we choose the private excitation to be Gaussian. (a) Under the above premise, we provide sufficient conditions for the existence/nonexistence of a separator when the honest and corrupt policies are chosen as randomized and Markovian; see Theorem 3.3 for more details on these conditions. 4 (b) We also establish sufficient conditions for the existence and non-existence of a separator under the assumption that the honest and the corrupt policies are randomized and history dependent; see Theorem 3.4 in §3.2 for more information on these conditions. The results are extensively discussed by means of several remarks in §3.4 that provide insights into the technical aspects of the notion of separability and the established results.
(3) One of the key attributes of our results is that they are existential and rely on the statistics of infinitely long state trajectories; these conditions may not be verifiable in practice as they stand now.  and Theorem 3.4 -neither provide a mechanism for the detection of the specific malicious components nor do they provide an estimate of the number of such malicious components present therein. Notwithstanding their per se inapplicability, we regard our main results as baseline fundamental steps towards understanding the nature of the problem under consideration and assessing the boundary of possibilities. The topic of implementable algorithm design for the detection of malicious components will be addressed in subsequent articles.
Organization. §2 formally describes the CPS under consideration and sets down the underlying assumptions. The idea of separability of the state trajectories generated by the honest and corrupt policies is introduced in Definition 2.1 and we discuss its relation to detecting the presence of malicious actuators in the CPS. In §3 we focus our attention on linear CPSs and give sufficient conditions under which the existence and non-existence of a separator can be asserted. Moreover, we include several technical examples and remarks to discuss the important ideas related to the main results in §2 (Theorem 2.3) and §3 (Theorem 3.3 and Theorem 3.4). We give a summary of our results and a brief overview of possible future directions in §4.
Notations. Our notations are standard: For any set , the set c denotes its complement. The sets of real numbers and non-negative integers are denoted by ℝ and ℕ, respectively. If is a topological space, then ( ) denotes the Borel -algebra generated by the topology of . For any matrix , its determinant is denoted by det( ). Given a set , we define the indicator function of a set ⊂ by . Separability of state trajectories in a general context §2.1. System description. We consider a CPS with nodes where the th node represents agent described by a scalar stochastic linear system with the following data: ∈ ℝ is the state of the th agent at time ; ∈ ℝ denotes the control action applied to the th agent at time ; ((2.1)-c) the scalars ( ) , =1 ∈ ℝ and ∈ ℝ denote the system and control parameters corresponding to the th agent of the CPS; ((2.1)-d) the set N denotes the set of in-neighbours of the th agent in the network; 5 ((2.1)-e) ( ) 0 is a sequence of independent and identically distributed (i.i.d) random variables with some known distribution, denoting the process noise corresponding to the th agent; 6 ((2.1)-f) 0 is the initial state of the th agent with some known distribution and is assumed to be independent of the sequence ( ) 0 .
The overall system of agents can be expressed in a compact form as: In-neighbors of the th agent refers to the set of all neighboring agents that can directly influence the evolution of the th agent. 6 See §3 for further details regarding the distribution chosen.
Let the admissible state-space be denoted by and the admissible action space be denoted by respectively. The control action at time is chosen by the honest actuators according to a pre-determined admissible control strategy or policy . In this article we focus on the class of randomized policies, both Markovian and history-dependent. § 2.2. The concept of separability. We introduce an idea of separability of sample paths generated by a cyber-physical system (CPS) and to establish its connection to its security. The idea of separability traces back to the theory of disjoint dynamical systems [Fur67], 7 but in what follows we formally define the term separability for completeness.
Recall the scalar stochastic linear system (2.1) along with its associated data ((2.1)-a)-((2.1)-f) and its compact form (2.2). Let Ω ≔ ℕ denote the sample space with being the state-space, and let (Ω ) denote the Borel -algebra generated in a standard fashion on Ω . 8 An element ∈ Ω , called a sample path or a state trajectory, is a sequence of the form Let us distinguish between two classes of sample paths: • is influenced by all honest components, and the distribution of these paths is P ; •˘ is influenced by at least one malicious component, and the distribution of these paths is P˘ . 9 Of course, the 's are generated under the influence of the honest policy ≔ ( ) ∈ℕ and the˘ 's are generated under the influence of the corrupt policy (˘ ) ∈ℕ . 10 Our objective here is to separate or distinguish these two classes of sample paths, to which end we introduce the following definition.
Consider the infinitely long sample paths 1 and 2 , where 1 corresponds to the case when all the actuators are honest and 2 corresponds to the case when at least one actuator is malicious, respectively, with different sets of characteristic properties and underlying statistics. The above definition concerns with the existence of a Boolean classifier Σ such that if 1 and 2 are fed into Σ, then we get the values 0 and 1, respectively. Note 7 The idea of the disjointedness of the support of two measures is motivated from [LPP15]. 8 See [ABFG + 93, Sect. 2.1] for more information on the construction of the filtered measurable space. 9 We refer the readers to Appendix A for the construction of the measures P and P˘ . 10 Here the class of all policies is denoted by Π . We refer the readers to [HLL12] for more details on different sub-classes of Π. The honest policy ( ) ∈ℕ refers to the case where all the actuators are honest, i.e., policy under no-attack scenario. On the other hand, corrupt policy (˘ ) ∈ℕ refers to the case of attack by some malicious adversary, where at least one actuator is assumed to be under attack. that the existence of such a separator Σ is independent of the specific (infinitely long) sample paths 1 and 2 fed to it. As a result and as mentioned in §1, in practice the above idea is not verifiable in an online fashion: firstly, it requires infinitely long memory which cannot be obtained in a real scenario, and secondly, it addresses posterior properties in the sense that the classification of the two infinitely long sample paths with different statistics can only be done after they have been generated.
R 2.2. Let us discuss when the presence of malicious components in (2.2) can be ascertained from the notion of separability in the sense of Definition 2.1. To that end, we define the following sets If˘ ≠ , there exists a sample path˜ ∈˘ , generated under the influence of˘ such that Σ(˜ ) = 1, which immediately asserts the presence of malicious components in the system. Let us now examine the case when˘ = , in detail: Firstly, it refers to the existence of a separator Σ, wherein the sets Ω and are considered to be separated -a trivial case that is not accounted in Definition 2.1. In fact Definition 2.1 concerns the existence of a non-trivial separator (in the sense that˘ ≠ ) and will be termed as separator in the sequel, unless specified otherwise. Finally, we emphasize that the fact = does not guarantee the non-existence of malicious components in the system (2.2). It only infers to the conditions that the process of detecting the presence of malicious components in (2.2) cannot be accomplished with certainty via the technique of separation of the two different classes of sample paths -and˘ .
Clearly, a separator in the sense of Definition 2.1 may not always exist. Therefore, one of the natural questions to ask is under what conditions the existence/non-existence of a separator can be asserted in the context of large-scale CPS (2.2). In §3 we answer these questions in the affirmative; in particular, we apply the idea of separability in detecting the presence of malicious components in CPSs, understand the various caveats of this approach, and provide asymptotic guarantees for the existence/non-existence of a separator. Development of finite-time algorithms to construct a separator, although an important problem from the perspective of a practitioner, will be considered elsewhere.
In the sequel we proceed with the assumption that the policy employed by the honest actuators and the malicious actuators belong to the class of all randomized and nonstationary policies Π R . In §3.1 and §3.2 we treat the case where the class of policies is considered to be randomized-Markovian and randomized history dependent, respectively. Now we state our chief observation which provides a set of sufficient conditions that guarantee the existence of a separator. Consider the filtered measurable space Ω , (Ω ), G ∈ℕ .
Let us define the measures Q andQ on the measurable space (Ω , G ), ∈ ℕ: for any ∈ G of the form = 0 × 1 × · · · × −1 × with Borel sets ∈ ( ) for each ∈ ℕ, we have where the measures Q andQ are the restrictions of P and P˘ , respectively, to G for each ∈ ℕ. See Appendix A for an exposition on the existence of the measures Q (and Q ) and P (and P˘ ).
Consider the cyber-physical system (2.2) along with its associated data ((2.1)-a)-((2.1)-f). Let denote an initial probability distribution on , and let ≔ ( ) ∈ℕ and˘ ≔ (˘ ) ∈ℕ denote admissible control policies corresponding to the case of all honest actuators and to the case of at least one malicious actuator, respectively. If the following conditions hold: then there exists a separator in the sense of Definition 2.1: there exists a Borel measurable map Σ : Ω −→ 0, 1 such that for P -almost every we have Σ( ) = 0 and for P˘ -almost every˘ we have Σ(˘ ) = 1, where and˘ are sample paths generated by the honest and malicious actuators, respectively.

P
. Under the given hypothesis, invoking Jessen's theorem [Str11, Theorem 5.2.20] shows that the limiting probability measures P and P˘ are mutually singular, i.e., P ⊥ P˘ , which implies that there exist a Borel measurable set ∈ (Ω ) such that The random variable Ω ∋ ↦ → 1 { ( 0 , 1 ,...) ∈ c } ( ) is a Borel measurable function. By definition, it separates the support of the probability measure P and P˘ , which proves the assertion.

R
2.4. Note that from [Str11, Theorem 5.2.20], the converse of Theorem 2.3 is also true under the assumption that Q ≪Q for each ∈ ℕ. To see that, it is enough to establish that the probability measures P and P˘ being mutually singular is equivalent to existence of a separator, defined in Definition 2.1. The necessary condition is verified in the proof of Theorem 2.3. For the sufficient condition, if there exists a Borel measurable function Σ : Ω −→ 0, 1 such that for P -almost every we have Σ( ) = 0 and for P˘ -almost every˘ we have Σ(˘ ) = 1, then the measures P and P˘ are mutually singular. Indeed, we observe that Ω = ∪˘ where the sets and˘ are defined in Remark 2.2 and from Definition 2.1, it follows that P˘ (˘ ) = 1 and P ( ) = 1 proving the sufficient condition.
The following auxiliary results are used in the proof of Theorem 3.3 and Theorem 3.4 ahead in §3.

Sufficient conditions for the (non)existence of a separator
We confine our attention to the case of linear CPSs where the private excitation and the process noise are Gaussian random vectors: Consider the case of -agents where each agent admits the dynamics (2.1) and the overall system can be expressed as (2.2).
A 3.1. Throughout the sequel, the following standing assumptions are imposed: (3.1-a) The private excitation ( ) ∈ℕ is a Gaussian random process with i.i.d. entries and 0 ∼ N(0, V ); its distribution is made public and is known to the malicious attackers. Moreover, it is assumed to be independent of all other random vectors in the system. We assume that the components of are uncorrelated for every , i.e., for every distinct , ∈ {1, . . . , } we have uncorrelated with ; in other words, V is a diagonal matrix. (3.1-b) The process noise ( ) ∈ℕ is also a Gaussian random process with i.i.d. entries and 0 ∼ N(0, V ). Moreover, for every , it is assumed that for every distinct , ∈ {1, 2, . . . , } we have uncorrelated with ; R 3.2. We draw attention to the fact that the precise functional form of the policies, as long as they belong to the class of all randomized policies, be it for the honest actuators or the corrupt ones, is not our concern; in other words, the results obtained -Theorem 3.3 and Theorem 3.4 in §3.1 and §3.2, respectively, are agnostic to the specific functional nature of the randomized policy played by the actuators (honest and corrupt). A few specific instances correspond to the cases where the attackers may choose to remain stealthy to learn the system parameters or it may know the system parameters from the very beginning. In addition to that, in certain specific cases, it may cut-off the communication channel between the controllers and the actuators or it may add an exogenous input signal -a popular type of cyber-attack that is extremely difficult to detect. We stress that many other attack structures are possible in the current setting. §3.1. Sufficient conditions for (non)separability under Π RM . The set Π RM consists of randomized and Markovian policies. Recall that from Definition C.3 that such a policy is a sequence ( ) ∈ℕ on the set given the current past satisfying ( ; ) = 1.
In the absence of any attack or in the no-attack situation, the control actions ( ) ∈ℕ admit a realization = ℓ + for every ∈ ℕ. Here the sequence (ℓ ) ∈ℕ corresponds to the pre-determined control actions (obtained via policies) designed under normal circumstances (no-attack scenario), and the sequence ( ) ∈ℕ denotes the private excitation injected into the system to add a layer of security against malicious attacks. The predetermined control actions (ℓ ) ∈ℕ take the form: ℓ = ( ), where each policy map (·) is a Borel measurable function; its specific functional form is not relevant to us. Similarly, the pre-determined control actions (l ) ∈ℕ under the attack situation take the form ℓ =˘ (˘ ), where˘ is a Borel measurable function whose specific functional form is not relevant in the main result. Hence, the input that the honest actuator applies at time is given by the expression where (·) corresponds to the th component of (·). In a similar manner, when the CPS is under attack, the input that a malicious actuator applies at time is where˘ (·) is the th component of the Borel measurable function˘ (·).
Note that a malicious attacker can hijack a subset of the actuators =1 at ∈ ℕ. Without loss of generality, let < be the number of actuators hijacked by malicious attackers. For the sake of analysis, we stack the control actions generated by the malicious actuators and the control actions generated by the honest actuators separately. That is, under the attack scenario, the control action˘ generated by the admissible policy˘ is partitioned as ˘ ,1 , ,2 where˘ ,1 = ˘ 1 ,˘ 2 , · · · ,˘ and ,2 = +1 , +2 , · · · , denote the actions employed by the malicious actuators and by the − honest actuators, respectively. Similarly, the control action generated by the admissible policy is partitioned as ,1 , ,2 , where ,1 = 1 , 2 , · · · , .
Next we provide sufficient conditions for the (non)existence of a separator in the sense of Definition 2.1 when Π RM is considered. 3.3. Consider the CPS (2.2) consisting of agents connected over a network along with its associated data ((2.1)-a)-((2.1)-f). Let < denote the number of actuators in the network that are under attack when the CPS is hijacked by an adversary. We stipulate that each agent is influenced either directly or indirectly by at least one of the honest actuators. Suppose that Assumption 3.1 holds and all actuators employ randomized Markovian policies. Let ( ) ∈ℕ and (˘ ) ∈ℕ denote, respectively, the admissible control policies corresponding to the case of all honest actuators and to the case of at least one actuator being malicious.
Define the quantity where for each ∈ ℕ, • and ˘ ˘ denote the conditional expectation of and˘ given the current history −1 and˘ −1 , respectively, and • min, and˘ max, denote the minimum and maximum eigenvalues corresponding to the conditional variance of and˘ given the current history −1 and˘ −1 , respectively.
Then we have the following assertions: A proof of Theorem 3.3 will be given in Appendix B. § 3.2. Sufficient conditions for (non)separability under Π RH . The set Π RH consists of randomized and history dependent policies. Recall that from Definition C.3 that such a policy is a sequence ( ) ∈ℕ on the set given the entire past ℎ satisfying (ℎ ; ) = 1. 11 Here we adhere to various notations in §3.1: Let us consider ℎ ≔ ( 0 , . . . , ) and ℎ ≔ (˘ 0 , . . . ,˘ ) that denote the entire history under and˘ , respectively, till the th time instant. The control actions generated by the admissible policies and˘ , when the system (2.2) operates under no-attack condition and when it is under attack by the adversaries, are given by the expression here the policy maps are partitioned as ,1 = ( 1 , · · · , ), ,2 = ( +1 , · · · , ) and ,1 = (˘ 1 , · · · ,˘ ); the private excitation injected into the system is also partitioned as ,1 = 1 , 2 · · · , and ,2 = +1 , +2 · · · , .
11 Note that the history process = ( ) ∈ℕ is defined in Appendix A.
Next we provide sufficient conditions for the (non)existence of a separator in the sense of Definition 2.1 when Π RH is considered.
T 3.4. Consider the CPS (2.2) consisting of agents connected over a network along with its associated data ((2.1)-a)-((2.1)-f). Let < denote the number of actuators in the network that are under attack when the CPS is hijacked by an adversary. We stipulate that each agent is influenced either directly or indirectly by at least one of the honest actuators. Suppose that Assumption 3.1 holds and all actuators employ randomized history dependent policies. Let ( ) ∈ℕ and (˘ ) ∈ℕ denote the history dependent admissible control policies corresponding to the case of all honest actuators and to the case of at least one actuator being malicious, respectively. Define the quantity where for each ∈ ℕ, • and ˘ denote the conditional expectation of and˘ given the entire history ℎ −1 andh −1 , respectively; • min, and˘ max, denote the minimum and maximum eigenvalues corresponding to the conditional variance of and˘ , given their entire history ℎ −1 andh −1 , respectively.
Then we have the following assertions: We document a few situations that justify the assumptions made in Theorem 3.3 and Theorem 3.4. As mentioned in §1, the defense mechanism adopted against malicious attacks involves injecting a random process (called private excitation) ( ) ∈ℕ superimposed with the control action ( ) ∈ℕ , as in, e.g., = ( )+ for ∈ ℕ. 12 We focus on the influence of the underlying network structure on the separability of the sample paths generated by the system (2.2). In order to understand the effect of the network structure, we consider the following example. 3.5 (Effect of the network structure). Let us assume that the system matrix = diag( 11 , 22 ) and the input matrix = diag( 1 , 2 ) are given where 1 , 2 ≠ 0. 13 We further assume that the noise sequence 1 ∈ℕ and 2 ∈ℕ are independent of each other, and their joint distribution is given by P . Note that the matrix depicts the communication between the agents. In this specific case, the system is decoupled, i.e., the agents do not communicate with each other. We also assume that the admissible control action admits a state-feedback law given by = ,1 1 + ,1 , ,2 2 + ,2 . Under malicious attack the control action is given by˘ = ˘ ,1˘ 1 , ,2˘ 2 + ,2 . Fix = 1. From Corollary 2.6, the probability measuresQ 1 and Q 1 are written as and where the set 1 ≔] − ∞, 1 ]×] − ∞, 2 ]. Observe that if the process noise is distributed as 1 0 , 2 0 ∼ Ber( ) × Ber( ), then choosing the private excitation ( , ) ∈ℕ ∼ Ber( ) for = 1, 2, will not guarantee absolute continuity of the measureQ in (3.14) with respect to the measure Q in (3.13). Hence, the assertion in Theorem 2.3 will not hold.
The above example demonstrates that to ensure the absolute continuity of the measure Q with respect toQ , it is necessary that each agent is influenced either directly or indirectly by at least one of the honest actuators. This condition enables the private excitation to affect all the agents corresponding to the malicious actuators (if exists). In the aforementioned example, since the CPS is decoupled, the private excitation injected using the honest actuator does not affect the agent corresponding to the malicious actuator. 3.6 (Choice of the private excitation). The objective here is to understand how a particular class of private excitation plays a pivot role in ensuring separability of the state trajectories under the policies and˘ . In other words, we are interested in observing what class of private excitation is useful against malicious attacks. For instance, in Example 3.5 discussed above, if we consider the matrix to be non-diagonal such that each agent is influenced either directly or indirectly by at least one of the honest actuators. Then, it is easy to observe that if the private excitation is such that its distribution is fully supported on ℝ, then Q 1 ≪Q 1 is guaranteed.   4-b)), respectively. The positive results motivates us to choose a certain class of private excitation of suitable statistics. Indeed, as designers one's objective must be to ensure that the Radon-Nikodym derivative (2.5) (if it exists) converges to zero almost surely for each ∈ ℕ, and the only apparatus available to them is private excitation. One must, therefore, carefully choose the private excitation so that the hypotheses of (3.3-c) (and (3.4-c)) hold. The negative results can be interpreted as the fundamental limitations of the chosen statistics for the private excitation, which is also crucial for a designer. To elaborate, firstly, if the designer has access to the information that the system may be subjected to certain types of malicious attacks satisfying the hypotheses of (3.3-a) and (3.3-b) (also (3.4-a) and (3.4-b)), they will refrain from injecting a Gaussian process as private excitation -an a priori advantage. Secondly, the negative results also suggest that it is essential to go beyond the regime of Gaussian private excitation to secure the CPS against intelligent and sophisticated attacks, an issue that will be addressed in subsequent articles.

R
3.8. In this work we use the idea of injecting private excitation into the CPS (2.2) to protect against actuator attacks with the objective of shaping the statistics of the resulting output signal along with some other mild conditions to ensure the existence of a separator in sense of Definition 2.1. Intuitively, these sufficient conditions aid in the classification of two specific classes of state trajectories -'s generated under a noattack situation and the˘ 's generated under an attack situation, without any error. The method developed in this article depends on the nature of the infinitely long sample -an important characteristic that allows us to give negative results as sufficient conditions for the non-separability of sample paths in (2.2). Consequently, our results are fundamentally different from the ones provided in [MS09,MWS15,WMS14] where the watermarks specific to an attack type are injected as an extra authentication signal to minimize its damage so that the resulting output remains correlated to the watermarking signal, and in [SK16a] where watermarking signals are injected into the system so that the malicious output signals are restricted to pass certain tests and prevent the malicious "nodes" from causing excessive distortion in order to remain undetected.

R
3.9. The proofs of Theorem 3.3 and Theorem 3.4 make use of Jessen's theorem [Str11, Theorem 5.2.20] which gives necessary and sufficient conditions for the mutual singularity of two measures P and P˘ . Specifically, we check the local absolute continuity [Shi19, Definition 1, p. 165, vol. 2] of P with respect to P˘ , and the convergence of the corresponding Radon-Nikodym derivative (if it exists) almost surely. As per our knowledge, the results pertaining to almost everywhere convergence are presented only in [SK16a] and later as an extension in [HPVA17] where the authors have provided certain tests that the output signal must satisfy in order to remain stealthy. • The hypothesis of (3.3-a) (and (3.4-a)) refers to the fact that if the corrupt policy˘ is such that the cumulative weighted energy of the path deviation under the influence of and˘ gradually become comparable in the limit → +∞, then it is not possible to separate the corresponding state trajectories ( ) ∈ℕ and (˘ ) ∈ℕ . Note that it is not required by the state trajectories ( ) ∈ℕ and (˘ ) ∈ℕ to be identical across ∈ ℕ, which would have been extremely restrictive. Moreover, we emphasize that the value "1" is rigid without which the term =1˘ being finite across ∈ ℕ, needs to be stipulated. This is considered in Theorem (3.3-b). • Hypotheses of (3.3-b) (and (3.4-b)) state that if the ratio of the cumulative weighted energy of the path deviation under the influence of and˘ , respectively, is uniformly bounded across ∈ ℕ, and the cumulative weighted energy of the path deviation under the influence of˘ is also finite then we cannot guarantee the existence of a separator in sense of Definition 2.1. 14 • While the assertions in (3.3-a) and (3.3-b) (also (3.4-a) and (3.4-b)) are negative results, part (3.3-c) (and (3.4-c)) is a positive result that asserts the existence of a separator. It states that trajectories ( ) ∈ℕ and (˘ ) ∈ℕ under and˘ , respectively, can be separated in sense of Definition 2.1, if the cumulative weighted energy of the path deviation ( − ) ∈ℕ under increases monotonically over ∈ ℕ at a relatively faster rate than the corresponding cumulative weighted energy of (˘ − ˘ ) ∈ℕ under˘ . A careful scrutiny suggests that a glimpse of the two noteworthy features of Definition 2.1, namely, asymptotic properties and path properties, get reflected in the assertions of Theorem 3.3. §4. Conclusions and future work In this article we introduced a novel idea of separability of state trajectories and demonstrated how it can be put to use in the context of security of large-scale CPS by detecting the presence of malicious actuators almost surely in them. As a defense strategy we used the mechanism of injecting a random process with suitable statistics, by the honest actuators, into the system for the purpose of separating sample paths generated by the honest policy and the corrupt policy˘ . As contributions, we provide positive and negative results for the existence of a separator when the underlying system is assumed to be linear time-invariant and the policies are chosen to be randomized-Markovian and randomized history dependent. As a natural future direction, we aim to derive conditions for the existence/non-existence of a separator when the honest and corrupt policies do not belong to the same class. Future work also includes developing efficient finite-time algorithms for the detection of malicious actuators.
Appendix A. Existence of the measure P and P˘ This appendix includes a proof of the existence of P and P˘ corresponding to and , respectively. The idea is to define a canonical space Ω and project it to Ω : Define the canonical sample space by Ω ≔ ( × ) ℕ where the sets and denote the state-space and the action space, respectively; and Ω corresponds to Borel -algebra induced by the product topology on Ω. An element ∈ Ω is a sequence of the form ≔ 0 , 0 , 1 , 1 , . . . . Define a filtration F ∈ℕ of (Ω) by F ≔ ( ) such that F ⊂ F +1 for every ∈ ℕ where ( ) ≔ ℎ = ( 0 , 0 , 1 , 1 , . . . , ). It is easy to 14 The condition sup ∈ℕ is reminiscent of the boundedness hypothesis in Stochastic Approximation [Bor08] which has now become standard. see that Given an initial distribution on ( ) and an admissible policy = ( ) ∈ℕ , there exists [HLL12, Proposition C.10 and Remark C.11] a probability measure R on the measurable space (Ω, (Ω)) such that for each ∈ ℕ, for all 0 ∈ ( ), ∈ ( ), and 1 ∈ ( ); where denotes the admissible policy.

Appendix B. Proofs
Here we establish the proof of Lemma 2.5

P
. From (A.16), we start by observing that for every ∈ ℕ, ∫ where (1) and (2) above follows from (A.15). From (2.3), Using (B.17) and proceeding backward in a similar way, we obtain where ℎ = ( 0 , . . . , ) and the first assertion follows immediately by observing that R 0 ∈ 0 , 0 ∈ , 1 ∈ 1 , 1 ∈ , . . . , which proves the second assertion. §B.1. Proof of Theorem 3.3. Next, we establish some auxiliary results for the proof of Theorem 3.3.  . Recall that by hypothesis, all policies are in Π RM , and consequently, the closed loop process ( ) ∈ℕ is Markovian. Firstly, we consider the case where the CPS (2.2) has at least one malicious actuator. Given =˘ , under the corrupted policy (˘ ) ∈ℕ , we have +1 = ˘ + + where denotes the process noise at ∈ ℕ and ≔ ( ) ∈ℕ denotes the control process; +1 is the sum of two Gaussian random vectors and , with the conditional mean ˘ Secondly, we consider the case when all the actuators are honest. Arguments similar to the preceding case hold when an honest policy ( ) is adopted. Consequently, +1 is a conditional Gaussian random vector given with the conditional mean and the conditional variance V = V ⊤ + V . This completes the proof.
In the next lemma we establish absolute continuity of the measure Q with respect toQ for each ∈ ℕ; see Appendix C for a brief description on some of the terms used in the subsequent results. P . The proof proceeds by the principle of mathematical induction (we shall adhere to various notations in Theorem 3.3). Our base case is = 1, and from Lemma B.1 it follows that 1 is a conditional Gaussian random vector given 0 under the influence of the policies ( ) ∈ℕ and (˘ ) ∈ℕ . Observe that from (2.7) where the term on the right-hand side of ( * ) is well-defined and can be expanded as Here the quantities V 1 and V˘ ˘ 1 are the conditional variances of 1 given 0 , under the policy ( ) ∈ℕ and (˘ ) ∈ℕ respectively. This implies that for any ∈ G 1 of the form = 0 × 1 where 0 , 1 ∈ ( ), we have that is, for any ∈ G 1 of the above form,Q 1 = 0 implies Q 1 = 0. This proves that the assertion holds for = 1. Suppose now that the assertion holds for an arbitrary but fixed ∈ ℕ. Then we have to show that Q +1 ≪Q +1 in our induction step. To that end, from (2.7) we write where, the term on the right-hand side of ( † †) is well-defined and admits a closed form expression. The equality ( †) follows from the induction step for an arbitrary but fixed ∈ ℕ and ( † †) follows from (2.7). Choosing ∈ G +1 of the form = 0 × 1 × · · · × +1 where 0 , . . . , +1 ∈ ( ), we integrate on both sides on the set to arrive at Arguments similar to the base case = 1 apply here, which proves the assertion for the ( + 1) th induction step.
We now proceed to the proof of Theorem 3.3. P T 3.3. Recall that we stack the control actions generated by the malicious actuators and by the honest actuators separately: for each ∈ ℕ, the control actions corresponding to the corrupt policy are partitioned as˘ = ˘ ,1 , ,2 wherȇ ,1 = ˘ 1 , · · · ,˘ and ,2 = +1 , · · · , . Consequently, the input matrix and the covariance matrices V and V are also partitioned as = diag , where ∈ ℝ × and ∈ ℝ ( − ) × ( − ) , and The proof of Theorem 3.3 is based on the fact that under Assumption 3.1, the random vector is conditionally Gaussian given its past −1 (see Lemma B.1 for the proof). Observe that Lemma B.2 guarantees that Q ≪Q for every ∈ ℕ. This implies that for every ∈ ℕ the Radon-Nikodym derivative dQ dQ exists P˘ -almost surely and is given by the expression This implies that where is defined in (3.11) and we abbreviate hold at this stage and we assert that a separator in the sense of Definition 2.1 does not exist.
We proceed to (3.3-c). Assume that the hypotheses hold. Then we write  B.3. Observe that in the proof of Theorem 3.3, the statistics of the private excitation is chosen in such a way that det V˘ ˘ < det V can be ascertained for every ∈ ℕ, which plays a crucial role in the rest of the proof. §B.2. Sketch of the proof of Theorem 3.4. The proof of Theorem 3.4 is based on the fact that given the history ℎ −1 , the random vector ( ) ∈ℕ is conditional Gaussian under Assumption 3.1, which is the main thesis of next Lemma.
The rest of the proofs of Lemma B.4 and Theorem 3.4 proceed along the same lines as those of Lemma B.1 and Theorem 3.3, respectively; we have omitted them for brevity. Observe that Q ≪Q for every ∈ ℕ: the proof again follows via the principle of mathematical induction.

Appendix C. Background on Probability theory
Consider two probability measures P, P ′ on the same measurable space (Ω, G).
• Support of a probability measure P is defined by the closure of the set ∈ G P( ) ≠ 0 .