Accurate Approximate Diagnosis of (Controllable) Stochastic Systems

Abstract

Diagnosis of partially observable stochastic systems prone to faults was introduced in the late nineties. Diagnosability may be specified in different ways: exact diagnosability requires that almost surely a fault is detected and that no fault is erroneously claimed; approximate diagnosability tolerates a small error probability when claiming a fault; last, accurate approximate diagnosability guarantees that the error probability can be chosen arbitrarily small. While all three notions were studied for passive systems such as observable Markov chains, only the exact notion was considered for systems equipped with a controller. As the approximate notion of diagnosability was shown to be undecidable in passive systems, in this article, we complete the picture by deciding the accurate approximate diagnosability for controllable observable Markov chains. More precisely, we show how to adapt the accurate approximate notion to the active setting and establish EXPTIME-completeness of the associated decision problem. We also show how to measure the set of faulty paths that are detected under the accurate approximate notion in the passive setting.

A AA-Disclosure Problem for oMC

Theorem 3

The AA-disclosure problem for finite oMC is PSPACE-complete.

We decompose the proof of the theorem in the two following proposition, each establishing one direction.

Proposition 6

The AA-disclosure problem for finite oMC is in PSPACE.

Proof

To establish this result, we first build an exponential size oMC which contains additional information: the set of states the system could be in after the observation sequence. Then we show that there are two kinds of BSCC in this new oMC: the ones that are reached by paths that almost surely have an AA-disclosing observation sequence, and the ones that are reached by paths that do not correspond to AA-disclosing observation sequences. We can then use the existing results for the AA-diagnosability problem to determine the status of each BSCC. Therefore, computing the AA-disclosure of the oMC is equivalent to computing the probability to reach the “AA-disclosing” BSCC, which can be done in NC in the size of the oMC, thus giving an overall PSPACE algorithm.

Let $\mathcal {M}= (S,p,\mathsf {O})$ be a finite oMC and $\mu _0$ be an initial distribution. We build a new oMC $\mathcal {M}'= (S',p',\mathsf {O}')$ which has the same behaviour as $\mathcal {M}$ but where the states are enriched with an additional information (the set of states the system can be in, given the produced observation sequence):

• $S' = S \times 2^S$;

• For $(s,B), (s',B')\in S',$ $p'((s',B')\mid (s,B))= p(s'\mid s)$ if $B' = \cup _{q\in B}{\mathsf {Supp}}(p(q))\cap \mathsf {O}^{-1}(\mathsf {O}(s'))$ else, $p'((s',B')\mid (s,B))=0$;

• For $(s,B)\in S'$, $\mathsf {O}'(s,B)=\mathsf {O}(s)$.

We define the initial distribution $\mu '_0$ for $\mathcal {M}'$ by $\mu '_0(s,{\mathsf {Supp}}(\mu _0)\cap \mathsf {O}^{-1}(\mathsf {O}(s)))=\mu _0(s)$ for all $s\in S$. There is a one-to-one correspondence between the paths of $\mathcal {M}(\mu _0)$ and $\mathcal {M}'(\mu '_0)$: every path $\rho =s_0s_1\cdots s_n$ of $\mathcal {M}(\mu _0)$ is associated to an unique path $\rho '=(s_0,B_0)(s_1,B_1)\cdots (s_n,B_n)$ with $\mathsf {O}(\rho )=\mathsf {O}(\rho ')$, $\mathbf{P}_{\mathcal {M}(\mu _0)}(\rho )=\mathbf{P}_{\mathcal {M}'(\mu '_0)}(\rho ')$ and $B_n$ contains the set of states of S that can be reached with a path of observation $\mathsf {O}(\rho )$. Due to the latter property, $B_n$ only depends on $\mathsf {O}(\rho )$ and is called the belief associated to $\mathsf {O}(\rho )$.

Let $(s,B)\in S'$ such that $s\in \mathsf {S^{F}}$ and (s, B) belongs to a BSCC of $\mathcal {M}'$. We claim that either for every path $\rho$ ending in (s, B), $\mathbf{P}(\{\rho '\in \mathsf {Path}(\mathcal {M}'(\mu '_0))\mid \rho \preceq \rho ' \wedge \mathsf {O}(\rho ')\in D^{\mathsf{AA}}\})=0$ or for every path $\rho$ ending in (s, B), $\mathbf{P}(\{\rho '\in \mathsf {Path}(\mathcal {M}'(\mu '_0))\mid \rho \preceq \rho ' \wedge \mathsf {O}(\rho ')\in D^{\mathsf{AA}}\})=\mathbf{P}(\rho )$. In other words, there are two categories of BSCC composed of faulty states: the good ones, that almost surely accurate approximately disclose the fault, and the bad ones that do not accurate approximately disclose the fault at all. Moreover, a BSCC containing the state (s, B) do not disclose the fault at all iff there exists a state $s' \in B$ such that $s'$ belongs to a BSCC of $\mathcal {M}$, $s'\not \in \mathsf {S^{F}}$ and $d(\mathcal {M}(\mathbf {1}_{s}),\mathcal {M}(\mathbf {1}_{s'}))<1$.

Let (s, B) be a state belonging to a BSCC of $\mathcal {M}'$. Assume that for all $s'\in B$ such that $s'$ belongs to a BSCC of $\mathcal {M}$ and $s'\not \in \mathsf {S^{F}}$ we have $d(\mathcal {M}(\mathbf {1}_{s}),\mathcal {M}(\mathbf {1}_{s'}))=1$. We denote $B'= (B{\setminus }\mathsf {S^{F}})\cup \{s\}$, and define $\mathcal {M}''$ by removing the path leading to a faulty state (aka, a path either starts faulty or forever remain correct). Then as s belongs to a BSCC of $\mathcal {M}$, we can directly use Theorem 2 to obtain that for any initial distribution $\mu _1$ of support $B'$, we have that $\mathcal {M}''(\mu _1)$ is AA-diagnosable. As the limitation to the states of $B{\setminus }B'$ and the transformation from $\mathcal {M}$ to $\mathcal {M}''$ can only increase the failure proportion, this ensures that $\mathbf{P}(\{\rho '\in \mathsf {Path}(\mathcal {M}'(\mu '_0))\mid \rho \preceq \rho ' \wedge \mathsf {O}(\rho ')\in Disc^{\mathsf{AA}}\})=\mathbf{P}(\rho )$.

Conversely, if there exists a state $s'\in B$ such that $s'$ belongs to a BSCC of B, $s'\not \in \mathsf {S^{F}}$ and $d(\mathcal {M}(\mathbf {1}_{s}),\mathcal {M}(\mathbf {1}_{s'}))<1$, then one can rely on the proof of Lemma A of [8] to obtain the result. For the sake of pedagogy, we present the proof here in the simpler case where B does not contain any faulty state beside s. Using Proposition 2 and the correspondence between $\mathcal {M}$ and $\mathcal {M}'$, one deduces that there exists $\rho _{(s,B)} \in \mathsf {fPath}(\mathcal {M}(\mathbf {1}_{(s,B)}))$ and $\alpha >0$ such that for all $w\in \varSigma ^*$ with $\mathsf {O}(\rho )\le w$

$$\begin{aligned} \mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s,B)})}&(\{\rho '\in \mathsf {fPath}(\mathcal {M}'(\mathbf {1}_{(s,B)}))\mid \rho _{(s,B)}\preceq \rho ' \wedge \mathsf {O}(\rho ') =w\})\end{aligned}$$

(1)

$$\begin{aligned}&\le \alpha \mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s',B)})}(\{\rho '\in \mathsf {fPath}(\mathcal {M}'(\mathbf {1}_{(s',B)}))\mid \mathsf {O}(\rho ') =w\}). \end{aligned}$$

(2)

Therefore, for all $w\in \varSigma ^*$ and initial distribution $\mu _1$ of support B we have:

$$\begin{aligned} \mathsf {Fprop}_{\mathcal {M}'(\mu _1)}(w)\le&\,\frac{\mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s,B)})}(w)}{\mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s,B)})}(w) + \frac{\mu _1(s')}{\mu _1(s)} \mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s',B)})}(w)} \end{aligned}$$

(3)

$$\begin{aligned} \le \,&\frac{ \varepsilon _w + \displaystyle \sum _{\rho \mid \mathsf {O}(\rho \rho _{(s,B)})\le w} \frac{\alpha \mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s,B)})}(\rho )}{\mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s,B)})}(\rho _{(s,B)})}\mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s',B)})}(w^\rho )}{\mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s,B)})}(w) + \frac{\mu _1(s')}{\mu _1(s)} \mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s',B)})}(w)} \end{aligned}$$

(4)

where $w^\rho$ is such that $w= \mathsf {O}(\rho )w^\rho$, the first term $\varepsilon _w = \mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s,B)})} (\{\rho \in \mathsf {fPath}(\mathcal {M}(\mathbf {1}_{(s,B)}) \mid \not \!\!\exists \rho _1,\rho _2, \rho = \rho _1 \rho _{(s,B)} \rho _2 \wedge \mathsf {O}(\rho )=w\})$ is the probability of the set of paths with observation w that do not contain the infix $\rho _{(s,B)}$ and the second term relies on the bound from Eq. 2 to bound the probability of every other paths. As with probability 1, a path of $\mathcal {M}'(\mathbf {1}_{(s,B)})$ visits (s, B) infinitely often, it will almost surely contain a $\rho _{(s,B)}$ subpath, more precisely: the value $\frac{\varepsilon _w}{{\mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s,B)})}(w)}}$ almost surely converges to 0 when |w| diverges to $\infty$. Let $w\in \varSigma ^\omega$, if $\mathsf {Fprop}_{\mathcal {M}'(\mu _1)}(w_{\downarrow n})\xrightarrow {n\xrightarrow {}\infty } 1$ then, for all $\rho$ such that $\mathsf {O}(\rho \rho _{(s,B)})\le w$ we have that $\frac{\mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s',B)})}(w^\rho _{\downarrow n})}{\mathbf{P}_{\mathcal {M}'(\mathbf {1}_{(s,B)})}(w_{\downarrow n})}$ converges to 0, thus, due to Eq. 4, $\varepsilon _{w_{\downarrow n}}$ does not converge to 0, which can only happen with probability 0. Therefore $\mathsf {Fprop}_{\mathcal {M}'(\mu _1)}(w_{\downarrow n})$ almost surely does not converge to 1. This implies that $\mathbf{P}\{\rho '\in \mathsf {Path}(\mathcal {M}'(\mu '_0))\mid \rho \preceq \rho ' \wedge \mathsf {O}(\rho ')\in D^{AA}\}=0$.

This result establishes that the BSCC of $\mathcal {M}'$ are partitioned between the good ones that accurately approximately and almost surely disclose the fault and the bad ones that do not accurately approximately disclose it at all. Moreover, given a state $(s_0,B_0)$ belonging to a BSCC of $\mathcal {M}'$, if there exists a state $s_0'\in B_0$ such that $s_0'$ belongs to a BSCC of B, $s_0'\not \in \mathsf {S^{F}}$ and $d(\mathcal {M}(\mathbf {1}_{s_0}),\mathcal {M}(\mathbf {1}_{s_0'}))<1$, then for any state $(s_1, B_1)$ belonging to the same BSCC, one can find a state $s_1'\in B_1$ satisfying a similar property with respect to $s_1$. In other words, for every BSCC of $\mathcal {M}'$, we only need to check a single state (s, B) of the BSCC to identify whether the BSCC is disclosing or not. Furthermore, this check can be done by testing the distance 1 between copies of $\mathcal {M}$ starting in s and copies starting in some of the states in B. There is thus at most linearly many tests to do, each of which can be done in polynomial time in the size of $\mathcal {M}$.

Therefore, one can obtain the value of $Disc^{AA}(\mathcal {M}'(\mu '_0))$ by computing the probability to reach the good BSCC, which is known to be possible in PTIME in the size of $\mathcal {M}'$. In fact, as computing this probability amount to solve a linear system of equations, this can even be done in NC  [11, 21]. The oMC $\mathcal {M}'$ being exponential in the size of $\mathcal {M}$, and as NC blown up to the exponential is equal to PSPACE  [10], this yields a PSPACE algorithm. As $Disc^{AA}(\mathcal {M}(\mu _0))= Disc^{AA}(\mathcal {M}'(\mu '_0))$, this allows us to solve the AA-disclosure problem.    $\square$

Proposition 7

The AA-disclosure problem for finite oMC is PSPACE-hard.

Proof

We now establish the hardness by reducing the universality problem for non-deterministic finite automaton (NFA), which is known to be PSPACE-complete [19].

An NFA is a tuple $\mathcal {A}= (Q,\varSigma ,T,q_0,F)$ where Q is the set of states, $q_0$ is the initial state, F is the set of accepting states, $\varSigma$ is the alphabet and $T\in Q\times \varSigma \times Q$ is the transition function. An NFA is universal if for all $w=a_1a_2\dots a_n\in \varSigma ^n$, there exists a path $q_0a_1q_1a_2\dots q_n$ such that $q_n\in F$ and for all $1\le i\le n, (q_{i-1},a_i,q_i)\in T$.

Fig. 3.

From NFA $\mathcal {A}$ to incomplete oMC $\hat{\mathcal {A}}$. The label next to the state is its name. We will not always display the state’s name so as not to overload the figure.

Let $\mathcal {A}= (Q,\varSigma ,T,q_0,F)$ be an NFA. W.l.o.g. we can assume that $F=Q$ and $\varSigma =\{a,b\}$. Our first step is to push the observations onto the states (as shown in Fig. 3). From $\mathcal {A}$ we define the incomplete oMC $\hat{\mathcal {A}}= (S_A,p_A,O_A)$ and the initial distribution $\mu ^A_0$ such that:

• $S_A = Q \times \varSigma$;

• for $(q,c), (q',d)\in S_A,$ if $(q,d,q')\in T$, then $p_A((q',d)\mid (q,c))= \frac{1}{|S_A|+1}$, else $p_A((q',d)\mid (q,c))=0$;

• for $(q,c)\in S_A, O_A(q,c)=c$;

• for $(q',d)\in S_A,$ if $(q_0,d,q')\in T$, then $\mu ^A_0(q',d)= \frac{1}{|S_A|+1}$, else $\mu ^A_0(q',d)=0$.

This oMC is incomplete as none of the distributions $\mu ^A_0$ and $p_A(\cdot \mid s)$ (for $s\in S_A$) sum to 1. We now build the oMC $\mathcal {M}= (S,p,O)$ represented in Fig. 4 where

• $S=S_A\cup \{s_\sharp ,f_a,f_b,f_\sharp \}$;

• given $s,s'\in S_A$, $p(s'\mid s)=p_A(s'\mid s)$, $p(s_\sharp \mid s) = 1 - \sum _{s'\in S_A}p(s'\mid s)$, for $h\in \{f_a,f_b\}$ and $g\in \{f_a,f_b,f_\sharp \}$, $p(g\mid h)=1/3$ and $p(f_\sharp \mid f_\sharp )=p(s_\sharp \mid s_\sharp )=1$;

• for $s\in S_A$, $O(s)=O_A(s)$, $O(s_\sharp )=O(f_\sharp )=\sharp , O(f_a)=a$ and $O(f_b)=b$.

We also define $\mu _0$ as $\mu _0(s) =\mu ^A_0(s)$ for $s\in S_A$ and $\mu _0(f_a)=\mu _0(f_b) = \frac{1 -\sum _{s\in S_A}\mu _0(s)}{2}$.

Fig. 4.

A reduction for PSPACE-hardness of the AA-disclosure problem.

Choosing $\mathsf {S^{F}}= \{f_\sharp \}$, let us show that $\mathcal {A}$ is not universal iff $Disc^{\mathsf{AA}}(\mathcal {M}(\mu _0))>0$.

Suppose first that $\mathcal {A}$ is not universal. There thus exists a word $w\in \varSigma ^*$ such that no path starting in $S_A$ has observation sequence w. As there exists one faulty path $\rho$ (starting in either $f_a$ or $f_b$) associated to $w\sharp$, we have $\mathsf {Fprop}_{\mathcal {M}(\mu _0)}(w\sharp )=1$. Therefore $Disc^{\mathsf{AA}}(\mathcal {M}(\mu _0))\ge \mathbf{P}_{\mathcal {M}(\mu _0)}(\rho )>0$.

Conversely, assume that $\mathcal {A}$ is universal. Let $\rho$ be a path ending in $f_\sharp$ with observation sequence $\mathsf {O}(\rho )= w\sharp$ for some $w\in \varSigma ^*$. As $\mathcal {A}$ is universal, there exists a finite path $\rho '$ in $\hat{\mathcal {A}}$ with observation sequence w. As for every state s of $\hat{\mathcal {A}}$, $p(s_\sharp \mid s)>0$, $\rho '$ can be extended into a finite path $\rho ''$ ending in $s_\sharp$ with observation $w\sharp$. Thus, $\mathsf {Fprop}_{\mathcal {M}(\mu _0)}(w\sharp )<1$. Moreover, every path ending with a $\sharp$ remains with probability 1 in either $s_\sharp$ or $f_\sharp$, due to this for every $k\ge 2$, $\mathsf {Fprop}_{\mathcal {M}(\mu _0)}(w\sharp ^{k})=\mathsf {Fprop}_{\mathcal {M}(\mu _0)}(w\sharp ).$ Therefore, $w\sharp ^\omega \not \in D^{\mathsf{AA}}$. This implies that no infinite path visiting $f_\sharp$ corresponds to an AA-disclosing observation sequence. $f_\sharp$ being the only faulty state, $Disc^{\mathsf{AA}}(\mathcal {M}(\mu _0))=0.$    $\square$