Probabilistic Reasoning About Simply Typed Lambda Terms

Abstract

Reasoning with uncertainty has gained an important role in computer science, artificial intelligence and cognitive science. These applications urge for development of formal models which capture reasoning of probabilistic features. We propose a formal model for reasoning about probabilities of simply typed lambda terms. We present its syntax, Kripke-style semantics and axiomatic system. The main results are the corresponding soundness and strong completeness, which rely on two key facts: the completeness of simple type assignment and the existence of a maximal consistent extension of a consistent set.

Appendix Proofs

Proof of Theorem 3 . Suppose that $T\cup \{\phi \}\vdash \psi$. We use transfinite induction on the length of a proof.

If the length of a proof is equal to 1, then $\psi$ is either an axiom or $\psi \in T\cup \{\phi \}$.

• (a) If $\psi$ is an axiom:

  • $T\vdash \psi$      Ax

  • $T\vdash \psi \Rightarrow (\phi \Rightarrow \psi )$      Ax

  • $T\vdash \phi \Rightarrow \psi$      MP,

• (b) If $\psi \in T$:

  • $T\vdash \psi$      Hyp

  • $T\vdash \psi \Rightarrow (\phi \Rightarrow \psi )$      Ax

  • $T\vdash \phi \Rightarrow \psi$      MP,

• (c) If $\psi \in \{\phi \}$:

  • $T\vdash \phi \Rightarrow \phi$      Ax.

Now, suppose that the length of a proof is $k>1$. Formula $\psi$ can belong to the set $T\cup \{\phi \}$, but then the proof is the same as above. Therefore, suppose that the formula $\psi$ is obtained by an application of some inference rule from the Inference Rules II.

First, if $\psi$ is obtained by an application of Rule II.1 from $T,\phi \vdash \psi _1$ and $T,\phi \vdash \psi _1\Rightarrow \psi$:

• $T\vdash \phi \Rightarrow \psi _1$      Ind. Hyp.

• $T\vdash \phi \Rightarrow (\psi _1\Rightarrow \psi )$      Ind. Hyp.

• $T\vdash (\phi \Rightarrow (\psi _1\Rightarrow \psi ))\Rightarrow ((\phi \Rightarrow \psi _1)\Rightarrow (\phi \Rightarrow \psi ))$      Taut.

• $T\vdash (\phi \Rightarrow \psi _1)\Rightarrow (\phi \Rightarrow \psi )$      MP

• $T\vdash \phi \Rightarrow \psi$      MP

Next, let us consider the case $\psi =P_{\ge 1}\alpha$ is obtained from $T\cup \{\phi \}$ by an application of Rule II.2. In that case:

• $T,\phi \vdash \alpha$,

• $T,\phi \vdash P_{\ge 1}\alpha$ by IR II.2.

However, since $\alpha \in \mathsf{For_B}$ and $\phi \in \mathsf{For_P}$ (otherwise, $\phi \Rightarrow P_{\ge 1}\alpha$ would not make sense), $\phi$ cannot affect the proof of $\alpha$ from $T\cup \{\phi \}$, and we have:

• (1) $T\vdash \alpha$      Hyp.

• (2) $T\vdash P_{\ge 1}\alpha$      IR II.2

• (3) $T\vdash P_{\ge 1}\alpha \Rightarrow (\phi \Rightarrow P_{\ge 1}\alpha )$      Taut.

• (4) $T\vdash \phi \Rightarrow P_{\ge 1}\alpha$      MP.

Finally, let us consider the case $\psi = \psi _1\Rightarrow P_{\ge s}\alpha$ is obtained from $T\cup \{\phi \}$ by an application of Rule II.3. Then:

• (1) $T,\phi \vdash \psi _1\Rightarrow P_{\ge s-\frac{1}{k}}\alpha$, for all $k\ge \frac{1}{s}$      Hyp.

• (2) $T\vdash \phi \Rightarrow (\psi _1\Rightarrow P_{\ge s-\frac{1}{k}}\alpha )$      Ind.Hyp.

• (3) $T\vdash (\phi \wedge \psi _1)\Rightarrow P_{\ge s-\frac{1}{k}}\alpha$      Taut.

• (4) $T\vdash (\phi \wedge \psi _1)\Rightarrow P_{\ge s}\alpha$      IR II.3

• (5) $T\vdash \phi \Rightarrow \psi$ Taut.$\square$

Proof of Theorem 4 . Our goal is to show that every instance of an axiom scheme holds in every model and that the inference rules preserve the validity. The Axiom 1 holds in every model because of the completeness of classical propositional logic.

By the Definition of the finitely additive probability measure we have that $\mu ([\alpha ])\ge 0$ for all $\alpha \in \mathsf{For_B}$. Hence, $\mathcal M\,\models \, P_{\ge 0} \alpha$, for every model $\mathcal M$ and the Axiom 2 is valid.

Let us consider the Axiom 3. Suppose that $P_{\le r} \alpha$ holds in model $\mathcal M=\langle W,\rho ,\xi ,H,\mu \rangle$ and $s>r$. It means that $\mu ([\alpha ])\le r$. Since $s>r$, we obtain $\mu ([\alpha ])< s$, that is $\mathcal M\,\models \, P_{<s} \alpha$.

Similarly, for the Axiom 4, suppose that $\mathcal M \,\models \, P_{<s} \alpha$. Then, we have $\mu ([\alpha ])<s$, that implies $\mu ([\alpha ])\le s$. Thus, $\mathcal M \,\models \, P_{\le s}\alpha$.

Next, let us consider Axiom 5. Suppose that in a model $\mathcal M=\langle W,\rho ,\xi ,H,\mu \rangle$,

$$P_{\ge r}\alpha , P_{\ge s}\beta \; \; \text{ and } \; \; P_{\ge 1}\lnot (\alpha \vee \beta )$$

hold. Then, $\mu ([\alpha ])\ge r$, $\mu ([\beta ])\ge s$ and $[\alpha ]$ and $[\beta ]$ are disjoint sets. Since $\mu$ is a finitely additive measure, we have that

$$\mu ([\alpha ]\cup [\beta ])=\mu ([\alpha \vee \beta ])=\mu ([\alpha ])+\mu ([\beta ]).$$

Thus, $\mathcal M\,\models \, P_{\ge min\{1,r+s\}}(\alpha \vee \beta )$, so Axiom 5 holds in the model $\mathcal M$.

Now, let us consider the Axiom 6. Suppose that $P_{\le r} \alpha$, $P_{< s} \beta$ hold in a model $\mathcal M=\langle W,\rho ,\xi ,H,\mu \rangle$. Then, $\mu ([\alpha ])\le r$ and $\mu ([\beta ])< s$. From

$$[\alpha ]=([\alpha ] \cap (W\setminus [\beta ])) \cup [\alpha \wedge \beta ],$$

follows that

$$\mu ([\alpha ])\ge \mu ([\alpha ] \cap (W\setminus [\beta ])).$$

Since $[\alpha \vee \beta ]=([\alpha ] \cap (W\setminus [\beta ])) \cup [\beta ]$, we have that

$$\mu ([\alpha \vee \beta ])\le \mu ([\alpha ])+\mu ([\beta ])<r+s.$$

Therefore, $\mathcal M \,\models \, P_{<r+s} (\alpha \vee \beta )$.

Finally, for the Axiom 7, suppose that $P_{\ge 1}(\alpha \Rightarrow \beta )$ holds in a model $\mathcal M=\langle W,\rho ,\xi ,H,\mu \rangle$. Then, the set of all worlds in which $\alpha$ holds, but $\beta$ does not hold has the measure 0, i.e., $\mu ([\alpha ] \cap (W\setminus [\alpha \Rightarrow \beta ]))=0$. From

$$[\alpha ]=([\alpha ]\cap (W\setminus [\alpha \Rightarrow \beta ]))\cup ([\alpha ] \cap [\alpha \Rightarrow \beta ])$$

follows that $\mu ([\alpha ])=\mu ([\alpha ] \cap [\alpha \Rightarrow \beta ])$ and, since $[\alpha ] \cap [\alpha \Rightarrow \beta ]\subseteq [\beta ]$, we have that $\mu ([\alpha ]) \le \mu ([\beta ])$. Thus, $\mathcal M \,\models \, P_{\ge s} \alpha \Rightarrow P_{\ge s} \beta$ and Axiom 7 holds in $\mathcal M$.

The proof that Inference Rules I are sound can be found in [13].

Inference Rules II:

• Rule II.1 is validity-preserving for the same reason as in classical logic.

• Rule II.2: suppose that $\alpha$ holds in $\mathcal M=\langle W,\rho ,\xi ,H,\mu \rangle$, then $[\alpha ]=W$, and therefore $\mu ([\alpha ])=1$, so $\mathcal M\,\models \, P_{\ge 1}\alpha$.

• Rule II.3: Suppose that $\mathcal M\,\models \,\phi \Rightarrow P_{\ge s-\frac{1}{k}}\alpha$ whenever $k\ge \frac{1}{s}$. If $\mathcal M\not \models \phi$, then obviously $\mathcal M\,\models \,\phi \Rightarrow P_{\ge s}\alpha$. Otherwise $\mathcal M\,\models \, P_{\ge s-\frac{1}{k}}\alpha$ for every $k\ge \frac{1}{s}$, so $\mathcal M\,\models \, P_{\ge s} \alpha$ because of the Archimedean properties of the set of reals. $\square$

Proof of Lemma 1

• (1) If $T\cup \{\phi \}\vdash \bot$, and $T\cup \{\lnot \phi \}\vdash \bot$, then by Deduction theorem we have $T\vdash \lnot \phi$ and $T\vdash \phi$. Contradiction.

• (2) Suppose that for all $n>\frac{1}{s}$:

  $$T,\phi \Rightarrow \lnot P_{\ge s-\frac{1}{n}}\alpha \vdash \bot .$$

  Therefore, by Deduction theorem and propositional reasoning, we have

  $$T\vdash \phi \Rightarrow P_{\ge s-\frac{1}{n}}\alpha ,$$

  and by an application of Rule II.3 we obtain $T\vdash \phi \Rightarrow P_{\ge s}\alpha$. Contradiction with the fact that $\lnot (\phi \Rightarrow P_{\ge s}\alpha )\in T$. $\square$

Proof of Lemma 2

(1) Consequence of Definition 7.4.

(2) Let $t=sup\{s\mid P_{\ge s}\alpha \in T\}\in \mathsf{S}.$ By the monotonicity of a measure, for each $s\in S$, $s<t$, $T\vdash P_{\ge s}\alpha$. Using Inference rule 3, we obtain

$$T\vdash P_{\ge t}\alpha .$$

T is a maximal consistent set of formulas, so, from (1), we have that

$$P_{\ge t}\alpha \in T.$$

$\square$

Proof of Lemma 3

• (1) The prove that H is an algebra is straightforward using that $W=[\alpha \vee \lnot \alpha ]$, $[\alpha ]^C=[\lnot \alpha ]$ and $[\alpha ]\cup [\beta ]=[\alpha \vee \beta ]$.

• (2) It suffices to prove that $[\alpha ]\subset [\beta ]$ implies $\mu ([\alpha ])\le \mu ([\beta ])$. According to the completeness of the propositional logic, we have that $[\alpha ]\subset [\beta ]$ means that $\alpha \Rightarrow \beta \in \mathsf{Cn_B}(T)$, and then also $P_{\ge 1}(\alpha \Rightarrow \beta )\in T^\star$. By axiom 7, we obtain that for each $s\in S$, $P_{\ge s}\alpha \Rightarrow P_{\ge s}\beta \in T^\star$, so $\mu ([\alpha ])\le \mu ([\beta ])$.

• (3) $P_{\ge 0}\alpha$ is an axiom, so $\mu ([\alpha ])\ge 0$.

• (4) For any $\alpha \in T$, we have that $\alpha \vee \lnot \alpha \in \mathsf{Cn_B}(T)$ and $P_{\ge 1}(\alpha \vee \lnot \alpha )\in T^\star$, therefore, we obtain that $W=[\alpha \vee \lnot \alpha ]$ and $\mu (W)=1$. Since $P_{\ge 1}(\alpha \vee \lnot \alpha )=P_{\ge 1-0}(\alpha \vee \lnot \alpha )=P_{\le 0}\lnot (\alpha \vee \lnot \alpha )=P_{\le 0}(\lnot \alpha \wedge \alpha )$ $=\lnot P_{> 0}(\lnot \alpha \wedge \alpha ),$ using that $P_{\ge t}\alpha \Rightarrow P_{> s}\alpha$, for $t>s$, we obtain that

  $$sup \{s\mid P_{\ge s}(\lnot \alpha \wedge \alpha )\in T^\star \}=0,$$

  and $\mu (\emptyset )=0.$

• (5) Let $\mu ([\alpha ])=\sup \{s\mid P_{\ge s}\alpha \in T^\star \}=r$. If $r=1$, then from Lemma 2 we obtain $P_{\ge 1}\alpha \in T^\star$. Therefore, $\lnot P_{>0}\lnot \alpha \in T^\star .$ Again, using the fact that $P_{\ge t}\alpha \Rightarrow P_{> s}\alpha$, for $t>s$, we obtain that $\mu ([\lnot \alpha ])=0.$ Now, suppose that $r<1.$ Then, for each rational number $r'\in (r,1]$, $\lnot P_{\ge r'}\alpha = P_{<r'}\alpha \in T^\star$. By Axiom 4, we obtain that $P_{\le r'}\alpha , P_{\ge 1-r'}\lnot \alpha \in T^\star$. If there is some rational number $r''\in [0,r)$ such that $P_{\ge 1-r''}\lnot \alpha \in T^\star$, then $\lnot P_{> r''}\alpha \in T^\star$, contradiction. Thus,

  $$\sup \{s\mid P_{\ge s}\lnot \alpha \in T^\star \}=1-\sup \{s\mid P_{\ge s}\alpha \in T^\star \},$$

  i.e., $\mu ([\alpha ])=1-\mu ([\lnot \alpha ])$.

• (6) Let $[\alpha ]\cap [\beta ]=\emptyset$, and let $\mu ([\alpha ])=r$ and $\mu ([\beta ])=s$. From the fact that $[\beta ]\subset [\lnot \alpha ]$, using steps (2) and (5), we obtain that $r+s\le r+(1-r)=1$. Suppose that both $r>0$ and $s>0$. Using properties of the supremum, for every rational number $r'\in [0,r)$, and for every rational number

  $s'\in [0,s)$, we have that $P_{\ge r'}\alpha , P_{\ge s'}\beta \in T^\star$. By the Axiom 5, we know that $P_{\ge r'+s'}(\alpha \vee \beta )\in T^\star$. Therefore, $r+s\le t_0=\sup \{t\mid P_{\ge t}(\alpha \vee \beta \in T^\star )\}$. In the case that $r+s=1$, the statement holds obviously, so suppose that $r+s<1$. If $r+s<t_0$, then for every rational number $t'\in (r+s,t_0)$ we have $P_{\ge t'}(\alpha \vee \beta )\in T^\star$. There exists rational numbers $r''>r$ and $s''>s$, such that:

  $$\lnot P_{\ge r''}\alpha , P_{< r''}\alpha \in T^\star \quad , \lnot P_{\ge s''}\alpha , P_{< s''}\alpha \in T^\star ,$$

  and

  $$r''+s''=t'\le 1.$$

  By Axiom 4, we obtain $P_{\le r''}\in T^\star$. Using Axiom 6, we get

  $$P_{\le r''+s''}(\alpha \vee \beta )\in T^\star ,\quad \lnot P_{\ge r''+s''}(\alpha \vee \beta )\in T^\star ,$$

  and

  $$\lnot P_{\ge t'}(\alpha \vee \beta )\in T^\star .$$

  Contradiction. Hence, $r+s=t_0$ and we obtain that

  $\mu ([\alpha ]\cup [\beta ])=\mu ([\alpha ]) + \mu ([\beta ])$. Finally, if we suppose that $r=0$ or $s=0$, we can reason as above, where $r'=0$ or $s'=0$.$\square$