Axiom selection over large theory based on new first-order formula metrics

Abstract

Axiom selection is a task that selects the most likely useful axioms from a large-scale axiom set for proving a given conjecture. Existing axiom selection methods either solely take shallow symbols into account or strongly dependent on previous successful proofs from homologous problems. To address these problems, we introduce a new metric to evaluate the dissimilarity between formulae and utilize it as an evaluator in the selection task. Firstly, we propose a substitution-based metric to compute the dissimilarity between terms. It is a pseudo-metric and can capture the in-depth syntactic difference trigged by both functional and variable subterms. We then extend it to atoms and prove the atom metric also to be a pseudo-metric. Treating formulae as atom sets, we define three kinds of dissimilarity metrics between formulae. Finally, we design and implement conjecture-oriented axiom selection methods based on newly proposed formula metrics. The experimental evaluation is conducted on the MPTP2078 benchmark and demonstrates dissimilarity-based axiom selection improves E prover’s performance. In the best case, it increases the ratio of successful proofs from 30.90% to 42.25%.

Appendix A

Proof Proof of Lemma 1

We need to prove that d^′ satisfies all properties a pseudo-metric has if function S^′ satisfies 1 - 6.

Let t₁, t₂ be arbitrary two terms. By condition 1, obviously,

$$d'(t_{1},t_{2})\geq_{2}(0,0).$$

If t₂ = t₁, t₁ is the lgg of itself. The substitution mapping a term to itself is an empty substitution or a renaming substitution. By conditions 2 and 3,

$$d'(t_{1},t_{1})=_{2}(0,0).$$

According to the definition of d^′, the symmetry is also satisfied. That is,

$$d'(t_{1},t_{2})=_{2}d'(t_{2},t_{1}).$$

Finally, we prove that d^′ satisfies the triangle inequality.

As shown in Fig. 2, t₁, t₂, and t₃ are arbitrary three terms, lgg(t₁,t₂) = t₄, lgg(t₁,t₃) = t₅, lgg(t₂,t₃) = t₆, lgg(t₅,t₆) = t₇. Here, t₃ must be a unified term of t₅ and t₆. For convenience, we use 𝜃_ij to denote the substitution which maps from term t_i to term t_j.

Fig. 2

Proof structure of triangle inequality

By the definition of d^′,

$$d'(t_{1},t_{2})=_{2}S'(\theta_{41})+S'(\theta_{42}),$$

$$d'(t_{1},t_{3})=_{2}S'(\theta_{51})+S'(\theta_{53}),$$

$$d'(t_{2},t_{3})=_{2}S'(\theta_{62})+S'(\theta_{63}).$$

By condition 5,

$$S'(\theta_{41})\leq_{2} s'(\theta_{71}), \quad S'(\theta_{42})\leq_{2} S'(\theta_{72}).$$

By condition 4,

$$S'(\theta_{71})\leq_{2} S'(\theta_{75})+S'(\theta_{51}),\quad S'(\theta_{72})\leq_{2} S'(\theta_{76})+S'(\theta_{62}).$$

By condition 6,

$$S'(\theta_{75})+S'(\theta_{76})\leq_{2} S'(\theta_{53})+S'(\theta_{63}).$$

Hence,

$$ \begin{aligned} d'(t_{1},t_{2})&=_{2}S'(\theta_{41})+S'(\theta_{42})\leq_{2} S'(\theta_{71})+S'(\theta_{72}) \\ &\leq_{2} S'(\theta_{75})+S'(\theta_{51})+S'(\theta_{76})+S'(\theta_{62}) \\ &\leq_{2}S'(\theta_{51})+S'(\theta_{53})+S'(\theta_{62})+S'(\theta_{63}) \\ &=_{2}d'(t_{1},t_{3})+d'(t_{2},t_{3}). \end{aligned} $$

□

Proof Proof of Theorem 1

By lemma 1, we only need to ensure that the function S_T satisfies 1 - 6.

• 1. For any substitution 𝜃, obviously, by (3) and (4), we have

  $$S_{f}(\theta_{f})\geq0 \text{and} S_{v}(\theta_{v})\geq0.$$

  Hence, by (5),

  $$S_{T}(\theta)\geq_{2}(0,0).$$

• 2. For an empty substitution ε, both ε_f and ε_v are also empty substitutions. Hence, S_f(ε_f), S_v(ε_v) are empty sums, and we have

  $$S_{T}(\varepsilon)=_{2}(0,0).$$

• 3. For a renaming substitution η and t₁η = t₂, we have η_f = ε and η_v = η. Suppose that η_v = η = {X₁↦Y₁,...,X_n↦Y_n}, where Y₁, ..., Y_n are distinct variables in t₂ and ∀X_i↦Y_i ∈ η_v, $occ_{t_{1}}^{+}(X_{i})=occ_{t_{2}}^{+}(Y_{i})$ and $g(w_{v}(occ_{t_{2}}^{+}(Y_{i})-occ_{t1}^{+}(X_{i})))=g(0)=0$. Hence,

  $$ S_{T}(\eta)=_{2}(0,0).$$

• 4. As shown in Fig. 3 , for arbitrary three terms t₁, t₂ and t₃, which satisfy:

  $$t_{1}\theta_{1}=t_{2}, t_{2}\theta_{2}=t_{3} \text{and} t_{1}\theta=t_{3}. $$

  Fig. 3

  

  Triangle structure

  To prove the inequality of S_T(𝜃) ≤₂S_T(𝜃₁) + S_T(𝜃₂), we would like to prove the conclusion of $S_{f}(\theta _{f})\leq S_{f}(\theta _{1_{f}})+S_{f}(\theta _{2_{f}})$ at first.

  • (a) 𝜃_f = ∅. Hence, $\theta _{1_{f}}=\boldsymbol {\emptyset }$ and $\theta _{2_{f}}=\boldsymbol {\emptyset }$. We have

    $S_{f}(\theta _{f})=S_{f}(\theta _{1_{f}})+S_{f}(\theta _{2_{f}})$.

  • (b) 𝜃_f≠∅. Suppose that Ran(𝜃_f) = {u₁,...,u_m}(m ≥ 1). In 𝜃, ∀u_i ∈ Ran(𝜃_f), let $V_{t_{1}}(u_{i})=\{X_{i1}, ..., X_{ik}\}(k\geq 1)$, denoting the set of variables substituted by u_i in t₁. $\forall X_{ij}\in V_{t_{1}}(u_{i})$, let $occ_{t_{1}}(X_{ij})=n_{ij}(\geq 1)$ and ${\sum }_{j=1}^{k}n_{ij}=n_{i}$. Hence, S_f(𝜃_f) also can be presented as:

    $S_{f}(\theta _{f})={\sum }_{f\in \bigcup _{i=1}^{m}F(u_{i})}w_{f}(f)({\sum }_{i=1}^{m}n_{i}occ_{u_{i}}(f))$.

    For any $f\in \bigcup _{i=1}^{m}F(u_{i})$, ${\sum }_{i=1}^{m}n_{i}occ_{u_{i}}(f)$ is considered as the number of new occurrences of f under 𝜃_f. Obviously, the sum of the number of new occurrences of f under $\theta _{1_{f}}$ and $\theta _{2_{f}}$ equals to ${\sum }_{i=1}^{m}n_{i}occ_{u_{i}}(f)$. Hence,

    $S_{f}(\theta _{f})=S_{f}(\theta _{1_{f}})+S_{f}(\theta _{2_{f}})$.

  Next, we start to prove $S_{v}(\theta _{v})\leq S_{v}(\theta _{1_{v}})+S_{v}(\theta _{2_{v}})$.

  • (a) 𝜃_v is a renaming substitution or an empty substitution. $\theta _{1_{v}}$ and $\theta _{2_{v}}$ are either empty substitution or renaming substitution (should not be empty at the same time). By the previous proof, we know

    $S_{v}(\theta _{v})=S_{v}(\theta _{1_{v}})=S_{v}(\theta _{2_{v}})=0$.

    Hence,

    $S_{v}(\theta _{v})=S_{v}(\theta _{1_{v}})+S_{v}(\theta _{2_{v}})$.

  • (b) Otherwise, suppose that 𝜃_v = {X₁↦Z₁,...,X_n↦Z_n}. ∀X_i↦Z_i ∈ 𝜃_v, let $\theta _{v_{i}}=\{X_{i}\mapsto Z_{i}\}$, then $\bigcup _{i=1}^{n}\theta _{v_{i}}=\theta _{v}$ and $\theta _{v_{i}}\cap \theta _{v_{j}}=\boldsymbol {\emptyset }(i\neq j)$. Obviously,

    $S_{v}(\theta _{v})={\sum }_{i=1}^{n}S_{v}(\theta _{v_{i}})$.

    For any $\theta _{v_{i}}$, we have

    $S_{v}(\theta _{v_{i}})=g(w_{v}(occ_{t_{3}}^{+}(Z_{i})-occ_{t_{1}}^{+}(X_{i})))$.

    It must exist substitutions $\theta _{1_{v_{i}}}=\{X_{i}\mapsto Y_{i}\}\subseteq \theta _{1_{v}}$ (Y_i may equal to X_i) and $\theta _{2_{v_{i}}}=\{Y_{i}\mapsto Z_{i}\}\subseteq \theta _{2_{v}}$ (Y_i may equal to Z_i) such that

    $X_{i}\theta _{1_{v_{i}}}\theta _{2_{v_{i}}}=Z_{i}$.

    If Y_i = X_i or Y_i = Z_i, then $\theta _{1_{v_{i}}}$ or $\theta _{2_{v_{i}}}$ is an empty substitution. Obviously,

    $occ_{t_{2}}^{+}(Y_{i})\geq occ_{t_{1}}^{+}(X_{i})$ and $occ_{t_{3}}^{+}(Z_{i})\geq occ_{t_{2}}^{+}(Y_{i})$.

    By the properties of function g, we have

    $g(w_{v}(occ_{t_{3}}^{+}(Z_{i})-occ_{t_{1}}^{+}(X_{i})))=g(w_{v}(occ_{t_{3}}^{+}(Z_{i})-occ_{t_{2}}^{+}(Y_{i})+occ_{t_{2}}^{+}(Y_{i})-occ_{t_{1}}^{+}(X_{i})))\leq g(w_{v}(occ_{t_{3}}^{+}(Z_{i})-occ_{t_{2}}^{+}(Y_{i})))+g(w_{v}(occ_{t_{2}}^{+}(Y_{i})-occ_{t_{1}}^{+}(X_{i})))$.

    Hence,

    $S_{v}(\theta _{v_{i}})\leq S_{v}(\theta _{1_{v_{i}}})+S_{v}(\theta _{2_{v_{i}}})$ and $S_{v}(\theta _{v})\leq S_{v}(\theta _{1_{v}})+S_{v}(\theta _{2_{v}})$.

  As a result,

  S_T(𝜃) ≤₂S_T(𝜃₁) + S_T(𝜃₂).

• 5. Under the same assumptions in 4, it is clear that $S_{f}(\theta _{2_{f}})\leq S_{f}(\theta _{f})$.

  • (a) $S_{f}(\theta _{2_{f}})<S_{f}(\theta _{f})$. It is obvious that S_T(𝜃₂) <₂S_T(𝜃).

  • (b) $S_{f}(\theta _{2_{f}})=S_{f}(\theta _{f})$. Suppose that 𝜃_v = {X₁↦Z₁,...,X_n↦Z_n}. ∀X_i↦Z_i ∈ 𝜃_v, there must exist substitutions $\{X_{i}\mapsto Y_{i}^{\prime }\}\subseteq \theta _{1_{v}}$ and $\{Y_{i}^{\prime }\mapsto Z_{i}\}\subseteq \theta _{2_{v}}$. Let $\theta _{2_{v}}^{\prime }=\bigcup _{i}\{Y_{i}^{\prime }\mapsto Z_{i} | occ_{t_{3}}^{+}(Z_{i})\geq occ_{t_{2}}^{+}(Y_{i}^{\prime })\}$, $\theta _{2_{v}}^{\prime \prime }=\bigcup _{j}\{Y_{j}^{\prime \prime }\mapsto Z_{j}^{\prime \prime } | Y_{j}^{\prime \prime } \in Ran(\theta _{1_{f}})\wedge occ_{t_{2}}^{+}(Y_{j}^{\prime \prime })=occ_{t_{3}}^{+}(Z_{j}) \}$. We can assert that $\theta _{2_{v}}=\theta _{2_{v}}^{\prime }\cup \theta _{2_{v}}^{\prime \prime }$. Hence,

    $S_{v}(\theta _{2_{v}})\leq S_{v}(\theta _{v})$.

  As a result,

  S_T(𝜃₂) ≤₂S_T(𝜃).

• 6. As shown in Fig. 4, t₁ and t₂ are two terms which can be unified, lgg(t₁,t₂) = t₃, and t₄ is a unified term of t₁ and t₂ (not need to be the most general unified term). For proving the conclusion of S_T(ρ₁) + S_T(ρ₂) ≤₂S_T(φ₁) + S_T(φ₂), we start to prove $S_{f}(\rho _{1_{f}})+S_{f}(\rho _{2_{f}})\leq S_{f}(\varphi _{1_{f}})+S_{f}(\varphi _{2_{f}})$ at first.

  • (a) $\rho _{1_{f}}=\boldsymbol {\emptyset }$ and $\rho _{2_{f}}=\boldsymbol {\emptyset }$. Obviously,

    $S_{f}(\rho _{1_{f}})+S_{f}(\rho _{2_{f}})\leq S_{f}(\varphi _{1_{f}})+S_{f}(\varphi _{2_{f}})$.

  • (b) $\rho _{1_{f}}\neq \boldsymbol {\emptyset }$ or $\rho _{2_{f}}\neq \boldsymbol {\emptyset }$. If $\rho _{1_{f}}\neq \boldsymbol {\emptyset }$, suppose that $Ran(\rho _{1_{f}})=\{u_{1}^{\prime }, ..., u_{h}^{\prime }\}$ and $Ran(\varphi _{2_{f}})=\{u_{1}, ..., u_{l}\}$. $\forall u_{i}^{\prime }\in Ran(\rho _{1_{f}})$, let $V_{t_{3}}(u_{i}^{\prime })=\{X_{i1}, ..., X_{ik}\}(k\geq 1)$ under ρ₁. $\forall X_{ij}\in V_{t_{3}}(u_{i}^{\prime })$, $occ_{t_{3}}(X_{ij})=n_{ij}^{\prime }$ and ${\sum }_{j=1}^{k}n_{ij}^{\prime }=n_{i}^{\prime }$. $\forall u_{r}\in Ran(\varphi _{2_{f}})$, let $V_{t_{2}}(u_{r})=\{Z_{r1}, ..., Z_{rs}\}(s\geq 1)$ under φ₂. $\forall Z_{rj}\in V_{t_{2}}(u_{r})$, $occ_{t_{2}}(Z_{rj})=n_{rj}$ and ${\sum }_{j=1}^{s}n_{rj}=n_{r}$. By the previous proof, we have $S_{f}(\rho _{1_{f}})={\sum }_{f\in \bigcup _{i=1}^{h}F(u_{i}^{\prime })}w_{f}(f)({\sum }_{i=1}^{h}n_{i}^{\prime }occ_{u_{i}^{\prime }}(f))$, $S_{f}(\varphi _{2_{f}})={\sum }_{f\in \bigcup _{r=1}^{l}F(u_{r})}w_{f}(f)({\sum }_{r=1}^{l}n_{r}occ_{u_{r}}(f))$.

    In ρ₁, $\forall X\in \bigcup _{i=1}^{m}V_{t_{3}}(u_{i}^{\prime })$, X is substituted by a functional term. Because t₁ and t₂ can be unified, X must be substituted by a variable in ρ₂. Hence, $\exists Z\in \bigcup _{r=1}^{l}V_{t_{2}}(u_{r})$, such that Xρ₁φ₁ = Xρ₂φ₂ = Zφ₂.

    That is, $\forall f\in \bigcup _{i=1}^{h}F(u_{i}^{\prime })$, there must exist some $u_{r}\in Ran(\varphi _{2_{f}})$ such that f ∈ u_r. Hence,

    $\bigcup _{i=1}^{h}F(u_{i}^{\prime })\subseteq \bigcup _{r=1}^{l}F(u_{r})$ ${\sum }_{i=1}^{h}n_{i}^{\prime }occ_{u_{i}^{\prime }}(f)\leq {\sum }_{r=1}^{l}n_{r}occ_{u_{r}}(f)$.

    As a consequence,

    $S_{f}(\rho _{1_{f}})\leq S_{f}(\varphi _{2_{f}})$.

    In the same way, if $\rho _{2_{f}}\neq \boldsymbol {\emptyset }$, we have

    $S_{f}(\rho _{2_{f}})\leq S_{f}(\varphi _{1_{f}})$.

    In a conclusion,

    $S_{f}(\rho _{1_{f}})+S_{f}(\rho _{2_{f}})\leq S_{f}(\varphi _{1_{f}})+S_{f}(\varphi _{2_{f}})$.

  Next, we would like to prove $S_{v}(\rho _{1_{v}})+S_{v}(\rho _{2_{v}})\leq S_{v}(\varphi _{1_{v}})+S_{v}(\varphi _{2_{v}})$ when $S_{f}(\rho _{1_{f}})+S_{f}(\rho _{2_{f}})=S_{f}(\varphi _{1_{f}})+S_{f}(\varphi _{2_{f}})$. In this case, we have

  $S_{f}(\rho _{1_{f}})=S_{f}(\varphi _{2_{f}})$ and $S_{f}(\rho _{2_{f}})=S_{f}(\varphi _{1_{f}})$,

  • (a) $S_{f}(\rho _{1_{f}})=S_{f}(\varphi _{2_{f}})=\boldsymbol {\emptyset }$ and $S_{f}(\rho _{2_{f}})=S_{f}(\varphi _{1_{f}})=\boldsymbol {\emptyset }$. We have

    $S_{v}(\rho _{1_{v}})\leq S_{v}(\varphi _{2_{v}})$, $S_{v}(\rho _{2_{v}})\leq S_{v}(\varphi _{1_{v}})$.

    Hence,

    $S_{v}(\rho _{1_{v}})+S_{v}(\rho _{2_{v}})\leq S_{v}(\varphi _{1_{v}})+S_{v}(\varphi _{2_{v}})$.

  • (b) $S_{f}(\rho _{1_{f}})=S_{f}(\varphi _{2_{f}}) \neq \boldsymbol {\emptyset }$ or $S_{f}(\rho _{2_{f}})=S_{f}(\varphi _{1_{f}})\neq \boldsymbol {\emptyset }$. If $S_{f}(\rho _{1_{f}})=S_{f}(\varphi _{2_{f}})\neq \boldsymbol {\emptyset }$, suppose that $Ran(\rho _{1_{f}})=\{u_{1}^{\prime }, ..., u_{h}^{\prime }\}$. Let $V_{t_{3}}(u_{i}^{\prime })=\{X_{i1}, ..., X_{ik}\}(k\geq 1)$ under ρ₁, we can assert that $\{X_{i1}\mapsto Y_{i1}, ..., X_{ik}\mapsto Y_{ik}\}\subseteq \rho _{2_{v}}$, where $occ_{t_{3}}^{+}(X_{ij})=occ_{t_{2}}^{+}(Y_{ij})(1\leq j \leq k)$. For any singleton element $\rho _{2_{v_{j}}}\in \rho _{2_{v}}/\bigcup _{i}\{X_{i1}\mapsto Y_{i1}, ..., X_{ik}\mapsto Y_{ik}\}$, Because t₁ and t₂ can be unified, there must exist a set $\varphi _{1_{v_{j}}}$ such that $S_{v}(\rho _{2_{v_{j}}})\leq S_{v}(\varphi _{1_{v_{j}}})$. Hence,

    $S_{v}(\rho _{2_{v}})\leq S_{v}(\varphi _{1_{v}})$.

    In the same way, if $S_{f}(\rho _{2_{f}})=S_{f}(\varphi _{1_{f}})\neq \boldsymbol {\emptyset }$, we have

    $S_{v}(\rho _{1_{v}})\leq S_{v}(\varphi _{2_{v}})$.

    In a conclusion,

    $S_{v}(\rho _{1_{v}})+S_{v}(\rho _{2_{v}})\leq S_{v}(\varphi _{1_{v}})+S_{v}(\varphi _{2_{v}})$.

  Hence,

  $$S_{T}(\rho_{1})+S_{T}(\rho_{2})\leq_{2} S_{T}(\varphi_{1})+S_{T}(\varphi_{2}).$$

□

Proof Proof of theorem 2

Given the function d_T, d_A must satisfy the properties 1 and 2 of pseudo-metric with no doubt. In the next proof, we only prove that d_A satisfies the triangle inequality.

• 1. A₁, A₂ are compatible.

  • (a) If A₃ is is compatible with A₁ and A₂, the triangle inequality is satisfied obviously.

  • (b) If A₃ is incompatible with A₁ and A₂, we have

    $d_{A}(A_{1},A_{3})=_{2}(+\infty ,+\infty )$, $\qquad ~~~~~ d_{A}(A_{2},A_{3})=_{2}(+\infty ,+\infty )$,

    and $d_{A}(A_{1},A_{3})+d_{A}(A_{2},A_{3})=_{2}(+\infty ,+\infty )$.

    Hence,

    $d_{A}(A_{1},A_{2})<_{2}(+\infty ,+\infty )=_{2}d_{A}(A_{1},A_{3})+d_{A}(A_{2},A_{3})$.

• 2. A₁, A₂ are incompatible.

  • (a) If A₃ is incompatible with A₁ and A₂, we have

    d_A(A₁,A₂) =₂d_A(A₁,A₃) + d_A(A₂,A₃).

  • (b) If A₃ is compatible with A₁ or A₂ (A₃ only compatible with one atom). Here, we assume that A₃ is compatible with A₁,

    $d_{A}(A_{1},A_{3})<_{2}(+\infty ,+\infty )$, $~~~~~d_{A}(A_{2},A_{3})=_{2}(+\infty ,+\infty )$,

    and

    $d_{A}(A_{1},A_{3})+d_{A}(A_{2},A_{3})=_{2}(+\infty ,+\infty )$.

    Hence,

    d_A(A₁,A₂) =₂d_A(A₁,A₃) + d_A(A₂,A₃).

In a conclusion,

d_A(A₁,A₂) ≤₂d_A(A₁,A₃) + d_A(A₂,A₃). □

Proof Proof of Theorem 3

Given the atom metric d_A, it is obvious that $d_{F_{m}}$ satisfies the properties 1 and 2. Next, we continue to prove $d_{F_{m}}$ satisfies the triangle inequality.

For a formula F with the corresponding atom set D, D_rF = {A|∃B ∈ D,d_A(A,B) ≤ r} is a set of atoms that are at most r far away from one atom in F. observe that

$d_{F_{m}}(F_{1},F_{2})=min\{r|D_{1}\subseteq D_{r}F_{2} \wedge D_{2}\subseteq D_{r}F_{1}\}$

Given a formula F₃ with the corresponding atom set D₃, if

$D_{3}\subseteq D_{r}F_{1}\wedge D_{1}\subseteq D_{r}F_{3} \wedge D_{3}\subseteq D_{s}F_{2}\wedge D_{2}\subseteq D_{s}F_{3}$,

then

∀A ∈ D₁,∃C ∈ D₃, (d_A(A,C) ≤ r ∧∃B ∈ D₂,d_A(C,B) ≤ s).

In this case, $D_{1}\subseteq D_{r+s}F_{2}$. We can also prove $D_{2}\subseteq D_{r+s}F_{1}$.

Fig. 4

Parallelogram structure

Hence, if $d_{F_{m}}(F_{1},F_{3})\leq r$ and $d_{F_{m}}(F_{3},F_{2})\leq s$, then $d_{F_{m}}(F_{1},F_{2})\leq r+s$, and consequently $d_{F_{m}}(F_{1},F_{2}) \leq d_{F_{m}}(F_{1}, F_{3}) + d_{F_{m}}(F_{3}, F_{2})$. □