INTRODUCTION

The paper concerns the so-called classical decision problem [1], in particular, the algorithmic complexity of fragments of elementary theories [2, 3]. Below we deal with theories of a binary predicate, both in languages with a binary predicate letter only, and those enriched with some additional tools.

The classical logic of a binary predicate is undecidable [4, Chapter 21]. Note that the corresponding proof [4] uses infinitely many individual variables; at the same time, to prove that the first-order logic is undecidable, it suffices to use only three individual variables and, alongside a binary predicate letter, infinitely many unary letters [5]. If we only allow one of the following: only two variables [6, 7], only unary predicated letters and equality [4, Chapter 21], or only guarded formulas (where binary letters are allowed only with certain limitations) [8], then we obtain decidable fragments. A natural question, thus, arises concerning decidability of the logic and theories of a single binary predicate in languages with a finite number (at least, three) of variables.

A similar situation applies to first-order languages enriched with operators motivated by applications. For example, the first-order logic with equality and the transitive closure operator is \(\Pi _{1}^{1}\)-hard in languages with two variables, but the proof uses several binary predicate letters and infinitely many unary ones [9]; again, the question arises concerning the computational complexity of theories of a binary predicate with a finite number (in this case, at least two) of variables.

The answer to the question about the decidability of the logic of a binary predicate in languages with at least three variables (and without additional operators) follows from [10]: it is undecidable (see [10, clause (ii) of Section 4.8]). Similar results for the classical logic of a binary predicate in languages with two variables, enriched with additional operators, are unknown to the author.

We shall describe a construction providing us with a short proof, firstly, of the undecidability of many theories of a binary predicate in languages with three individual variables (in particular, \(\Sigma _{1}^{0}\)-completeness of the logic of a binary predicate and \(\Pi _{1}^{0}\)-completeness of the theory of finite models of a binary predicate), and secondly, \(\Pi _{1}^{1}\)-hardness of the validity problem for languages with a binary predicate letter, equality, and two individual variables, enriched with the operators of composition and transitive closure. The construction will consist of modelling tiling problems [11, 12]; note that this is a well-known method, with applications both in algebra [13, 14] and in logic [1, 15–18].

An undecidable tiling problem. Tiles are squares, all of the same size; type \(t\) of a tile is determined by the colors \(lt\), \(rt\), \(ut\), and \(dt\) of its edges. The following tiling problem is \(\Pi _{1}^{0}\)-complete [11]: given a set T = {t0, …, tn} of tile types, we are to determine if there exists a T‑tiling, i.e., a map \(f:\mathbb{N} \times \mathbb{N} \to T\) such that \(rf(i,j)\, = \,lf(i + 1,j)\) and \(uf(i,j) = df(i,j + 1)\), for every \(i,j \in \mathbb{N}\).

Modelling a tiling. Fix a binary predicate letter P and introduce the following abbreviations: \(rx = xPx\), \(\bar {r}x = \neg rx\), \(gx = \exists y{\kern 1pt} (ry \wedge xPy)\), \(\bar {g}x = \neg gx\), and

$$\begin{gathered} xEy = \forall z{\kern 1pt} ((xPz \leftrightarrow yPz) \wedge (zPx \leftrightarrow zPy)); \\ xSy = xPy \wedge \neg xEy \wedge \forall z{\kern 1pt} (xPz \wedge zPy \to zEx \vee zEy); \\ xHy = xSy \wedge (rx \leftrightarrow ry); \\ xVy = xSy \wedge (rx \leftrightarrow \bar {r}y). \\ \end{gathered} $$

If rx, then x is reflexive, and if \(\bar {r}x\), then x is irreflexive; \(gx\) means that x belongs to an \(\mathbb{N} \times \mathbb{N}\) grid, i.e., x is a tile-holder; \(E\) induces a congruence; \(S\) means a single \(P\)-step; and \(H\) and \(V\) are understood as moving, respectively, rightwards and upwards. To obtain a definition of \(xSz\), replace, in the definition of \(xSy\), both \(y\) with \(z\) and \(z\) with \(y\); analogously for \(ySz\), \(zSy\), \(xEz\), etc. For a property \(u\), let \({{\forall }_{u}}x{\kern 1pt} A = \forall x{\kern 1pt} (ux \to A)\) and \({{\exists }_{u}}x{\kern 1pt} A\) = \(\exists x{\kern 1pt} (ux \wedge A)\).

Define \(G\) be the conjunction of the formulas  \(\forall x\forall y\forall z{\kern 1pt} (xPy \wedge yPz \to \) xPz), \({{\forall }_{g}}x{{\exists }_{g}}y{\kern 1pt} xHy\), \({{\forall }_{g}}x{{\exists }_{g}}y{\kern 1pt} xVy\), \({{\forall }_{g}}x{{\forall }_{g}}y\) \((\exists z{\kern 1pt} (xHz \wedge zVy)\) \( \leftrightarrow \) \(\exists z{\kern 1pt} (xVz\) \( \wedge \) \(zHy))\), and \({{\exists }_{g}}x{\kern 1pt} \bar {r}x\). It should be clear that if \(G\) is true in a model \(M\), then \(M\) contains an \(\mathbb{N} \times \mathbb{N}\) grid whose rows are determined by H and columns by \(V\); the elements of the first row are irreflexive, the elements of the second row are reflexive, the the elements of the third row are irreflexive, etc.

Let \({{h}^{y}}x = \neg \exists y{\kern 1pt} xPy\), \(t_{0}^{y}x\) = \({{\exists }_{{\bar {g}}}}y{\kern 1pt} (xSy \wedge {{h}^{x}}y)\), \(t_{{k + 1}}^{y}x\) = \({{\exists }_{{\bar {g}}}}y{\kern 1pt} (xSy \wedge t_{k}^{x}y)\), where \(k \in \mathbb{N}\). Formula \(t_{m}^{y}x\) says that \(x\) sees an \(S\)-successor outside the grid, and that the said successor sees a dead end in \(m\) \(S\)-steps. We write \({{t}_{m}}x\) and \({{t}_{m}}y\) instead of \(t_{m}^{y}x\) and \(t_{m}^{x}y\). The intended meaning of \({{t}_{m}}x\) is that x contains a tile of type \({{t}_{m}}\).

Define \({{t}_{T}}\) be the conjunction of the following formulas:

$$\begin{gathered} {{\forall }_{g}}x\;\mathop \vee \limits_{i = 0}^n \;({{t}_{i}}x \wedge \;\mathop \wedge \limits_{j \ne i} \;\neg {{t}_{j}}x); \\ {{\forall }_{g}}x\;\mathop \vee \limits_{i = 0}^n \;({{t}_{i}}x \to {{\forall }_{g}}y{\kern 1pt} (xHy \to \;\mathop \vee \limits_{{\text{r}}{{t}_{i}} = {\text{l}}{{t}_{j}}} \;{{t}_{j}}y)); \\ {{\forall }_{g}}x\;\mathop \vee \limits_{i = 0}^n \;({{t}_{i}}x \to {{\forall }_{g}}y{\kern 1pt} (xVy \to \;\mathop \vee \limits_{{\text{u}}{{t}_{i}} = {\text{d}}{{t}_{j}}} \;{{t}_{j}}y)). \\ \end{gathered} $$

Lemma 1. \(G \wedge {{t}_{T}}\) is satisfiable \( \Leftrightarrow \) there exists a T-tiling.

Observe that, in the arguments given above, we can make do with positive formulas only: first, replace \(G \wedge {{t}_{T}}\) with its negation and the satisfiability problem with the refutability problem; then, replace every negation with implication to the formula \(\forall x\forall y{\kern 1pt} xPy\).

Theories of a binary predicate. By Lemma 1 and the observation concerning positive formulas, we obtain the following refinement of the Church’s theorem [19]:

• the positive fragment of the classical predicate logic is \(\Sigma _{1}^{0}\)-complete in the language with a single binary predicate letter and three individual variables.

Theorem 1 can be extended to theories of a binary predicate: it suffices to make minor adjustments in the above-given encoding. For a class \(C\) of models of a binary predicate, let us denote by \(Th(C)\) the elementary theory of the class \(C\). Let \(F\), \(If\), \(R\), \(Ir\), \(S\), \(As\), \(T\), and \(It\) be the classes of, respectively, finite, infinite, reflexive, irreflexive, symmetric, antisymmetric, transitive, and intransitive models of a binary predicate. If \(X\) and \(Y\) are classes of models, then we will write \(XY\) rather than \(X \cap Y\). So, \(IfRST\) is the class of infinite models that are reflexive, symmetric, and transitive.

Theorem 1. Let \(C\) be a class of models of a binary predicate containing at least one of the following classes: \(IfIrT\), \(IfIt\), \(IfRS\), \(IfIrS\). Then the positive fragment of \(Th(C)\) is \(\Sigma _{1}^{0}\)-hard in the language with three individual variables.

As a corollary, the elementary theories of the classes If, \(R\), \(Ir\), \(S\), \(As\), \(T\), and \(It\) are not decidable in languages with three individual variables.

Using a similar argument, we obtain the following refinement of Trakhtenbrot’s theorem [20, 22]:

• the positive fragment of the theory of finite models is \(\Pi _{1}^{0}\)-complete in languages with a single binary predicate letter and three individual variables.

To prove this statement, it is enough to notice that one can encode with a suitable tiling problem the problem of non-termination of Turing machines on the empty tape, and then to adjust the formulas given above so that they say that the first row of a tiling corresponds to the initial configuration of a Turing machine on the empty tape and that the initial tiling segment does not contain a tile corresponding to a final state of the machine; note that another possible way is to use the effective inseparability [21] and Theorem 1.

A more general statement is also true.

Theorem 2. Let \(C\) be a class of finite models of a binary predicate containing \(FR\) or \(FIrAsT\). Then the positive fragment of \(Th(C)\) is \(\Pi _{1}^{0}\)-hard in languages with three individual variables.

Enriching the language. We shall now show that logics and theories in enriched languages can be highly undecidable, even with only two variables. Consider the tiling problem where the \(T\)-tiling f is required to additionally satisfy the condition that the set \(\{ j \in \mathbb{N}:f(0,j) = {{t}_{0}}\} \) is infinite. This problem is known to be \(\Sigma _{1}^{1}\)-complete [12, Theorem 6.4].

Extend the language of first-order logic with the transitive closure operator and denote by \({{P}^{ + }}\) the transitive closure of \(P\). For a formula \(A\), define \(A'\) to be the formula obtained from A by uniformly replacing \(P\) with \({{P}^{ + }}\). Let \(lx = \neg \exists y{\kern 1pt} yH'x\). Define G* be the conjunction of the formulas \(\forall x\forall y{\kern 1pt} (xPy \wedge \neg yPx \to xS{\kern 1pt} '{\kern 1pt} z)\), \({{\forall }_{{g{\kern 1pt} '{\kern 1pt} }}}x{{\exists }_{{g{\kern 1pt} '{\kern 1pt} }}}y{\kern 1pt} xH{\kern 1pt} '{\kern 1pt} y\), \({{\forall }_{{g{\kern 1pt} '{\kern 1pt} }}}x{{\exists }_{{g{\kern 1pt} '{\kern 1pt} }}}y{\kern 1pt} xV{\kern 1pt} '{\kern 1pt} y\), \({{\forall }_{{g{\kern 1pt} '{\kern 1pt} }}}x{{\exists }_{{g{\kern 1pt} '{\kern 1pt} }}}y{\kern 1pt} (\exists z{\kern 1pt} (xH{\kern 1pt} '{\kern 1pt} z \wedge zV{\kern 1pt} '{\kern 1pt} y)\) \( \leftrightarrow \) \(\exists z{\kern 1pt} (xV{\kern 1pt} '{\kern 1pt} z\) \( \wedge \) \(zH{\kern 1pt} '{\kern 1pt} y))\), \({{\exists }_{{g{\kern 1pt} '{\kern 1pt} }}}x{\kern 1pt} (\bar {r}x \wedge lx)\), \({{\forall }_{{g{\kern 1pt} '{\kern 1pt} }}}x{\kern 1pt} (lx \to \) \({{\forall }_{{g{\kern 1pt} '{\kern 1pt} }}}\)y(xVy → ly)), and \(t_{T}^{ * }\) to be the conjunction of \(t_{T}^{'}\) and the formula \({{\forall }_{{g'}}}x{\kern 1pt} (lx \to {{\exists }_{{g'}}}y{\kern 1pt} (\neg xE{\kern 1pt} '{\kern 1pt} y \wedge x{{P}^{ + }}y \wedge ly \wedge t_{0}^{'}y))\). Then, \(G{\text{*}} \wedge t_{T}^{ * }\) is satisfiable if, and only if, there exists a \(T\)-tiling with the extra condition. This proves \(\Pi _{1}^{1}\)-hardness of a number of theories of a binary predicate letter with three variables; note that the transitive closure operator was applied only to atomic formulas.

If, in addition, we have equality and composition in the language, then, using ideas from [9], we can describe the same tiling problem with formulas containing only two variables and only one binary predicate letter. The third variable is used in the definitions of \(E\) and \(S\), as well as in one of the conjuncts of the formula \(G\). Replace \(E\) with equality. Using equality and the transitive closure operator, one can define the following properties of a binary relation: functionality, surjectivity, and disjointness of its domain and its range, see [9]. This allows us to divide steps alongside \(H\) and \(V\) into “even” steps \({{H}_{0}}\) and \({{V}_{0}}\) and “odd” steps \({{H}_{0}}\), \({{H}_{1}}\), \({{V}_{0}}\), and \({{V}_{1}}\); also, we can use \(P\) instead of \(S\) in \({{t}_{m}}x\); then, there is no need for \(S\) any more. The conjunct of \(G\) with three variables can be replaced with \({{\forall }_{{g'}}}x{\kern 1pt} {{\exists }_{{g'}}}y{\kern 1pt} (x[H \circ V]y \wedge x[V \circ H]y)\), where xHy = \(x{{H}_{0}}y \vee x{{H}_{1}}y\) and \(xVy = x{{V}_{0}}y \vee x{{V}_{1}}y\). Again, we obtain \(\Pi _{1}^{1}\)-hardness of theories, but this time with two variables.

As a result, we obtain the following theorem.

Theorem 3. The validity problem for languages with two individual language, binary predicate letter, equality and the operators of composition and transitive closure is \(\Pi _{1}^{1}\)-hard.

Note that the validity problem for such languages in the class of all finite models is in the class \(\Pi _{1}^{0}\), since it is possible to effectively enumerate both all formulas and all finite models (up to isomorphism), which makes it possible to construct an effective enumeration of the set of refutable formulas.

DISCUSSION

Note that slightly weakened versions of Theorems 1 and 2 can be obtained from [10] in view of the results presented in [2, 3, 22] and in other papers (see, for example, [23; 24, Appendix]). Thus, in [10], the undecidability of the logic of a binary predicate in the language with three individual variables is proved, and using the translations from [2, 3], one can obtain undecidability (and, in view of [22], \(\Sigma _{1}^{0}\)-hardness or \(\Pi _{1}^{0}\)-hardness) of various theories of a binary predicate in languages with a finite (sometimes, perhaps, quite large) number of individual variables; the idea is to eliminate some variables in translations when nested quantifiers appear in formulas, roughly as in the definition of formulas \({{t}_{m}}(x)\) above, i.e., to reuse a variable \(x\) in the recursive clause for a formula \(\varphi \) whenever \(x\) does not occur freely in \(\varphi \), rather than introducing a new variable.

We also note that a lot of attention has been devoted (see, for example, [1, 3]) to the study of the computational properties of elementary fragments defined by quantifier prefixes from some regular set; if the said set is infinite, then it contains arbitrarily long quantifier prefixes; hence, the corresponding fragment of the language contains infinitely many individual variables. Sets of quantifier prefixes leading to undecidability of the logic of a binary predicate determined are infinite. Thus, a natural question arises: is it possible to extract from the above construction a proof of the undecidability of some fragment of the logic of a binary predicate defined by a finite set of quantifier prefixes? The answer is negative: the formulas \(t_{m}^{y}(x)\) use nested quantifiers for the variables x and y, and their quantifier depth depends on \(m\); therefore, converting formulas of the form \({{t}_{T}}\) to prefix normal form and increasing the number of elements in T, we get an increase in the length of the quantifier prefix, which leads to an increase in the number of variables in the resultant formula.