Computing tighter bounds on the n-queens constant via Newton’s method

Abstract

In recent work Simkin shows that bounds on an exponent occurring in the famous n-queens problem can be evaluated by solving convex optimization problems, allowing him to find bounds far tighter than previously known. In this note we use Simkin’s formulation, a sharper bound developed by Knuth, and a Newton method that scales to large problem instances, to find even sharper bounds.

Appendices

Appendix

A Details of convex problems

In this appendix we give the details of the lower and upper bound problems, as well as an approximate upper bound problem. We define the variables in their natural notation, leaving it to the reader to re-arrange these into a single vector variable x. In a similar way, we describe the linear constraints in their natural notation, leaving it to the reader to translate these into $Ax=b$.

Fig. 1

A chessboard with all of its triangles labeled. This chessboard is used to interpret the $n=8$ problem. The red line represents one of the diagonals of the chessboard. The blue line represents one of the anti-diagonals of the chessboard

1.1 A.1 Common notation

In this section we describe variables and notation that are shared by the lower and upper bound problems.

Chessboard triangle variables. In both problems, the variable x consists of 4 $n \times n$ matrices N, E, S, W, and some additional slack variables. The i, jth entry in N, E, S, W is interpreted as a value associated with the North, East, South, or West triangle, respectively, formed by dividing each square of an $n\times n$ chessboard into 4 right triangles. We index these matrices starting at 0, diverging from the notation used in [1, 5] which use indexing that begins at 1. Figure 1 shows the $n=8$ case.

Diagonal sum operators. We introduce operators ${\mathcal {D}}_k : \mathbf {R}^{n\times n} \rightarrow \mathbf {R}$ and ${\mathcal {A}}_k : \mathbf {R}^{n\times n} \rightarrow \mathbf {R}$ defined for $k \in \{-n, -n+1, -n +2, \ldots , -1, 0, 1, \ldots , n-2, n-1, n\}$, where ${\mathcal {D}}_k(Z)$ is the sum of the kth diagonal of Z, and ${\mathcal {A}}_k(Z)$ is the sum of the kth anti-diagonal of Z. For example ${\mathcal {D}}_0 Z = \sum _{i=0}^{n-1} Z_{ii} = \mathbf{Tr}{Z}$, ${\mathcal {D}}_1 Z = \sum _{i=1}^{n-1} Z_{i,i-1}$, and ${\mathcal {A}}_{-1} Z = \sum _{i=1}^{n-1} Z_{n-i,i}$. Note that ${\mathcal {D}}_{n} Z ={\mathcal {D}}_{-n} Z = 0$. These are illustrated in Fig. 1: ${\mathcal {D}}_{-2} N$ is the sum of the entries in the North triangles the red line passes through, and ${\mathcal {A}}_1 E$ is the sum of the entries in the East triangles the blue line passes through.

Negative entropy. Following [5], we define the function $g:\mathbf {R}_+\rightarrow \mathbf {R}$ as $g(x)=x \log x$ for $x>0$, and $g(0) = 0$. (This is the negative entropy function [7, p.72].)

1.2 A.2 Lower bound problem

This problem formulation is taken from [1, Claim 6.3], except that Simkin maximizes a concave function and we minimize its negative, a convex function.

Slack variables We introduce the following slack variables,

$$\begin{aligned} d_k = \frac{1}{n} - {\mathcal {D}}_{k} (S+W) - {\mathcal {D}}_{k+1} (N+E), \quad k \in \{-n, -n+1, \ldots , n - 1\}, \end{aligned}$$

and

$$\begin{aligned} a_k = \frac{1}{n} - {\mathcal {A}}_{k} (S+E) - {\mathcal {A}}_{k+1} (N+W), \quad k \in \{-n, -n+1, \ldots , n - 1\}. \end{aligned}$$

The quantities $d_k$ and $a_k$ are the sums along the diagonals and anti-diagonals, respectively, of the chessboard. In Fig. 1, $d_{-1}$ includes contributions from all triangles the red line passes through and $a_{0}$ includes contributions from triangles the blue line passes through.

These equations form 4n entries in A, b.

Objective. The objective function is

$$\begin{aligned}&\sum _{i=0}^{n-1} \sum _{j=0}^{n-1} \left( g(N_{i,j}) + g(E_{i,j}) + g(S_{i,j}) + g(W_{i,j})\right) \\&\quad + \sum _{k=-n}^{n-1} \left( g(d_k) + g(a_k)\right) + 4 \log n + 2 \log 2 + 3. \end{aligned}$$

Constraints. Simkin also introduces the constraints

$$\begin{aligned} \sum _{j=0}^{n-1} N_{i,j}+E_{i,j}+S_{i,j}+W_{i,j} = \frac{1}{n}, \quad i \in \{0, 1, \ldots , n-1\}, \end{aligned}$$

and

$$\begin{aligned} \sum _{i=0}^{n-1} N_{i,j}+E_{i,j}+S_{i,j}+W_{i,j} = \frac{1}{n}, \quad j \in \{0, 1, \ldots , n-1\}. \end{aligned}$$

These constraints are linearly dependent, with co-rank one, so we delete the first constraint with $i = 0$ to obtain a total of $2n - 1$ constraints that we include in A, b.

Properties. This problem has $n^2$ entries in each of N, E, S, W and 2n entries in each of d, a. Accordingly, the total number of variables is $p=4n^2 + 4n$. We have 4n constraints affecting the slack variables, and $2n - 1$ constraints affecting only N, E, S, W for a total of $q= 6n - 1$ constraints.

The objective is a sum of the negative entropy of individual optimization variables, making it separable and strictly convex.

The constraint with the most variables are the row and column constraints, which involve 4n variables. Each triangle is in at most 1 column constraint, 1 row constraint, 1 diagonal constraint, and 1 anti-diagonal constraint. Therefore, each column of A can have at most 4 entries.

1.3 A.3 Upper bound problem

We use Knuth’s formulation of the Xqueenon problem [5], except that he maximizes a concave function and we minimize its negative, a convex function.

Slack variables. We introduce the slack variables

$$\begin{aligned} d^{SW}_{k}= & {} 1 -\frac{1}{2n} {\mathcal {D}}_{k}\left( S + W\right) , \quad k \in \{-n+1, -n+2, \ldots , n-1\},\\ d^{NE}_{k}= & {} 1 -\frac{1}{2n} {\mathcal {D}}_{k}\left( N + E\right) , \quad k \in \{-n+1, -n+2, \ldots , n -1\},\\ a^{SE}_{k}= & {} 1 -\frac{1}{2n} {\mathcal {A}}_{k}\left( S + E\right) , \quad k \in \{-n+1, -n+2, \ldots , n-1\}, \end{aligned}$$

and

$$\begin{aligned} a^{NW}_{k} = 1 -\frac{1}{2n} {\mathcal {A}}_{k}\left( N + W\right) , \quad k \in \{-n+1, -n+2, \ldots , n -1\}. \end{aligned}$$

These equations form $8n - 4$ entries in A, b.

Objective. For ease of notation, let

$$\begin{aligned} d^{SW}_{-n} = d^{NE}_{n} = a^{SE}_{-n} = a^{NW}_{n} = 1. \end{aligned}$$

Our objective function is

$$\begin{aligned} 3 + L_0(N, E, S, W) + L_-(d^{SW}, d^{NE}) + L_+(a^{SE}, a^{NW}), \end{aligned}$$

where

$$\begin{aligned}&L_0(N, E, S, W) = \frac{1}{4n^2} \sum _{i=0}^{n-1}\sum _{j=0}^{n-1}\\&\quad \left( g(N_{i,j}) + g(E_{i,j}) +g(S_{i,j}) +g(W_{i,j})\right) ,\\&\quad L_-(d^{SW}, d^{NE}) = \frac{1}{n}\sum _{k=-n+1}^{n} \int _{0}^1 g\left( (1 - y)d_{k-1}^{SW} + y d_{k}^{NE}\right) \,dy, \end{aligned}$$

and

$$\begin{aligned} L_+(a^{SE}, a^{NW}) = \frac{1}{n}\sum _{k=-n+1}^{n} \int _{0}^1 g\left( (1 - y)a_{k-1}^{SE} + y a_{k}^{NW}\right) \,dy. \end{aligned}$$

Using a symbolic solver, we were able to generate closed-form expressions for the integrals and their partial derivatives [13].

In order to make the matrix block-diagonal, $d^{SW}_k$ and $d^{NE}_k$ must be interleaved in x. Similar interleaving applies to $a^{SE}_k$ and $a^{NW}_k$.

Constraints. In addition to the $8n - 4$ equations involving the slack variables, Knuth requires the following conditions on N and S,

$$\begin{aligned} \sum\limits_{{j = 0}}^{{n - 1}} {N_{{i,j}} } = & n,\quad i \in \{ 0,1, \ldots ,n - 1\} , \\ \sum\limits_{{j = 0}}^{{n - 1}} {S_{{i,j}} } = & n,\quad i \in \{ 0,1, \ldots ,n - 1\} , \\ \end{aligned}$$

and

$$\begin{aligned} \sum _{i=0}^{n-1} N_{i,j} + S_{i,j} = 2n, \quad j \in \{0, 1, \dots , n-1\}. \end{aligned}$$

As any of these equations are linearly dependent on all the others, we choose to eliminate the first column constraint on N.

On E and W, Knuth requires

$$\begin{aligned} \sum\limits_{{i = 0}}^{{n - 1}} {E_{{i,j}} } = & n,\quad j \in \{ 0,1, \ldots ,n - 1\} , \\ \sum\limits_{{i = 0}}^{{n - 1}} {W_{{i,j}} } = & n,\quad j \in \{ 0,1, \ldots ,n - 1\} , \\ \end{aligned}$$

and

$$\begin{aligned} \sum _{j=0}^{n-1} E_{i,j} + W_{i,j} = 2n, \quad i \in \{0, 1, \dots , n-1\}. \end{aligned}$$

As with N and S, one of these equations is linearly dependent, and we choose to eliminate the first row constraint on E.

Properties. This problem has $n^2$ entries in each of N, E, S, W and $2n- 1$ entries in each of $d^{SW}_k, d^{NE}_k, a^{SE}_k, a^{NW}_k$. This forms a total of $p = 4n^2 + 8n - 4$ variables. We also have the $8n - 4$ constraints involving the slack variables and $6n - 2$ of the other constraints for a total of $q = 14n - 6$ constraints. The objective is block separable as each variable appears in only one term of the objective function and no term has more than two variables. The rows of A with the most entries are the entries along the diagonal and anti-diagonal, which contain 2n entries of N, E, S, W and 1 slack variable. Each column of A has at most 4 non-zero entries: 1 from its row constraint, 1 from its column constraints, 1 from its diagonal term, and 1 from its anti-diagonal term. Columns associated with slack variables have one non-zero entry.

1.4 A.4 Approximate upper bound problem

In the initial phase of computing $U_n$ we solve a problem with a diagonal Hessian that approximates the upper bound problem. We do this by applying Jensen’s inequality to the integrals in the diagonal and anti-diagonal terms of the objective.

After applying this approximation, the integral terms of the objective function become

$$\begin{aligned} g\left( \frac{1}{2}d^{SW}_{k-1} +\frac{1}{2}d^{NE}_k\right) , \end{aligned}$$

and

$$\begin{aligned} g\left( \frac{1}{2}a^{SE}_{k-1} +\frac{1}{2}a^{NW}_k\right) . \end{aligned}$$

We introduce new slack variables $d_k = \frac{1}{2}d^{SW}_{k-1} +\frac{1}{2}d^{NE}_k$ and $a_k = \frac{1}{2}a^{SE}_{k-1} +\frac{1}{2}a^{NW}_k$ and then replace the integral terms with $g(d_k)$ and $g(a_k)$ appropriately.

All other constraints and terms of the objective function are the same.