On some determinants involving the tangent function

Abstract

Let p be an odd prime and let $a,b\in {\mathbb {Z}}$ with $p\not \mid ab$. In this paper,we mainly evaluate

$$\begin{aligned} T_p^{(\delta )}(a,b,x):=\det \left[ x+\tan \pi \frac{aj^2+bk^2}{p}\right] _{\delta \leqslant j,k\leqslant (p-1)/2}\ \ (\delta =0,1). \end{aligned}$$

For example, in the case $p\equiv 3\ ({\textrm{mod}}\ 4)$, we show that $T_p^{(1)}(a,b,0)=0$ and

$$\begin{aligned} T_p^{(0)}(a,b,x)={\left\{ \begin{array}{ll} 2^{(p-1)/2}p^{(p+1)/4}&{}\text {if}\ (\frac{ab}{p})=1, \\ p^{(p+1)/4}&{}\text {if}\ (\frac{ab}{p})=-1,\end{array}\right. } \end{aligned}$$

where $(\frac{\cdot }{p})$ is the Legendre symbol. When $(\frac{-ab}{p})=-1$, we also evaluate the determinant $\det [x+\cot \pi \frac{aj^2+bk^2}{p}]_{1\leqslant j,k\leqslant (p-1)/2}.$ In addition, we pose several conjectures one of which states that for any prime $p\equiv 3\ ({\textrm{mod}}\ 4)$, there is an integer $x_p\equiv 1\ ({\textrm{mod}}\ p)$ such that

$$\begin{aligned}\det \left[ \sec 2\pi \frac{(j-k)^2}{p}\right] _{0\leqslant j,k\leqslant p-1}=-p^{(p+3)/2}x_p^2.\end{aligned}$$