1 Introduction and Definitions

Let \({\mathbb {D}}:=\{ z \in {\mathbb {C}} : |z|<1 \}\) and \(\overline{\mathbb {D}}:=\{z\in {\mathbb {C}}: |z|\le 1\},\) and for \(r>0,\) let \({\mathbb {T}}_r:=\{ z \in {\mathbb {C}} : |z|=r\}\) and \({\mathbb {T}}:={\mathbb {T}}_1.\) Denote by \({\mathcal {H}}\) be the class of all analytic functions f in \(\mathbb {D}\) and by \({\mathcal {A}}\) the subclass of \({\mathcal {H}}\) with f normalized such that \(f(0)=0\) and \(f'(0)=1,\) so that f(z) is of the form

$$\begin{aligned} f(z)= \sum _{n=1}^{\infty }a_nz^n, \quad a_1:=1,\ z\in \mathbb {D}. \end{aligned}$$
(1)

Let \({\mathcal {S}}\) be the subclass of \({\mathcal {A}}\) consisting of univalent functions.

For \(q,n\in {\mathbb {N}},\) consider the matrix \(T_{q,n}(f)\) with \(f\in {\mathcal {A}}\) given by (1) defined by

$$\begin{aligned} T_{q,n}(f):=\left[ \begin{matrix} a_n&{} a_{n+1}&{}\ldots &{} a_{n+q-1}\\ {\overline{a}}_{n+1}&{} a_n&{}\ldots &{} a_{n+q-2}\\ \vdots &{} \vdots &{} \vdots &{}\vdots \\ {\overline{a}}_{n+q-1}&{}{\overline{a}}_{n+q-2}&{}\ldots &{}a_n\end{matrix}\right] , \end{aligned}$$

where \({\overline{a}}_k:=\overline{a_k}.\) In the case when \(a_n\) is a real number, \(T_{q,n}(f)\) is called an Hermitian Toeplitz matrix.

In recent years, a great many papers have been devoted to the estimation of determinants whose entries are coefficients of functions in \({\mathcal {A}}\) or its subclasses. Hankel matrices, i.e., square matrices which have constant entries along the reverse diagonal and the generalized Zalcman functional \(J_{m,n}(f):=a_{m+n-1}-a_ma_n,\ m,n\in \mathbb {N},\) are of particular interest (see, e.g., [5, 6, 8, 13, 15, 16, 18,19,20, 25]). Also of interest are the determinants of symmetric Toeplitz matrices, the study of which was initiated in [1].

In [9, 11, 14], research was investigated into the study of Hermitian Toeplitz determinants whose entries are the coefficients of functions in subclasses of \({\mathcal {A}}.\)

In this paper, we continue this research by computing the sharp upper and lower bounds of the second- and third-order Hermitian Toeplitz determinants over some subclasses of close-to-convex functions, but first noting that the following general result was proved in [11].

Theorem 1

([11]) Let \({\mathcal {F}}\) be a subclass of \({\mathcal {A}}\) such that \(\{f\in {\mathcal {F}}: a_2=0\}\not =\emptyset \) and \(A_2({\mathcal {F}}):=\max \{|a_2|: f\in {\mathcal {F}}\}\) exists. Then

$$\begin{aligned} 1-A_2^2({\mathcal {F}})\le \det T_{2,1}(f) \le 1. \end{aligned}$$

Both inequalities are sharp.

We next define the classes of close-to-convex functions considered in this paper. First denote by \({\mathcal {S}}^*\) the subclass of \({\mathcal {S}}\) consisting of the starlike functions, i.e., \(f\in {\mathcal {S}}^*\) if and only if \(f\in {\mathcal {A}}\) and

$$\begin{aligned} {{\,\mathrm{Re}\,}}\frac{zf'(z)}{f(z)}>0,\quad z\in {\mathbb {D}}. \end{aligned}$$

A function \(f\in {\mathcal {A}}\) is called close-to-convex if there exist \(g\in {\mathcal {S}}^*\) and \(\delta \in \mathbb {R}\) such that

$$\begin{aligned} {{\,\mathrm{Re}\,}}\frac{\mathrm {e}^{\mathrm {i}\delta }zf'(z)}{g(z)}>0,\quad z\in {\mathbb {D}}. \end{aligned}$$
(2)

The class \({\mathcal {C}}\) of all close-to-convex functions (which is necessarily a subclass \({\mathcal {S}}\)), was introduced by Kaplan [12] (see also [10, Vol. II, p. 3]), where the following geometrical interpretation was given: \(f\in {\mathcal {A}}\) is close-to-convex if and only if there are no sections of the curve \(f(\mathbb {T}_r),\) for every \(r\in (0,1),\) in which tangent vector turns backward through an angle not less than \(\pi \) (cf. [10, Vol. II, p. 4]). Lewandowski [22, 23] proved that the class of close-to-convex functions is identical with the class of linearly accessible functions introduced by Biernacki [3].

Given \(g\in {\mathcal {S}}^*\) and \(\delta \in {\mathbb {R}},\) let \({\mathcal {C}}_\delta (g)\) be the subclass of \({\mathcal {C}}\) of all f satisfying (2). The four classes \({\mathcal {C}}_0(g_i),\ i=1,\ldots ,4,\) where

$$\begin{aligned} g_1(z):=\frac{z}{1-z^2},\quad g_2(z):=\frac{z}{(1-z)^2},\quad g_3(z):=\frac{z}{1+z+z^2} \end{aligned}$$

and

$$\begin{aligned} g_4(z):=\frac{z}{1-z},\quad z\in {\mathbb {D}}, \end{aligned}$$

are particularly interesting and have been studied by various authors (e.g., [2, 7, 17]). In [14], the sharp bounds of the second- and third-order Hermitian Toeplitz determinants were found for the classes \({\mathcal {C}}_0(g_1)\) and \({\mathcal {C}}_0(g_2).\) In this paper, we will do the same for the other two classes, i.e., for \({\mathcal {C}}_0(g_3)=:{\mathcal {F}}_1\) and \({\mathcal {C}}_0(g_4)=:{\mathcal {F}}_2\) which f in view of (2) satisfy the conditions

$$\begin{aligned} {{\,\mathrm{Re}\,}}\left\{ \left( 1+z+z^2\right) f'(z)\right\} >0,\quad z\in {\mathbb {D}}, \end{aligned}$$
(3)

and

$$\begin{aligned} {{\,\mathrm{Re}\,}}\left\{ (1-z)f'(z)\right\} >0,\quad z\in {\mathbb {D}}, \end{aligned}$$
(4)

respectively. We note here that in [21] the classes \({\mathcal {C}}(\delta ,\xi _1,\xi _2),\) where \(\delta \in (-\pi /2,\pi /2),\) \(\xi _1,\xi _2\in {\overline{\mathbb {D}}},\) of univalent functions were introduced by generalizing Robertson’s condition for convexity in the direction of the imaginary axis [26]. In particular, the class \({\mathcal {C}}(0,(-1-\sqrt{3}\mathrm {i})/2,(-1+\sqrt{3}\mathrm {i})/2)\) is identical to the class \({\mathcal {F}}_1\) and the class \({\mathcal {C}}(0,0,1)\) is identical to the class \({\mathcal {F}}_2.\) A geometrical property of functions in classes \({\mathcal {C}}(\delta ,\xi _1,\xi _2)\) relating to the hyperbolic or parabolic family of arcs related to \(\xi _1\) and \(\xi _2\) was presented in [21].

Let \({\mathcal {P}}\) be the class of all \(p\in {\mathcal {H}}\) of the form

$$\begin{aligned} p(z)=1+\sum _{n=1}^\infty c_nz^n,\quad z\in \mathbb {D}, \end{aligned}$$
(5)

having a positive real part in \(\mathbb {D}.\)

2 Lemmas

In the proof of our main result, we will use the following lemma, see ([4, 24, p. 166]).

Lemma 1

If \(p \in {{\mathcal {P}}}\) is of the form (5), then

$$\begin{aligned} |c_n|\le 2,\quad n\in \mathbb {N}. \end{aligned}$$
(6)

Moreover,

$$\begin{aligned} c_1 = 2\zeta _1 \end{aligned}$$
(7)

and

$$\begin{aligned} c_2 = 2\zeta _1^2 + 2(1-|\zeta _1|^2)\zeta _2 \end{aligned}$$
(8)

for some \(\zeta _i \in \overline{{\mathbb {D}}}\), \(i \in \{ 1,2 \}\).

For \(\zeta _1 \in {\mathbb {T}}\), there is a unique function \(p \in {{\mathcal {P}}}\) with \(c_1\) as in (7), namely

$$\begin{aligned} p(z) = \frac{1+\zeta _1 z}{1-\zeta _1 z}, \quad z\in {\mathbb {D}}. \end{aligned}$$

For \(\zeta _1\in {\mathbb {D}}\) and \(\zeta _2 \in {\mathbb {T}}\), there is a unique function \(p \in {{\mathcal {P}}}\) with \(c_1\) and \(c_2\) as in (7) and (8), namely

$$\begin{aligned} p(z) = \frac{1+( {\overline{\zeta }}_1 \zeta _2 +\zeta _1 )z + \zeta _2 z^2}{1+( {\overline{\zeta }}_1 \zeta _2 -\zeta _1 )z - \zeta _2 z^2}, \quad z\in {\mathbb {D}}. \end{aligned}$$
(9)

3 The Class \(\mathcal {F}_1\)

Let \(f \in {\mathcal {F}}_1\) be the form (1). Then by (3) there exists \(p\in {{\mathcal {P}}}\) of the form (5) such that

$$\begin{aligned} \left( 1+z+z^2\right) f'(z) =p(z),\quad z\in \mathbb {D}. \end{aligned}$$
(10)

Substituting (1) and (5) into (10) and equating the coefficients, we obtain

$$\begin{aligned} a_2 =\frac{1}{2}\left( -1+ c_1\right) , \quad a_3 =\frac{1}{3}\left( -c_1+ c_2\right) . \end{aligned}$$
(11)

Hence, using (6) it follows that \(A_2({\mathcal {F}}_1)=3/2\) with \(f_1\in \mathcal {F}_1\) satisfying

$$\begin{aligned} \left( 1+z+z^2\right) f_1'(z)=\frac{1-z}{1+z},\quad z\in \mathbb {D}, \end{aligned}$$

i.e.,

$$\begin{aligned} f_1(z)=\int _0^z\frac{1-\xi }{1+2\xi +2\xi ^2+\xi ^3}d\xi =z-\frac{3}{2}z^2+\cdots ,\quad z\in \mathbb {D}. \end{aligned}$$

Also \(f_2\in \mathcal {F}_1\) such that

$$\begin{aligned} \left( 1+z+z^2\right) f_2'(z)=\frac{1}{1-z},\quad z\in \mathbb {D}, \end{aligned}$$

i.e.,

$$\begin{aligned} f_2(z)=\int _0^z\frac{d\xi }{1-\xi ^3}=z+\frac{1}{4}z^4+\cdots ,\quad z\in \mathbb {D}, \end{aligned}$$
(12)

serves as the extreme function, since \(a_2=0\). Thus, Theorem 1 gives

Theorem 2

If \(f\in \mathcal {F}_1\), then

$$\begin{aligned} -\frac{5}{4}\le \det T_{2,1}(f) \le 1. \end{aligned}$$

Both inequalities are sharp.

We now find the upper and lower bounds of \(\det T_{3,1}(f)\) in the class \(\mathcal {F}_1\), first noting that

$$\begin{aligned} \det T_{3,1}(f)=\begin{vmatrix} 1&a_2&a_3\\ {\overline{a}}_2&1&a_2\\ {\overline{a}}_3&{\overline{a}}_2&1\end{vmatrix}=2{{\,\mathrm{Re}\,}}\left( a_2^2 {\overline{a}}_3\right) -2 |a_2|^2-|a_3|^2+1. \end{aligned}$$
(13)

Theorem 3

If \(f \in \mathcal {F}_1,\) then

$$\begin{aligned} -1\le \det T_{3,1}(f) \le 1. \end{aligned}$$
(14)

Both inequalities are sharp.

Proof

We first find the upper bound.

By (11) and (6), we see that \(|a_2| \le 3/2\) and \(|a_3| \le 4/3.\) Since \({{\,\mathrm{Re}\,}}(a_2^2 {\overline{a}}_3) \le |a_2|^2|a_3|,\) it follows from (13) that

$$\begin{aligned} \det T_{3,1}(f) \le F\left( |a_2|,|a_3|\right) , \end{aligned}$$
(15)

where

$$\begin{aligned} F(x,y):=2x^2 y -2x^2 - y^2 +1,\quad (x,y)\in \left[ 0,3/2\right] \times \left[ 0,4/3\right] . \end{aligned}$$

Observe now that the point (1, 1) is the unique solution in \((0,3/2)\times (0,4/3)\) of the system of equations

$$\begin{aligned} \begin{aligned}&\frac{\partial F}{\partial x}=4x(y-1)=0,\\&\frac{\partial F}{\partial y}=2(x^2-y)=0. \end{aligned} \end{aligned}$$

However,

$$\begin{aligned} \frac{\partial ^2F}{\partial x^2}(1,1)\frac{\partial ^2 F}{\partial y^2}(1,1)-\left( \frac{\partial F}{\partial x\partial y}(1,1)\right) ^2=-16<0, \end{aligned}$$

so (1, 1) is a saddle point of F.

We now consider F on the boundary of \([0,3/2]\times [0,4/3].\)

  1. (1)

    On the side \(x=0,\)

    $$\begin{aligned} F(0,y)=1-y^2 \le 1,\quad 0\le y\le \frac{4}{3}. \end{aligned}$$
  2. (2)

    On the side \(x=3/2,\)

    $$\begin{aligned} F\left( \frac{3}{2},y\right) =-\frac{7}{2}+\frac{9}{2}y-y^2 \le F\left( \frac{3}{2},\frac{4}{3}\right) =\frac{13}{18},\quad 0\le y\le \frac{4}{3}. \end{aligned}$$
  3. (3)

    On the side \(y=0,\)

    $$\begin{aligned} F(x,0)=1-2x^2 \le 1,\quad 0\le x\le \frac{3}{2}. \end{aligned}$$
  4. (4)

    On the side \(y=4/3\)

    $$\begin{aligned} F\left( x,\frac{4}{3}\right) =-\frac{7}{9}+\frac{2}{3}x^2 \le \frac{13}{18},\quad 0\le x\le \frac{3}{2}. \end{aligned}$$

Therefore, the inequality \(F(x,y) \le 1\) holds for all \((x,y)\in [0,3/2]\times [0,4/3],\) which in view of (15) gives the upper bound.

For the lower bound, we substitute (7) and (8) into (11) to obtain

$$\begin{aligned} a_2 =\frac{1}{2}\left( -1+2\zeta _1\right) , \quad a_3 =\frac{1}{3}\left[ -2\zeta _1+2\zeta _1^2 +2\left( 1-|\zeta _1|^2\right) \zeta _2\right] \end{aligned}$$

with \(\zeta _i \in \overline{{\mathbb {D}}},\) \(i=1,2.\) Therefore, from (13) we have

$$\begin{aligned} \det T_{3,1}(f)=\frac{1}{18}\left( \varPsi _1 + \varPsi _2\right) , \end{aligned}$$
(16)

where

$$\begin{aligned} \varPsi _1 :=9 -20|\zeta _1|^2 +16|\zeta _1|^4 -8(1-|\zeta _1|^2)^2 |\zeta _2|^2 \end{aligned}$$
(17)

and

$$\begin{aligned} \begin{aligned} \varPsi _2:=&30{{\,\mathrm{Re}\,}}\zeta _1+6{{\,\mathrm{Re}\,}}\left( \zeta _1^2\right) -32|\zeta _1|^2{{\,\mathrm{Re}\,}}\zeta _1 -8\left( 1-|\zeta _1|^2\right) {{\,\mathrm{Re}\,}}\left( \zeta _1 {\overline{\zeta }}_2\right) \\&+6\left( 1-|\zeta _1|^2\right) {{\,\mathrm{Re}\,}}\zeta _2 +8\left( 1-|\zeta _1|^2\right) {{\,\mathrm{Re}\,}}\left( \zeta _1^2{\overline{\zeta }}_2\right) . \end{aligned} \end{aligned}$$

We now consider various cases.

A. Suppose that \(\zeta _1\zeta _2\not =0.\) Then \(\zeta _1 =r \mathrm {e}^{\mathrm {i}\theta }\) and \(\zeta _2 = s \mathrm {e}^{\mathrm {i}\psi }\) with \(r,s\in (0,1]\) and \(\theta ,\psi \in [0,2\pi ).\) Then

$$\begin{aligned} \varPsi _2 = \varPsi _3+\varPsi _4, \end{aligned}$$
(18)

where

$$\begin{aligned} \begin{aligned} \varPsi _3&:= 30r\cos \theta +6r^2\cos 2\theta -32r^3\cos \theta \\&= -6r^2 +30r \cos \theta -32r^3 \cos \theta +12r^2\cos ^2\theta \end{aligned} \end{aligned}$$
(19)

and

$$\begin{aligned} \begin{aligned} \varPsi _4&:= -8rs\left( 1-r^2\right) \cos (\theta -\psi ) +6s\left( 1-r^2\right) \cos \psi +8r^2s\left( 1-r^2\right) \cos (2\theta -\psi )\\&=2s(1-r^2) \sqrt{\kappa _1^2 +\kappa _2^2} \sin (\psi +\alpha ), \end{aligned} \end{aligned}$$
(20)

where \(\alpha \in \mathbb {R}\) satisfies

$$\begin{aligned} \cos \alpha = \frac{\kappa _1}{ \sqrt{\kappa _1^2 +\kappa _2^2} },\quad \sin \alpha = \frac{\kappa _2}{ \sqrt{\kappa _1^2 +\kappa _2^2} } \end{aligned}$$
(21)

with

$$\begin{aligned} \kappa _1 := -4r\sin \theta +4r^2\sin 2\theta ,\quad \kappa _2 := 3 - 4r\cos \theta +4r^2\cos 2\theta . \end{aligned}$$
(22)

Since \(\sin (\psi +\alpha ) \ge -1\) and \(s\le 1\), we have

$$\begin{aligned} \begin{aligned} \varPsi _4&\ge -2\left( 1-r^2\right) \sqrt{\kappa _1^2 +\kappa _2^2} \\&= -2\left( 1-r^2\right) \sqrt{9 -8r^2 +16r^4 -24r\cos \theta -32r^3\cos \theta +48r^2\cos ^2\theta }. \end{aligned} \end{aligned}$$

Therefore, from (18)–(20), we obtain

$$\begin{aligned} \begin{aligned} \varPsi _2 =&\varPsi _3+\varPsi _4\\ \ge&30r\cos \theta +6r^2\cos 2\theta -32r^3\cos \theta \\&-2\left( 1-r^2\right) \sqrt{9 -8r^2 +16r^4 -24r\cos \theta -32r^3\cos \theta +48r^2\cos ^2\theta }. \end{aligned} \end{aligned}$$
(23)

Noting that \(|\zeta _2|\le 1\) from (17) we have

$$\begin{aligned} \varPsi _1 \ge 1 -4r^2 +8r^4. \end{aligned}$$
(24)

Thus, from (16), (23) and (24) it follows that

$$\begin{aligned} 18 \det T_{3,1}(f) \ge G(r,\cos \theta ), \quad r\in (0,1], \ \theta \in [0,2\pi ), \end{aligned}$$
(25)

where

$$\begin{aligned} G(t,x) := g_1(t,x) -2(1-t^2)\sqrt{g_2(t,x)} \end{aligned}$$

with

$$\begin{aligned} g_1(t,x) := 1-10t^2+8t^4+30tx -32t^3x +12t^2x^2 \end{aligned}$$

and

$$\begin{aligned} g_2(t,x) := 9 -8t^2 +16t^4 -24tx -32t^3x +48t^2x^2 \end{aligned}$$
(26)

for \(t\in [0,1]\) and \(x\in [-1,1].\)

Let \(\varOmega :=[0,1]\times [-1,1].\) Now we will show that

$$\begin{aligned} \min \left\{ G(t,x) : (t,x) \in \varOmega \right\} = -18. \end{aligned}$$

A1. We next deal with the critical points of G in the interior of \(\varOmega \), i.e., in \((0,1)\times (-1,1).\) Note that \(g_2(t,x)\ge 0.\) Moreover, \(g_2(t,x)=0\) holds only for \(t=\sqrt{3}/2\) and \(x=\sqrt{3}/3.\)

A1.1. When \(t=\sqrt{3}/2\) and \(x=\sqrt{3}/3,\) from (17), (19)and (20) we have

$$\begin{aligned} \varPsi _1=3-\frac{1}{2}|\zeta _2|^2,\quad \varPsi _3=\frac{3}{2}, \quad \varPsi _4=0, \end{aligned}$$

which gives

$$\begin{aligned} \det T_{3,1}(f)=-\frac{1}{36}(-9+|\zeta _2|^2)\ge \frac{2}{9}. \end{aligned}$$

A1.2. We now consider case \(g_2(t,x)>0.\) Differentiating G with respect to x yields

$$\begin{aligned} 0 = \frac{\partial G}{\partial x}(t,x) = \frac{\partial g_1}{\partial x}(t,x) -(1-t^2) (g_2(t,x))^{-1/2} \frac{\partial g_2}{\partial x}(t,x). \end{aligned}$$
(27)

A1.2.a. Assume first that \(\partial g_1/\partial x=0.\) Then from (27), it follows that \(\partial g_2/\partial x=0,\) which is possible only for \(t=\sqrt{6}/2>1\) and \(x=\sqrt{6}/4.\)

A1.2.b. Assume next that \(\partial g_1/\partial x\ne 0.\) Then we can write (27) as

$$\begin{aligned} g_2(t,x)^{1/2} = \frac{\left( 1-t^2\right) \dfrac{\partial g_2}{\partial x}(t,x)}{ \dfrac{\partial g_1}{\partial x}(t,x)} = \frac{ 4\left( 1-t^2\right) \left( -3-4t^2+12tx\right) }{ 15-16t^2+12tx }, \end{aligned}$$
(28)

or equivalently, by substituting (26), as

$$\begin{aligned} \begin{aligned} \varPhi =\varPhi (t,x):=\,&3840t^8-12800t^7x+24576t^6x^2-23040t^5x^3+6912t^4x^4\\&-9600t^6+16128t^5x-21888t^4x^2+13824t^3x^3+10112t^4\\&-2784t^3x+1152t^2x^2-6216t^2-1008tx+1881=0. \end{aligned} \end{aligned}$$
(29)

Now note that by (28) the inequality

$$\begin{aligned} 0<\frac{\left( g_2(t,x)\right) ^{1/2}}{4\left( 1-t^2\right) } = \frac{ -3-4t^2+12tx }{ 15-16t^2+12tx } \end{aligned}$$
(30)

holds for

$$\begin{aligned} 0< t<\sqrt{\frac{3}{2}}\quad \text {and}\quad \frac{4t^2+3}{12t}<x<\frac{16t^2-15}{12t}. \end{aligned}$$

Differentiating G with respect to t, we have

$$\begin{aligned} \frac{\partial G}{\partial t}(t,x) = \frac{ \partial g_1 }{ \partial t}(t,x) +4t\left( g_2(t,x)\right) ^{1/2} - \left( 1-t^2\right) \left( g_2(t,x)\right) ^{-1/2} \frac{ \partial g_2 }{ \partial t}(t,x). \end{aligned}$$
(31)

By (28) and (31), we get

$$\begin{aligned} \begin{aligned} \frac{\partial G}{\partial t}(t,x)=-\frac{72t}{\left( 12xt-4t^2-3\right) \left( 12xt-16t^2+15\right) } H(t,x), \end{aligned} \end{aligned}$$

where for \((t,x)\in (0,1)\times (-1,1),\)

$$\begin{aligned} H(t,x) := \left( -3+4t^2-4tx\right) \left( 9-18t^2+8t^4+6tx+8t^3x-24t^2x^2\right) . \end{aligned}$$

Therefore, each critical point of G satisfies

$$\begin{aligned} -3+4t^2-4tx =0 \end{aligned}$$
(32)

or

$$\begin{aligned} 9-18t^2+8t^4+6tx+8t^3x-24t^2x^2=0. \end{aligned}$$
(33)

I. Assume that (32) holds, then \( x=x(t)= (-3+4t^2)/(4t). \) Thus, by (29) we see that

$$\begin{aligned} \varPhi \left( t,\frac{-3+4t^2}{4t}\right) = -8(2t+1)(2t-1)\left( 4t^2-5\right) \left( 2t^2-3\right) ^2=0 \end{aligned}$$

occurs only when \(t=1/2.\) Thus, \(x=x(1/2 )=-1 .\) However, it can be seen that the right side of (30) equals to \(-2\) for \(t=1/2\) and \(x=-1,\) which means that then the inequality (30) is not true. Therefore, G does not have critical point in the interior of \(\varOmega \) in the case of (32).

II. Suppose now that (33) is satisfied. Since Eq. (33) is a quadratic in x with \(\Delta :=225-408t^2+208t^4>0,\ t\in (0,1),\) it has two roots, namely

$$\begin{aligned} x_i= x_i(t) = \frac{ 3+4t^2 + (-1)^{i+1} \sqrt{ 225-408t^2+208t^4 }}{ 24t }, \quad i=1,2. \end{aligned}$$
(34)

a. Let \(x=x_1.\) Then \(\varPhi (t,x_1(t))=0\) is equivalent to the equation

$$\begin{aligned} \begin{aligned}&\left( -561 +2204t^2 -2736t^4 +1088t^6\right) \sqrt{ 225-408t^2+208t^4 }\\&\quad = 11169 -48552t^2 +80288t^4 -59520t^6 +16640t^8. \end{aligned} \end{aligned}$$
(35)

Squaring both sides of (35) leads to

$$\begin{aligned} 2304\gamma _1(t)\gamma _2^2(t) = 0, \end{aligned}$$
(36)

where for \(t\in (0,1),\)

$$\begin{aligned} \gamma _1(t) :=289-408t^2+208t^4 \end{aligned}$$

and

$$\begin{aligned} \gamma _2(t) := 9-27t^2+26t^4-8t^6=-(1-t)(1+t)\left( 3-4t^2\right) \left( 3-2t^2\right) . \end{aligned}$$

We see that there is a unique root \(t=\sqrt{3}/2\) of the Eq. (36), which also satisfies (35). Then by (34), \(x_1(\sqrt{3}/2)=\sqrt{3}/3.\) However, this case was discussed in A1.1.

b. Let \(x=x_2.\) Then \(\varPhi (t,x_2(t))=0\) is equivalent to the equation

$$\begin{aligned} \begin{aligned}&-\left( -561 +2204t^2 -2736t^4 +1088t^6\right) \sqrt{ 225-408t^2+208t^4 }\\&\quad = 11169 -48552t^2 +80288t^4 -59520t^6 +16640t^8. \end{aligned} \end{aligned}$$
(37)

Squaring both sides of (37) yields again the Eq. (36) having a unique root \(t=\sqrt{3}/2,\) which does not satisfy (37)

A2. It therefore remains to consider G on the boundary of \(\varOmega \).

(1) On the side \(t=0\),

$$\begin{aligned} G(0,x) \equiv -5,\quad x\in [-1,1]. \end{aligned}$$

(2) On the side \(t=1\),

$$\begin{aligned} G(1,x)=-1-2x+12x^2 \ge G\left( 1,\frac{1}{12}\right) = -\frac{13}{12},\quad x\in [-1,1]. \end{aligned}$$

(3) On the side \(x=-1\),

$$\begin{aligned} G(t,-1)=-5-38t+40t^3+16t^4=:\varrho _1(t),\quad t\in [0,1]. \end{aligned}$$

Since \(\varrho _1'(t) = 0\) occurs only when \(t=1/2\) and \(\varrho _1''(1/2)=168>0\), we have

$$\begin{aligned} \varrho _1(t) \ge \varrho _1(1/2)= -18, \quad t\in [0,1]. \end{aligned}$$

(4) On the side \(x=1\),

$$\begin{aligned} G(t,1)=-5+38t-40t^3+16t^4=:\varrho _2(t),\quad t\in [0,1]. \end{aligned}$$

Since \(\varrho _2'(t) = 0\) occurs only when \(t=(19-\sqrt{57})/16 \in [0,1] \) and \(\varrho _2''((19-\sqrt{57})/16)=(57-27\sqrt{57})/2<0\), we have

$$\begin{aligned} \varrho _2(t) \ge \varrho _2(0) = -5, \quad t\in [0,1]. \end{aligned}$$

B. Suppose that \(\zeta _1=0.\) Then

$$\begin{aligned} \varPsi _1=9-8|\zeta _2|^2,\quad \varPsi _2=6{{\,\mathrm{Re}\,}}\zeta _2, \end{aligned}$$

and therefore,

$$\begin{aligned} \det T_{3,1}(f)=\frac{1}{18}(9+6{{\,\mathrm{Re}\,}}\zeta _2-8|\zeta _2|^2)\ge -\frac{5}{18}. \end{aligned}$$
(38)

C. Suppose that \(\zeta _2=0\) and \(\zeta _1=r \mathrm {e}^{\mathrm {i}\theta }\not =0,\) where \(r\in (0,1]\) and \(\theta \in [0,2\pi ).\) Then

$$\begin{aligned} \varPsi _1=9 -20|\zeta _1|^2 +16|\zeta _1|^4=9 -20r^2 +16r^4 \end{aligned}$$

and

$$\begin{aligned} \varPsi _2= 30{{\,\mathrm{Re}\,}}\zeta _1+6{{\,\mathrm{Re}\,}}\left( \zeta _1^2\right) -32|\zeta _1|^2{{\,\mathrm{Re}\,}}\zeta _1=30r\cos \theta +6r^2\cos 2\theta -32r^3\cos \theta . \end{aligned}$$

Thus,

$$\begin{aligned} 18\det T_{3,1}(f)=G(r,\cos \theta ), \quad r\in (0,1], \ \theta \in [0,2\pi ), \end{aligned}$$

where

$$\begin{aligned} G(t,x) :=9 -26t^2 +16t^4+30tx-32t^3x+12t^2x^2 \end{aligned}$$

for \(t\in (0,1]\) and \(x\in [-1,1].\) Set

$$\begin{aligned} x_w:=-\frac{15-16t^2}{12t},\quad t\in (0,1]. \end{aligned}$$

Note that \(-1<x_w\) occurs for \(t>(\sqrt{69}-3)/8= 0.663328\cdots ,\) and \(x_w<1\) holds for \(t\in (0,1).\) Hence, for \(t\in \left( (\sqrt{69}-3)/8,1\right) \) we have

$$\begin{aligned} G(t,x)\ge G(t,x_w)=-\frac{39}{4}+14t^2-\frac{16}{3}t^4=:\phi _1(t). \end{aligned}$$

Since \(\phi _1\) is increasing in \(\left( (\sqrt{69}-3)/8,1\right) ,\) so

$$\begin{aligned} \phi _1(t) \ge -\frac{1}{32}\left( 123+3\sqrt{69}\right) = -4.622495\cdots \end{aligned}$$

for \(t\in \left( (\sqrt{69}-3)/8,1\right) . \) Further, \(x_w<-1\) occurs for \(t\in \left( 0,(\sqrt{69}-3)/8\right) .\) Hence,

$$\begin{aligned} G(t,x)\ge G(t,-1)=9-30t-14t^2+32t^3+16t^4=:\phi _2(t). \end{aligned}$$

Since \(\phi _2'(t) = 0\) have a unique root \(t_0=(-2+\sqrt{19})/4=0.589724\cdots \) and \(\phi _2''(t_0)=152>0\), then

$$\begin{aligned} \phi _2(t)\ge \phi _2\left( \frac{1}{4}\left( -2+\sqrt{19}\right) \right) =-\frac{81}{16}=-5.062500\cdots , \quad t\in \left( 0,\frac{1}{8}\left( \sqrt{69}-3\right) \right) . \end{aligned}$$

Summarizing, form Parts A–C it follows the lower bound in (14).

We discuss now sharpness of (14). The function \(f_2\) defined by (12), for which \(a_2=a_3=0,\) is extremal for the upper bound in (14). It is observed from (16), (23), (24) and (25) that equality for the lower bound in (14) holds when the following conditions are satisfied:

$$\begin{aligned} r=\frac{1}{2}, \quad \cos \theta = -1, \quad s=1,\quad \sin (\psi +\alpha )=-1, \end{aligned}$$
(39)

where \(\alpha \) is determined by the condition (21) with \(\kappa _1\) and \(\kappa _2\) given in (22). Thus, \(\theta =\pi ,\) \(\alpha =\pi /2\) and \(\psi =\pi .\) Consequently, \(\zeta _1=-1/2\) and \(\zeta _2=-1,\) which in view of (9) holds for the function

$$\begin{aligned} p(z)=\frac{1-z^2}{1+z+z^2},\quad z\in {\mathbb {D}}, \end{aligned}$$

in the class \(\mathcal {P}.\) Therefore, the extremal function f in the class \({\mathcal {F}}_1\) for the lower bound in (14) satisfies (10) with p given as above, having the coefficients \(a_2=-1\) and \(a_3=0.\) \(\square \)

4 The Class \(\mathcal {F}_2\)

Let \(f \in \mathcal {F}_2\) be the form (1). Then by (4) there exists \(p\in {{\mathcal {P}}}\) of the form (5) such that

$$\begin{aligned} (1-z) f'(z) =p(z),\quad z\in \mathbb {D}. \end{aligned}$$
(40)

Putting the series (1) and (5) into (40) by equating the coefficients, we get

$$\begin{aligned} a_2 = \frac{1}{2}(1+c_1), \quad a_3 =\frac{1}{3}( 1+c_1+c_2). \end{aligned}$$
(41)

Hence and by (6), it follows that \(A_2(\mathcal {F}_2)=3/2\) with the extremal function \(f_1\in \mathcal {F}_2\) such that

$$\begin{aligned} (1-z)f_1'(z) =\frac{1+z}{1-z},\quad z\in \mathbb {D}. \end{aligned}$$
(42)

Observe also that \(a_2=0\) for the function \(f_2\in \mathcal {F}_2\) such that

$$\begin{aligned} (1-z)f_2'(z)=\frac{1}{1+z},\quad z\in \mathbb {D}. \end{aligned}$$

Therefore, by Theorem 1 we have

Theorem 4

If \(f\in \mathcal {F}_2\), then

$$\begin{aligned} -\frac{5}{4}\le \det T_{2,1}(f) \le 1. \end{aligned}$$

Both inequalities are sharp.

Now we estimate \(\det T_{3,1}(f)\) for functions in the class \(\mathcal {F}_2.\)

Theorem 5

If \(f \in \mathcal {F}_2,\) then

$$\begin{aligned} \det T_{3,1}(f) \le \frac{11}{9} . \end{aligned}$$
(43)

The inequality is sharp.

Proof

By (41) and (6), we see that \(|a_2| \le 3/2\) and \(|a_3| \le 5/3.\) As in the proof of Theorem 3, the inequality (15) holds with the function

$$\begin{aligned} F(x,y):=2x^2 y -2x^2 - y^2 +1,\quad (x,y)\in [0,3/2]\times [0,5/3]. \end{aligned}$$

Repeating argumentation in the proof of Theorem 3, we see that the function F does not have any relative maxima in \((0,3/2)\times (0,5/3).\)

We consider F on the boundary of \([0,3/2]\times [0,5/3].\)

  1. (1)

    On the side \(x=0\),

    $$\begin{aligned} F(0,y)=1-y^2 \le 1,\quad 0\le y\le \frac{5}{3}. \end{aligned}$$
  2. (2)

    On the side \(x=3/2\),

    $$\begin{aligned} F\left( \frac{3}{2},y\right) = -\frac{7}{2}+\frac{9}{2}y-y^2 \le F\left( \frac{3}{2},\frac{5}{3}\right) = \frac{11}{9},\quad 0\le y\le \frac{5}{3}. \end{aligned}$$
  3. (3)

    On the side \(y=0\),

    $$\begin{aligned} F(x,0)=1-2x^2 \le 1,\quad 0\le x\le \frac{3}{2}. \end{aligned}$$
  4. (4)

    On the side \(y=5/3\),

    $$\begin{aligned} F\left( x,\frac{5}{3}\right) = -\frac{16}{9} + \frac{4}{3}x^2 \le F\left( \frac{3}{2},\frac{5}{3}\right) = \frac{11}{9},\quad 0\le x\le \frac{3}{2}. \end{aligned}$$

Therefore, the inequality \(F(x,y) \le 11/9\) holds for all \((x,y)\in [0,3/2]\times [0,5/3]\) which in view of (15) shows (43).

For the function \(f_1\) given by (42), \(a_2=3/2\) and \(a_3=5/3\) which makes equality in (43). \(\square \)

Theorem 6

If \(f \in \mathcal {F}_2,\) then

$$\begin{aligned} \det T_{3,1}(f)\ge \frac{1}{44}\left( 32-31\sqrt{3}\right) = -0.493035\cdots \end{aligned}$$
(44)

The inequality is sharp.

Proof

Substituting (7) and (8) into (41) yields

$$\begin{aligned} a_2 = \frac{1}{2}\left( 2\zeta _1+1\right) ,\quad a_3 =\frac{1}{3}\left[ 1 +2\zeta _1 +2\zeta _1^2 +2\left( 1-|\zeta _1|^2\right) \zeta _2\right] , \end{aligned}$$

for some \(\zeta _i \in \overline{{\mathbb {D}}}\) (\(i=1,2\)). Therefore, from (13) we get

$$\begin{aligned} \det T_{3,1}(f)= \frac{1}{18}\left( \varPsi _1 + \varPsi _2\right) , \end{aligned}$$
(45)

where

$$\begin{aligned} \varPsi _1 := 10-20|\zeta _1|^2+ 16|\zeta _1|^4 -8\left( 1-|\zeta _1|^2\right) ^2 |\zeta _2|^2 \end{aligned}$$
(46)

and

$$\begin{aligned} \begin{aligned} \varPsi _2:=&-26{{\,\mathrm{Re}\,}}\zeta _1+10{{\,\mathrm{Re}\,}}\left( \zeta _1^2\right) +32|\zeta _1|^2{{\,\mathrm{Re}\,}}\zeta _1 +8\left( 1-|\zeta _1|^2\right) {{\,\mathrm{Re}\,}}\left( \zeta _1 {\overline{\zeta }}_2\right) \\&-2\left( 1-|\zeta _1|^2\right) {{\,\mathrm{Re}\,}}\zeta _2 +8(1-|\zeta _1|^2){{\,\mathrm{Re}\,}}\left( \zeta _1^2 {\overline{\zeta }}_2\right) . \end{aligned} \end{aligned}$$

A. Suppose that \(\zeta _1\zeta _2\not =0.\) Thus, \(\zeta _1 =r \mathrm {e}^{\mathrm {i}\theta }\) and \(\zeta _2 = s \mathrm {e}^{\mathrm {i}\psi }\) with \(r,s\in (0,1]\) and \(\theta ,\psi \in [0,2\pi ).\) Then

$$\begin{aligned} \varPsi _2 = 2\left( \varPsi _3+\varPsi _4\right) , \end{aligned}$$
(47)

where

$$\begin{aligned} \begin{aligned} \varPsi _3&:= -13r\cos \theta +5r^2\cos 2\theta +16r^3\cos \theta \\&= -5r^2 -13r \cos \theta +16r^3 \cos \theta +10r^2\cos ^2\theta \end{aligned} \end{aligned}$$
(48)

and

$$\begin{aligned} \begin{aligned} \varPsi _4&:= 4rs\left( 1-r^2\right) \cos (\theta -\psi ) -s\left( 1-r^2\right) \cos \psi +4r^2s\left( 1-r^2\right) \cos (2\theta -\psi )\\&=s\left( 1-r^2\right) \sqrt{\kappa _1^2 +\kappa _2^2} \sin (\psi +\alpha ), \end{aligned} \end{aligned}$$
(49)

where \(\alpha \) is the quantity satisfying (21) with

$$\begin{aligned} \kappa _1 := 4r\sin \theta +4r^2\sin 2\theta ,\quad \kappa _2 := -1 + 4r\cos \theta +4r^2\cos 2\theta . \end{aligned}$$
(50)

Since \(\sin (\psi +\alpha ) \ge -1\) and \(s\le 1\), we have

$$\begin{aligned} \begin{aligned} \varPsi _4&\ge -\left( 1-r^2\right) \sqrt{\kappa _1^2 +\kappa _2^2} \\&= -\left( 1-r^2\right) \sqrt{1 +24r^2 +16r^4 -8r\cos \theta +32r^3\cos \theta -16r^2\cos ^2\theta }. \end{aligned} \end{aligned}$$

Therefore, from (47), (48) and (49), we get

$$\begin{aligned} \begin{aligned} \frac{1}{2} \varPsi _2 =&\varPsi _3+\varPsi _4\\ \ge&-13r\cos \theta +5r^2\cos 2\theta +16r^3\cos \theta \\&-\left( 1-r^2\right) \sqrt{1 +24r^2 +16r^4 -8r\cos \theta +32r^3\cos \theta -16r^2\cos ^2\theta }. \end{aligned} \end{aligned}$$
(51)

Taking into account that \(|\zeta _2|\le 1\) from (46) we have

$$\begin{aligned} \varPsi _1 \ge 2\left( 1 -2r^2 +4r^4\right) . \end{aligned}$$
(52)

Thus, from (45), (51) and (52) it follows that

$$\begin{aligned} 9 \det T_{3,1}(f) \ge G(r,\cos \theta ), \quad r\in (0,1], \ \theta \in [0,2\pi ), \end{aligned}$$
(53)

where

$$\begin{aligned} G(t,x) := g_1(t,x) -\left( 1-t^2\right) \sqrt{g_2(t,x)} \end{aligned}$$

with

$$\begin{aligned} g_1(t,x) := 1-7t^2+4t^4 -13tx +16t^3x +10t^2x^2 \end{aligned}$$

and

$$\begin{aligned} g_2(t,x) := 1 +24t^2 +16t^4 -8tx +32t^3x -16t^2x^2 \end{aligned}$$
(54)

for \(t\in [0,1]\) and \(x\in [-1,1].\)

Let \(\varOmega :=[0,1]\times [-1,1]\) and

$$\begin{aligned} \varTheta := \frac{9}{44}\left( 32-31\sqrt{3}\right) = -4.4373\cdots . \end{aligned}$$

Now we will show that

$$\begin{aligned} \min \left\{ G(t,x) : (t,x) \in \varOmega \right\} = \varTheta . \end{aligned}$$

A1. For this, we first we find the critical points of G in the interior of \(\varOmega \), i.e., in \((0,1)\times (-1,1).\) Since \(g_2(t,x)=0\) holds only for \(t=(\sqrt{2}-1)/2\) and \(x=1,\) it follows that \(g_2(t,x)> 0\) for \((t,x)\in (0,1)\times (-1,1).\)

Differentiating G with respect to x yields

$$\begin{aligned} 0 = \frac{\partial G}{\partial x}(t,x) = \frac{\partial g_1}{\partial x}(t,x) -\frac{1}{2}(1-t^2) (g_2(t,x))^{-1/2} \frac{\partial g_2}{\partial x}(t,x). \end{aligned}$$
(55)

A1.1 Assume first that \(\partial g_1/\partial x=0.\) Then by (55), it follows that \(\partial g_2/\partial x=0,\) which is possible only for \(t=\sqrt{2}/2\) and \(x=\sqrt{2}/4.\) From (53), we have

$$\begin{aligned} 9 \det T_{3,1}(f) \ge G\left( \frac{\sqrt{2}}{2},\frac{\sqrt{2}}{4}\right) =-\frac{17}{8}-\frac{3}{2}\sqrt{2}= -4.246320\ldots \end{aligned}$$

A1.2 Assume now that \(\partial g_1/\partial x\ne 0.\) Then we can write the Eq. (55) as

$$\begin{aligned} g_2(t,x)^{1/2} = \frac{\left( 1-t^2\right) \dfrac{\partial g_2}{\partial x}(t,x)}{2 \dfrac{\partial g_1}{\partial x}(t,x)} = \frac{ 4\left( 1-t^2\right) \left( -1+4t^2-4tx\right) }{ -13+16t^2+20tx }, \end{aligned}$$
(56)

or equivalently, by substituting (54), as

$$\begin{aligned} \begin{aligned} \varPhi =\varPhi (t,x) :=&3840t^8+18944t^7x+22528t^6x^2+2560t^5x^3-6400t^4x^4\\&+128t^6-9472t^5x-4992t^4x^2+5120t^3x^3-7552t^4\\&-2336t^3x+1600t^2x^2+3800t^2-2000tx+153=0. \end{aligned} \end{aligned}$$
(57)

Furthermore, note that by (56) the inequality

$$\begin{aligned} 0<\frac{\left( g_2(t,x)\right) ^{1/2}}{4\left( 1-t^2\right) }= \frac{ -1+4t^2-4tx }{ -13+16t^2+20tx } \end{aligned}$$
(58)

is true for

$$\begin{aligned} 0< t< \frac{\sqrt{2}}{2}\quad \text {and} \quad \frac{4t^2-1}{4t}<x<\frac{13-16t^2}{20t} \end{aligned}$$

or

$$\begin{aligned} \frac{\sqrt{2}}{2}<t<1\quad \text {and}\quad \left( x<\frac{4t^2-1}{4t}\quad \text {or}\quad x>-\frac{16t^2-13}{20t}\right) . \end{aligned}$$

Differentiating G with respect to t yields

$$\begin{aligned} \frac{\partial G}{\partial t}(t,x) = \frac{ \partial g_1 }{ \partial t}(t,x) +2t(g_2(t,x))^{1/2} - \frac{1}{2}\left( 1-t^2\right) (g_2(t,x))^{-1/2} \frac{ \partial g_2 }{ \partial t}(t,x). \end{aligned}$$
(59)

By (56) and (59), we get

$$\begin{aligned} \begin{aligned} \frac{\partial G}{\partial t}(t,x)=-\frac{36t}{\left( -4xt+4t^2-1\right) \left( 20xt+16t^2-13\right) } H(t,x), \end{aligned} \end{aligned}$$

where

$$\begin{aligned} H(t,x) := \left( -3+4t^2+4tx\right) \left( -11+6t^2+8t^4+2tx+24t^3x+40t^2x^2\right) . \end{aligned}$$

Therefore, each critical point of G satisfies

$$\begin{aligned} -3+4t^2+4tx =0 \end{aligned}$$
(60)

or

$$\begin{aligned} -11+6t^2+8t^4+2tx+24t^3x+40t^2x^2=0. \end{aligned}$$
(61)

I. Assume that (60) holds. Then \( x=x(t)= (3-4t^2)/(4t). \) Thus, by (57) we see that

$$\begin{aligned} \varPhi \left( t,\frac{3-4t^2}{4t}\right) = -8(48t^4-104t^2+39)(2t^2-1)^2=0 \end{aligned}$$

occurs only when \(t={\hat{t}}_i,\ i=1,2,\) where

$$\begin{aligned} {\hat{t}}_1 := \frac{1}{6} \sqrt{39-6\sqrt{13} } = 0.6945\cdots ,\quad {\hat{t}}_2 := \frac{\sqrt{2}}{2} = 0.7071\cdots \end{aligned}$$

Thus,

$$\begin{aligned} x=x\left( {\hat{t}}_1 \right) = {\hat{x}}_1 = \frac{-2+\sqrt{13}}{\sqrt{39-6\sqrt{13}}} = 0.3852\cdots \end{aligned}$$

and

$$\begin{aligned} x=x\left( {\hat{t}}_2 \right) = {\hat{x}}_2 = \frac{\sqrt{2}}{4} = 0.3535\cdots . \end{aligned}$$

However, it can be seen that

$$\begin{aligned} \frac{ -1+4{\hat{t}}_1^2-4{\hat{t}}_1{\hat{x}}_1 }{ -13+16{\hat{t}}_1^2+20{\hat{t}}_1{\hat{x}}_1 }=-2 <0, \end{aligned}$$

which means that the inequality (58) is not satisfied for \(t={\hat{t}}_1\) and \(x={\hat{x}}_1\). Note that the case \(t={\hat{t}}_2\) and \(x={\hat{x}}_2\) reduces to A1.1.

Therefore, G does not have critical point in the interior of \(\varOmega .\)

II. Suppose now that (61) is satisfied. Equation (61) as a quadratic equation of x with \(\Delta :=441-216t^2-176t^4>0,\ t\in (0,1)\) has two roots, namely

$$\begin{aligned} x_i= x_i(t) = \frac{ -\left( 1+12t^2\right) + (-1)^{i+1} \sqrt{ 441-216t^2-176t^4 }}{ 40t }, \quad i=1,2. \end{aligned}$$
(62)

a. Let \(x=x_1.\) Then \(\varPhi (t,x_1(t))=0\) is equivalent to the equation

$$\begin{aligned} \begin{aligned}&\left( -603 -940t^2 -9936t^4 +7104t^6 \right) \sqrt{ 441-216t^2-176t^4 }\\&\quad = -2313 -96696t^2 +20384t^4 -44928t^6 +92928t^8. \end{aligned} \end{aligned}$$
(63)

Squaring the both sides of (63) leads to

$$\begin{aligned} 518400\gamma _1(t)\gamma _2^2(t) = 0, \end{aligned}$$
(64)

where for \(t\in (0,1),\)

$$\begin{aligned} \gamma _1(t) :=-299+648t^2+528t^4 \end{aligned}$$

and

$$\begin{aligned} \gamma _2(t) := 1+t^2-10t^4+8t^6=(1-t)(1+t)\left( 1+4t^2\right) \left( 1-2t^2\right) . \end{aligned}$$

Thus, there are two roots \(t_1\) and \(t_2\) in (0, 1) of the Eq. (64), namely

$$\begin{aligned} t_1:= \frac{\sqrt{2}}{2 } = 0.707\cdots ,\quad t_2:= \frac{1}{66}\sqrt{-2673+2442\sqrt{3}} = 0.59779\cdots . \end{aligned}$$
(65)

For \(t=t_1\), by (62), \({\tilde{x}}_1:=x_1(t_1)=\sqrt{2}/4\) and this case was discussed in A1.1.

For \(t=t_2\), by (62),

$$\begin{aligned} {\tilde{x}}_2:=x_1\left( t_2\right) = \frac{21+13\sqrt{3}}{2\sqrt{-2673+2442\sqrt{3}}} =0.551477\cdots . \end{aligned}$$
(66)

It can be verified that \({\tilde{\Phi }}(t_2,{\tilde{x}}_2) =0\) and the inequality (58) holds for \(t=t_2\) and \(x={\tilde{x}}_2\). Therefore, G has a critical point at \((t_2,{\tilde{x}}_2)\).

b. Let \(x=x_2.\) Then \(\varPhi (t,x_2(t))=0\) is equivalent to the equation

$$\begin{aligned} \begin{aligned}&-\left( -603 -940t^2 -9936t^4 +7104t^6 \right) \sqrt{ 441-216t^2-176t^4 }\\&\quad = -2313 -96696t^2 +20384t^4 -44928t^6 +92928t^8. \end{aligned} \end{aligned}$$
(67)

Squaring both sides of (67) yields again the Eq. (64) having roots \(t_1\) and \(t_2\) given by (65), which do not satisfy (67).

Therefore, by A, B1 and B2, the function G has a unique critical point at \((t_2,{\tilde{x}}_2)\). Denote

$$\begin{aligned} \lambda _1:= & {} \frac{\partial ^2 G}{\partial t^2}\left( t_2,{\tilde{x}}_2\right) =-\frac{30910188}{351923}+\frac{90657386}{1055769}\sqrt{3}= 60.89649\cdots ,\\ \lambda _2:= & {} \frac{\partial ^2 G}{\partial t \partial x}\left( t_2,{\tilde{x}}_2\right) =-\frac{211055}{1177}+\frac{396373}{3531}\sqrt{3},\\ \lambda _3= & {} \frac{\partial ^2 G}{\partial x^2}\left( t_2,{\tilde{x}}_2\right) =-\frac{308371}{1177}+\frac{551104}{3531}\sqrt{3}. \end{aligned}$$

Since \(\lambda _1 >0\) and

$$\begin{aligned} \lambda _1 \lambda _3 - \lambda _2^2=-\frac{7933032}{1177}+\frac{4769748}{1177}\sqrt{3}= 279.02623\cdots >0, \end{aligned}$$

the function G has a local minimum at \((t_2,{\tilde{x}}_2).\)

A2. It remains to consider G in the boundary of \(\Omega \).

(1) On the side \(t=0\),

$$\begin{aligned} G(0,x) \equiv 0 > \Theta ,\quad x\in [-1,1]. \end{aligned}$$

(2) On the side \(t=1\),

$$\begin{aligned} G(1,x)=-2+3x+10x^2\ge G\left( 1,-\frac{3}{20}\right) = -\frac{89}{40} > \Theta ,\quad x\in [-1,1]. \end{aligned}$$

(3) On the side \(x=-1\),

$$\begin{aligned} G(t,-1)=t\left( 9+8t-12t^2\right) \ge G(0,-1) = 0 > \Theta ,\quad t\in [0,1]. \end{aligned}$$

(4) On the side \(x=1\),

$$\begin{aligned} G(t,1)=1 -13t +3t^2 +16t^3 +4t^4 -\left( 1-t^2\right) \left| -1+4t+4t^2\right| =:\varrho (t). \end{aligned}$$

When \(t \in \left[ 0,(-1+\sqrt{2})/2\right] =:I_1\), we have \(-1+4t+4t^2 \le 0\). Therefore,

$$\begin{aligned} \varrho (t) =t\left( -9+8t+12t^2\right) \ge G\left( \frac{\sqrt{2}-1}{2},1\right) =-\sqrt{2}\approx -1.41421> \Theta , \quad t\in I_1. \end{aligned}$$

When \(t\in \left[ (-1+\sqrt{2})/2,1\right] =:I_2\), we have \(-1+4t+4t^2 \ge 0\) and \(\varrho (t) = (-2+t+2t^2) (-1+8t+4t^2) \). Since \(\varrho '(t) = 0\) occurs only when \(t=1/2 \in I_2\) and \(\varrho ''(1/2)=80>0\),we have

$$\begin{aligned} \varrho (t) \ge \varrho \left( \frac{1}{2}\right) = -4 > \Theta , \quad t\in I_2. \end{aligned}$$

B. Suppose that \(\zeta _1=0.\) Then

$$\begin{aligned} \varPsi _1=10-8|\zeta _2|^2,\quad \varPsi _2=-2{{\,\mathrm{Re}\,}}\left( \zeta _2\right) , \end{aligned}$$

and therefore,

$$\begin{aligned} \det T_{3,1}(f)=\frac{1}{18}\left[ 10-8|\zeta _2|^2-2{{\,\mathrm{Re}\,}}\left( \zeta _2\right) \right] \ge 0. \end{aligned}$$

C. Suppose that \(\zeta _2=0\) and \(\zeta _1\not =0.\) Then

$$\begin{aligned} \varPsi _1= 10-20|\zeta _1|^2+ 16|\zeta _1|^4, \quad \varPsi _2= -26{{\,\mathrm{Re}\,}}\zeta _1+10{{\,\mathrm{Re}\,}}(\zeta _1^2) +32|\zeta _1|^2{{\,\mathrm{Re}\,}}\zeta _1. \end{aligned}$$

Thus, taking \(\zeta _1=r \mathrm {e}^{\mathrm {i}\theta },\) where \(r\in (0,1]\) and \(\theta \in [0,2\pi )\) we have

$$\begin{aligned} \varPsi _1 = 10 -20r^2 +16r^4 \quad \varPsi _2=-26r\cos \theta +10r^2\cos 2\theta +32r^3\cos \theta . \end{aligned}$$

Then

$$\begin{aligned} 9\det T_{3,1}(f)=G(r,\cos \theta ), \quad r\in (0,1], \ \theta \in [0,2\pi ), \end{aligned}$$

where

$$\begin{aligned} G(t,x) :=5 -15t^2 +8t^4-13tx +16t^3x +10t^2x^2 \end{aligned}$$

for \(t\in (0,1]\) and \(x\in [-1,1].\) Set

$$\begin{aligned} x_w=\frac{13-16t^2}{20t},\quad t\in (0,1]. \end{aligned}$$

Note that \(-1<x_w\) occurs for \(t\in (0,1)\) and \(x_w<1\) holds for \(t>(\sqrt{77}-5)/8=0.471870\cdots .\) Hence, for \(t\in \left( (\sqrt{77}-5)/8,1\right) \) we have

$$\begin{aligned} G(t,x)\ge G(t,x_w)=\frac{31}{40}-\frac{23}{5}t^2+\frac{8}{5}t^4=:\phi _1(t). \end{aligned}$$

Since \(\phi _1\) is decreasing in \(\left( (\sqrt{77}-5)/8,1\right) ,\)

$$\begin{aligned} \phi _1(t) \ge -\frac{89}{40}= -2.225, \quad t\in \left( \frac{1}{8}\left( \sqrt{77}-5\right) ,1\right) . \end{aligned}$$

On the other hand, \(x_w>1\) occurs for \(t\in \left( 0,(\sqrt{77}-5)/8\right) .\) Hence,

$$\begin{aligned} G(t,x)\ge G(t,1)=5-13t-5t^2+16t^3+8t^4=:\phi _2(t). \end{aligned}$$

Since \(\phi _2\) is decreasing in \((0,(\sqrt{77}-5)/8)\),

$$\begin{aligned} \phi _2(t)\ge \frac{33}{64}-\frac{5}{64}\sqrt{77}= -0.169919\cdots , \quad t\in \left( 0,\frac{1}{8}\left( \sqrt{77}-5\right) \right) . \end{aligned}$$

Summarizing, form Parts A–C it follows that the inequality (44) holds.

It remains to show that the inequality (44) is sharp. It is observed from (45), (51), (52) and (53) that \(9\det T_{3,1}(f) = \Theta \) holds when the following conditions are satisfied:

$$\begin{aligned} r=t_2, \quad \cos \theta = {\tilde{x}}_2, \quad s=1,\quad \sin (\psi +\alpha )=-1, \end{aligned}$$
(68)

where \(t_2\) and \({\tilde{x}}_2\) are given by (65) and (66), and where \(\alpha \) is determined by the condition (21) with \(\kappa _1\) and \(\kappa _2\) given by (50).

Now set \(\theta =\mathrm{Arccos}({\tilde{x}}_2)\) so that it satisfies the second condition in (68). Then \(\kappa _1 = 3.309903\cdots >0\) and \(\kappa _2 = -0.241293\cdots <0\). Thus, (21) is satisfied if we take

$$\begin{aligned} \alpha = - \mathrm{Arccos}\left( \frac{ \kappa _1 }{ \sqrt{ \kappa _1^2 + \kappa _2^2 } } \right) =-0.0727716 \cdots . \end{aligned}$$

Thus, if we put

$$\begin{aligned} \psi = \frac{3\pi }{2} - \alpha = 4.7851606\cdots , \end{aligned}$$

then \(\psi \) satisfies the fourth condition in (68). Now consider a function \({\tilde{p}}\) which has the form (9) with \(\zeta _1 = t_2\mathrm {e}^{\mathrm {i}\theta }\) and \(\zeta _2 = \mathrm {e}^{\mathrm {i}\psi }\). Since \(\zeta _1\in {\mathbb {D}}\) and \(\zeta _2 \in {\mathbb {T}}\), from Lemma 1 it follows that \({\tilde{p}}\in \mathcal {P}\), and so the extremal function f in the class \({\mathcal {F}}_2\) for which equality in (44) holds satisfies (40) with \(p:={\tilde{p}}.\) \(\square \)