The Bound of the Hankel Determinant of the Third Kind for Starlike Functions

Abstract

In the present paper, the estimate of the third Hankel determinant

$$\begin{aligned} \begin{aligned} H_{3,1}(f)&= \begin{vmatrix} a_{1}&a_{2}&a_{3} \\ a_{2}&a_{3}&a_{4} \\ a_{3}&a_{4}&a_{5} \end{vmatrix} \end{aligned} \end{aligned}$$

for the class of starlike functions, i.e., for the class of analytic functions f standardly normalized such that ${{\mathrm{Re}}}(zf'(z)/f(z)) > 0,\ z\in {{\mathbb {D}}}:=\{z \in {\mathbb {C}} : |z|<1\},$ is improved.

1 Introduction

Let ${{\mathcal {H}}}$ be a class of analytic functions in ${\mathbb {D}}:= \left\{ z \in {\mathbb {C}} : |z|<1 \right\}$ and let ${{\mathcal {A}}}$ be its subclass normalized by $f(0):=0$, $f'(0):=1,$ i.e., of the form

$$\begin{aligned} f(z) = \sum _{n=1}^{\infty }a_{n}z^{n},\quad a_1:=1,\ z\in {\mathbb {D}}. \end{aligned}$$

(1.1)

Let ${{\mathcal {S}}}^{*}$ denote the class of starlike functions, namely, the subclass of ${{\mathcal {A}}}$ consisting of functions f such that

$$\begin{aligned} {{\mathrm{Re}}}\frac{zf'(z)}{f(z)} > 0,\quad z\in {\mathbb {D}}. \end{aligned}$$

(1.2)

Given $q,n \in {\mathbb {N}},$ the Hankel determinants $H_{q,n}(f)$ of Taylor’s coefficients of functions $f \in {{\mathcal {A}}}$ of the form (1.1) are defined as

$$\begin{aligned} H_{q,n}(f) := \begin{vmatrix} a_{n}&a_{n+1}&\cdots&a_{n+q-1} \\ a_{n+1}&a_{n+2}&\cdots&a_{n+q} \\ \vdots&\vdots&\vdots&\vdots \\ a_{n+q-1}&a_{n+q}&\cdots&a_{n+2(q-1)} \end{vmatrix}. \end{aligned}$$

Particularly, the third Hankel determinant $H_{3,1}(f)$ is given by

$$\begin{aligned} H_{3,1}(f) := \begin{vmatrix} a_{1}&a_{2}&a_{3} \\ a_{2}&a_{3}&a_{4} \\ a_{3}&a_{4}&a_{5} \end{vmatrix} = a_{3}( a_{2}a_{4} - a_{3}^{2} ) - a_{4}( a_{4} - a_{2}a_{3} ) + a_{5}( a_{3} - a_{2}^{2} ). \end{aligned}$$

(1.3)

To find the growth of the Hankel determinant $H_{q,n}(f)$ dependent on q and n for the whole class ${\mathcal {S}}\subset {\mathcal {A}}$ of univalent functions as well as for its subclasses is an interesting problem to study. For the class ${\mathcal {S}}$ some important result was shown by Pommerenke [13]. For fixed q and n the growth problem can be reduced to an estimate of the Hankel determinant for the selected subclasses of ${\mathcal {A}}.$ Recently many authors examined the Hankel determinant $H_{2,2}(f)=a_2a_4-a_3^2$ of order 2 (see, e.g., [3, 4, 6, 8, 12]). Note also that $H_{2,1}(f)=a_3-a_2^2.$ Thus the Hankel determinant $H_{2,1}(f)$ reduces to the well-known coefficient functional which for ${\mathcal {S}}$ was estimated in 1916 by Bieberbach (see, e.g., [5, Vol. I, p. 35]).

The problem to find the upper bound of the Hankel determinant $H_{3,1}(f)$ of order 3 is more sophisticated if we expect to get sharp result. From (1.3) by using the triangle inequality we get at once the following inequality

$$\begin{aligned} |H_{3,1}(f)|\le |a_3||H_{2,2}(f)|+|a_{4}||a_{4} - a_{2}a_{3}| + |a_{5}||H_{2,1}(f)|. \end{aligned}$$

(1.4)

This simple observation allowed to estimate of $|H_{3,1}(f)|$ for compact subclasses ${\mathcal {F}}$ of ${\mathcal {A}}$ by various authors (see, e.g., [2, 15,16,17,18]). However, these results are far from sharpness. If case when a given subclass ${\mathcal {F}}$ of ${\mathcal {A}}$ has a representation with using the Carathéodory class ${\mathcal {P}}$, i.e., the class of functions $p \in {{\mathcal {H}}}$ of the form

$$\begin{aligned} p(z) = 1 + \sum _{n=1}^{\infty }c_{n}z^{n}, \quad z\in {\mathbb {D}}, \end{aligned}$$

(1.5)

having a positive real part in ${\mathbb {D}},$ the coefficients of functions in ${\mathcal {F}}$ have a suitable representation expressed by the coefficients of functions in ${\mathcal {P}}.$ Therefore to get the upper bound of each term in (1.4) cited authors based their computing on the well-known formulas on coefficient $c_2$ (e.g., [14, p. 166]) and on the formula $c_3$ due to Libera and Zlotkiewicz [9].

In order to improve the bound of $|H_{3,1}(f)|$ we have to use directly formula (1.3), where we need to apply a formula for $c_4,$ similar to the formulas (2.1) and (2.2). In a recent paper [7] the authors found such a formula for $c_4.$ According to the authors’ knowledge, formulas for the coefficients $c_n$ for $n \ge 5$ analogous to the formulas (2.1) and (2.2) are not known.

Basing on the formulas for $c_2,\ c_3$ and $c_4,$ we improve the known estimate of the Hankel determinant $H_{3,1}(f)$ in the class ${{\mathcal {S}}}^{*}$ of starlike functions. We show that $|H_{3,1}(f)|\le 8/9.$ Estimating each term of the right hand of (1.4) Babalola [1] showed that $|H_{3,1}(f)|\le 16.$ In [19] Zaprawa by a suitable grouping and using Lemma 1 due to Livingston [11] proved that $|H_{3,1}(f)|\le 1.$

2 Main Result

The basis for proof of the main result is the following lemma. It contains the well-known formula for $c_2$ (e.g., [14, p. 166]), the formula for $c_3$ due to Libera and Zlotkiewicz [9, 10] and the formula for $c_4$ found by the authors [7].

Lemma 2.1

If $p \in {{\mathcal {P}}}$ is of the form (1.5) with $c_1\ge 0,$ then

$$\begin{aligned} 2c_2= & {} c_1^2 + (4-c_1^2)\zeta , \end{aligned}$$

(2.1)

$$\begin{aligned} 4c_3= & {} c_1^3 +(4-c_1^2)c_1\zeta (2-\zeta ) + 2(4-c_1^2)( 1 - |\zeta |^2) \eta \end{aligned}$$

(2.2)

and

$$\begin{aligned} \begin{aligned} 8c_4&= c_1^4+(4-c_1^2)\zeta \left[ c_1^2(\zeta ^2-3\zeta +3)+4\zeta \right] \\&-\,4(4-c_1^2)(1-|\zeta |^2)\left[ c_1(\zeta -1)\eta +\overline{\zeta }\eta ^2-\left( 1-|\eta |^2\right) \xi \right] \end{aligned} \end{aligned}$$

(2.3)

for some $\zeta ,\eta ,\xi \in \overline{{\mathbb {D}}}:=\{z\in {\mathbb {C}}:|z|\le 1 \}.$

Now, we will estimate the third-order Hankel determinant $H_{3,1}(f)$ for $f \in {{\mathcal {S}}}^{*}$. To this end, the following propositions are required.

Proposition 2.2

Let $\Theta : [0,3] \times [0,1] \rightarrow {\mathbb {R}}$ be a function defined by

$$\begin{aligned} \Theta (t,x) := 96\theta _{1}(x) - 8\theta _{2}(x) t + 3\theta _{3}(x) t^{2}, \end{aligned}$$

(2.4)

where for $x\in [0,1],$

$$\begin{aligned} \theta _{1}(x):= & {} 2 +8x -x^{2} -6x^{3}, \\ \theta _{2}(x):= & {} 16 + 67x -34x^{2} -53x^{3} + 2x^{4} \end{aligned}$$

and

$$\begin{aligned} \theta _{3}(x) := 12 + 19x - 21x^{2} - 17x^{3} + 3x^{4}. \end{aligned}$$

Then $\Theta (t,x) > 0$ for $0 \le t \le 3$ and $0 \le x \le 1$.

Proof

At first, note that the polynomial $\theta _{3}$ has a unique zero $x=:x_{1}\approx 0.9314$ in (0, 1). Since $x_1\in (0.92,0.95)$ and for $x\in (0.92,0.95),$

$$\begin{aligned}&\theta _{2}(x)>16 + 67\cdot (0.92)-34\cdot (0.95)^2 -53\cdot (0.95)^3 + 2\cdot (0.92)^4\\&\quad =2.94691092>0, \end{aligned}$$

it follows that

$$\begin{aligned} \frac{\partial }{\partial t} \Theta (t,x_1) = -8 \theta _{2}(x_1) \not =0. \end{aligned}$$

For $x \not = x_1$, $(\partial /\partial t) \Theta (t,x)=0$ occurs at

$$\begin{aligned} t= \frac{ 4\theta _{2}(x) }{ 3\theta _{3}(x) } =: t_{0}(x). \end{aligned}$$

We have

$$\begin{aligned} \frac{\partial }{\partial x}\Theta (t,x) \Big |_{t=t_{0}(x)} = \frac{16 \theta _{4}(x)}{9\theta _{3}^{2}(x)}, \end{aligned}$$

where

$$\begin{aligned} \begin{aligned} \theta _{4}(x)&:= 54\theta _{1}'(x)\theta _{3}^{2}(x) - 6\theta _{2}'(x) \theta _{2}(x) \theta _{3}(x) + 3\theta _{3}'(x) \theta _{2}^{2}(x) \\&= -3\left( 128 + 31896x - 18709x^{2} -133828x^{3} -3737x^{4} +198602x^{5} \right. \\&\quad \left. +74185x^{6}-91136x^{7} -54071x^{8} -2774x^{9} +668x^{10} \right. \\&\quad \left. +48x^{11} \right) ,\quad x\in (0,1). \end{aligned} \end{aligned}$$

The polynomial $\theta _{4}$ has exactly two zeros in (0, 1),  namely, $x=:x_2\approx 0.533701$ and $x=:x_3\approx 0.811327.$ We will now show that

$$\begin{aligned} t_{0}(x)>3,\quad x\in [0.5,0.9]. \end{aligned}$$

(2.5)

Since $x_1>0.9,$ so $\theta _3(x)>0,$ for $x\in [0.5,0.9]$ and the inequality (2.5) is equivalent to

$$\begin{aligned} 4\theta _2(x)-9\theta _3(x)>0,\quad x\in [0.5,0.9]. \end{aligned}$$

The above one can be equivalently written as

$$\begin{aligned} 19x^4+59x^3-53x^2-97x+44<0,\quad x\in [0.5,0.9]. \end{aligned}$$

As the polynomial on the left hand of the above inequality has a unique zero $x\approx 0.40928$ in [0, 1],  the above inequality is true, so is the inequality (2.5). Thus the function $\Theta$ has no critical point in $(0,3) \times (0,1).$ Hence it is sufficient to show that $\Theta > 0$ on the boundary of $[0,3] \times [0,1]$. We can easily check that the following inequalities hold:

$$\begin{aligned} \Theta (t,0)= & {} 4(48-32t+9t^{2}) \ge \frac{704}{9}, \quad t\in [0,3], \\ \Theta (t,1)= & {} 4(72+4t-3t^{2}) \ge 228, \quad t\in [0,3], \\ \Theta (0,x)= & {} 96(2+8x-x^{2}-6x^{3}) \ge 192, \quad x\in [0,1], \end{aligned}$$

and

$$\begin{aligned} \begin{aligned} \Theta (3,x)&= 3\left( 44 -109x +51x^{2} +79x^{3} +11x^{4} \right) \\&= 3\left( 44(1-x)^{3} +x(23 -81x +123x^{2}) +11x^{4} \right) \\&\ge 3\left( 44(1-x)^{3} + \frac{1585}{164}x +11x^{4} \right) > 0, \quad x \in [0,1]. \end{aligned} \end{aligned}$$

Thus the proof of the proposition is completed. $\square$

Proposition 2.3

Let $\Psi : [1,4] \times [0,1] \rightarrow {\mathbb {R}}$ be a function defined by

$$\begin{aligned} \Psi (t,x) := 16 \psi _{1}(x) + 8 \psi _{2}(x) t + 3 \psi _{3}(x) t^{2}, \end{aligned}$$

(2.6)

where for $x\in [0,1],$

$$\begin{aligned} \psi _{1}(x):= & {} -2 + 27x + 21x^{2} - 37x^{3} +x^{4}, \\ \psi _{2}(x):= & {} 10 - 12x - 9x^{2} + 20x^{3} +x^{4} \end{aligned}$$

and

$$\begin{aligned} \psi _{3}(x) := x(3-5x-x^{2}-x^{3}). \end{aligned}$$

Then $\Psi (t,x) > 0$ for $1 \le t \le 4$ and $0 \le x \le 1$.

Proof

At first, note that the function $\psi _{3}$ has a unique zero $x=:x_{1}\approx 0.51839$ in (0, 1). Since $x_1\in (0.5,0.6)$ and for $x\in (0.5,0.6),$

$$\begin{aligned} \psi _{2}(x)> 10 -12\cdot (0.6) -9\cdot (0.6)^2 + 20\cdot (0.5)^3 + (0.5)^4=2.1225>0, \end{aligned}$$

(2.7)

it follows that

$$\begin{aligned} \frac{\partial }{\partial t} \Psi (t,x_1) = 8\psi _{2}(x_1) \not =0. \end{aligned}$$

For $x \not = x_1$, $(\partial /\partial t) \Psi (t,x)=0$ occurs at

$$\begin{aligned} t = \frac{-4\psi _{2}(x)}{3\psi _{3}(x)} =: t_{0}(x). \end{aligned}$$

We have

$$\begin{aligned} \frac{\partial }{\partial x}\Psi (t,x) \Big |_{t=t_{0}(x)} = \frac{16 \psi _{4}(x)}{3 \psi _{3}^{2}(x)}, \end{aligned}$$

where

$$\begin{aligned} \begin{aligned} \psi _{4}(x)&:= 3\psi _{1}'(x)\psi _{3}^{2}(x) - 2\psi _{2}'(x) \psi _{2}(x) \psi _{3}(x) + \psi _{3}'(x) \psi _{2}^{2}(x) \\&= 300 - 1000x + 1737x^{2} - 4912x^{3} + 2009x^{4} + 13706x^{5} - 17777x^{6} \\&\quad + 6596x^{7}- 1541x^{8} + 546x^{9} -184x^{10} + 16x^{11},\quad x\in (0,1). \end{aligned} \end{aligned}$$

The polynomial $\psi _{4}$ has a unique zero $x=:x_{2}\approx 0.388025$ in (0, 1). Since $x_1>0.5,$ so $\psi _3(x)>0,$ for $x\in (0,0.5).$ Additionally, since $\psi _2$ has no zero in (0, 1),  the inequality (2.7) is true on [0, 1]. Thus $t_0(x)<0$ for $x\in (0,0.5)$ and in consequence, the function $\Psi$ has no critical point in $(1,4) \times (0,1).$ Hence it is sufficient to show that $\Psi >0$ on the boundary of $[1,4] \times [0,1]$. We can easily check that the following inequalities hold:

$$\begin{aligned} \Psi (t,0)= & {} -32 + 80t \ge 48, \quad t \in [1,4], \\ \Psi (t,1)= & {} 160 + 80t -12t^{2} \ge 228, \quad t \in [1,4], \\ \Psi (4,x)= & {} 96(3+2x-2x^{2}) \ge 288, \quad x \in [0,1], \end{aligned}$$

and

$$\begin{aligned} \begin{aligned} \Psi (1,x)&= 3\left( 16 +115x +83x^{2} -145x^{3} +7x^{4} \right) \\&= 3\left( 16 + 53x^2 + 7x^{4} + 115x(1-x^{2}) + 30x^{2}(1-x) \right) \\&\ge 48, \quad x \in [0,1]. \end{aligned} \end{aligned}$$

Thus the proof of the proposition is completed. $\square$

Proposition 2.4

Let $\Phi : [3,4] \times [0,1] \rightarrow {\mathbb {R}}$ be a function defined by

$$\begin{aligned} \Phi (t,x) := 48\phi _{1}(x) +8\phi _{2}(x)t -3\phi _{3}(x)t^{2}, \end{aligned}$$

(2.8)

where for $x\in [0,1],$

$$\begin{aligned} \phi _{1}(x):= & {} 1 +7x +x^{2} -3x^{3}, \\ \phi _{2}(x):= & {} 5 -19x +10x^{2} +5x^{3} +x^{4} \end{aligned}$$

and

$$\begin{aligned} \phi _{3}(x) := x(-3 +5x +x^{2} +x^{3}). \end{aligned}$$

Then $\Phi (t,x) > 0$ for $3 \le t \le 4$ and $0 \le x \le 1$.

Proof

Since $\phi _{3}=-\psi _3,$ by the part of proof of Proposition 2.3, we at once have

$$\begin{aligned} \frac{\partial }{\partial t} \Phi (t,x_{1}) = 8\phi _{2}(x_{1}) \not =0. \end{aligned}$$

For $x \not = x_1$, $(\partial /\partial t)\Phi (t,x)=0$ occurs at

$$\begin{aligned} t= \frac{ 4\phi _{2}(x) }{ 3\phi _{3}(x) } =: t_{0}(x). \end{aligned}$$

We have

$$\begin{aligned} \frac{\partial }{\partial x}\Phi (t,x) \Big |_{t=t_{0}(x)} = \frac{16 \phi _{4}(x)}{3 \phi _{3}^{2}(x)}, \end{aligned}$$

where

$$\begin{aligned} \begin{aligned} \phi _{4}(x)&:= 9\phi _{1}'(x)\phi _{3}^{2}(x) + 2\phi _{2}'(x) \phi _{2}(x) \phi _{3}(x) - \phi _{3}'(x) \phi _{2}^{2}(x) \\&= 75 -250x +59x^{2} +532x^{3} -893x^{4} +558x^{5} -269x^{6} +844x^{7} \\&\quad -366x^{8} +16x^{9} -46x^{10} +4x^{11},\quad x\in (0,1). \end{aligned} \end{aligned}$$

The polynomial $\phi _4$ has exactly two zeros in (0, 1),  namely $x=:x_2\approx 0.414034$ and $x=:x_3\approx 0.663886.$ We have $t_{0}(x_2) \approx 3.59845$ and $t_{0}(x_3) = -2.95522.$ Therefore the function $\Phi$ has a unique critical point $(t_0(x_2),x_2)$ in $(3,4) \times (0,1)$. For $(t,x)\in [3.58,3.61]\times [0.39,0.43]$ by simple computing, we show that $\Phi (t,x)>0.$ Thus, particularly $\Phi (t_0(x_2),x_2)>0.$ Therefore it is sufficient to show that $\Phi >0$ on the boundary of $[3,4] \times [0,1].$ We can easily check that the following inequalities hold:

$$\begin{aligned} \Phi (t,0)= & {} 8(6+5t) \ge 168 , \quad t \in [3,4], \\ \Phi (t,1)= & {} 4\left( 72 +4t -3t^{2} \right) \ge 160, \quad t \in [3,4], \\ \Phi (4,x)= & {} 16\left( 13 -8x +8x^{2} -2x^{3} -x^{4} \right) \ge 16(2 +8x^{2}) \ge 32, \quad x \in [0,1], \end{aligned}$$

and

$$\begin{aligned} \Phi (3,x) = 3\left( 56 -13x +51x^{2} -17x^{3} -x^{4} \right) \ge 3(25+ 51x^{2}) \ge 75, \quad x \in [0,1]. \end{aligned}$$

Thus the proof of the proposition is completed. $\square$

Finally, we estimate now the third-order Hankel determinant $H_{3,1}(f)$ for $f \in {{\mathcal {S}}}^{*}$.

Theorem 2.5

If $f \in {{\mathcal {S}}}^{*}$ is the form (1.1), then

$$\begin{aligned} |H_{3,1}(f)| \le \frac{8}{9}. \end{aligned}$$

(2.9)

Proof

Let $f \in {{\mathcal {S}}}^{*}$ be of the form (1.1). Then by (1.2) we have

$$\begin{aligned} zf'(z) = f(z)p(z), \quad z\in {\mathbb {D}}, \end{aligned}$$

(2.10)

for some function $p \in {{\mathcal {P}}}$ of the form (1.5). Since the class ${{\mathcal {P}}}$ is invariant under the rotations, we may assume that $c:=c_1 \in [0,2]$ (e.g., [5, Vol. I, p. 80, Theorem 3]). Putting the series (1.1) and (1.5) into (2.10) and by equating the coefficients we get

$$\begin{aligned} a_{2}=c, \quad a_{3}=\frac{1}{2} \left( c^{2} +c_{2} \right) , \quad a_{4}=\frac{1}{6}\left( c^{3} + 3cc_{2} + 2c_{3} \right) \end{aligned}$$

and

$$\begin{aligned} a_{5} = \frac{1}{24} \left( c^{4} + 6c^{2}c_{2} +8cc_{3} + 3c_{2}^{2} +6c_{4} \right) . \end{aligned}$$

Hence

$$\begin{aligned} \begin{aligned} H_{3,1}(f)&= -a_{3}^{3} + 2a_{2}a_{3}a_{4} - a_{4}^{2} -a_{2}^{2}a_{5} + a_{3}a_{5} \\&= \frac{1}{144} \left( -c^{6} + 3c^{4}c_{2} - 9c_{2}^{3} +8c^{3}c_{3} + 24cc_{2}c_{3} -16c_{3}^{2} \right. \\&\quad \left. +18c_{2}c_{4} -9c^{2}c_{2}^{2} -18c^{2}c_{4} \right) . \end{aligned} \end{aligned}$$

Now using the equalities (2.1)–(2.3), by straightforward algebraic computation we have

$$\begin{aligned} H_{3,1}(f) = \frac{1}{1152} (c^{2}-4) \left[ \gamma _{1}(c,\zeta ) + \gamma _{2}(c,\zeta )\eta + \gamma _{3}(c,\zeta )\eta ^{2} + \Gamma (c,\zeta ,\eta ) \xi \right] , \end{aligned}$$

(2.11)

where for $\zeta ,\,\eta ,\,\xi \in \overline{{\mathbb {D}}},$

$$\begin{aligned} \gamma _{1}(c,\zeta ):= & {} c^{2}\zeta \left[ -3c^{2} + \left( 44-5c^{2} \right) \zeta + \left( 40-c^{2} \right) \zeta ^{2} \right] - c^{2}\left( 4-c^{2} \right) \zeta ^{4}, \\ \gamma _{2}(c,\zeta ):= & {} -4c\left( 1- |\zeta |^{2} \right) \left[ 3c^{2} + 4\left( 5+c^{2} \right) \zeta - \left( 4-c^{2} \right) \zeta ^{2} \right] , \\ \gamma _{3}(c,\zeta ):= & {} 32 \left( 4-c^{2} \right) -28|\zeta |^{2} \left( 4-c^{2} \right) -36c^{2} {\overline{\zeta }} \left( 1-|\zeta |^{2} \right) - 4\left( 4-c^{2} \right) |\zeta |^{4}, \end{aligned}$$

and

$$\begin{aligned} \Gamma (c,\zeta ,\eta ) := 36\left[ c^{2} + \left( c^{2}-4 \right) \zeta \right] \left( 1-|\zeta |^{2} \right) \left( 1-|\eta |^{2} \right) . \end{aligned}$$

Setting $x:=|\zeta | \in [0,1],$$y:=|\eta | \in [0,1]$ and taking into account that $|\xi |\le 1,$ from (2.11) we get

$$\begin{aligned} \begin{aligned}&|H_{3,1}(f)| \\&\quad \le \frac{1}{1152} \left( 4-c^{2} \right) \left[ | \gamma _{1}(c,\zeta ) | + | \gamma _{2}(c,\zeta ) | |\eta | + | \gamma _{3}(c,\zeta ) | |\eta |^{2} + |\Gamma (c,\zeta ,\eta )| \right] \\&\quad \le \frac{1}{1152} (4-c^2)F(c,x,y), \end{aligned} \end{aligned}$$

(2.12)

where

$$\begin{aligned} F(c,x,y) := f_{1}(c,x) + f_{2}(c,x)y + f_{3}(c,x)y^{2} + f_{4}(c,x)\left( 1-y^{2} \right) , \end{aligned}$$

(2.13)

with

$$\begin{aligned} f_{1}(c,x):= & {} c^{2}x \left[ 3c^{2} + (44-5c^{2})x + (40-c^{2}) x^{2} \right] + c^{2}(4-c^{2})x^{4}, \\ f_{2}(c,x):= & {} 4c(1-x^{2}) \left[ 3c^{2} + 4(5+c^{2})x + (4-c^{2})x^{2} \right] , \\ f_{3}(c,x):= & {} 32(4-c^{2}) + 28x^{2}(4-c^{2}) + 36c^{2}x(1-x^{2}) + 4(4-c^{2})x^{4} \end{aligned}$$

and

$$\begin{aligned} f_{4}(c,x) := 36 \left[ c^{2}+(4-c^{2})x \right] (1-x^{2}). \end{aligned}$$

Now, we will show that

$$\begin{aligned} (4-c^2)F(c,x,y) \le 1024 \end{aligned}$$

(2.14)

for $c\in [0,2],\ x\in [0,1]$ and $y\in [0,1].$

I. Assume first that $c\in [1,2].$ Then by (2.13) we have

$$\begin{aligned} \begin{aligned}&F(c,x,y) \\&\quad \le f_{1}(c,x) + cf_{2}(c,x)y + f_{3}(c,x)y^{2} + f_{4}(c,x)\left( 1-y^{2} \right) \\&\quad = f_{1}(c,x) + f_{4}(c,x) + cf_{2}(c,x)y + (f_{3}(c,x)-f_4(c,x))y^{2} \\&\quad =: F_1(c,x,y) . \end{aligned} \end{aligned}$$

(2.15)

(a) Consider the case $f_{3}(c,x)\ge f_4(c,x)$ in $[1,2]\times [0,1].$ Let

$$\begin{aligned} \Omega _{1} := \left\{ (c,x) \in [1,2]\times [0,1]: f_{3}(c,x) \ge f_{4}(c,x) \right\} . \end{aligned}$$

By (2.15) we get

$$\begin{aligned} \begin{aligned} F_1(c,x,y)&\le F_1(c,x,1)\\&= f_{1}(c,x) + cf_{2}(c,x) + f_{3}(c,x),\quad (c,x)\in \Omega _1,\ y\in [0,1]. \end{aligned} \end{aligned}$$

Set $t:=4-c^2.$ Clearly, $t\in [0,3].$ Define

$$\begin{aligned} {\tilde{F}}_{1}(t,x):=tF_1(\sqrt{4-t},x,1), \quad (\sqrt{4-t},x)\in \Omega _1. \end{aligned}$$

A simple computing yields

$$\begin{aligned} \begin{aligned} {\tilde{F}}_{1}(t,x)&= t \left\{ (4-t)x \left[ 12-3t+(24+5t)x+(36+t)x^2 \right] \right. \\&\quad + t(4-t)x^4+32t+28tx^2+36(4-t)x(1-x^2) + 4tx^4 \\&\left. \quad + 4(4-t)(1-x^2) \left[ 12-3t+4(9-t)x+tx^2 \right] \right\} \\&= 96(2 + 8x - x^{2} - 6x^{3})t - 4(16 + 67x - 34x^{2} - 53x^{3} + 2x^{4})t^{2} \\&\quad + (12+19x-21x^{2}-17x^{3} + 3x^{4})t^{3},\quad (\sqrt{4-t},x)\in \Omega _1. \end{aligned} \end{aligned}$$

Hence and by Proposition 2.2 we have

$$\begin{aligned} \frac{\partial }{\partial t}{\tilde{F}}_{1}(t,x) = \Theta (t,x)>0,\quad (\sqrt{4-t},x)\in \Omega _1, \end{aligned}$$

where the function $\Theta$ is defined by (2.4). Thus the function $[0,3]\ni t\mapsto {\tilde{F}}_{1}(t,\cdot )$ is increasing, and therefore we have

$$\begin{aligned} {\tilde{F}}_{1}(t,x) \le {\tilde{F}}_{1}(3,x) = 9\left( 36+45x+41x^2 -31x^3 +x^4 \right) < 1024,\quad x\in [0,1]. \end{aligned}$$

(2.16)

Indeed, the last inequality is true since, as easy to verify the inequality

$$\begin{aligned} -700+405x+369x^2-279x^3+9x^4<0,\quad x\in [0,1], \end{aligned}$$

holds. Thus the inequality (2.16) confirms the inequality (2.14).

(b) Consider the case $f_{3}(c,x)< f_4(c,x)$ in $[1,2]\times [0,1].$ Let

$$\begin{aligned} \Omega _{2} := \left\{ (c,x) \in [1,2]\times [0,1]: f_{3}(c,x) < f_{4}(c,x) \right\} . \end{aligned}$$

Since $f_{2}(c,x)\ge 0$ in $[1,2]\times [0,1],$ so

$$\begin{aligned} \sigma :=\frac{-c f_{2}(c,x)}{2(f_{3}(c,x) - f_{4}(c,x))}\ge 0,\quad (c,x)\in \Omega _2. \end{aligned}$$

If $\sigma \ge 1,$ i.e., if $cf_{2}(c,x)+2(f_{3}(c,x) - f_{4}(c,x))\ge 0,$ then

$$\begin{aligned} \begin{aligned} F_1(c,x,y)&\le F_1(c,x,1)\\&= f_{1}(c,x) + cf_{2}(c,x) + f_{3}(c,x),\quad (c,x)\in \Omega _2,\ y\in [0,1]. \end{aligned} \end{aligned}$$

and repeating the argumentation of Part (a) we get the inequality (2.14).

If $\sigma <1,$ i.e., if $cf_{2}(c,x)+2(f_{3}(c,x) - f_{4}(c,x)) < 0,$ then

$$\begin{aligned} \begin{aligned} F_1(c,x,y)&\le F_1(c,x,\sigma )=\frac{-c^2f_2^2(c,x)}{4(f_{3}(c,x) - f_{4}(c,x))}+f_{1}(c,x)+f_{4}(c,x)\\&\le \frac{\left[ -2(f_3(c,x)-f_4(c,x)) \right] ^2}{4(f_{3}(c,x) - f_{4}(c,x))}+f_{1}(c,x)+f_{4}(c,x)\\&\le f_{1}(c,x)+f_3(c,x)+2f_{4}(c,x)=:F_2(c,x),\quad (c,x)\in \Omega _2. \end{aligned} \end{aligned}$$

Set $t:=c^2.$ Clearly, $t\in [1,4].$ Define

$$\begin{aligned} {\tilde{F}}_{2}(t,x) := (4-t)F_2(\sqrt{t},x),\quad (\sqrt{t},x)\in \Omega _2. \end{aligned}$$

A simple computing yields

$$\begin{aligned} \begin{aligned} {\tilde{F}}_{2}(t,x)&= (4-t) \left\{ tx \left[ 3t+(44-5t)x+(40-t)x^2 \right] +t(4-t)x^4 \right. \\&\quad + 32(4-t)+28x^2(4-t)+36tx(1-x^2)+4(4-t)x^4\\&\quad \left. + 72\left[ t+(4-t)x \right] (1-x^2) \right\} \\&= -\left\{ -64\left( 8+18x+7x^2-18x^3+x^4 \right) \right. \\&\quad + 16 \left( -2 + 27x + 21x^{2} - 37x^{3} +x^{4} \right) t \\&\quad + 4 \left( 10 -12x -9x^{2} + 20x^{3} +x^{4} \right) t^2 \\&\quad \left. + x\left( 3-5x-x^{2}-x^{3} \right) t^3 \right\} ,\quad (\sqrt{t},x)\in \Omega _2. \end{aligned} \end{aligned}$$

Hence and by Proposition 2.3 we have

$$\begin{aligned} \frac{\partial }{\partial t}{\tilde{F}}_{2}(t,x) = -\Psi (t,x)<0,\quad (\sqrt{t},x)\in \Omega _2, \end{aligned}$$

where the function $\Psi$ is defined by (2.6). Thus the function $[1,4]\ni t\mapsto {\tilde{F}}_{2}(t,\cdot )$ is decreasing, and therefore we have

$$\begin{aligned} {\tilde{F}}_{2}(t,x) \le {\tilde{F}}_{2}(1,x) = 9\left( 56+85x+17x^2-71x^3+5x^4 \right) < 1024,\quad x\in [0,1]. \end{aligned}$$

(2.17)

Indeed, the last inequality is true since, as easy to verify the inequality

$$\begin{aligned} -520+765x+153x^2-639x^3+45x^4<0,\quad x\in [0,1], \end{aligned}$$

holds. Thus the inequality (2.17) confirms the inequality (2.14).

II. Assume that $c\in (0,1).$ Then by (2.13) we have

$$\begin{aligned} \begin{aligned}&F(c,x,y) \\&\quad \le f_{1}(c,x) + \frac{1}{c}f_{2}(c,x)y + f_{3}(c,x)y^{2} + f_{4}(c,x)\left( 1-y^{2} \right) \\&\quad = f_{1}(c,x) + f_{4}(c,x) + \frac{1}{c}f_{2}(c,x)y + (f_{3}(c,x)-f_4(c,x))y^{2} \\&\quad =: F_3(c,x,y). \end{aligned} \end{aligned}$$

(2.18)

(a) Consider the case $f_{3}(c,x)\ge f_4(c,x)$ in $(0,1)\times [0,1].$ Let

$$\begin{aligned} \Omega _{3} := \left\{ (c,x) \in (0,1)\times [0,1]: f_{3}(c,x) \ge f_{4}(c,x) \right\} . \end{aligned}$$

By (2.18) we get

$$\begin{aligned} \begin{aligned} F_3(c,x,y)&\le F_3(c,x,1)\\&= f_{1}(c,x) + \frac{1}{c}f_{2}(c,x) + f_{3}(c,x),\quad (c,x)\in \Omega _3,\ y\in [0,1]. \end{aligned} \end{aligned}$$

Set $t:=4-c^2.$ Clearly, $t\in (3,4).$ Define

$$\begin{aligned} {\tilde{F}}_{3}(t,x):=tF_3(\sqrt{4-t},x,1), \quad (\sqrt{4-t},x)\in \Omega _3. \end{aligned}$$

A simple computing yields

$$\begin{aligned} \begin{aligned} {\tilde{F}}_{3}(t,x)&= t \left\{ 3(4-t)^2x+(4-t)(24+5t)x^2+(4-t)(36+t)x^3+(4-t)tx^4 \right. \\&\quad + 4(1-x^2)\left[ 12-3t+4(9-t)x+tx^2\right] \\&\quad \left. + 32t+28tx^2+36(4-t)x(1-x^2)+4tx^4 \right\} \\&= 48 \left( 1 +7x +x^{2} -3x^{3} \right) t + 4\left( 5 -19x +10x^{2} +5x^{3} +x^{4} \right) t^2 \\&\quad - x\left( -3 +5x +x^{2} +x^{3} \right) t^3 ,\quad (\sqrt{4-t},x)\in \Omega _3. \end{aligned} \end{aligned}$$

Hence and by Proposition 2.4 we have

$$\begin{aligned} \frac{\partial }{\partial t}{\tilde{F}}_{3}(t,x) = \Phi (t,x)>0,\quad (\sqrt{4-t},x)\in \Omega _3, \end{aligned}$$

where the function $\Phi$ is defined by (2.8). Thus the function $(3,4)\ni t\mapsto {\tilde{F}}_{1}(t,\cdot )$ is increasing, and therefore we have

$$\begin{aligned} {\tilde{F}}_{3}(t,x) \le 512+320x+512x^2-320x^3 \le 1024. \end{aligned}$$

(2.19)

Indeed, the last inequality is true since so is the following one

$$\begin{aligned} -512+320x+512x^2-320x^3=(1-x^2)(320x-512)\le 0,\quad x\in [0,1]. \end{aligned}$$

Thus the inequality (2.19) confirms the inequality (2.14).

(b) Consider the case $f_{3}(c,x)< f_4(c,x)$ in $(0,1)\times [0,1]$ which is equivalent to

$$\begin{aligned} \begin{aligned}&32-8c^2+28x^2-7c^2x^2+9c^2x-9c^2x^3+4x^4-c^2x^4\\&\quad < 9c^2-9c^2x^2+36x-9c^2x-36x^3+9c^2x^3 \end{aligned} \end{aligned}$$

(2.20)

for $c\in (0,1)$ and $x\in [0,1].$ Note that

$$\begin{aligned} 17-18x-2x^2+18x^3+x^4>0,\quad x\in [0,1]. \end{aligned}$$

Thus the inequality (2.20) can be written as

$$\begin{aligned} c^2>\frac{32-36x+28x^2+36x^3+4x^4}{17-18x-2x^2+18x^3+x^4},\quad c\in (0,1),\ x\in [0,1]. \end{aligned}$$

(2.21)

However,

$$\begin{aligned} \frac{32-36x+28x^2+36x^3+4x^4}{17-18x-2x^2+18x^3+x^4}\ge 1,\quad x\in [0,1]. \end{aligned}$$

(2.22)

Indeed, the above inequality is equivalent to

$$\begin{aligned} 32-36x+28x^2+36x^3+4x^4\ge 17-18x-2x^2+18x^3+x^4,\quad x\in [0,1], \end{aligned}$$

which by simplifying is equivalent to the true inequality

$$\begin{aligned} (x-1)^4+10x^3+4(x-1)^2+6x\ge 0,\quad x\in [0,1]. \end{aligned}$$

Thus by (2.21) and (2.22) it follows that $c\ge 1$ which contradicts the assumption.

III. At the end assume that $c=0.$ Then by (2.13) we have

$$\begin{aligned} \begin{aligned} F(0,x,y)&= 16\left( (8-9x+7x^2+9x^3+x^4)y^2+9x(1-x^2)\right) \\&\le 16(8+7x^2+x^4) \le 256,\quad x\in [0,1],\ y\in [0,1]. \end{aligned} \end{aligned}$$

Summarizing, from all considering cases it follows that the inequality (2.14) holds which together with (2.12) shows (2.9). $\square$

Remark 2.6

Although the constant 8 / 9 improves essentially the estimates found in [1] and [19], it is not the best possible. To find the sharp estimate of the Hankel determinant $H_{3,1}(f)$ for starlike functions is still an open problem.