1 Introduction

In this paper, we study the existence of entire k-convex radial solutions to the following problem of Hessian type system:

$$ \textstyle\begin{cases} \sigma _{k} (\lambda (D^{2} u+\mu \vert \nabla u \vert I ) )=p( \vert x \vert ) f_{1}(u)f_{2}(v), & x \in B_{1}(0), \\ \sigma _{l} (\lambda (D^{2} v+\nu \vert \nabla v \vert I ) )=q( \vert x \vert ) g_{1}(u)g_{2}(v), & x \in B_{1}(0), \\ u=v=0,& x\in \partial B_{1}(0), \end{cases} $$
(1.1)

where \(k,l=1,2,\ldots, N\), \(\mu , \nu \ge 0\) are constants, \(B_{1}(0)\) is the unit ball in \(\mathbb{R}^{N}\), for any \(N \times N\) real symmetric matrix A, \(\lambda (A)\) denotes the eigenvalues of A, \(D^{2} u(x)= ( \frac{\partial ^{2} u(x)}{\partial x_{i} \partial x_{j}} )\) denotes the Hessian matrix of the function \(u \in C^{2} (\overline{B_{1}(0)} )\), ∇u denotes the gradient of u, and \(\sigma _{k}(\lambda )=\sum_{1 \leq i_{1}<\cdots <i_{k} \leq N} \lambda _{i_{1}} \cdots \lambda _{i_{k}}\) denotes the kth elementary symmetric function of \(\lambda = (\lambda _{1},\ldots, \lambda _{N} ) \in \mathbb{R}^{N}\).

For p, q, \(f_{1}\), \(f_{2}\), \(g_{1}\), \(g_{2}\), we introduce the following conditions:

  1. (H1)

    \(p,q\in C([0,1],(0,+\infty ))\). \(f_{1},f_{2},g_{1},g_{2}\in C((-\infty ,0],[0,+\infty ))\) are decreasing.

  2. (H2)

    For any \(a>0\), the integral \(\int _{-\infty }^{-a} \frac{d\tau }{(f_{1}(\tau )f_{2}(\tau ))^{\frac{1}{k}}+(g_{1}(\tau )g_{2}(\tau ))^{\frac{1}{l}}}\) is divergent.

  3. (H3)

    For any \(a>0\), the integral \(\int _{-a}^{0} \frac{d\tau }{(f_{1}(\tau )f_{2}(\tau ))^{\frac{1}{k}}+(g_{1}(\tau )g_{2}(\tau ))^{\frac{1}{l}}}\) is divergent.

Denote

$$ \Gamma _{k}:= \bigl\{ \lambda \in \mathbb{R}^{N}: \sigma _{j}(\lambda )>0, 1 \leq j \leq k \bigr\} . $$

We say that a function \(u \in C^{2} (\overline{B_{1}(0)} )\) is k-convex in \(B_{1}(0)\) if \(\lambda (D^{2} u(x) ) \in \Gamma _{k}\) for all \(x \in B_{1}(0)\).

In (1.1), if \(\mu =0\) and \(f_{2}(v)\equiv 1\), the first equation in the system becomes the following k-Hessian type equation:

$$ \sigma _{k} \bigl(\lambda \bigl(D^{2} u \bigr) \bigr)=p \bigl( \vert x \vert \bigr) f_{1}(u); $$
(1.2)

if \(\mu =\nu =0\) and \(f_{1}(u)=g_{2}(v)\equiv 1\), the system becomes the following coupling k-Hessian system:

$$ \textstyle\begin{cases} \sigma _{k} (\lambda (D^{2} u ) )=p( \vert x \vert ) f_{2}(v), \\ \sigma _{l} (\lambda (D^{2} v ) )=q( \vert x \vert ) g_{1}(u). \end{cases} $$
(1.3)

Related to k-Hessian equations, if \(k=1\) the k-Hessian equations become the well-known Laplacian equations, and if \(k=N\) the k-Hessian equations become the Monge–Ampère equations. Concerning Laplacian equations and Monge–Ampère equations, there are a great number of research papers, see for examples [1, 6, 7, 22] and the references therein. Here we specially mention Keller [15], Osserman [21], and Lair and Wood [17] for Laplacian equations and Cheng and Yau [2] and Laser and McKenna [19] for Monge–Ampère equations. Similar situations occur for coupling k-Hessian system (1.3), although in this case there are not so many research papers. Here we only mention Lair and Wood [18] and Cîrstea and Rădulescu [3] for coupling Laplacian systems and Wang and An [24] and Zhang and Qi [26] for coupling Monge–Ampère systems.

For general k-Hessian equation (1.2), when \(p \equiv 1\) and \(f(u)=u^{\gamma k}\), \(\gamma >1\), Jin, Li, and Xu [13] showed the nonexistence of entire k-convex positive solutions. When \(p \equiv 1\), Ji and Bao [11] gave necessary and sufficient conditions on the existence of entire positive k-convex radial solutions. If we generalize \(p( \vert x \vert )f(u)\) to \(f(x,u)\), de Oliveira, do Ó, and Ubilla obtained the existence of k-convex radial solutions in the case of supercritical nonlinearity by means of variational techniques (see [5] and the references therein for research in this direction). For general k-Hessian equation (1.2) and coupling k-Hessian system (1.3), Zhang and Zhou [27] obtained several results on the existence of entire positive k-convex radial solutions. We refer to the papers of Feng and Zhang [8] and Gao, He, and Ran [9] and the references therein for research on coupling k-Hessian system (1.3).

It is obvious that the k-Hessian type equation

$$ \sigma _{k} \bigl(\lambda \bigl(D^{2} u+\mu \vert \nabla u \vert I \bigr) \bigr)=p \bigl( \vert x \vert \bigr) f(u) $$

is a generalization of k-Hessian equation (1.2), but it is a special case of the following fully nonlinear Hessian equation:

$$ F \bigl(\lambda \bigl(D^{2} u+A(x,u,\nabla u) \bigr) \bigr)=f(x,u, \nabla u). $$
(1.4)

See Guan and Jiao [10] and Jiang and Trudinger [12] and the references therein for research on fully nonlinear Hessian equation (1.4). Here we also want to mention the work of Dai [4] for similar study.

Inspired by the works above, and as we know that now there are no papers on the problem of k-Hessian type system (1.1), we obtain the following results in this paper.

Theorem 1.1

Under conditions (H1) and (H2), if \(f_{1}(0)g_{2}(0)\neq0\) and \(f_{2}(0)+g_{1}(0)\neq0\), then problem (1.1) admits an entire k-convex radial solution \((u,v) \in C^{2} (\overline{B_{1}(0)} )\times C^{2} ( \overline{B_{1}(0)} )\).

Remark 1.1

In the case of \(f_{1}(0)g_{2}(0)=0\), if \(f_{1}(0)=g_{2}(0)=0\), then there is a trivial solution \((u,v)=(0,0)\) to problem (1.1); if \(f_{1}(0)=0\) or \(g_{2}(0)=0\), then there is a semi-trivial solution \((u,v)=(0,v)\) or \((u,v)=(u,0)\) to problem (1.1); moreover, the semi-trivial solution may become trivial if \(f_{1}(0)=0\) with \(g_{1}(0)=0\) or \(g_{2}(0)=0\) with \(f_{2}(0)=0\).

In the case of \(f_{2}(0)+g_{1}(0)=0\), there is a trivial solution \((u,v)=(0,0)\) to problem (1.1).

Theorem 1.2

Under conditions (H1) and (H3), problem (1.1) admits no entire k-convex radial solution \((u,v) \in C^{2} (\overline{B_{1}(0)} )\times C^{2} ( \overline{B_{1}(0)} )\).

Remark 1.2

In this case, \(f_{1}(0)f_{2}(0)=g_{1}(0)g_{2}(0)=0\), and there is a trivial solution \((u,v)=(0,0)\) to problem (1.1).

2 Preliminaries

In this section, we give some preliminary results which will be used to prove the main results in the next section.

Lemma 2.1

Assume \(\varphi (r) \in C^{2}[0, 1]\) with \(\varphi ^{\prime }(0)=0\). Then, for \(u(x)=\varphi (r)\), there holds that \(u \in C^{2} (\overline{B_{1}(0)} )\) and

$$ \lambda \bigl(D^{2} u+\eta \vert \nabla u \vert I \bigr)= \textstyle\begin{cases} (\varphi ^{\prime \prime }(r)+\eta \varphi ^{\prime }(r), ( \frac{1}{r}+\eta )\varphi ^{\prime }(r),\ldots, ( \frac{1}{r}+\eta )\varphi ^{\prime }(r) ), &r \in (0, 1], \\ (\varphi ^{\prime \prime }(0), \varphi ^{\prime \prime }(0),\ldots, \varphi ^{\prime \prime }(0) ), &r=0, \end{cases} $$

and further

$$ \begin{aligned} &\sigma _{k} \bigl(\lambda \bigl(D^{2} u+\eta \vert \nabla u \vert I \bigr) \bigr) \\ &\quad = \textstyle\begin{cases} C_{N-1}^{k-1} (\varphi ^{\prime \prime }(r)+\eta \varphi ^{\prime }(r)) ( (\frac{1}{r}+\eta )\varphi ^{\prime }(r) )^{k-1}+C_{N-1}^{k} ( (\frac{1}{r}+\eta )\varphi ^{\prime }(r) )^{k}, &r \in (0, 1], \\ C_{N}^{k} (\varphi ^{\prime \prime }(0) )^{k}, &r=0, \end{cases}\displaystyle \end{aligned} $$

where \(C_{N}^{k}=\frac{N!}{k!(N-k)!}\).

Proof

It is immediate that, for \(x \neq 0\), \(1 \leq i, j \leq N\),

$$ \frac{\partial u(x)}{\partial x_{i}}= \biggl( \frac{\varphi ^{\prime }(r)}{r} \biggr) x_{i} $$

and

$$ \frac{\partial ^{2} u(x)}{\partial x_{i} \partial x_{j}}= \biggl( \frac{\varphi ^{\prime \prime }(r)}{r^{2}} \biggr) x_{i} x_{j}- \biggl( \frac{\varphi ^{\prime }(r)}{r^{3}} \biggr) x_{i} x_{j}+ \biggl( \frac{\varphi ^{\prime }(r)}{r} \biggr) \delta _{i j}. $$

Further if define

$$ \frac{\partial u(0)}{\partial x_{i}}=0,\qquad \frac{\partial ^{2} u(0)}{\partial x_{i} \partial x_{j}}=\varphi ^{ \prime \prime }(0) \delta _{i j}, $$

then \(u \in C^{2} (\overline{B_{1}(0)} )\).

Now it is easy to show the two equalities for \(\lambda (D^{2} u+\eta \vert \nabla u \vert I )\) and \(\sigma _{k} (\lambda (D^{2} u+\eta \vert \nabla u \vert I ) )\). □

Lemma 2.2

Let \(f\in C(-\infty ,0]\) be decreasing. Assume that \(\varphi \in C^{0}[0, 1] \cap C^{1}(0, 1]\) is a solution of the Cauchy problem

$$ \textstyle\begin{cases} \varphi ^{\prime }(r)= (\frac{k}{C_{N-1}^{k-1}}\mathrm{e}^{-\psi _{k, \eta }(r)} \int _{0}^{r} \mathrm{e}^{\psi _{k,\eta }(s)} \frac{s^{k-1}p(s)}{(1+\eta s)^{k-1}} f(\varphi (s)) \,d s )^{ \frac{1}{k}},\quad 0< r< 1, \\ \varphi (1)=0, \end{cases} $$

where

$$ \psi _{k,\eta }(r)=\frac{k}{C_{N-1}^{k-1}} \bigl(C_{N}^{k} \eta r+C_{N-1}^{k} \ln r \bigr). $$

Then \(\varphi \in C^{2}[0, 1]\), and it satisfies the problem

$$ \textstyle\begin{cases} C_{N-1}^{k-1}\varphi ^{\prime \prime }(r) (\varphi ^{\prime }(r) )^{k-1}r + (C_{N}^{k}\eta r+C_{N-1}^{k} ) ( \varphi ^{\prime }(r) )^{k} =\frac{r^{k}p(r)}{(1+\eta r)^{k-1}} f( \varphi (r)), \quad 0< r< 1, \\ \varphi ^{\prime }(0)=0. \end{cases} $$

Furthermore, if φ is nontrivial, i.e., \(\varphi (r)<0\) for \(0\le r<1\), then

$$ \lambda _{r}:= \biggl(\varphi ^{\prime \prime }(r)+\eta \varphi ^{ \prime }(r), \biggl(\frac{1}{r}+\eta \biggr)\varphi ^{\prime }(r),\ldots, \biggl(\frac{1}{r}+\eta \biggr)\varphi ^{\prime }(r) \biggr) \in \Gamma _{k} $$

for \(0 \leq r<1\).

Proof

It is easy to see that \(\varphi (r) \in C^{2}[0, 1]\).

From

$$ \varphi ^{\prime }(r)= \biggl(\frac{k}{C_{N-1}^{k-1}}\mathrm{e}^{-\psi _{k, \eta }(r)} \int _{0}^{r} \mathrm{e}^{\psi _{k,\eta }(s)} \frac{s^{k-1}p(s)}{(1+\eta s)^{k-1}} f \bigl(\varphi (s) \bigr) \,d s \biggr)^{ \frac{1}{k}} $$

we have

$$ \bigl(\varphi ^{\prime }(r) \bigr)^{k}=\frac{k}{C_{N-1}^{k-1}} \mathrm{e}^{-\psi _{k,\eta }(r)} \int _{0}^{r} \mathrm{e}^{\psi _{k,\eta }(s)} \frac{s^{k-1}p(s)}{(1+\eta s)^{k-1}} f \bigl(\varphi (s) \bigr) \,d s, $$

and further differentiating with respect to r we have

$$ C_{N-1}^{k-1}\varphi ^{\prime \prime }(r) \bigl(\varphi ^{\prime }(r) \bigr)^{k-1}r + \bigl(C_{N}^{k} \eta r+C_{N-1}^{k} \bigr) \bigl( \varphi ^{\prime }(r) \bigr)^{k} =\frac{r^{k}p(r)}{(1+\eta r)^{k-1}} f \bigl( \varphi (r) \bigr). $$

If φ is nontrivial, it is easy to see that φ is increasing, so for \(0 \leq r<1\) we conclude \(\varphi (r)<\varphi (1)=0\), \(f(\varphi (r))>f(\varphi (1))\ge 0\) and further

$$ \sigma _{k} (\lambda _{r} )=f \bigl(\varphi (r) \bigr)>0 \quad \text{for }0 \leq r< 1. $$

By the properties of kth elementary symmetric functions (see for example [20]), we know \(\sigma _{j} (\lambda _{r} )>0\) for \(1\le j< k\) and \(0 \leq r<1\). Therefore we conclude the lemma. □

3 Proofs of the main results

In this section, we prove the main results in this paper, i.e., the existence and nonexistence of entire k-convex radial solutions for problem (1.1).

Proof of Theorem 1.1

From the system

$$ \textstyle\begin{cases} C_{N-1}^{k-1}u^{\prime \prime }(r) (u^{\prime }(r) )^{k-1}r + (C_{N}^{k}\mu r+C_{N-1}^{k} ) (u^{\prime }(r) )^{k} =\frac{r^{k}p(r)}{(1+\mu r)^{k-1}} f_{1}(u(r))f_{2}(v(r)), \\ C_{N-1}^{l-1}v^{\prime \prime }(r) (v^{\prime }(r) )^{l-1}r + (C_{N}^{l}\nu r+C_{N-1}^{l} ) (v^{\prime }(r) )^{l} =\frac{r^{l}q(r)}{(1+\nu r)^{l-1}} g_{1}(u(r))g_{2}(v(r)), \end{cases} $$

we get

$$ \textstyle\begin{cases} u^{\prime }(r)= (\frac{k}{C_{N-1}^{k-1}}\mathrm{e}^{-\psi _{k,\mu }(r)} \int _{0}^{r} \mathrm{e}^{\psi _{k,\mu }(s)} \frac{s^{k-1}p(s)}{(1+\mu s)^{k-1}} f_{1}(u(s))f_{2}(v(s)) \,d s )^{ \frac{1}{k}}, \\ v^{\prime }(r)= (\frac{l}{C_{N-1}^{l-1}}\mathrm{e}^{-\psi _{l,\nu }(r)} \int _{0}^{r} \mathrm{e}^{\psi _{l,\nu }(s)} \frac{s^{l-1}q(s)}{(1+\nu s)^{l-1}} g_{1}(u(s))g_{2}(v(s)) \,d s )^{ \frac{1}{k}}, \end{cases} $$

furthermore we have

$$ \textstyle\begin{cases} u(r)=\int _{1}^{r} (\frac{k}{C_{N-1}^{k-1}}\mathrm{e}^{-\psi _{k, \mu }(t)} \int _{0}^{t} \mathrm{e}^{\psi _{k,\mu }(s)} \frac{s^{k-1}p(s)}{(1+\mu s)^{k-1}} f_{1}(u(s))f_{2}(v(s)) \,d s )^{ \frac{1}{k}}\,dt, \\ v(r)=\int _{1}^{r} (\frac{l}{C_{N-1}^{l-1}}\mathrm{e}^{-\psi _{l, \nu }(t)} \int _{0}^{t} \mathrm{e}^{\psi _{l,\nu }(s)} \frac{s^{l-1}q(s)}{(1+\nu s)^{l-1}} g_{1}(u(s))g_{2}(v(s)) \,d s )^{ \frac{1}{k}}\,dt. \end{cases} $$

Define

$$ \mathcal{L}(u,v) (r)= \begin{pmatrix} \int _{1}^{r} (\frac{k}{C_{N-1}^{k-1}}\mathrm{e}^{-\psi _{k,\mu }(t)} \int _{0}^{t} \mathrm{e}^{\psi _{k,\mu }(s)} \frac{s^{k-1}p(s)}{(1+\mu s)^{k-1}} f_{1}(u(s))f_{2}(v(s)) \,d s )^{ \frac{1}{k}}\,dt \\ \int _{1}^{r} (\frac{l}{C_{N-1}^{l-1}}\mathrm{e}^{-\psi _{l,\nu }(t)} \int _{0}^{t} \mathrm{e}^{\psi _{l,\nu }(s)} \frac{s^{l-1}q(s)}{(1+\nu s)^{l-1}} g_{1}(u(s))g_{2}(v(s)) \,d s )^{ \frac{1}{k}}\,dt \end{pmatrix}^{T}, $$

then we need only to find a fixed point of \(\mathcal{L}\). Here we use the monotone iterative method to find such a fixed point.

It is easy to show that \(\mathcal{L}\) is a mapping from \(C^{2}[0,1]\times C^{2}[0,1]\) to \(C^{2}[0,1]\times C^{2}[0,1]\), and it is continuous on \(C[0,1]\times C[0,1]\).

Let \(\{u_{n}\}\) and \(\{v_{n}\}\) be the sequence of continuous functions defined by

$$ \textstyle\begin{cases} u_{0}(r)=0, \\ v_{0}(r)=0, \\ u_{n}(r)=\int _{1}^{r} (\frac{k}{C_{N-1}^{k-1}}\mathrm{e}^{-\psi _{k, \mu }(t)} \int _{0}^{t} \mathrm{e}^{\psi _{k,\mu }(s)} \frac{s^{k-1}p(s)}{(1+\mu s)^{k-1}} f_{1}(u_{n-1}(s))f_{2}(v_{n-1}(s)) \,d s )^{\frac{1}{k}}\,dt, \\ v_{n}(r)=\int _{1}^{r} (\frac{l}{C_{N-1}^{l-1}}\mathrm{e}^{-\psi _{l, \nu }(t)} \int _{0}^{t} \mathrm{e}^{\psi _{l,\nu }(s)} \frac{s^{l-1}q(s)}{(1+\nu s)^{l-1}} g_{1}(u_{n-1}(s))g_{2}(v_{n-1}(s)) \,d s )^{\frac{1}{k}}\,dt. \end{cases} $$

It is easy to see that \(u_{n}\) and \(v_{n}\) are decreasing on \([0,1]\) for \(n>1\) and by induction \(\{u_{n}\}\) and \(\{v_{n}\}\) are decreasing as well, i.e., \(u_{n+1}(r)< u_{n}(r)\) and \(v_{n+1}(r)< v_{n}(r)\) for \(0\leq r<1\) and \(n\ge 1\).

By condition (H1), for each \(0< r<1\) and \(n>1\),

$$\begin{aligned} 0 < &u_{n}^{\prime }(r) \\ =& \biggl(\frac{k}{C_{N-1}^{k-1}}\mathrm{e}^{-\psi _{k,\mu }(r)} \int _{0}^{r} \mathrm{e}^{\psi _{k,\mu }(s)} \frac{s^{k-1}p(s)}{(1+\mu s)^{k-1}} f_{1} \bigl(u_{n-1}(s) \bigr)f_{2} \bigl(v_{n-1}(s) \bigr) \,d s \biggr)^{\frac{1}{k}} \\ \le &C(N,k,p) \bigl(f_{1} \bigl(u_{n}(r) \bigr)f_{2} \bigl(v_{n}(r) \bigr) \bigr)^{ \frac{1}{k}} \\ \le &C(N,k,p) \bigl(f_{1} \bigl(u_{n}(r)+v_{n}(r) \bigr)f_{2} \bigl(u_{n}(r)+v_{n}(r) \bigr) \,d s \bigr)^{\frac{1}{k}}, \end{aligned}$$

where \(C(N,k,p)\) is a constant dependent on N, k, and p.

Similarly,

$$ 0< v_{n}^{\prime }(r)\le C(N,l,q) \bigl(g_{1} \bigl(u_{n}(r)+v_{n}(r) \bigr)g_{2} \bigl(u_{n}(r)+v_{n}(r) \bigr) \bigr)^{\frac{1}{l}} $$

and further

$$\begin{aligned} \begin{aligned} 0&< \bigl(u_{n}(r)+v_{n}(r) \bigr)^{\prime } \\ &\le C(N,k,l,p,q) \bigl( \bigl(f_{1} \bigl(u_{n}(r)+v_{n}(r) \bigr)f_{2} \bigl(u_{n}(r)+v_{n}(r) \bigr) \bigr)^{\frac{1}{k}} \\ & \quad{} + \bigl(g_{1} \bigl(u_{n}(r)+v_{n}(r) \bigr)g_{2} \bigl(u_{n}(r)+v_{n}(r) \bigr) \bigr)^{\frac{1}{l}} \bigr), \end{aligned} \end{aligned}$$
(3.1)

i.e.,

$$\begin{aligned} 0 < &\frac{ (u_{n}(r)+v_{n}(r) )^{\prime }}{ (f_{1}(u_{n}(r)+v_{n}(r))f_{2}(u_{n}(r)+v_{n}(r)) )^{\frac{1}{k}} + (g_{1}(u_{n}(r)+v_{n}(r))g_{2}(u_{n}(r)+v_{n}(r)) )^{\frac{1}{l}}} \\ \le & C(N,k,l,p,q), \end{aligned}$$

where \(C(N,l,q)\) and \(C(N,k,l,p,q)\) are constants dependent on N, l, q and N, k, l, p, q, respectively.

Integrating from 1 to r, we have

$$ \int _{0}^{u_{n}(r)+v_{n}(r)} \frac{d\tau }{ (f_{1}(\tau )f_{2}(\tau ) )^{\frac{1}{k}} + (g_{1}(\tau )g_{2}(\tau ) )^{\frac{1}{l}}}\ge -C(N,k,l,p,q). $$
(3.2)

By condition (H2), denote

$$ F(w)= \int _{0}^{w} \frac{d\tau }{ (f_{1}(\tau )f_{2}(\tau ) )^{\frac{1}{k}} + (g_{1}(\tau )g_{2}(\tau ) )^{\frac{1}{l}}}, $$

then F is continuous and increasing on \((-\infty ,0]\), and it has an inverse function \(F^{-1}\). From (3.2), we have

$$ F^{-1} \bigl(-C(N,k,l,p,q) \bigr)\le u_{n}(r)+v_{(}r) \le 0 $$

for \(0\le r\le 1\) and \(n\ge 1\).

By condition (H1) and (3.1), we have for \(n\ge 1\)

$$\begin{aligned} 0 < & \bigl(u_{n}(r)+v_{n}(r) \bigr)^{\prime } \\ \le &C(N,k,l,p,q) \Bigl(\max_{F^{-1}(-C(N,k,l,p,q))\le w \le 0} \bigl( \bigl(f_{1}(w)f_{2}(w) \bigr)^{\frac{1}{k}} + \bigl(g_{1}(w)g_{2}(w) \bigr)^{\frac{1}{l}} \bigr) \Bigr) \\ =&C(N,k,l,p,q, f_{1},f_{2},g_{1},g_{2}), \end{aligned}$$

where \(C(N,k,l,p,q, f_{1},f_{2},g_{1},g_{2})\) is a constant dependent on N, k, l, p, q, \(f_{1}\), \(f_{2}\), \(g_{1}\), and \(g_{2}\). So \(\{u_{n}\}\) and \(\{v_{n}\}\) are bounded in \(C^{1}[0,1]\) and by Arzela–Ascoli theorem \(\{u_{n}\}\) and \(\{v_{n}\}\) have convergent subsequences (still denoted by \(\{u_{n}\}\) and \(\{v_{n}\}\)) in \(C[0,1]\). Denote

$$\begin{aligned}& u(r)=\lim_{n\rightarrow +\infty }u_{n}(r), \\& v(r)=\lim_{n\rightarrow +\infty }v_{n}(r). \end{aligned}$$

By the continuity of \(\mathcal{L}\) on \(C[0,1]\times C[0,1]\), from

$$ (u_{n},v_{n})=\mathcal{L}(u_{n-1},v_{n-1}), $$

we conclude that \((u,v)\) is a fixed point of \(\mathcal{L}\) after letting \(n\rightarrow +\infty \). □

Proof of Theorem 1.2

We prove by contradiction. Suppose that \((u,v)\) is a k-convex radial solution to problem (1.1). Then u and v are decreasing on \([0,1]\). For \(0< r< 1\), by Lemma 2.2 we can get

$$\begin{aligned}& 0< u^{\prime }(r)\le C(N,k,p) \bigl(f_{1} \bigl(u(r)+v(r) \bigr)g_{2} \bigl(u(r)+v(r) \bigr) \bigr)^{\frac{1}{k}}, \\& 0< v^{\prime }(r)\le C(N,l,q) \bigl(g_{1} \bigl(u(r)+v(r) \bigr)g_{2} \bigl(u(r)+v(r) \bigr) \bigr)^{\frac{1}{l}}. \end{aligned}$$

So

$$\begin{aligned} 0 < &\frac{ (u(r)+v(r) )^{\prime }}{ (f_{1}(u(r)+v(r))f_{2}(u(r)+v(r)) )^{\frac{1}{k}} + (g_{1}(u(r)+v(r))g_{2}(u(r)+v(r)) )^{\frac{1}{l}}} \\ \le & C(N,k,l,p,q). \end{aligned}$$

Integrating for 0 to 1, we have

$$ 0< \int ^{0}_{u(0)+v(0)} \frac{d\tau }{ (f_{1}(\tau )f_{2}(\tau ) )^{\frac{1}{k}} + (g_{1}(\tau )g_{2}(\tau ) )^{\frac{1}{l}}}\le C(N,k,l,p,q), $$

which contradicts condition (H3). Now we finish the proof. □

At the end of this section, we give some examples for the sake of clearly understanding the results in this paper.

Assume that α, β, \(\alpha _{1}\), \(\beta _{1}\), \(\alpha _{2}\), and \(\beta _{2}\) are positive.

Example 3.1

If \(\alpha _{1} +\beta _{1}\le k\) and \(\alpha _{2} +\beta _{2}\le l\), then the following problem admits an entire k-convex radial solution \((u,v) \in C^{2} (\overline{B_{1}(0)} )\times C^{2} ( \overline{B_{1}(0)} )\):

$$ \textstyle\begin{cases} \sigma _{k} (\lambda (D^{2} u+\mu \vert \nabla u \vert I ) )=(1+ \vert x \vert )^{\alpha } (1+ \vert u \vert )^{\alpha _{1}} \vert v \vert ^{\beta _{1}}, & x \in B_{1}(0), \\ \sigma _{l} (\lambda (D^{2} v+\nu \vert \nabla v \vert I ) )=(1+ \vert x \vert )^{\beta } (1+ \vert u \vert )^{\alpha _{2}}(1+ \vert v \vert )^{\beta _{2}}, & x \in B_{1}(0), \\ u=v=0,& x\in \partial B_{1}(0). \end{cases} $$

Example 3.2

If \(\alpha _{1} +\beta _{1}\ge k\) and \(\alpha _{2} +\beta _{2}\ge l\), then the following problem admits no entire k-convex radial solution \((u,v) \in C^{2} (\overline{B_{1}(0)} )\times C^{2} ( \overline{B_{1}(0)} )\):

$$ \textstyle\begin{cases} \sigma _{k} (\lambda (D^{2} u+\mu \vert \nabla u \vert I ) )=(1+ \vert x \vert )^{\alpha } \vert u \vert ^{\alpha _{1}} \vert v \vert ^{\beta _{1}}, & x \in B_{1}(0), \\ \sigma _{l} (\lambda (D^{2} v+\nu \vert \nabla v \vert I ) )=(1+ \vert x \vert )^{\beta } \vert u \vert ^{\alpha _{2}} \vert v \vert ^{\beta _{2}}, & x \in B_{1}(0), \\ u=v=0,& x\in \partial B_{1}(0). \end{cases} $$

4 Conclusion

In this paper, by converting the existence of an entire solution to the existence of a fixed point of a continuous mapping, we establish the existence of entire k-convex radial solutions for a Hessian type system. Moreover the nonexistence of entire k-convex radial solutions is also obtained. In the process of obtaining the existence of entire k-convex radial solutions, we utilize the monotone iterative method. By different fixed point theorems (such as the ones in [14] and [25]) or different methods (such as degree theory in [16] and the regularization method in [23]), we may get different results on Hessian type systems. In our opinion, it is interesting to fulfil this kind of works in the future.