$(H,F)$-Closed set and coupled coincidence point theorems for a generalized compatible in partially G-metric spaces

Abstract

In this work, we present a notion of an $(H,F)$-closed set and prove the existence of a coupled coincidence point theorem for a pair $\{F,H\}$ of mappings $F,H:X\times X\to X$ with φ-contraction mappings in partially ordered metric spaces without H-increasing property of F and mixed monotone property of H. We give some examples of a nonlinear contraction mapping, which is not applied to the existence of coupled coincidence point by H using the mixed monotone property and H-increasing property of F. We also show the uniqueness of a coupled coincidence point of the given mappings. Further, we apply our results to the existence and uniqueness of a coupled coincidence point of the given mappings in partially ordered G-metric spaces with H-increasing property of F and mixed monotone property of H. These results generalize some recent results in the literature.

1 Introduction

The existence of a fixed point for the contraction type of mappings in partially ordered metric spaces has been first studied by Ran and Reurings [1]. Moreover, they established some new results and presented some applications to matrix equations. In 1987, Guo and Lakshmikantham [2] introduced the concept of a coupled fixed point. Later, Bhaskar and Lakshmikantham [3] introduced the concept of the mixed monotone property for contractive operators. They also showed some applications on the existence and uniqueness of the coupled fixed point theorems for mappings which satisfy the mixed monotone property in partially ordered metric spaces. Lakshimikantham and Ćirić [4] extended the results in [3] by defining the mixed g-monotonicity and studied the existence and uniqueness of coupled coincidence point for such a mappings which satisfy the mixed monotone property in partially ordered metric spaces. As a continuation of this work, many authors conducted research on the coupled fixed point theory and coupled coincidence point theory in partially ordered metric spaces and different spaces. We refer the reader for example to [4–31].

In 2006, Mustafa and Sims [32] introduced the notion of a G-metric spaces as a generalization of the concept of a metric spaces and proved the analog of the Banach contraction mapping principle in the context of G-metric spaces. For examples of extensions and applications of these works see [33–46]. In 2011, Choudhury and Maity [47] proved the existence of a coupled fixed point theorem of nonlinear contraction mappings with mixed monotone property in partially ordered G-metric spaces. Aydi et al. [48] established coupled coincidence and coupled common fixed point results for a mixed g-monotone mapping satisfying nonlinear contractions in partially ordered G-metric spaces. They generalized the results obtained by Choudhury and Maily [47]. Later, Karapınar et al. [49] extended the results of coupled coincidence and coupled common fixed point theorem for a mixed g-monotone mapping obtained by Aydi et al. [48]. Many authors have studied coupled coincidence point and coupled common fixed point results for a mixed g-monotone mapping satisfying nonlinear contractions in partially ordered G-metric spaces (see, for example, [47–64]).

One of the interesting ways to developed coupled fixed point theory is to consider the mapping $F:X\times X\to X$ without the mixed monotone property. Recently, Sintunavarat et al. [29, 30] proved some coupled fixed point theorems for nonlinear contractions without mixed monotone property which extended the results of Bhaskar and Lakshmikantham [3] by using the concept of an F-invariant set due to Samet and Vetro [28]. Later, Batra and Vashistha [6] introduced an $(F,g)$-invariant set which is a generalization of an F-invariant set. Recently, Kutbi et al. [22] introduced the concept of an F-closed set which is weaker than the concept of an F-invariant set and proved some coupled fixed point theorems without the condition of F-invariant set and mixed monotone property. Very recently, Charoensawan and Thangthong [55] generalized and extended the coupled coincidence point theorem of nonlinear contraction mappings in partially ordered G-metric spaces without the mixed g-monotone property by using the concept of $(F^{\ast },g)$-invariant set in partially ordered G-metric spaces which are generalizations of the results of Aydi et al. [48]. In 2014, Hussain et al. [16] presented the new concept of generalized compatibility of a pair $\{F,G\}$ of mappings $F,G:X\times X\to X$ and proved some coupled coincidence point results of such a mapping without the mixed G-monotone property of F in partially ordered metric spaces which generalized some recent comparable results in the literature.

In this work, we introduce the concept of $(H,F)$-closed set and the notion of generalized compatibility of a pair $\{F,H\}$ of mapping $F,H:X\times X\to X$ in the setting of G-metric spaces. We also obtain a coupled coincidence point theorem for a pair $\{F,H\}$ with φ-contraction mappings in partially ordered metric spaces without H-increasing property of F and mixed monotone property of H. Our theorem generalizes and extends the very recent results obtained by Hussain et al. [16] and Karapınar et al. [49].

2 Preliminaries

In this section, we give some definitions, propositions, examples and remarks which are useful for main results in our paper. Throughout this paper, $(X,⪯)$ denotes a partially ordered set with the partial order ⪯. By $x⪯y$, we mean $y⪰x$. Let $(x,⪯)$ be a partially ordered set, the partial order ⪯₂ for the product set $X\times X$ defined in the following way, for all $(x,y),(u,v)\in X\times X$:

$$(x,y){⪯}_2(u,v)\Rightarrow H(x,y)⪯H(u,v)\text{and}H(v,u)⪯H(y,x),$$

where $H:X\times X\to X$ is one-one.

We say that $(x,y)$ is comparable to $(u,v)$ if either $(x,y){⪯}_2(u,v)$ or $(u,v){⪯}_2(x,y)$.

Definition 2.1 [32]

Let X be a nonempty set and $G:X\times X\times X\to {\mathbb{R}}^{+}$ be a function satisfying the following properties:

• (G1) $G(x,y,z)=0$ if $x=y=z$.

• (G2) $0<G(x,x,y)$ for all $x,y\in X$ with $x\ne y$.

• (G3) $G(x,x,y)\le G(x,y,z)$ for all $x,y,z\in X$ with $y\ne z$.

• (G4) $G(x,y,z)=G(x,z,y)=G(y,z,x)=\cdots$ (symmetry in all three variables).

• (G5) $G(x,y,z)\le G(x,a,a)+G(a,y,z)$ for all $x,y,z,a\in X$ (rectangle inequality).

Then the function G is called a generalized metric, or more specially, a G-metric on X, and the pair $(X,G)$ is called a G-metric space.

Example 2.2 Let $(X,d)$ be a metric space. The function $G:X\times X\times X\to [0,+\mathrm{\infty })$, defined by $G(x,y,z)=d(x,y)+d(y,z)+d(z,x)$, for all $x,y,z\in X$, is a G-metric on X.

Definition 2.3 [32]

Let $(X,G)$ be a G-metric space, and let $(x_n)$ be a sequence of points of X. We say that $(x_n)$ is G-convergent to $x\in X$ if ${lim}_{n,m\to \mathrm{\infty }}G(x,x_n,x_m)=0$, that is, for any $\epsilon >0$, there exists $N\in \mathbb{N}$ such that $G(x,x_n,x_m)<\epsilon$, for all $n,m\ge N$. We call x the limit of the sequence $(x_n)$ and write $x_n\to x$ or ${lim}_{n\to \mathrm{\infty }}{x}_n=x$.

Proposition 2.4 [32]

Let $(X,G)$ be a G-metric space, the following are equivalent:

• (1) $(x_n)$ is G-convergent to x.

• (2) $G(x_n,x_n,x)\to 0$ as $n\to +\mathrm{\infty }$.

• (3) $G(x_n,x,x)\to 0$ as $n\to +\mathrm{\infty }$.

• (4) $G(x_n,x_m,x)\to 0$ as $n,m\to +\mathrm{\infty }$.

Definition 2.5 [32]

Let $(X,G)$ be a G-metric space. A sequence $(x_n)$ is called a G-Cauchy sequence if, for any $\epsilon >0$, there exists $N\in \mathbb{N}$ such that $G(x_n,x_m,x_l)<\epsilon$, for all $n,m,l\ge N$. That is, $G(x_n,x_m,x_l)\to 0$ as $n,m,l\to +\mathrm{\infty }$.

Proposition 2.6 [32]

Let $(X,G)$ be a G-metric space, the following are equivalent:

• (1) the sequence $(x_n)$ is G-Cauchy;

• (2) for any $\epsilon >0$, there exists $N\in \mathbb{N}$ such that $G(x_n,x_m,x_m)<\epsilon$, for all $n,m\ge N$.

Proposition 2.7 [32]

Let $(X,G)$ be a G-metric space. A mapping $f:X\to X$ is G-continuous at $x\in X$ if and only if it is G-sequentially continuous at x, that is, whenever $(x_n)$ is G-convergent to x, $(f(x_n))$ is G-convergent to $f(x)$.

Definition 2.8 [32]

A G-metric space $(X,G)$ is called G-complete if every G-Cauchy sequence is G-convergent in $(X,G)$.

Definition 2.9 [47]

Let $(X,G)$ be a G-metric space. A mapping $F:X\times X\to X$ is said to be continuous if for any two G-convergent sequences $(x_n)$ and $(y_n)$ converging to x and y, respectively, $(F(x_n,y_n))$ is G-convergent to $F(x,y)$.

In 2009, Lakshmikantham and Ćirić [4] introduced the concept of a mixed g-monotone mapping and a coupled coincidence point as follows.

Definition 2.10 [4]

Let $(X,⪯)$ be a partially ordered set and $F:X\times X\to X$ and $g:X\to X$. We say F has the mixed g-monotone property if for any $x,y\in X$

$$x_1,x_2\in X,g(x_1)⪯g(x_2)\text{implies}F(x_1,y)⪯F(x_2,y),$$

and

$$y_1,y_2\in X,g(y_1)⪯g(y_2)\text{implies}F(x,y_1)⪰F(x,y_2).$$

Definition 2.11 [4]

An element $(x,y)\in X\times X$ is called a coupled coincidence point of mappings $F:X\times X\to X$, and $g:X\to X$ if $F(x,y)=g(x)$ and $F(y,x)=g(y)$.

Definition 2.12 [4]

Let X be a nonempty set and $F:X\times X\to X$ and $g:X\to X$. We say F and g are commutative if $g(F(x,y))=F(g(x),g(y))$ for all $x,y\in X$.

Let Φ denote the set of functions $\phi :[0,\mathrm{\infty })\to [0,\mathrm{\infty })$ satisfying

• 1. $\phi (t)<t$ for all $t>0$,

• 2. ${lim}_{r\to t^{+}}\phi (r)<t$ for all $t>0$.

In 2012, Karapınar et al. [49] proved the following theorems.

Theorem 2.13 [49]

Let $(X,⪯)$ be a partially ordered set and G be a G-metric on X such that $(X,G)$ is a complete G-metric space. Suppose that there exist $\phi \in \mathrm{\Phi }$, $F:X\times X\to X$, and $g:X\to X$ such that

$$\begin{array}{r}[G(F(x,u),F(y,v),F(z,w))+G(F(u,x),F(v,y),F(w,z))]\\ \le \phi (G(g(x),g(y),g(z))+G(g(u),g(v),g(w)))\end{array}$$

for all $x,y,u,v,z,w\in X$ for which $g(x)⪰g(y)⪰g(z)$ and $g(u)⪯g(v)⪯g(w)$. Suppose also that F is continuous and has the mixed g-monotone property, $F(X\times X)\subseteq g(X)$, and g is continuous and commutes with F. If there exists $(x_0,y_0)\in X\times X$ such that

$$g(x_0)⪯F(x_0,y_0)\mathit{\text{and}}g(y_0)⪰F(y_0,x_0),$$

then there exists $(x,y)\in X\times X$ such that $g(x)=F(x,y)$ and $g(y)=F(y,x)$, that is, F and g have a coupled coincidence point.

Hussain et al. [16] introduced the concept of H-increasing and $\{F,H\}$ generalized compatible as follows.

Definition 2.14 [16]

Suppose that $F,H:X\times X\to X$ are two mappings. F is said to be H-increasing with respect to ⪯ if for all $x,y,u,v\in X$, with $H(x,y)⪯H(u,v)$, we have $F(x,y)⪯F(u,v)$.

Definition 2.15 [16]

An element $(x,y)\in X\times X$ is called a coupled coincidence point of mappings $F,H:X\times X\to X$ if $F(x,y)=H(x,y)$ and $F(y,x)=H(y,x)$.

Definition 2.16 [16]

Let $(X,d)$ be a metric space and $F,H:X\times X\to X$. We say that the pair $\{F,H\}$ is generalized compatible if

$$\{\begin{array}{ll}d(F(H(x_n,y_n),H(y_n,x_n)),H(F(x_n,y_n),F(y_n,x_n)))\to 0& \text{as }n\to +\mathrm{\infty },\\ d(F(H(y_n,x_n),H(x_n,y_n)),H(F(y_n,x_n),F(x_n,y_n)))\to 0& \text{as }n\to +\mathrm{\infty },\end{array}$$

whenever $(x_n)$ and $(y_n)$ are sequences in X such that

$$\{\begin{array}{l}{lim}_{n\to \mathrm{\infty }}F(x_n,y_n)={lim}_{n\to \mathrm{\infty }}H(x_n,y_n)=t_1,\\ {lim}_{n\to \mathrm{\infty }}F(y_n,x_n)={lim}_{n\to \mathrm{\infty }}H(y_n,x_n)=t_2.\end{array}$$

Definition 2.17 [16]

Let $F,H:X\times X\to X$ be two maps. We say that the pair $\{F,H\}$ is commuting if

$$F(H(x,y),H(y,x))=H(F(x,y),F(y,x))\text{for all }x,y\in X.$$

It is easy to see that a commuting pair is generalized compatible but the converse is not true in general.

Let ϒ denote the set of all functions $\varphi :[0,\mathrm{\infty })\to [0,\mathrm{\infty })$ such that:

• (i) ϕ is continuous and increasing,

• (ii) $\varphi (t)=0$ if and only if $t=0$,

• (iii) $\varphi (t+s)\le \varphi (t)+\varphi (s)$, for all $t,s\in [0,\mathrm{\infty })$.

Let Ψ be the set of all functions $\varphi :[0,\mathrm{\infty })\to [0,\mathrm{\infty })$ such that ${lim}_{t\to r}\psi (t)>0$ for all $r>0$ and ${lim}_{t\to 0^{+}}\psi (t)=0$.

Recently, Hussain et al. [16] proved the coupled coincidence point for such mappings involving $(\psi ,\varphi )$-contractive condition as follows.

Theorem 2.18 [16]

Let $(X,⪯)$ be a partially ordered set and M be a nonempty subset of $X^4$ and let there exists d, a metric on X such that $(X,d)$ is a complete metric space. Assume that $F,H:X\times X\to X$ are two generalized compatible mappings such that F is H-increasing with respect to ⪯, H is continuous and has the mixed monotone property. Suppose that for any $x,y\in X$, there exist $u,v\in X$ such that $F(x,y)=H(u,v)$ and $F(y,x)=H(v,u)$. Suppose that there exist $\varphi \in \mathrm{\Upsilon }$ and $\psi \in \mathrm{\Psi }$ such that the following holds:

$$\begin{array}{rl}\varphi \left(d(F(x,y),F(u,v))\right)\le & \frac{1}{2}\varphi (d(H(x,y),H(u,v))+d(H(y,x),H(v,u)))\\ -\psi \left(\frac{d(H(x,y),H(u,v))+d(H(y,x),H(v,u))}{2}\right)\end{array}$$

for all $x,y,u,v\in X$ with $H(x,y)⪯H(u,v)$ and $H(y,x)⪰H(v,u)$.

Also suppose that either

• (a) F is continuous or

• (b) X has the following properties: for any two sequences $\{x_n\}$ and $\{y_n\}$ with

  • (i) if a non-decreasing sequence $\{x_n\}\to x$, then $x_n⪯x$ for all n,

  • (ii) if a non-increasing sequence $\{y_n\}\to y$, then $y⪯y_n$ for all n.

If there exists $(x_0,y_0)\in X\times X$ with

$$H(x_0,y_0)⪯F(x_0,y_0)\mathit{\text{and}}H(y_0,x_0)⪰F(y_0,x_0),$$

then there exists $(x,y)\in X\times X$ such that $H(x,y)=F(x,y)$ and $H(y,x)=F(y,x)$, that is, F and H have a coupled coincidence point.

In order to remove the mixed monotone property, Batra and Vashistha [6] introduced the following property.

Definition 2.19 [6]

Let $(X,d)$ be a metric space and $F:X\times X\to X$, $g:X\to X$ be given mappings. Let M be a nonempty subset of $X^4$. We say that M is an $(F,g)$-invariant subset of $X^4$ if and only if, for all $x,y,z,w\in X$,

• (i) $(x,y,z,w)\in M\iff (w,z,y,x)\in M$.

• (ii) $(g(x),g(y),g(z),g(w))\in M\Rightarrow (F(x,y),F(y,x),F(z,w),F(w,z))\in M$.

Kutbi et al. [22] introduced the notion of F-closed set which extended the notion of F-invariant set as follows.

Definition 2.20 [22]

Let $F:X\times X\to X$ be a mapping, and let M be a subset of $X^4$. We say that M is an F-closed subset of $X^4$ if, for all $x,y,u,v\in X$,

$$(x,y,u,v)\in M\Rightarrow (F(x,y),F(y,x),F(u,v),F(v,u))\in M.$$

Inspired by above definitions, we give the notion of a $(H,F)$-closed set which is useful for our main results.

Definition 2.21 Let $F,H:X\times X\to X$ be two mappings and let M be a subset of $X^6$. We say that M is an $(H,F)$-closed subset of $X^6$ if, for all $x,y,z,u,v,w\in X$,

$$\begin{array}{r}(H(x,u),H(u,x),H(y,v),H(v,y),H(z,w),H(w,z))\in M\\ \Rightarrow (F(x,u),F(u,x),F(y,v),F(v,y),F(z,w),F(w,z))\in M.\end{array}$$

Definition 2.22 Let $H:X\times X\to X$ be a mapping and M be a subset of $X^6$. We say that M satisfies the transitive property if and only if for all $x,y,z,u,v,w,a,b,c,d\in X$,

$$\begin{array}{r}(H(x,u),H(u,x),H(y,v),H(v,y),H(a,b),H(b,a))\in M\text{and}\\ (H(a,b),H(b,a),H(c,d),H(d,c),H(z,w),H(w,z))\in M\\ \Rightarrow (H(x,u),H(u,x),H(y,v),H(v,y),H(z,w),H(w,z))\in M.\end{array}$$

Definition 2.23 Let $F,H:X\times X\to X$ be two mappings. We say that the pair $\{F,H\}$ is generalized compatible if $(x_n)$ and $(y_n)$ are sequences in X such that for some $x,y\in X$

$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}H(x_n,y_n)=\underset{n\to \mathrm{\infty }}{lim}F(x_n,y_n)=x\text{and}\\ \underset{n\to \mathrm{\infty }}{lim}H(y_n,x_n)=\underset{n\to \mathrm{\infty }}{lim}F(y_n,x_n)=y\end{array}$$

imply

$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}F(H(x_n,y_n),H(y_n,x_n))=\underset{n\to \mathrm{\infty }}{lim}H(F(x_n,y_n),F(y_n,x_n))\text{and}\\ \underset{n\to \mathrm{\infty }}{lim}F(H(y_n,x_n),H(x_n,y_n))=\underset{n\to \mathrm{\infty }}{lim}H(F(y_n,x_n),F(x_n,y_n)).\end{array}$$

Remark The set $M=X^6$ is trivially $(H,F)$-closed set, which satisfies the transitive property.

Example 2.24 Let $(X,G)$ be a G-metric space endowed with a partial order ⪯. Let $F,H:X\times X\to X$ are two generalized compatible mappings such that F is H-increasing with respect to ⪯, H is continuous and has the mixed monotone property. Define a subset $M\subseteq X^6$ by

$$M=\{(x,u,y,v,z,w)\in X^6:x⪰y⪰z,\text{and }u⪯v⪯w\}.$$

Let $(H(x,u),H(u,x),H(y,v),H(v,y),H(z,w),H(w,z))\in M$. It is easy to see that, since F is H-increasing with respect to ⪯, we have $F(x,u)⪰F(y,v)⪰F(z,w)$ and $F(u,x)⪯F(v,y)⪯F(w,z)$, this implies that

$$(F(x,u),F(u,x),F(y,v),F(v,y),F(z,w),F(w,z))\in M.$$

Then M is $(H,F)$-closed subset of $X^6$, which satisfies the transitive property.

3 Main results

Now, we state our first result which successively guarantees a coupled coincidence point.

Theorem 3.1 Let $(X,\le )$ be a partially ordered set and G be a G-metric on X such that $(X,G)$ is a complete G-metric space and M be a nonempty subset of $X^6$. Assume that $F,H:X\times X\to X$ are two generalized compatible mappings such that H is continuous and for any $x,y\in X$, there exists $u,v\in X$ such that $F(x,y)=H(u,v)$ and $F(y,x)=H(v,u)$. Suppose that there exists $\phi \in \mathrm{\Phi }$ such that the following holds:

$$\begin{array}{r}[G(F(x,u),F(y,v),F(z,w))+G(F(u,x),F(v,y),F(w,z))]\\ \le \phi (G(H(x,u),H(y,v),H(z,w))+G(H(u,x),H(v,y),H(w,z)))\end{array}$$

(1)

for all $x,y,z,u,v,w\in X$ with $(H(x,u),H(u,x),H(y,v),H(v,y),H(z,w),H(w,z))\in M$.

Suppose also that either

• (a) F is continuous;

• (b) for any two sequences $\{x_n\}$ and $\{y_n\}$ with for all $n⩾1$

  $$\begin{array}{r}(x_{n+1},y_{n+1},x_{n+1},y_{n+1},x_n,y_n)\in M\mathit{\text{and}}\\ H(x_n,y_n)\to H(x,y),H(y_n,x_n)\to H(y,x)\\ \mathit{\text{implies}}\\ (H(x_n,y_n),H(y_n,x_n),H(x,y),H(y,x),H(x,y),H(y,x))\in M,\end{array}$$

If there exists $(x_0,y_0)\in X\times X$ such that

$$(F(x_0,y_0),F(y_0,x_0),F(x_0,y_0),F(y_0,x_0),H(x_0,y_0),H(y_0,x_0))\in M$$

and M is an $(H,F)$-closed, then there exists $(x,y)\in X\times X$ such that $H(x,y)=F(x,y)$ and $H(y,x)=F(y,x)$, that is, F and H have a coupled coincidence point.

Proof Let $x_0,y_0\in X$ be such that

$$(F(x_0,y_0),F(y_0,x_0),F(x_0,y_0),F(y_0,x_0),H(x_0,y_0),H(y_0,x_0))\in M.$$

From the assumption, there exists $(x_1,y_1)\in X\times X$ such that

$$F(x_0,y_0)=H(x_1,y_1)\text{and}F(y_0,x_0)=H(y_1,x_1).$$

Again from the assumption, we can choose $x_2,y_2\in X$ such that

$$F(x_1,y_1)=H(x_2,y_2)\text{and}F(y_1,x_1)=H(y_2,x_2).$$

By repeating this argument, we can construct two sequences ${\{x_n\}}_{n=1}^{\mathrm{\infty }}$ and ${\{y_n\}}_{n=1}^{\mathrm{\infty }}$ in X such that

$$F(x_n,y_n)=H(x_{n+1},y_{n+1})\text{and}F(y_n,x_n)=H(y_{n+1},x_{n+1})\text{for all }n\ge 1.$$

(2)

Since

$$(F(x_0,y_0),F(y_0,x_0),F(x_0,y_0),F(y_0,x_0),H(x_0,y_0),H(y_0,x_0))\in M,$$

and M is an $(H,F^{\ast })$-closed, we get

$$\begin{array}{c}(F(x_0,y_0),F(y_0,x_0),F(x_0,y_0),F(y_0,x_0),H(x_0,y_0),H(y_0,x_0))\hfill \\ =(H(x_1,y_1),H(y_1,x_1),H(x_1,y_1),H(y_1,x_1),H(x_0,y_0),H(y_0,x_0))\in M\hfill \\ \Rightarrow (F(x_1,y_1),F(y_1,x_1),F(x_1,y_1),F(y_1,x_1),F(x_0,y_0),F(y_0,x_0))\hfill \\ \phantom{\Rightarrow }=(H(x_2,y_2),H(y_2,x_2),H(x_2,y_2),H(y_2,x_2),H(x_1,y_1),H(y_1,x_1))\in M.\hfill \end{array}$$

Again, using the fact that M is a $(H,F)$-closed, we have

$$\begin{array}{c}(H(x_2,y_2),H(y_2,x_2),H(x_2,y_2),H(y_2,x_2),H(x_1,y_1),H(y_1,x_1))\in M\hfill \\ \Rightarrow (F(x_2,y_2),F(y_2,x_2),F(x_2,y_2),F(y_2,x_2),F(x_1,y_1),F(y_1,x_1))\hfill \\ \phantom{\Rightarrow }=(H(x_3,y_3),H(y_3,x_3),H(x_3,y_3),H(y_3,x_3),H(x_2,y_2),H(y_2,x_2))\in M.\hfill \end{array}$$

Continuing this process, for all $n\ge 0$ we obtain

$$\begin{array}{r}(H(x_{n+1},y_{n+1}),H(y_{n+1},x_{n+1}),H(x_{n+1},y_{n+1}),\\ H(y_{n+1},x_{n+1}),H(x_n,y_n),H(y_n,x_n))\in M.\end{array}$$

(3)

Let

$$\begin{array}{rl}{\delta }_n=& G(H(x_{n+1},y_{n+1}),H(x_{n+1},y_{n+1}),H(x_n,y_n))\\ +G(H(y_{n+1},x_{n+1}),H(y_{n+1},x_{n+1}),H(y_n,x_n)).\end{array}$$

(4)

We can suppose that ${\delta }_n>0$ for all $n\ge 0$. If not, $(x_n,y_n)$ will be a coupled coincidence point and the proof is finished. From (1), (2), and (3), we have

$$\begin{array}{rl}{\delta }_n=& G(F(x_n,y_n),F(x_n,y_n),F(x_{n-1},y_{n-1}))\\ +G(F(y_n,x_n),F(y_n,x_n),F(y_{n-1},x_{n-1}))\\ \le & \phi (G(H(x_n,y_n),H(x_n,y_n),H(x_{n-1},y_{n-1}))\\ +G(H(y_n,x_n),H(y_n,x_n),H(y_{n-1},x_{n-1})))\\ =& \phi ({\delta }_{n-1}).\end{array}$$

(5)

This implies that

$${\delta }_n\le \phi ({\delta }_{n-1}).$$

(6)

Since $\varphi (t)<t$ for all $t>0$, it follows that $\{{\delta }_n\}$ is decreasing sequence. Therefore, there is some $\delta \ge 0$ such that ${lim}_{n\to \mathrm{\infty }}{\delta }_n=\delta$.

We shall prove that $\delta =0$. Assume, to the contrary, that $\delta >0$. Then by letting $n\to \mathrm{\infty }$ in (6) and using the properties of the map φ, we get

$$\delta =\underset{n\to \mathrm{\infty }}{lim}{\delta }_n\le \underset{n\to \mathrm{\infty }}{lim}\phi ({\delta }_{n-1})=\underset{{\delta }_{n-1}\to {\delta }^{+}}{lim}\phi ({\delta }_{n-1})<\delta .$$

A contradiction, thus $\delta =0$, and hence

$$\begin{array}{rl}\underset{n\to \mathrm{\infty }}{lim}{\delta }_n=& \underset{n\to \mathrm{\infty }}{lim}[G(H(x_{n+1},y_{n+1}),H(x_{n+1},y_{n+1}),H(x_n,y_n))\\ +G(H(y_{n+1},x_{n+1}),H(y_{n+1},x_{n+1}),H(y_n,x_n))]=0.\end{array}$$

(7)

Next, we prove that ${\{H(x_n,y_n)\}}_{n=1}^{\mathrm{\infty }}$ and ${\{H(y_n,x_n)\}}_{n=1}^{\mathrm{\infty }}$ are Cauchy sequences in the G-metric space $(X,G)$. Suppose, to the contrary, that at least of ${\{H(x_n,y_n)\}}_{n=1}^{\mathrm{\infty }}$ and ${\{H(y_n,x_n)\}}_{n=1}^{\mathrm{\infty }}$ is not Cauchy sequence in $(X,G)$. Then there exists an $\epsilon >0$ for which we can find subsequences $\{H(x_{m(k)},y_{m(k)})\}$, $\{H(x_{n(k)},y_{n(k)})\}$ of ${\{H(x_n,y_n)\}}_{n=1}^{\mathrm{\infty }}$ and $\{H(y_{m(k)},x_{m(k)})\}$, $\{H(y_{n(k)},x_{n(k)})\}$ of ${\{H(y_n,x_n)\}}_{n=1}^{\mathrm{\infty }}$, respectively, with $n(k)>m(k)\ge k$ such that

$$\begin{array}{rl}{r}_k:=& G(H(x_{m(k)},y_{m(k)}),H(x_{m(k)},y_{m(k)}),H(x_{n(k)},y_{n(k)}))\\ +G(H(y_{m(k)},x_{m(k)}),H(y_{m(k)},x_{m(k)}),H(y_{n(k)},x_{n(k)}))\ge \epsilon .\end{array}$$

(8)

Further, corresponding to $n(k)$, we can choose $m(k)$ in such a way that it is the smallest integer with $m(k)>n(k)\ge K$ and satisfying (8). Then

$$\begin{array}{r}G(H(x_{m(k)-1},y_{m(k)-1}),H(x_{m(k)-1},y_{m(k)-1}),H(x_{n(k)},y_{n(k)}))\\ +G(H(y_{m(k)-1},x_{m(k)-1}),H(y_{m(k)-1},x_{m(k)-1}),H(y_{n(k)},x_{n(k)}))<\epsilon .\end{array}$$

(9)

Using the rectangle inequality and (9), we have

$$\begin{array}{rl}\epsilon \le & r_k\\ \le & G(H(x_{m(k)},y_{m(k)}),H(x_{m(k)},y_{m(k)}),H(x_{m(k)-1},y_{m(k)-1}))\\ +G(H(x_{m(k)-1},y_{m(k)-1}),H(x_{m(k)-1},y_{m(k)-1}),H(x_{n(k)},y_{n(k)}))\\ +G(H(y_{m(k)},x_{m(k)}),H(y_{m(k)},x_{m(k)}),H(y_{m(k)-1},x_{m(k)-1}))\\ +G(H(y_{m(k)-1},x_{m(k)-1}),H(y_{m(k)-1},x_{m(k)-1}),H(y_{n(k)},x_{n(k)}))\\ <& {\delta }_{m(k)-1}+\epsilon .\end{array}$$

(10)

Letting $k\to +\mathrm{\infty }$ and using (7), we obtain

$$\begin{array}{rl}\underset{k\to \mathrm{\infty }}{lim}{r}_k=& \underset{k\to +\mathrm{\infty }}{lim}[G(H(x_{m(k)},y_{m(k)}),H(x_{m(k)},y_{m(k)}),H(x_{n(k)},y_{n(k)}))\\ +G(H(y_{m(k)},x_{m(k)}),H(y_{m(k)},x_{m(k)}),H(y_{n(k)},x_{n(k)}))]\\ =& \epsilon .\end{array}$$

(11)

Again, by the rectangle inequality, we have

$$\begin{array}{rl}{r}_k\le & G(H(x_{m(k)},y_{m(k)}),H(x_{m(k)},y_{m(k)}),H(x_{m(k)+1},y_{m(k)+1}))\\ +G(H(x_{m(k)+1},y_{m(k)+1}),H(x_{m(k)+1},y_{m(k)+1}),H(x_{n(k)+1},y_{n(k)+1}))\\ +G(H(x_{n(k)+1},y_{n(k)+1}),H(x_{n(k)+1},y_{n(k)+1}),H(x_{n(k)},y_{n(k)}))\\ +G(H(y_{m(k)},x_{m(k)}),H(y_{m(k)},x_{m(k)}),H(y_{m(k)+1},x_{m(k)+1}))\\ +G(H(y_{m(k)+1},x_{m(k)+1}),H(y_{m(k)+1},x_{m(k)+1}),H(y_{n(k)+1},x_{n(k)+1}))\\ +G(H(y_{n(k)+1},x_{n(k)+1}),H(y_{n(k)+1},x_{n(k)+1}),H(y_{n(k)},x_{n(k)}))\\ =& {\delta }_{n(k)}+G(H(x_{m(k)},y_{m(k)}),H(x_{m(k)},y_{m(k)}),H(x_{m(k)+1},y_{m(k)+1}))\\ +G(H(x_{m(k)+1},y_{m(k)+1}),H(x_{m(k)+1},y_{m(k)+1}),H(x_{n(k)+1},y_{n(k)+1}))\\ +G(H(y_{m(k)},x_{m(k)}),H(y_{m(k)},x_{m(k)}),H(y_{m(k)+1},x_{m(k)+1}))\\ +G(H(y_{m(k)+1},x_{m(k)+1}),H(y_{m(k)+1},x_{m(k)+1}),H(y_{n(k)+1},x_{n(k)+1})).\end{array}$$

Using the fact that $G(x,x,y)\le 2G(x,y,y)$ for any $x,y\in X$, we obtain

$$\begin{array}{rl}{r}_k\le & {\delta }_{n(k)}+2{\delta }_{m(k)}\\ +G(H(x_{m(k)+1},y_{m(k)+1}),H(x_{m(k)+1},y_{m(k)+1}),H(x_{n(k)+1},y_{n(k)+1}))\\ +G(H(y_{m(k)+1},x_{m(k)+1}),H(y_{m(k)+1},x_{m(k)+1}),H(y_{n(k)+1},x_{n(k)+1})).\end{array}$$

(12)

Since $m(k)>n(k)$ and using (3), we have

$$\begin{array}{r}(H(x_{m(k)},y_{m(k)}),H(y_{m(k)},x_{m(k)}),H(x_{m(k)},y_{m(k)}),\\ H(y_{m(k)},x_{m(k)}),H(x_{m(k)-1},y_{m(k)-1}),H(y_{m(k)-1},x_{m(k)-1}))\in M\end{array}$$

and

$$\begin{array}{r}(H(x_{m(k)-1},y_{m(k)-1}),H(y_{m(k)-1},x_{m(k)-1}),H(x_{m(k)-1},y_{m(k)-1}),\\ H(y_{m(k)-1},x_{m(k)-1}),H(x_{m(k)-2},y_{m(k)-2}),H(y_{m(k)-2},x_{m(k)-2}))\in M.\end{array}$$

From the fact that M is an $(H,F)$-closed set which satisfies the transitive property, we have

$$\begin{array}{r}(H(x_{m(k)},y_{m(k)}),H(y_{m(k)},x_{m(k)}),H(x_{m(k)},y_{m(k)}),\\ H(y_{m(k)},x_{m(k)}),H(x_{m(k)-2},y_{m(k)-2}),H(y_{m(k)-2},x_{m(k)-2}))\in M.\end{array}$$

By this process, we can get

$$\begin{array}{r}(H(x_{m(k)},y_{m(k)}),H(y_{m(k)},x_{m(k)}),H(x_{m(k)},y_{m(k)}),\\ H(y_{m(k)},x_{m(k)}),H(x_{n(k)},y_{n(k)}),H(y_{n(k)},x_{n(k)}))\in M.\end{array}$$

Now, using (1), we have

$$\begin{array}{r}G(H(x_{m(k)+1},y_{m(k)+1}),H(x_{m(k)+1},y_{m(k)+1}),H(x_{n(k)+1},y_{n(k)+1}))\\ +G(H(y_{m(k)+1},x_{m(k)+1}),H(y_{m(k)+1},x_{m(k)+1}),H(y_{n(k)+1},x_{n(k)+1}))\\ =G(F(x_{m(k)},y_{m(k)}),F(x_{m(k)},y_{m(k)}),F(x_{n(k)},y_{n(k)}))\\ +G(F(y_{m(k)},x_{m(k)}),F(y_{m(k)},x_{m(k)}),F(y_{n(k)},x_{n(k)}))\\ \le \varphi (G(H(x_{m(k)},y_{m(k)}),H(x_{m(k)},y_{m(k)}),H(x_{n(k)},y_{n(k)}))\\ +G(H(y_{m(k)},x_{m(k)}),H(y_{m(k)},x_{m(k)}),H(y_{n(k)},x_{n(k)})))\\ \le \varphi (r_k).\end{array}$$

(13)

From (12) and (13), it follows that

$$r_k\le {\delta }_{n(k)}+2{\delta }_{m(k)}+\varphi (r_k).$$

(14)

Letting $k\to +\mathrm{\infty }$ in (14) and using (7) and (11) and ${lim}_{r\to t^{+}}\varphi (r)<t$ for all $t>0$, we have

$$\epsilon =\underset{k\to \mathrm{\infty }}{lim}{r}_k\le \underset{n\to \mathrm{\infty }}{lim}\varphi (r_k)=\underset{r_k\to {\epsilon }^{+}}{lim}\varphi (r_k)<\epsilon ,$$

which is a contradiction. This shows that ${\{H(x_n,y_n)\}}_{n=1}^{\mathrm{\infty }}$ and ${\{H(y_n,x_n)\}}_{n=1}^{\mathrm{\infty }}$ are Cauchy sequences in the G-metric space $(X,G)$. Since $(X,G)$ is complete and from (2), ${\{H(x_n,y_n)\}}_{n=1}^{\mathrm{\infty }}$ and ${\{H(y_n,x_n)\}}_{n=1}^{\mathrm{\infty }}$ are G-convergent, there exist $x,y\in X$ such that

$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}H(x_n,y_n)=\underset{n\to \mathrm{\infty }}{lim}F(x_n,y_n)=x\text{and}\\ \underset{n\to \mathrm{\infty }}{lim}H(y_n,x_n)=\underset{n\to \mathrm{\infty }}{lim}F(y_n,x_n)=y.\end{array}$$

(15)

Since the pair $\{F,G\}$ satisfies the generalized compatibility, from (15), we have

$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}F(H(x_n,y_n),H(y_n,x_n))=\underset{n\to \mathrm{\infty }}{lim}H(F(x_n,y_n),F(y_n,x_n))\text{and}\\ \underset{n\to \mathrm{\infty }}{lim}F(H(y_n,x_n),H(x_n,y_n))=\underset{n\to \mathrm{\infty }}{lim}H(F(y_n,x_n),F(x_n,y_n)).\end{array}$$

(16)

Suppose that assumption (a) holds. For all $n\ge 0$, from (16), we have

$$\begin{array}{rl}H(x,y)& =H(\underset{n\to \mathrm{\infty }}{lim}F(x_n,y_n),\underset{n\to \mathrm{\infty }}{lim}F(y_n,x_n))=\underset{n\to \mathrm{\infty }}{lim}H(F(x_n,y_n),F(y_n,x_n))\\ =\underset{n\to \mathrm{\infty }}{lim}F(H(x_n,y_n),H(y_n,x_n))=F(\underset{n\to \mathrm{\infty }}{lim}H(x_n,y_n),\underset{n\to \mathrm{\infty }}{lim}H(y_n,x_n))\\ =F(x,y)\end{array}$$

and

$$\begin{array}{rl}H(y,x)& =H(\underset{n\to \mathrm{\infty }}{lim}F(y_n,x_n),\underset{n\to \mathrm{\infty }}{lim}F(x_n,y_n))=\underset{n\to \mathrm{\infty }}{lim}H(F(y_n,x_n),F(x_n,y_n))\\ =\underset{n\to \mathrm{\infty }}{lim}F(H(y_n,x_n),H(x_n,y_n))=F(\underset{n\to \mathrm{\infty }}{lim}H(y_n,x_n),\underset{n\to \mathrm{\infty }}{lim}H(x_n,y_n))\\ =F(y,x).\end{array}$$

We have

$$H(x,y)=F(x,y)\text{and}H(y,x)=F(y,x).$$

Therefore, $(x,y)$ is a coupled coincidence point of F and H.

Suppose now assumption (b) holds. Since ${\{H(x_n,y_n)\}}_{n=1}^{\mathrm{\infty }}$ converges to x, ${\{H(y_n,x_n)\}}_{n=1}^{\mathrm{\infty }}$ converges to y, the pair $\{F,G\}$ satisfies the generalized compatibility, H is continuous and by (15), we have

$$\begin{array}{rl}\underset{n\to \mathrm{\infty }}{lim}H(H(x_n,y_n),H(y_n,x_n))& =H(x,y)\\ =\underset{n\to \mathrm{\infty }}{lim}H(F(x_n,y_n),F(y_n,x_n))\\ =\underset{n\to \mathrm{\infty }}{lim}F(H(x_n,y_n),H(y_n,x_n))\end{array}$$

(17)

and

$$\begin{array}{rl}\underset{n\to \mathrm{\infty }}{lim}H(H(y_n,x_n),H(x_n,y_n))& =H(y,x)\\ =\underset{n\to \mathrm{\infty }}{lim}H(F(y_n,x_n),F(x_n,y_n))\\ =\underset{n\to \mathrm{\infty }}{lim}F(H(y_n,x_n),H(x_n,y_n)).\end{array}$$

(18)

From (3), (17), (18), and assumption (b), for all $n⩾1$, we have

$$(H(x_n,y_n),H(y_n,x_n),H(x,y),H(y,x),H(x,y),H(y,x))\in M.$$

(19)

Then, by (1), (2), (19), and the triangle inequality, we have

$$\begin{array}{r}G(H(x,y),F(x,y),F(x,y))+G(H(y,x),F(y,x),F(y,x))\\ \le G(H(x,y),F(H(x_n,y_n),H(y_n,x_n)),F(H(x_n,y_n),H(y_n,x_n)))\\ +G(F(H(x_n,y_n),H(y_n,x_n)),F(x,y),F(x,y))\\ +G(H(y,x),F(H(y_n,x_n),H(x_n,y_n)),F(H(y_n,x_n),H(x_n,y_n)))\\ +G\left(F(H(y_n,x_n),H(x_n,y_n))y_n\right)),F(y,x),F(y,x))\\ \le \phi (G(H(H(x_n,y_n),H(y_n,x_n)),H(x,y),H(x,y))\\ +G(H(H(y_n,x_n),H(x_n,y_n)),H(y,x),H(y,x)))\\ +G(H(x,y),F(H(x_n,y_n),H(y_n,x_n)),F(H(x_n,y_n),H(y_n,x_n)))\\ +G(H(y,x),F(H(y_n,x_n),H(x_n,y_n)),F(H(y_n,x_n),H(x_n,y_n))).\end{array}$$

Letting now $n\to \mathrm{\infty }$ in the above inequality and using property of φ such that ${lim}_{r\to 0^{+}}\phi (r)=0$, we have

$$G(H(x,y),F(x,y),F(x,y))+G(H(y,x),F(y,x),F(y,x))=0,$$

which implies that $H(x,y)=F(x,y)$ and $H(y,x)=F(y,x)$. □

Next, we give an example to validate Theorem 3.1.

Example 3.2 Let $X=[0,1]$, $G(x,y,z)=|x-y|+|x-z|+|y-z|$, and $F,H:X\times X\to X$ be defined by

$$F(x,y)=\{\begin{array}{ll}\frac{x^2-y^2}{8}& \text{if }x\ge y,\\ 0& \text{if }x<y\end{array}$$

and

$$H(x,y)=\{\begin{array}{ll}x+y& \text{if }x\ge y,\\ 0& \text{if }x<y.\end{array}$$

Clearly, H does not satisfy the mixed monotone property and if $x>y$, $u=v\ne 0$, consider

$$\begin{array}{r}H(x,y)\le H(u,v)\Rightarrow x+y\le u+v\\ \text{but }F(x,y)=x^2-y^2=(x-y)(x+y)>0=F(u,v).\end{array}$$

Then F is not H-increasing.

Now, we prove that for any $x,y\in X$, there exist $u,v\in X$ such that $F(x,y)=H(u,v)$ and $F(y,x)=H(v,u)$. It is easy to see that we have the following cases.

Case 1: If $x=y$, then we have $F(y,x)=F(x,y)=0=H(0,0)$.

Case 2: If $x>y$, then $(x-y)x>(x-y)y$ and we have

$$F(x,y)=\frac{x^2-y^2}{8}=\frac{(x-y)x+(x-y)y}{8}=H(\frac{(x-y)x}{8},\frac{(x-y)y}{8})$$

and

$$F(y,x)=0=H(\frac{(x-y)y}{8},\frac{(x-y)x}{8}).$$

Case 3: If $y>x$, then $(y-x)y>(y-x)x$ and we have

$$F(y,x)=\frac{y^2-x^2}{8}=\frac{(y-x)y+(y-x)x}{8}=H(\frac{(y-x)y}{8},\frac{(y-x)x}{8})$$

and

$$F(x,y)=0=H(\frac{(y-x)x}{8},\frac{(y-x)y}{8}).$$

Now, we prove that the pair $\{F,G\}$ satisfies the generalized compatibility hypothesis. Let ${\{x_n\}}_{n=1}^{\mathrm{\infty }}$ and ${\{y_n\}}_{n=1}^{\mathrm{\infty }}$ be two sequences in X such that

$$\begin{array}{r}{t}_1=\underset{n\to \mathrm{\infty }}{lim}F(x_n,y_n)=\underset{n\to \mathrm{\infty }}{lim}H(x_n,y_n)\text{and}\\ t_2=\underset{n\to \mathrm{\infty }}{lim}F(y_n,x_n)=\underset{n\to \mathrm{\infty }}{lim}H(y_n,x_n).\end{array}$$

Then we must have $t_1=0=t_2$ and it is easy to prove that

$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}F(H(x_n,y_n),H(y_n,x_n))=\underset{n\to \mathrm{\infty }}{lim}H(F(x_n,y_n),F(y_n,x_n))\text{and}\\ \underset{n\to \mathrm{\infty }}{lim}F(H(y_n,x_n),H(x_n,y_n))=\underset{n\to \mathrm{\infty }}{lim}H(F(y_n,x_n),F(x_n,y_n)).\end{array}$$

Now, for all $x,y,z,u,v,w\in X$ with $(H(x,u),H(u,x),H(y,v),H(v,y),H(z,w),H(w,z))\in M=X^6$ and let $\phi :[0,+\mathrm{\infty })\to [0,+\mathrm{\infty })$ be a function defined by $\phi (t)=\frac{t}{8}$, we have

$$\begin{array}{r}[G(F(x,u),F(y,v),F(z,w))+G(F(u,x),F(v,y),F(w,z))]\\ =|\frac{x^2-u^2}{8}-\frac{y^2-v^2}{8}|+|\frac{x^2-u^2}{8}-\frac{z^2-w^2}{8}|+|\frac{y^2-v^2}{8}-\frac{z^2-w^2}{8}|\\ +|\frac{u^2-x^2}{8}-\frac{v^2-y^2}{8}|+|\frac{u^2-x^2}{8}-\frac{w^2-z^2}{8}|+|\frac{v^2-y^2}{8}-\frac{w^2-z^2}{8}|\\ =2(|\frac{x^2-u^2}{8}-\frac{y^2-v^2}{8}|+|\frac{x^2-u^2}{8}-\frac{z^2-w^2}{8}|+|\frac{y^2-v^2}{8}-\frac{z^2-w^2}{8}|)\\ =2(|\frac{(x-u)(x+u)}{8}-\frac{(y-v)(y+v))}{8}|+|\frac{(x-u)(x+u)}{8}-\frac{(z-w)(z+w)}{8}|\\ +|\frac{(y-v)(y+v)}{8}-\frac{(z-w)(z+w)}{8}|)\\ \le \frac{1}{4}(|(x+u)-(y+v)|+|(x+u)-(z+w)|+|(y+v)-(z+w)|)\\ =\phi \left(2(|(x+u)-(y+v)|+|(x+u)-(z+w)|+|(y+v)-(z+w)|)\right)\\ =\phi (|(x+u)-(y+v)|+|(x+u)-(z+w)|+|(y+v)-(z+w)|\\ +|(x+u)-(y+v)|+|(x+u)-(z+w)|+|(y+v)-(z+w)|)\\ =\phi (G(H(x,u),H(y,v),H(z,w))+G(H(u,x),H(v,y),H(w,z))).\end{array}$$

Therefore, condition (1) is satisfied. Thus, all the requirements of Theorem 3.1 are satisfied and $(0,0)$ is a coupled coincidence point of F and G.

Next, we show the uniqueness of the coupled coincidence point and coupled fixed point of F and G.

Theorem 3.3 In addition to the hypotheses of Theorem 3.1, suppose that for every $(x,y),(z,t)\in X\times X$, there exists $(u,v)\in X\times X$ such that

$$\begin{array}{r}(H(u,v),H(v,u),H(x,y),H(y,x),H(x,y),H(y,x))\in M\mathit{\text{and}}\\ (H(u,v),H(v,u),H(z,t),H(t,z),H(z,t),H(t,z))\in M.\end{array}$$

Then F and H have a unique coupled coincidence point. Moreover, if the pair $\{F,H\}$ is commuting, then F and H have a unique coupled fixed point, that is, there exists a unique $(a,b)\in X^2$ such that

$$a=H(a,b)=F(a,b)\mathit{\text{and}}b=H(b,a)=F(b,a).$$

Proof From Theorem 3.1, we know that F and H have a coupled coincidence point. Suppose that $(x,y)$, $(z,t)$ are coupled coincidence points of F and H, that is,

$$\begin{array}{r}F(x,y)=H(x,y),F(y,x)=H(y,x)\text{and}\\ F(z,t)=H(z,t),F(t,z)=H(t,z).\end{array}$$

(20)

Now, we show that $H(x,y)=H(z,t)$ and $H(y,x)=H(t,z)$. By the hypothesis there exists $(u,v)\in X\times X$ such that

$$\begin{array}{r}(H(u,v),H(v,u),H(x,y),H(y,x),H(x,y),H(y,x))\in M\text{and}\\ (H(u,v),H(v,u),H(z,t),H(t,z),H(z,t),H(t,z))\in M.\end{array}$$

We put $u_0=u$ and $v_0=v$ and define two sequences ${\{H(u_n,v_n)\}}_{n=1}^{\mathrm{\infty }}$ and ${\{H(v_n,u_n)\}}_{n=1}^{\mathrm{\infty }}$ as follows:

$$F(u_n,v_n)=H(u_{n+1},v_{n+1})\text{and}F(v_n,u_n)=H(v_{n+1},u_{n+1})\text{for all }n\ge 0.$$

Since M is $(H,F)$-closed and

$$(H(u,v),H(v,u),H(x,y),H(y,x),H(x,y),H(y,x))\in M,$$

we have

$$\begin{array}{c}(H(u,v),H(v,u),H(x,y),H(y,x),H(x,y),H(y,x))\in M\hfill \\ =(H(u_0,v_0),H(v_0,u_0),H(x,y),H(y,x),H(x,y),H(y,x))\in M\hfill \\ \Rightarrow (F(u_0,v_0),F(v_0,u_0),F(x,y),F(y,x),F(x,y),F(y,x))\hfill \\ \phantom{\Rightarrow }=(H(u_1,v_1),H(v_1,u_1),H(x,y),H(y,x),H(x,y),H(y,x))\in M.\hfill \end{array}$$

From $(H(u_1,v_1),H(v_1,u_1),H(x,y),H(y,x),H(x,y),H(y,x))\in M$, if we use again the property of $(H,F)$-closedness, then

$$\begin{array}{c}(H(u_1,v_1),H(v_1,u_1),H(x,y),H(y,x),H(x,y),H(y,x))\in M\hfill \\ \Rightarrow (F(u_1,v_1),F(v_1,u_1),F(x,y),F(y,x),F(x,y),F(y,x))\hfill \\ \phantom{\Rightarrow }=(H(u_2,v_2),H(v_2,u_2),H(x,y),H(y,x),H(x,y),H(y,x))\in M.\hfill \end{array}$$

By repeating this process, we get

$$(H(u_n,v_n),H(v_n,u_n),H(x,y),H(y,x),H(x,y),H(y,x))\in M\text{for all }n\ge 0.$$

(21)

Using (1), (20), and (21), for all $n\ge 0$, we have

$$\begin{array}{r}G(H(u_{n+1},v_{n+1}),H(x,y),H(x,y))+G(H(v_{n+1},u_{n+1}),H(y,x),H(y,x))\\ =G(F(u_n,v_n),F(x,y),F(x,y))+G(F(v_n,u_n),F(y,x),F(y,x))\\ \le \phi (G(H(u_n,v_n),H(x,y),H(x,y))+G(H(v_n,u_n),H(y,x),H(y,x))).\end{array}$$

(22)

Using property that $\phi (t)<t$ and repeating this process, for all $n\ge 0$, we get

$$\begin{array}{r}G(H(u_{n+1},v_{n+1}),H(x,y),H(x,y))+G(H(v_{n+1},u_{n+1}),H(y,x),H(y,x))\\ \le {\phi }^n(G(H(u_1,v_1),H(x,y),H(x,y))+G(H(v_1,u_1),H(y,x),H(y,x))).\end{array}$$

(23)

From $\phi (t)<t$ and ${lim}_{r\to t^{+}}\phi (r)<t$, it follows that ${lim}_{n\to \mathrm{\infty }}{\phi }^n(t)=0$ for each $t>0$. Therefore, from (23), we have

$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}(G(H(u_{n+1},v_{n+1}),H(x,y),H(x,y))+G(H(v_{n+1},u_{n+1}),H(y,x),H(y,x)))\\ =0.\end{array}$$

(24)

This implies that

$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}G(H(u_{n+1},v_{n+1}),H(x,y),H(x,y))=0\text{and}\\ \underset{n\to \mathrm{\infty }}{lim}G(H(v_{n+1},u_{n+1}),H(y,x),H(y,x))=0.\end{array}$$

(25)

Similarly, we show that

$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}G(H(u_{n+1},v_{n+1}),H(z,t),H(z,t))=0\text{and}\\ \underset{n\to \mathrm{\infty }}{lim}G(H(v_{n+1},u_{n+1}),H(t,z),H(t,z))=0.\end{array}$$

(26)

From (25) and (26), we have

$$H(x,y)=H(z,t)\text{and}H(y,x)=H(t,z).$$

(27)

Now let the pair $\{F,H\}$ be commuting, we shall prove that F and H have a unique coupled fixed point. Since

$$F(x,y)=H(x,y)\text{and}F(y,x)=H(y,x),$$

(28)

and F and H commutes, we have

$$\begin{array}{r}H(H(x,y),H(y,x))=H(F(x,y),F(y,x))=F(H(x,y),H(y,x))\text{and}\\ H(H(y,x),H(x,y))=H(F(y,x),F(x,y))=F(H(y,x),H(x,y)).\end{array}$$

(29)

Denote $H(x,y)=a$ and $H(y,x)=b$. Then, by (28) and (29), one gets

$$H(a,b)=F(a,b)\text{and}H(b,a)=F(b,a).$$

(30)

Therefore, $(a,b)$ is a coupled coincidence point of F and H. Then, by (27) with $z=a$ and $t=b$, it follows that

$$a=H(x,y)=H(a,b)\text{and}b=H(y,x)=H(b,a).$$

(31)

Thus, $(a,b)$ is a coupled fixed point of H, by (28), $(a,b)$ is also a coupled fixed point of F. To prove the uniqueness, assume $(p,q)$ is another coupled fixed point of F and H. Then, by (27) and (31), we have

$$p=H(p,q)=H(a,b)=a\text{and}q=H(q,p)=H(b,a)=b.$$

 □

Next, we give some applications of our results to coupled coincidence point theorems.

Corollary 3.4 Let $(X,⪯)$ be a partially ordered set and M be a nonempty subset of $X^6$ and let there exists G be a G-metric on X such that $(X,G)$ is a complete G-metric space. Assume that $F,H:X\times X\to X$ are two generalized compatible mappings such that F is H-increasing with respect to ⪯, H is continuous and has the mixed monotone property. Suppose that for any $x,y\in X$, there exist $u,v\in X$ such that $F(x,y)=H(u,v)$ and $F(y,x)=H(v,u)$. Suppose that there exists $\phi \in \mathrm{\Phi }$ such that the following holds:

$$\begin{array}{r}[G(F(x,u),F(y,v),F(z,w))+G(F(u,x),F(v,y),F(w,z))]\\ \le \phi (G(H(x,u),H(y,v),H(z,w))+G(H(u,x),H(v,y),H(w,z)))\end{array}$$

for all $x,y,z,u,v,w\in X$ with $F(x,u)⪰F(y,v)⪰F(z,w)$ and $F(u,x)⪯F(v,y)⪯F(w,z)$.

Also suppose that either

• (a) F is continuous or

• (b) X has the following properties: for any two sequences $\{x_n\}$ and $\{y_n\}$ with

  • (i) if a non-decreasing sequence $\{x_n\}\to x$, then $x_n⪯x$ for all n,

  • (ii) if a non-increasing sequence $\{y_n\}\to y$, then $y⪯y_n$ for all n.

If there exists $(x_0,y_0)\in X\times X$ with

$$H(x_0,y_0)⪯F(x_0,y_0)\mathit{\text{and}}H(y_0,x_0)⪰F(y_0,x_0).$$

Then there exists $(x,y)\in X\times X$ such that $H(x,y)=F(x,y)$ and $H(y,x)=F(y,x)$, that is, F and H have a coupled coincidence point.

Proof We define the subset $M\subseteq X^6$ by

$$M=\{(x,u,y,v,z,w)\in X^6:x⪰y⪰z,\text{and }u⪯v⪯w\}.$$

From Example 2.24, M is a $(H,F)$-closed set which satisfies the transitive property. For all $x,y,z,u,v,w\in X$ with $H(x,u)⪰H(y,v)⪰H(z,w)$ and $H(u,x)⪯H(v,y)⪯H(w,z)$, we have $(H(x,u),H(u,x),H(y,v),H(v,y),H(z,w),H(w,z))\in M$. By (1), we get

$$\begin{array}{r}[G(F(x,u),F(y,v),F(z,w))+G(F(u,x),F(v,y),F(w,z))]\\ \le \phi (G(H(x,u),H(y,v),H(z,w))+G(H(u,x),H(v,y),H(w,z))).\end{array}$$

Since $(x_0,y_0)\in X\times X$ with

$$H(x_0,y_0)⪯F(x_0,y_0)\text{and}H(y_0,x_0)⪰F(y_0,x_0).$$

(32)

We have

$$(F(x_0,y_0),F(y_0,x_0),F(x_0,y_0),F(y_0,x_0),H(x_0,y_0),H(y_0,x_0))\in M.$$

Assumption (a) holds, and F is continuous. By assumption (a) of Theorem 3.1, we have $H(x,y)=F(x,y)$ and $H(y,x)=F(y,x)$.

Next, assumption (b) holds; since F is H-increasing with respect to ⪯, using (32) and (2), we have

$$H(x_n,y_n)⪯H(x_{n+1},y_{n+1})\text{and}H(y_n,x_n)⪰H(y_{n+1},x_{n+1})\text{for all }n.$$

Therefore

$$\begin{array}{r}(H(x_{n+1},y_{n+1}),H(y_{n+1},x_{n+1}),H(x_{n+1},y_{n+1}),\\ H(y_{n+1},x_{n+1}),H(x_n,y_n),H(y_n,x_n))\in M.\end{array}$$

From H is continuous and by (15), we have

$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}H(H(x_n,y_n),H(y_n,x_n))=H(x,y)\text{and}\\ \underset{n\to \mathrm{\infty }}{lim}H(H(y_n,x_n),H(x_n,y_n))=H(y,x).\end{array}$$

For any two sequences ${\{H(x_n,y_n)\}}_{n=1}^{\mathrm{\infty }}$ and ${\{H(y_n,x_n)\}}_{n=1}^{\mathrm{\infty }}$ such that ${\{H(x_n,y_n)\}}_{n=1}^{\mathrm{\infty }}$ is a non-decreasing sequence in X with $H(x_n,y_n)\to x$ and ${\{H(y_n,x_n)\}}_{n=1}^{\mathrm{\infty }}$ is a non-increasing sequence in X with $H(y_n,x_n)\to y$. Using assumption (b), we have

$$H(x_n,y_n)⪯x\text{and}H(y_n,x_n)⪰y\text{for all }n.$$

Since H has the mixed monotone property, we have

$$\begin{array}{r}H(H(x_n,y_n),H(y_n,x_n))⪯H(x,y),\\ H(H(y_n,x_n),H(x_n,y_n))⪰H(y,x).\end{array}$$

Therefore, we have

$$(H(x_n,y_n),H(y_n,x_n),H(x,y),H(y,x),H(x,y),H(y,x))\in M,$$

and so assumption (b) of Theorem 3.1 holds. Now, since all the hypotheses of Theorem 3.1 hold, then F and H have a coupled coincidence point. The proof is completed. □

Corollary 3.5 In addition to the hypotheses of Corollary 3.4, suppose that for every $(x,y),(z,t)\in X\times X$, there exists $(u,v)\in X\times X$ which is comparable to $(x,y)$ and $(z,t)$. Then F and H have a unique coupled coincidence point.

Proof We define the subset $M\subseteq X^6$ by

$$M=\{(x,u,y,v,z,w)\in X^6:x⪰y⪰z,\text{and }u⪯v⪯w\}.$$

From Example 2.24, M is an $(H,F)$-closed set which satisfies the transitive property. Thus, the proof of the existence of a coupled coincidence point is straightforward by following the same lines as in the proof of Corollary 3.4.

Next, we show the uniqueness of a coupled coincidence point of F and H.

Since for all $(x,y),(z,t)\in X\times X$, there exists $(u,v)\in X\times X$ such that

$$H(x,y)⪯H(u,v),H(y,x)⪰H(v,u)$$

and

$$H(z,t)⪯H(u,v),H(t,z)⪰H(v,u),$$

we can conclude that

$$\begin{array}{r}(H(u,v),H(v,u),H(x,y),H(y,x),H(x,y),H(y,x))\in M\text{and }\\ (H(u,v),H(v,u),H(z,t),H(t,z),H(z,t),H(t,z))\in M.\end{array}$$

Therefore, since all the hypotheses of Theorem 3.3 hold, F and H have a unique coupled coincidence point. The proof is completed. □