1 Introduction

Let \(R\) will be an associative ring with center \(Z.\) For any \(x,y\in R\) the symbol \([x,y]\) represents commutator \(xy-yx\) and the Jordan product \(xoy=xy+yx.\) Recall that a ring \(R\) is prime if \(xRy=0\) implies \(x=0\) or \(y=0.\) An additive mapping \(*:R\rightarrow R\) is called an involution if \((xy)^{*}=y^{*}x^{*}\) and \((x^{*})^{*}=x\) for all \(x,y\in R.\) A ring equipped with an involution is called a ring with involution or \(*\)-ring. A ring with an involution is said to \(*\)-prime if \(xRy=xRy^{*}=0\) or \(xRy=x^{*}Ry=0\) implies that \(x=0\) or \(y=0.\) Every prime ring with an involution is \(*\)-prime but the converse need not hold general. An example due to Oukhtite [7] justifies the above statement that is, \(R\) be a prime ring, \(S=R\times R^{o}\) where \(R^{o}\) is the opposite ring of \(R.\) Define involution \(*\) on \(S\) as \(*(x,y)=(y,x).\) \(S\) is \(*\)-prime, but not prime. This example shows that \(*\)-prime rings constitute a more general class of prime rings. In all that follows the symbol \(S_{a_{*}}(R),\) first introduced by Oukhtite, will denote the set of symmetric and skew symmetric elements of \(R,\) i.e. \(S_{a_{*}}(R)=\{x\in R\mid x^{*}=\pm x\}.\)

An additive subgroup \(J\) of \(R\) is said to be a Jordan ideal of \(R\) if \(uor\in J,\) for all \(u\in J,r\in R.\) A Jordan ideal is said to be a \(*\)-Jordan ideal if \(J^{*}=J.\) Let \(I\) be the ring of integers. Set

$$\begin{aligned} R=\left\{ \left( \begin{array} [c]{ll} a &{} 0\\ c &{} d \end{array} \right) \mid a,c,d\in I\right\} \end{aligned}$$

and

$$\begin{aligned} J=\left\{ \left( \begin{array}{ll} 0 &{} 0\\ c &{} 0 \end{array} \right) ~\mid ~c\in I\right\} \end{aligned}$$

We define the following maps on \(R:\) \(*\left( \small \begin{array}{ll} a &{} 0\\ c &{} d \end{array} \right) =\left( \small \begin{array} [c]{ll} d &{} 0\\ -c &{} a \end{array} \right) \). Then it is easy to see that \(J\) is a nonzero \(*\)-Jordan ideal of \(R.\)

An additive mapping \(d:R\rightarrow R\) is called a derivation if \(d(xy)=d(x)y+xd(y)\) holds for all \(x,y\in R.\) For a fixed \(a\in R,\) the mapping \(I_{a}:R\rightarrow R\) given by \(I_{a}(x)=[a,x]\) is a derivation which is said to be an inner derivation. An additive mapping \(f:R\rightarrow R\) is called a generalized derivation if there exists a derivation \(d:R\rightarrow R\) such that

$$\begin{aligned} f(xy)=f(x)y+xd(y),\quad \text {\ for all }x,y\in R. \end{aligned}$$

This definition was given by Bresar in [3]. Let \(\alpha \) and \(\beta \) be any two automorphisms of \(R.\) An additive mapping \(d:R\rightarrow R\) is called a \((\alpha ,\beta )\)-derivation if \(d(xy)=d(x)\alpha (y)+\beta (x)d(y)\) holds for all \(x,y\in R.\) Inspired by the definition \((\alpha ,\beta )\)-derivation, the notion of generalized derivation was extended as follows: Let \(\alpha \) and \(\beta \) be any two automorphisms of \(R.\) An additive mapping \(f:R\rightarrow R\) is called a generalized \((\alpha ,\beta )\)-derivation on \(R\) if there exists a \((\alpha ,\beta )\)-derivation \(d:R\rightarrow R\) such that

$$\begin{aligned} f(xy)=f(x)\alpha (y)+\beta (x)d(y),\quad \text { for all }x,y\in R. \end{aligned}$$

Of course a generalized (1,1)-derivation is a generalized derivation on \(R,\) where 1 is the identity mapping on R.

Let \(S\) be a nonempty subset of \(R\). A mapping \(F\) from \(R\) to \(R\) is called centralizing on \(S\) if \([F(x),x]\in Z,\) for all \(x\in S\) and is called commuting on \(S\) if \([F(x),x]=0,\) for all \(x\in S.\) The study of centralizing and commuting mappings was initiated by Posner in [12] (Posner’s second theorem). Several authors have proved commutativity theorems for prime rings or semiprime rings admitting automorphisms or derivations which are centralizing and commuting on appropriate subsets of \(R\) (see, e.g., [1, 2, 4, 6] and references therein). Recently, Oukhtite et al. [8, Theorem 1] proved Posner’s second theorem to rings with involution in the case of characteristic not 2: Let \(R\) be a \(2\)-torsion free \(*\)-prime ring and \(U\) a square closed \(*\)-Lie ideal of \(R.\) If \(R\) admits a nonzero derivation \(d\) centralizing on \(U,\) then \(U\subseteq Z\). They generalized this theorem to generalized derivations centralizing on Jordan ideals in rings with involution [11, Theorem 1]. Also, in [5], the authors investigated some commutativity theorems for Jordan ideals in rings with generalized derivation. In the present paper, we shall study these results Jordan ideals of \(*\)-prime rings with generalized \(( \alpha ,\beta )\)-derivation of \(R.\)

2 Preliminaries

Throughout the paper, \(R\) will be a 2-torsion free \(*\)-prime ring and \(\alpha ,\beta \in AutR.\) \(F\) be a nonzero generalized \(( \alpha ,\beta )\)-derivation of \(R\) with associated nonzero \(( \alpha ,\beta )\)-derivation \(d\) which commutes with \(*\) and \(J\) be a nonzero \(*\)-Jordan ideal and a subring of \(R.\) Also, we will make some extensive use of the basic commutator identities:

$$\begin{aligned}&[x,yz]=y[x,z]+[x,y]z\\&[xy,z]=[x,z]y+x[y,z]\\&[xy,z]_{\alpha ,\beta }=x[y,z]_{\alpha ,\beta }+[x,\beta (z)]y=x[y,\alpha (z)]+[x,z]_{\alpha ,\beta }y\\&[x,yz]_{\alpha ,\beta }=\beta (y)[x,z]_{\alpha ,\beta }+[x,y]_{\alpha ,\beta } \alpha (z)\\&xo(yz)=(xoy)z-y[x,z]=y(xoz)+[x,y]z\\&(xy)oz=x(yoz)-[x,z]y=(xoz)y+x[y,z]\\&( xo(yz)) _{\alpha ,\beta }=(xoy)_{\alpha ,\beta }\alpha (z) -\beta ( y) [x,z]_{\alpha ,\beta }=\beta ( y) (xoz)_{\alpha ,\beta }+[x,y]_{\alpha ,\beta }\alpha ( z) \\&( (xy)oz) _{\alpha ,\beta }=x(yoz)_{\alpha ,\beta }-[x,\beta ( z) ]y=(xoz)_{\alpha ,\beta }y+x[y,\alpha ( z) ] \end{aligned}$$

Lemma 1

[9, Lemma 2] Let \(R\) be a \(2\)-torsion free \(*\)-prime ring, \(J\) a nonzero \(*\)-Jordan ideal of \(R\) and \(a,b\in R.\) If \(aJb=a^{*}Jb=0\), then \(a=0\) or \(b=0\).

Lemma 2

[9, Lemma 3] Let \(R\) be a \(2\)-torsion free \(*\)-prime ring and \(J\) a nonzero \(*\)-Jordan ideal of \(R.\) If \([J,J]=0,\) then \(J\subseteq Z.\)

Lemma 3

[10, Lemma 3] Let \(R\) be a \(2\)-torsion free \(*\)-prime ring and \(J\) a nonzero \(*\)-Jordan ideal of \(R.\) If \(J\subseteq Z,\) then \(R\) is commutative.

Lemma 4

Let \(R\) be a \(2\)-torsion free \(*\)-prime ring, \(J\) a nonzero \(*\)-Jordan ideal of \(R\) and \(d\) a nonzero \(( \alpha ,\beta ) \)-derivation of \(R.\) If \(d\) commutes with \(*\) and \(d(J)=0,\) then \(R\) is commutative.

Proof

By the hypothesis, we obtain that

$$\begin{aligned} d(ur+ru)=0,\quad \text { for all }u\in J,r\in R. \end{aligned}$$

Expanding this term and using the hypothesis, we get

$$\begin{aligned} (d(r),u)_{\alpha ,\beta }=0,\quad \text { for all }u\in J,r\in R. \end{aligned}$$
(2.1)

Replacing \(r\) by \(2rv,v\in J\) in (2.1) and using (2.1), \(d(J)=0,\) we have

$$\begin{aligned} d(r)\alpha ([v,u])=0,\quad \text { for all }u,v\in J,r\in R. \end{aligned}$$

Substituting \(rs,s\in R\) for \(r\) in this equation and using this, we find that \(d(r)\alpha (s)\alpha ([v,u])=0,\) and so

$$\begin{aligned} d(r)R\alpha ([v,u])=0,\quad \text { for all }u,v\in J,r\in R. \end{aligned}$$
(2.2)

Writing \(r^{*}\) by \(r\) in the last equation, we get

$$\begin{aligned} d(r^{*})R\alpha ([v,u])=0,\quad \text { for all }u,v\in J,r\in R. \end{aligned}$$

Since \(d\) commutes with \(*\), the last equation follows

$$\begin{aligned} d(r)^{*}R\alpha ([v,u])=0,\quad \text { for all }u,v\in J,r\in R. \end{aligned}$$
(2.3)

Appliying the \(*\)-primenes of \(R,\) because of (2.2) and (2.3), we conclude that \(d(r)=0\) or \([v,u])=0,\) for all \(u,v\in J,r\in R.\) Since \(d\) is a nonzero \(( \alpha ,\beta )\)-derivation of \(R,\) we arrive at \([J,J]=0,\) and so \(R\) is commutative by Lemmas 2 and 3. \(\square \)

3 Results

Theorem 1

Let \(R\) be a \(2\)-torsion free \(*\)-prime ring and \(J\) a nonzero \(*\)-Jordan ideal and a subring of \(R.\) If \(R\) admits a nonzero generalized \((\alpha ,\beta )\)-derivation \(F\) associated with nonzero \((\alpha ,\beta )\)-derivation \(d\) which commutes with \(*\) such that \([F(u),u]_{\alpha ,\beta }=0,\) for all \(u\in J,\) then \(R\) is commutative.

Proof

Suppose that

$$\begin{aligned}{}[F(u),u]_{\alpha ,\beta }=0,\quad \text { for all }u\in J. \end{aligned}$$
(3.1)

Linearizing (3.1) and using this, we obtain that

$$\begin{aligned}{}[F(u),v]_{\alpha ,\beta }+[F(v),u]_{\alpha ,\beta }=0,\quad \text { for all }u,v\in J. \end{aligned}$$
(3.2)

Replacing \(v\) by \(vu\) in (3.1), we get

$$\begin{aligned}{}[F(u),vu]_{\alpha ,\beta }+[F(v)\alpha (u)+\beta (v)d(u),u]_{\alpha ,\beta }=0. \end{aligned}$$

That is

$$\begin{aligned}&[F(u),v]_{\alpha ,\beta }\alpha (u)+\beta (v)[F(u),u]_{\alpha ,\beta }+[F(v),u]_{\alpha ,\beta }\alpha (u)\\&\qquad + F(v)[\alpha (u),\alpha (u)]+\beta (v)[d(u),u]_{\alpha ,\beta }+[\beta (v),\beta (u)]d(u)=0,\\&\quad \text { for all }u,v\in J. \end{aligned}$$

Now combining (3.1) and (3.2) in the last equation, we find that

$$\begin{aligned} \beta (v)[d(u),u]_{\alpha ,\beta }+[\beta (v),\beta (u)]d(u)=0,\quad \text { for all }u,v\in J. \end{aligned}$$
(3.3)

Again replace \(v\) by \(vw\) in (3.3) and use (3.3), to get

$$\begin{aligned}{}[\beta (v),\beta (u)]\beta (w)d(u)=0,\quad \text { for all }u,v,w\in J. \end{aligned}$$

Since \(\beta \) is an automorphism of \(R,\) we see that

$$\begin{aligned}{}[ v,u] J\beta ^{-1}( d( u) ) =0, \quad \text { for all }u,v\in J. \end{aligned}$$

Since \(J\) is a nonzero \(*\)-Jordan ideal of \(R\) yields that

$$\begin{aligned}{}[v,u]^{*}J\beta ^{-1}( d( u) ) =0,\quad \text { for all }v\in J,u\in J\cap S_{a_{*}}(R). \end{aligned}$$

Hence we get

$$\begin{aligned}{}[ v,u] J\beta ^{-1}( d( u) ) =[v,u]^{*}J\beta ^{-1}( d( u) ) =0,\quad \text { for all }v\in J,u\in J\cap S_{a_{*}}(R). \end{aligned}$$
(3.4)

By Lemma 1, we get either \([ v,u] =0,\) for all \(v\in J\) or \(d( u) =0\) for each \(u\in J\cap S_{a_{*}}(R).\) Let \(u\in J,\) as \(u+u^{*},u-u^{*}\in J\cap S_{a_{*}}(R)\) and \([ v,u\pm u^{*}] =0,\) for all \(v\in J\) or \(d(u\pm u^{*})=0.\) Hence we have \([ v,u] =0\) or \(d(u)=0,\) for all \(u,v\in J.\) We obtain that \(J\) is union of two additive subgroups of \(U\) such that

$$\begin{aligned} K=\{u\in J\mid d(u)=0\} \end{aligned}$$

and

$$\begin{aligned} L=\{u\in J\mid [ v,u] =0,\quad \text { for all }v\in J\}. \end{aligned}$$

Morever, \(J\) is the set-theoretic union of \(K\) and \(L.\) But a group can not be the set-theoretic union of two proper subgroups, hence \(K=J\) or \(L=J.\) In the former case, we get \(R\) is commutative by Lemma 4. In the latter case, \([ J,J] =( 0).\) That is \(J\subseteq Z\) by Lemma 2 and \(R\) is commutative by Lemma 3. This completes the proof. \(\square \)

In the view of Theorem 1, we get the following result:

Corollary 1

Let \(R\) be a \(2\)-torsion free \(*\)-prime ring and \(J\) a nonzero \(*\)-Jordan ideal and a subring of \(R.\) If \(R\) admits a nonzero \((\alpha ,\beta )\)-derivation \(d\) which commutes with \(*\) such that \([d(u),u]_{\alpha ,\beta }=0,\) for all \(u\in J,\) then \(R\) is commutative.

Theorem 2

Let \(R\) be a \(2\)-torsion free \(*\)-prime ring and \(J\) a nonzero \(*\)-Jordan ideal and a subring of \(R.\) If \(R\) admits a nonzero generalized \((\alpha ,\beta )\)-derivation \(F\) associated with nonzero \((\alpha ,\beta )\)-derivation \(d\) which commutes with \(*\) such that \(F(u)\alpha (u)=\beta (u)d(u),\) for all \(u\in J,\) then \(R\) is commutative.

Proof

We have

$$\begin{aligned} F(u)\alpha (u)=\beta (u)d(u), \quad \text { for all }u\in J \end{aligned}$$
(3.5)

Replacing \(u\) by \(u+v\) in (3.5) and using this, we get

$$\begin{aligned} F(v)\alpha (u)+F(u)\alpha (v)=\beta (v)d(u)+\beta (u)d(v),\quad \text { for all }u,v\in J. \end{aligned}$$
(3.6)

Writting \(vu\) for \(v\) in (3.6) and using (3.6), we obtain that

$$\begin{aligned} 2\beta (v)d(u)\alpha (u)=\beta (uov)d(u), \quad \text { for all }u,v\in J. \end{aligned}$$

Taking \(wv\) instead of \(v\) in the above equation and using this, we have

$$\begin{aligned} \beta ([w,u])\beta (v)d(u)=0,\quad \text { for all }u,v,w\in J. \end{aligned}$$

Hence we arrive at

$$\begin{aligned}{}[w,u]J\beta ^{-1}(d(u))=0,\quad \text { for all }u,v,w\in J. \end{aligned}$$

Since \(J\) is a nonzero \(*\)-Jordan ideal of \(R\) yields that

$$\begin{aligned}{}[w,u]^{*}J\beta ^{-1}(d(u))=0,\quad \text { for all }\,u,v,w\in J. \end{aligned}$$

Therefore, we get

$$\begin{aligned}{}[w,u]J\beta ^{-1}(d(u))=[w,u]^{*}J\beta ^{-1}(d(u))=0, \quad \text {for all }v\in J, \,u\in J\cap S_{a_{*}}(R). \end{aligned}$$

The similar arguments as used after equation (3.4), we get the required result. \(\square \)

Theorem 3

Let \(R\) be a \(2\)-torsion free \(*\)-prime ring and \(J\) a nonzero \(*\)-Jordan ideal and a subring of \(R.\) If \(R\) admits a nonzero generalized \((\alpha ,\beta )\)-derivation \(F\) associated with nonzero \((\alpha ,\beta )\)-derivation \(d\) which commutes with \(*\) such that \(F(u^{2})=\pm \alpha (u^{2}),\) for all \(u\in J,\) then \(R\) is commutative.

Proof

Linearizing the hypothesis, we get

$$\begin{aligned} F(u)\alpha (v)+\beta (u)d(v)+F(v)\alpha (u)+\beta (v)d(u)=\alpha (uv+vu),\quad \text { for all }u,v\in J.\nonumber \\ \end{aligned}$$
(3.7)

Replacing \(v\) by \(vu,\) \(u\in J\) in (3.7) and appliying this equation, we arrive at

$$\begin{aligned} \beta (uov)d( u) =0,\quad \text { for all }u,v\in J. \end{aligned}$$
(3.8)

Writting \(vw\) for \(v\) in (3.8) and using (3.8), we obtain that

$$\begin{aligned} \beta ([u,v])\beta (w)d( u) =0,\quad \text { for all }u,v,w\in J, \end{aligned}$$

and so

$$\begin{aligned}{}[u,v]J\beta ^{-1}(d( u) )=0,\quad \text { for all }u,v\in J. \end{aligned}$$

Since \(J\) is a nonzero \(*\)-Jordan ideal of \(R\) yields that

$$\begin{aligned}{}[u,v]^{*}J\beta ^{-1}( d( u) ) =0, \quad \text { for all }v\in J,\, u\in J\cap S_{a_{*}}(R). \end{aligned}$$

Therefore, we get

$$\begin{aligned}{}[u,v]J\beta ^{-1}(d(u))=[u,v]^{*}J\beta ^{-1}(d(u))=0,\quad \text {for all }v\in J,\,u\in J\cap S_{a_{*}}(R). \end{aligned}$$

The similar arguments as used after equation (3.4), we get the required result. \(\square \)

Theorem 4

Let \(R\) be a \(2\)-torsion free \(*\)-prime ring and \(J\) a nonzero \(*\)-Jordan ideal and a subring of \(R.\) If \(R\) admits a nonzero generalized \((\alpha ,\beta )\)-derivation \(F\) associated with nonzero \((\alpha ,\beta )\)-derivation \(d\) which commutes with \(*\) such that \(F(u^{2})=2d(u)\alpha (u),\) for all \(u\in J,\) then \(R\) is commutative.

Proof

We get

$$\begin{aligned} F(u^{2})=2d(u)\alpha (u),\quad \text { for all }u\in J. \end{aligned}$$

That is

$$\begin{aligned} F(u)\alpha (u)+\beta (u)d(u)=2d(u)\alpha (u),\quad \text { for all }u\in J. \end{aligned}$$
(3.9)

Linearizing (3.9) and using this, we obtain

$$\begin{aligned}&F(u)\alpha (v)+F(v)\alpha (u)+\beta (u)d(v)+\beta (v)d(u)=2d(u)\alpha (v)+2d(v)\alpha (u),\nonumber \\&\qquad \text { for all }u,v\in J. \end{aligned}$$
(3.10)

Taking \(vu\) instead of \(v\) in (3.10) and using this equation, we have

$$\begin{aligned} \beta ( uov) d( u) =2\beta (v)d(u)\alpha (u),\quad \text { for all }u,v\in J. \end{aligned}$$
(3.11)

Letting \(v\) by \(wv\) in (3.11) and using (3.11), we arrive at

$$\begin{aligned} \beta ([u,w])\beta (v)d( u) =0,\quad \text { for all }u,v,w\in J. \end{aligned}$$
(3.12)

That is

$$\begin{aligned}{}[u,w]J\beta ^{-1}(d( u) )=0,\quad \text { for all }u,w\in J. \end{aligned}$$

Since \(J\) is a nonzero \(*\)-Jordan ideal of \(R\) yields that

$$\begin{aligned}{}[u,w]^{*}J\beta ^{-1}( d( u) ) =0,\quad \text { for all }w\in J,\,u\in J\cap S_{a_{*}}(R). \end{aligned}$$

Therefore, we get

$$\begin{aligned}{}[u,w]J\beta ^{-1}(d(u))=[u,w]^{*}J\beta ^{-1}(d(u))=0,\quad \text {for all }w\in J,\,u\in J\cap S_{a_{*}}(R). \end{aligned}$$

Further application of similar arguments as used after (3.4), we get the required result. \(\square \)

Theorem 5

Let \(R\) be a \(2\)-torsion free \(*\)-prime ring and \(J\) a nonzero \(*\)-Jordan ideal and a subring of \(R.\) If \(R\) admits a nonzero generalized \((\alpha ,\beta )\)-derivation \(F\) associated with nonzero \((\alpha ,\beta )\)-derivation \(d\) which commutes with \(*\) such that \(d(u^{2})=2F(u)\alpha (u),\) for all \(u\in J,\) then \(R\) is commutative.

Proof

Expanding \(d(u^{2})=2F(u)\alpha (u),\) we have

$$\begin{aligned} d(u)\alpha (u)+\beta (u)d(u)=2F(u)\alpha (u),\quad \text { for all }u\in J. \end{aligned}$$
(3.13)

Linearizing (3.13) and using this, we obtain

$$\begin{aligned}&d(u)\alpha (v)+d(v)\alpha (u)+\beta (u)d(v)+\beta (v)d(u)=2F(u)\alpha (v)+2F(v)\alpha (u),\nonumber \\&\quad \quad \text { for all }u,v\in J. \end{aligned}$$
(3.14)

Writing \(vu\) instead of \(v\) in (3.14) and using this equation, we have

$$\begin{aligned} \beta ( uov) d( u) =2\beta (v)d(u)\alpha (u),\quad \text { for all }u,v\in J. \end{aligned}$$
(3.15)

Replacing \(v\) by \(wv\) in the last equation and using this, we arrive at

$$\begin{aligned} \beta ([u,w])\beta (v)d( u) =0,\quad \text { for all }u,v,w\in J. \end{aligned}$$

The similar arguments as used after equation (3.12) in the proof of Theorem 4, we get the required result. \(\square \)

We can give the following corollary by Theorem 4 (or 5):

Corollary 2.

Let \(R\) be a 2-torsion free \(*\)-prime ring and \(J\) a nonzero \(*\)-Jordan ideal and a subring of \(R.\) If \(R\) admits a nonzero \((\alpha ,\beta )\)-derivation \(d\) which commutes with \(*\) such that \(d(u^{2})=2d(u)\alpha (u),\) for all \(u\in J,\) then \(R\) is commutative.