1 Introduction

Einstein’s General Relativity (GR) has been the most widely accepted theory capable of explaining a number of phenomena and the geometry of spacetime in general. However, recent experimental observations showed the existence of phenomena with deviations from GR predictions. Among them the present acceleration of the universe has been a big problem in cosmology lacking of a proper or satisfactory explanation till now. This problem was discovered around 22 years ago with the help of type Ia supernovae observations [1,2,3,4]. It put a big question on the viability of GR, specially at cosmological scale. However, if one still wishes to stick with GR then he has to bear with the problem of mysterious invisible and exotic dark energy, which is responsible for around 76% of total energy content of the universe. Moreover, a theoretical problem associated with GR is that it is not renormalizable based on the conventional methods. In order to overcome these drawbacks of GR different modifications have been proposed. In most of the cases the modifications were introduced to solve some specific problems and as expected/ eventually they redefined the spacetime geometry and imprinted some changes in other sectors/ranges also. These changes can be a measure of how much the new theory is deviating from GR. A very important result from GR is the Gravitational Waves (GWs). In modified theories of gravity, the properties of GWs also change and can result massive modes of polarization apart from the usual GR modes or tensor modes of polarization [5]. These massive modes travel with less speed than the tensor modes. Besides the presence of massive modes of polarizations, the generation and propagation of GWs are also affected in different modified theories of gravity. The study of these variations can be a good tool to test the modified theories of gravity. With the first detection of GWs in 2015 by the LIGO and Virgo collaborations [6] followed by many other detections till now, a new and promising direction of studying gravitational theories has begun. These experimental evidences of GWs put a new set of constraints on GR as well as on modified theories of gravity.

A straight forward and simple modification to GR is the f(R) theory of gravity. In this theory, the Ricci scalar R of Einstein–Hilbert action is replaced by a function of R. Till now many models of f(R) theory have been proposed. Some successful models capable of explaining the drawbacks of GR upto a significant range are the Starobinsky model [7], Hu–Sawicki model [8], and Tsujikawa model [9,10,11,12] or exponential gravity. The properties of GWs in these models have been studied earlier. These studies show that GWs in f(R) gravity vary significantly from those in GR. As mentioned above, in GR, there are two polarization modes of GWs, viz., the tensor plus mode and tensor cross mode. These modes are massless in nature and they propagate with the speed of light in spacetime. In metric formalism f(R) gravity, there exists a scalar degree of freedom in the theory and hence the total degrees of freedom in the theory increases, which leads to increase the polarization modes of GWs in such theories [5, 13]. It is found that total number of polarization modes of GWs that can exist in f(R) theories of gravity is 3 [14]. Recent studies show that there exist a massless breathing mode and a massive longitudinal mode in a mixed state as a single polarization mode of GWs in f(R) theories of gravity. The existence of this extra polarization mode can be checked with the help of modified Newman–Penrose scalars and also by geodesic deviation equations in the theory [14]. However, another study shows that the existence of third mixed polarization mode of GWs are model dependent and there exists a model in f(R) gravity where the massive longitudinal mode vanishes and the third polarization mode becomes a pure massless breathing mode [15].

In this work, we have used a new f(R) gravity model as a toy model and studied the behaviour of the potential and scalar field mass both in Jordan frame and Einstein frame. We have also checked the polarization modes of GWs present in this model and tried to constrain the model.

The paper is organized as follows. In the next section, we have introduced our new f(R) model along with motivations. The characteristics of this model and the behaviour of the associated scalar field both in Jordan frame and Einstein frame have been studied in this section. Also, in this section, we have checked the viability of the model in terms of solar system tests. In Sect. 3, we have compared the model with two other most viable models viz., the Starobinsky model and the Hu–Sawicki model. In Sect. 4, the model has been constrained. In Sect. 5, we have studied the polarization modes of GWs present in the model by using the perturbed field equation and the modified Newman–Penrose formalism. In Sect. 6, we have reviewed a way to detect the polarization modes of GWs experimentally and discussed the possibilities of experimental validation of the model. In the last section, we conclude the paper with a very brief discussion of the results and the future aspects of the model in such type of studies.

2 A new model of f(R) gravity

Although we have a pretty good number of f(R) gravity models, no f(R) gravity model can explain all cosmological and astrophysical aspects of the present universe completely. Moreover, as like GR, different f(R) gravity models come with different types of drawbacks. However, we should mention that including the Starobinsky and Hu–Sawicki models there are several viable models in f(R) gravity, which were proposed in order to overcome the drawbacks of GR. Although the asymptotic behaviour of such viable models are almost same, the functional forms are completely different. In such f(R) gravity models the modifications of geometry is done in a unique manner by different f(R) functions, which might result different and unique cosmological and astrophysical features in the intermediate curvature regime. Thus a new f(R) gravity model capable of explaining the drawbacks of GR might have some different implications in different perspectives and also can have drawbacks or anomalies in different realms. Furthermore, a f(R) gravity model with more parametric control is more suitable in this respect in the sense that such a model can be constrained properly with the observational data by controlling its parameters and hence can be used easily to overcome the drawbacks of GR. With these motivations, here we introduce another f(R) gravity model containing three parameters, over and above to the existing ones, as given by:

$$\begin{aligned} f(R) = R-\frac{\alpha }{\pi }\, R_c \cot ^{-1}\left( \tfrac{R_c^2}{R^2}\right) -\beta \, R_c \left[ 1-\exp \left( {-\,\tfrac{R}{R_c}}\right) \right] , \nonumber \\ \end{aligned}$$
(1)

where \(\alpha \) and \(\beta \) are two dimensionless positive constants and \(R_c\) is a characteristic curvature constant having dimensions same as curvature scalar R. This model has two correction terms:

$$\begin{aligned} \frac{\alpha }{\pi }\, R_c \cot ^{-1}\left( \tfrac{R_c^2}{R^2}\right) \;\; \text {and}\;\; \beta \, R_c \left[ 1-\exp \left( {-\,\tfrac{R}{R_c}}\right) \right] . \end{aligned}$$

The first correction factor is estimated by two parameters \(\alpha \) and \(R_c\). Similarly, the second correction factor has also two parameters \(\beta \) and \(R_c\) and it mimics the exponential f(R) gravity model. We’ll show that this model passes the basic requirements of a viable f(R) gravity model including the solar system tests.

The requirements for any f(R) gravity model to describe the late-time dark energy problem are [16,17,18,19]:

  1. (1)

    A sufficient and suitable chameleon mechanism which allows f(R) gravity to pass the constraints of local systems. In case of our model, we’ll show that it can pass the solar system tests. A detailed study of chameleon mechanism in this model is beyond the scope of this paper.

  2. (2)

    A late-time stable de-Sitter solution. The condition for the existence of de-Sitter solution for a model is

    $$\begin{aligned} 2 f(R_0) - R_0 f'(R_0)=0, \end{aligned}$$
    (2)

    where \(R_0\) is the de Sitter curvature. To ensure the stability of the de Sitter solution, the model needs to satisfy the following condition [20]:

    $$\begin{aligned} \frac{f'(R_0)}{f''(R_0)}>R_0. \end{aligned}$$
    (3)

    For simplicity, considering \(R_0= R_c\) in the Eq. (2) and solving for \(\beta \) we find,

    $$\begin{aligned} \beta = -\frac{e (2 \alpha -\pi (\alpha -2))}{(6-4 e) \pi }, \end{aligned}$$
    (4)

    where e is the natural exponential. This is the de Sitter solution for the case \(R_0= R_c\). Now the stability condition (3) gives,

    $$\begin{aligned} (e\, \alpha +\pi \, \beta ) (2\, e\, \alpha +2\, \pi \, \beta -e\, \pi )<0. \end{aligned}$$
    (5)

    Using the expression for \(\beta \) from Eq. (4) in the above expression, we find the allowed range of \(\alpha \) as

    $$\begin{aligned} -1.68381<\alpha <0.367545. \end{aligned}$$
    (6)

    Thus for \(R_0= R_c\), the model can have stable de Sitter solutions if \(\alpha \) lies in the above range. In Sect. 3, we explicitly showed that the model has a stable de Sitter solution and also compared it with two other viable models.

  3. (3)

    A positive effective gravitational coupling, leading to \(f'(R) > 0\). By putting our model in this condition gives,

    $$\begin{aligned} 1-\frac{2\, \alpha \, R\, R_c^3}{\pi R^4+\pi R_c^4}-\beta \exp \left( -\,\tfrac{R}{R_c}\right) > 0. \end{aligned}$$
  4. (4)

    A stable cosmological perturbation and a positivity of the GWs for the scalar mode, causing to \(f''(R) > 0\). Using our model in this condition we find,

    $$\begin{aligned} \frac{\beta \exp \left( -\,\tfrac{R}{R_c}\right) }{R_c}-\frac{2\, \alpha \, R_c^3 \left( R_c^4-3 R^4\right) }{\pi \left( R^4+R_c^4\right) ^2} > 0. \end{aligned}$$

    This condition ensures the absence of tachyonic instabilities in the model, i.e. this condition results \(\text {m}^2_\phi > 0\).

  5. (5)

    An asymptotic behaviour to the \(\Lambda CDM\) model in the large curvature region. In case of our model, at large curvature region i.e. at \(R \rightarrow \infty \), we have \(f(R)-R \rightarrow -\,\frac{1}{2} R_c (\alpha +2 \beta ) = \text {constant}\), which mimicks the \(\Lambda CDM\) model in the large curvature region. Again, at \(R \rightarrow 0\), we have \(f(R)-R \rightarrow 0\).

Thus it is seen that this model is capable of explaining the late-time dark energy problem. In the following part of this paper, we’ll study the behaviour of scalar degrees of freedom of the model as well as the nature of GWs in it.

2.1 Scalar degrees of freedom in Jordan frame

The action of a generic f(R) gravity model is given as

$$\begin{aligned} S = \dfrac{1}{2\kappa } \int d^4 x \sqrt{-g} f(R) + \int d^4 x \sqrt{-g}\, {\mathcal {L}}_m \left[ g^{\mu \nu }, {\bar{\psi }} \, \right] . \end{aligned}$$
(7)

In the above equation, the function f(R) stands for any arbitrary function of Ricci curvature scalar R, \(g_{\mu \nu }\) is the metric, \(\kappa ^2 = 8 \pi G = M_{pl}^{-2}\) and \(\hbar = c=1\). Here \(M_{pl} \approx 2 \times 10^{18}\, \text {GeV}\) is the reduced Planck’s mass. \({\mathcal {L}}_m \left[ g^{\mu \nu }, {\bar{\psi }} \, \right] \) is the Lagrangian for a matter field \({\bar{\psi }}\). Variation of the action (7) with respect to the metric gives the following field equation:

$$\begin{aligned}&f'(R)R_{\mu \nu } -\dfrac{1}{2} f(R)g_{\mu \nu } - \nabla _\mu \nabla _\nu f'(R) + g_{\mu \nu }\, \square f'(R) \nonumber \\&\quad = \kappa ^2\, T_{\mu \nu }(g^{\mu \nu }, {\bar{\psi }}), \end{aligned}$$
(8)

where \(\square \equiv \nabla ^\mu \nabla _\mu \), \(T_{\mu \nu }(g^{\mu \nu }, {\bar{\psi }}) = \dfrac{-\,2}{\sqrt{-g}}\dfrac{\delta \left( \sqrt{-g}\,{\mathcal {L}}_m\left[ g^{\mu \nu }, {\bar{\psi }} \, \right] \right) }{\delta g^{\mu \nu }}\) is the matter energy–momentum tensor and \(f'(R) = \partial _R f(R)\). Trace of Eq. (8) is

$$\begin{aligned} f'(R) R +3\, \square f'(R) - 2 f(R) = \kappa ^2\, T, \end{aligned}$$
(9)

where \(T = g^{\mu \nu } T_{\mu \nu }\) is the trace of \(T_{\mu \nu }\). It is seen that the trace of the field equation is dynamical. This equation also indicates the existence of an extra scalar degree of freedom in the theory. For a detailed study about this degree of freedom we would like to use our model given in the Eq. (1). Now, if we define a scalar field as

$$\begin{aligned} \phi = f'(R), \end{aligned}$$
(10)

then for our model the field \(\phi \) becomes,

$$\begin{aligned} \phi = 1-\frac{2 \alpha R_c^3}{\pi R^3 \left( \frac{R_c^4}{R^4}+1\right) }-\beta \,\exp \left( -\,\tfrac{R}{R_c}\right) . \end{aligned}$$
(11)

This shows that the scalar curvature R can be expressed as a function of the field \(\phi \). From this definition of scalar field \(\phi \), we may view the trace Eq. (9) as an effective scalar field equation of Klein-Gordon with the following identification [21]:

$$\begin{aligned} \dfrac{dV}{d\phi }&\equiv \dfrac{1}{3} \Big [ 2f(R(\phi )) - R(\phi ) f'(R(\phi )) \Big ] \nonumber \\&\equiv \dfrac{1}{3} \Big [ 2f(R(\phi )) - R(\phi ) \phi \Big ], \end{aligned}$$
(12)

where V is the potential of the scalar field \(\phi \). Thus, the trace Eq. (9) can be written as a Klein-Gordon type equation for the scalar field \(\phi \) as given by,

$$\begin{aligned} \square \phi = \dfrac{dV}{d \phi } + \dfrac{1}{3}\,\kappa ^2\, T = \dfrac{dV_{eff}}{d\phi }, \end{aligned}$$
(13)

where \(V_{eff}\) is effective potential of the field and is define as

$$\begin{aligned} \dfrac{d V_{eff}}{d \phi } = \dfrac{1}{3}\Big [ 2f(R(\phi )) - R(\phi )\phi +\kappa ^2\, T \Big ]. \end{aligned}$$
(14)

At far away from the source or in absence of any matter source \(V_{eff} \equiv V\). Again, from the stationary condition:

$$\begin{aligned} \dfrac{dV_{eff}}{d\phi } = 0, \end{aligned}$$
(15)

we can have \(\phi =\phi _0\) satisfying \(\phi _0 = f'(R_0)\). From this condition, the mass of the scalar field (or the scalaron mass) can be obtained by differentiating the Eq. (14) with respect to \(\phi \) as

$$\begin{aligned} m^2_{\phi } \equiv \dfrac{dV^2_{eff}}{d\phi ^2}\Big \vert _{\phi = \phi _0} = \dfrac{1}{3}\left[ \dfrac{f'(R_0)}{f''(R_0)} - R_0 \right] . \end{aligned}$$
(16)

Here \(R_0\) is the background curvature corresponding to \(\phi _0\). From the above equation, we can see that avoiding of tachyonic instabilities demands \(\frac{f'(R_0)}{f''(R_0)} - R_0 \ge 0\) and to keep the mass term finite we need \(f''(R_0) \ne 0\). For our model the mass term is found as

$$\begin{aligned} m^2_{\phi } =\left[ \frac{R_c \exp \left( R/R_c\right) \left( \pi \left( R^4+R_c^4\right) ^2-8 \alpha R^5 R_c^3\right) -\pi \beta (R+R_c) \left( R^4+R_c^4\right) ^2}{3 \pi \beta \left( R^4+R_c^4\right) ^2-6 \alpha R_c^4 \exp \left( R/R_c\right) \left( R_c^4-3 R^4\right) }\right] _{R\, =\, R_0}. \end{aligned}$$
(17)

For \(R_0 = 0\), this equation gives

$$\begin{aligned} m^2_{\phi }\big |_{R_0\, =\, 0} = \frac{\pi (\beta -1) R_c}{6 \alpha -3 \pi \beta }. \end{aligned}$$
(18)

This shows that the mass of the scalar field is non-vanishing even at a large distance away from the source or in the Minkowski space. The mass \(m_{\phi }\) of the scalar field depends on the model parameters \(\alpha \), \(\beta \) and \(R_c\). Figure 1 shows the variation of \(m^2_{\phi }\) with respect to \(R_0\) for different sets of model parameters. From the figure we see that, the mass of the scalar field increases rapidly with the increasing value of the background curvature after a hump in the curve for \(0<R_0<1\) region, which increases when the parameter \(\alpha \) takes value closer to parameter \(\beta \). By increasing the difference between \(\beta \) and \(\alpha \) (i.e. for \(\beta \gg \alpha )\) the hump can be minimized. An increase of \(\alpha \) increases the hump which occurs near the small curvature region as mentioned above and comparatively decreases the mass of the scalar field at higher curvature region.

Fig. 1
figure 1

Mass square of the scalar field as a function of the background curvature for different sets of values of \(\alpha \) and \(\beta \) with characteristic curvature constant \(R_c = 1\) in arbitrary units

2.1.1 Scalar tensor equivalence of the model

To see the origin of the scalar field in the theory, we would like to rewrite the action (7) by introducing a new auxiliary scalar field \(\psi \) as [14, 22]

$$\begin{aligned} S= & {} \dfrac{1}{2\kappa ^2}\int d^4 x \sqrt{-g}\, \Big [ f_\psi (\psi )R - \big \lbrace f_\psi (\psi )\psi - f(\psi ) \big \rbrace \Big ] \nonumber \\&+ \int d^4 x \sqrt{-g}\, {\mathcal {L}}_m\left[ g^{\mu \nu }, {\bar{\psi }} \; \right] , \end{aligned}$$
(19)

where \(f_\psi = \dfrac{\partial f \left( \psi \right) }{\partial \psi }\). Now, varying this equation with respect to the new auxiliary scalar field \(\psi \) we get,

$$\begin{aligned} f_{\psi \psi } \left( \psi \right) \left( R - \psi \right) = 0. \end{aligned}$$
(20)

For finiteness of the previously defined mass square term of the scalar field (see Eq. (16)), we have \(f''\left( R \right) \ne 0\), which is in terms of \(\psi \), \(f_{\psi \psi } \left( \psi \right) \ne 0\). With this condition the above equation gives, \(R=\psi \). Substituting of this result in the action (19) we can recover the original action (7). Moreover, the quantum stability condition demands that \(f'' \left( R \right) \ge 0\). This along with the finiteness condition of the mass of the scalar field demands that \(f'' \left( R \right) > 0\). Thus, it is always possible to have a scalar tensor representation of f(R) theory of gravity. Redefining the previously defined scalar field \(\phi \) in terms of the new auxiliary field \(\psi \) as

$$\begin{aligned} \phi = f_\psi \left( \psi \right) , \end{aligned}$$
(21)

the action (19) can be rewritten as

$$\begin{aligned} S= & {} \dfrac{1}{2\kappa ^2} \int d^4 x \sqrt{-g}\, \Big [ \phi R - U\left( \phi \right) \Big ] \nonumber \\&+ \int d^4 x \sqrt{-g}\, {\mathcal {L}}_m \left[ g_{\mu \nu }, {\bar{\psi }} \; \right] , \end{aligned}$$
(22)

where \(U\left( \phi \right) = f_\psi \left( \psi \right) \psi - f \left( \psi \right) = \phi \, \psi \left( \phi \right) - f \left( \psi \left( \phi \right) \right) \) is the potential of the scalar field. To be precise, this term \(f_\psi \left( \psi \right) \psi - f \left( \psi \right) \) originates the scalar field. Unless and otherwise this term equals to zero, there exists a scalar field in the theory. In terms of the Ricci scalar, this term reads,

$$\begin{aligned} V = U \left( \phi \right) = f'\left( R \right) R - f \left( R \right) \big |_{R\, =\, R_0}. \end{aligned}$$
(23)

For \(f\left( R \right) = R\) and hence for \(f'\left( R \right) = 1\), the action (22) recovers GR giving the potential term \(V = 0\). Using Eq. (1) in this Eq. (23), the scalar field potential for our model can be obtained as

$$\begin{aligned} V= & {} \frac{\alpha }{\pi }\, R_c \cot ^{-1}\left( \tfrac{R_c^2}{R^2}\right) \nonumber \\&+R \left[ -\,\frac{2 \alpha R_c^3}{\pi R^3 \left( \frac{R_c^4}{R^4}+1\right) }-\beta \exp \left( -\,\tfrac{R}{R_c}\right) +1\right] \nonumber \\&+\beta R_c \left[ 1-\exp \left( -\tfrac{R}{R_c}\right) \right] -R \; \Big | _{R \,=\, R_0}. \end{aligned}$$
(24)
Fig. 2
figure 2

Variation of the potential (24) as a function of the background curvature \(R_0\) for different sets of values of \(\alpha \) and \(\beta \) with \(R_c = 1\) in arbitrary units

The variation of the potential V with respect to \(R_0\) for different values of \(\alpha \) and \(\beta \) is shown in Fig. 2 in arbitrary units considering \(R_c = 1\). The Figure shows that the potential in Jordan frame increases gradually with respect to the background curvature with some initial deviations depending upon the values of \(\alpha \) and \(\beta \). In contrast to the case of mass square of the scalar field, the potential shows some slight amount of dip, but near to the same small curvature region, which increases slowly when the value of \(\alpha \) moves closer to the value of \(\beta \). In fact, this dip in the potential is responsible for the hump in the mass square curve for the corresponding values of \(\alpha \) and \(\beta \). This dip region of the potential curve almost eliminates in the case when \(\alpha \ll \beta \). Moreover, with the increasing values of both \(\alpha \) and \(\beta \), the potential comparatively increases after the dip region or without the dip region.

2.2 Model in Einstein frame

In order see the behaviour of our model (1) in Einstein frame, which is usually used to avoid the non-minimal coupling of gravity with the scalar field, we would like to study the model in this frame also. In the Einstein frame, the following conformal transformation of the metric is performed [23, 24]:

$$\begin{aligned} {\tilde{g}}_{\mu \nu } = f'(R) g_{\mu \nu }, \end{aligned}$$

which for our model takes the form:

$$\begin{aligned} {\tilde{g}}_{\mu \nu } = \left[ 1-\frac{2 \alpha R_c^3}{\pi R^3 \left( \frac{R_c^4}{R^4}+1\right) }-\beta \exp \left( -\,\tfrac{R}{R_c}\right) \right] g_{\mu \nu }. \end{aligned}$$
(25)

Consequently, in this frame with \({\mathcal {L}}_m = 0\), the action changes to [23, 24]

$$\begin{aligned} S= & {} \int d^4 x \sqrt{-g} \nonumber \\&\times \left[ \dfrac{1}{2\kappa ^2} {\tilde{R}} - \dfrac{1}{2}\, {\tilde{g}}^{\mu \nu }\, \nabla _\mu \phi _E \nabla _\nu \phi _E - V(\phi _E) \right] , \end{aligned}$$
(26)

where the scalar field

$$\begin{aligned} \phi _E= & {} -\, \sqrt{\dfrac{3}{2}} \dfrac{1}{\kappa } \ln f'(R) = -\, \sqrt{\dfrac{3}{2}} \dfrac{1}{\kappa } \ln \nonumber \\&\times \left[ 1-\frac{2 \alpha R_c^3}{\pi R^3 \left( \frac{R_c^4}{R^4}+1\right) }-\beta \exp \left( -\,\tfrac{R}{R_c}\right) \right] \end{aligned}$$
(27)

and \(V(\phi _E)\) is potential of the scalar field in this frame, and is given by,

$$\begin{aligned} V(\phi _E) = \dfrac{1}{2\kappa ^2} \dfrac{U}{f'(R)^2} = \dfrac{1}{2\kappa ^2} \dfrac{f'(R) R - f(R)}{f'(R)^2} \Big |_{R = R_0}.\nonumber \\ \end{aligned}$$
(28)

The Eq. (27) shows the dependency of the Einstein frame scalar field \(\phi _E\) on the scalar curvature R. Using our model, this potential (28) can be expressed as

$$\begin{aligned} V(\phi _E) = \dfrac{1}{2\kappa ^2}\, \frac{\pi \, R_c\,\chi \, e^x \Big [-\pi \, \beta \, \chi \left( x+1\right) +e^x \left( \pi \beta +\pi \beta \, x^4-2\, \alpha \, x^2\right) +\alpha \,\chi \, e^x \cot ^{-1}\left( x^{-\,2}\right) \Big ]}{\Big [e^x \left( \pi x^4-2\, \alpha \, x+\pi \right) -\pi \beta \chi \Big ]^2}, \end{aligned}$$
(29)

where \(x = R_0/R_c\) and \(\chi = x^4+1\). Hence, the mass square term of the scalar field in the Einstein frame is

$$\begin{aligned} \nonumber m^2(\phi _E)&= \dfrac{d^2 V(\phi _E)}{d \phi _E^2} = \dfrac{1}{3} \left[ \dfrac{1}{f''(R)} + \dfrac{R}{f'(R)} - \dfrac{4f(R)}{f'(R)^2} \right] _{R\, =\, R_0} \nonumber \\&= \frac{1}{3} R_c \left[ \frac{\pi \chi ^2 e^x}{2\, \alpha \, e^x \left( 3 x^4-1\right) +\pi \beta \chi ^2}\right. \nonumber \\&\quad \left. +\frac{\pi \chi e^x x}{e^x \left( \pi x^4-2\, \alpha \, x+\pi \right) -\pi \beta \chi } \right] \nonumber \\&\quad -\frac{4 R_c \left[ -\,\pi ^{-1}\alpha \cot ^{-1}\left( x^{-2}\right) + \beta \left( e^{-x}-1\right) +x\right] }{3\left[ 2\, \alpha \, x\left( \pi x^4+\pi \right) ^{-1}+\beta e^{-x}-1\right] ^2}.\nonumber \\ \end{aligned}$$
(30)

In Minkowski space i.e. for \(R_0 = 0\), this equation takes the form:

$$\begin{aligned} m^2(\phi _E) = \frac{\pi R_c}{3 (\pi \beta -2\,\alpha )}. \end{aligned}$$
(31)
Fig. 3
figure 3

Variation of scalar field potential as a function of background curvature in Einstein frame for different values of \(\alpha \) and \(\beta \) parameters with \(R_c = 1\) in arbitrary units

Fig. 4
figure 4

Variation of scalaron mass square as a function of background curvature in Einstein frame for different values of \(\alpha \) and \(\beta \) parameters with \(R_c = 1\) in arbitrary units

Fig. 5
figure 5

Variation of scalar field \(\phi _E\) as a function of scalar curvature in Einstein frame for different values of \(\alpha \) and \(\beta \) parameters with \(R_c = 1\) in arbitrary units

Fig. 6
figure 6

Variation of scalar field potential as a function of field \(\phi _E\) in Einstein frame for different values of \(\alpha \) and \(\beta \) parameters with \(R_c = 1\) in arbitrary units

From Eqs. (29) and (30), we see that although the expressions for the scalar field potential and scalaron mass square in Einstein frame are little bit complicated in comparison to that in the Jordan frame, their variations as a function of \(R_0\) are found to be almost similar as seen from Figs. 3 and 4 respectively. That is, the behaviours of potential and the mass term of the scalar field are almost identical in both frames. In Fig. 5, the variation of field \(\phi _E\) as a function of background curvature is shown. The field \(\phi _E\) at \(R_0=0\) and \(R_0 \rightarrow \infty \) is independent of \(\alpha \), whereas it is independent of \(\beta \) only at \(R_0 \rightarrow \infty \). Moreover, \(\phi _E\) is non-zero at \(R_0=0\) and tends to zero at \(R_0 \rightarrow \infty \), which is obvious from it’s expression. Again from Eqs. (11) and (27) it is clear that all these behaviours of \(\phi _E\) should be applicable to \(\phi \) also, but with positive values of \(\phi \) for all values of \(R_0\). Variation of the potential (29) as a function of \(\phi _E\) is shown in Fig. 6. It is seen from this figure that the minimum of the potential moves towards the higher value of \(\phi _E\) when \(\alpha \ll \beta \) than the case when \(\alpha \sim \beta \). Obviously, similar behaviour can be attributed to the potential (24) as a function of the field \(\phi \). Thus, because of the similarity of behaviours of the scalar field, it’s potential and mass square term in both Jordan and Einstein frames, the rest of the study in this paper is done in the Jordan frame only.

2.3 Solar system tests of the model

It is possible to recover GR by introducing the Chameleon mechanism in the theory. In this mechanism, the scalar field \(\phi = f'(R)\) is coupled with the matter density of the environment. Thus, when a model is used inside the solar system, due to presence of matter density, the scalar field coupled with it gains mass and hence allows the model to pass the solar system tests. Clearly, this mechanism implies that the functional form f(R) should have a very closer value to the Ricci scalar R, for R above or equal to the solar system scale. A model is considered viable and consistent if it passes the solar system tests. Guo has introduced several methods to test whether an f(R) gravity model passes the solar system tests or not in Jordan frame [25]. According to Guo, a model can pass solar system tests if it satisfies the following conditions:

$$\begin{aligned}&\left| \, \dfrac{f(R) - R}{R}\, \right| \ll 1, \end{aligned}$$
(32)
$$\begin{aligned}&\left| \, f'(R) - 1 \, \right| \ll 1, \end{aligned}$$
(33)
$$\begin{aligned}&R f''(R) \ll 1. \end{aligned}$$
(34)

We’ve calculated the above functions numerically for our model (see Table 1). These functions are plotted against background curvature in the units of \(R_c\) for different parameters (see Fig. 7). These indicate that the model can be made to pass the solar system tests by increasing the ratio \(R_0/R_c\) or by simply decreasing the parameter \(R_c\). However, a simple and effective way to make the model solar system viable is to decrease all the parameters (i.e. \(\alpha , \beta \) and \(R_c\)) sufficiently (see Table 1). Thus, within a viable range of parameters, the model can easily pass the solar system tests. This is another advantage of this model, which allows us to enlist the model as a solar system viable model.

Fig. 7
figure 7

Plots of solar system test functions with respect to \(R_0/R_c\)

Table 1 Numerical values of solar system test functions for specific values of \(R_0/R_c, \alpha \) and \(\beta \)

3 Comparison of the model with other viable models

In this section, we would like to compare the model with two other viable models viz., the Starobinsky model and the Hu- Sawicki model in terms of stability and nature of the models in local regime. At first, we would like study the de Sitter stability of our toy model. For our model the Eq. (2) takes the form:

$$\begin{aligned} \frac{2\, \alpha \, x^2}{x^4+1}+\pi \left[ \beta e^{-x} (x+2)+x -2 \beta \right] =2 \alpha \cot ^{-1}\left( x^{-\,2}\right) , \end{aligned}$$

where \(x = R_0/R_c\) and \(R_c \ne 0\). On solving this equation for \(\beta \), we get,

$$\begin{aligned} \beta = -\,\frac{e^x \left( -\,\pi x^5-2 \alpha x^2+2 \alpha \cot ^{-1}\left( \frac{1}{x^2}\right) +2 \alpha x^4 \cot ^{-1}\left( \frac{1}{x^2}\right) -\pi x\right) }{\pi \left( -\,x+2 e^x-2\right) \left( x^4+1\right) }.\nonumber \\ \end{aligned}$$
(35)
Fig. 8
figure 8

\(\beta \) as a function of x and \(\alpha \). The contour shows the de Sitter solutions of the model

The contour plot of \(\beta \) as a function of x and \(\alpha \) is shown in Fig. 8. The Starobinsky model and the Hu–Sawicki model are defined respectively as

$$\begin{aligned} f_S(R) = R-s_1 m_s^2 \left[ 1-\left( \frac{R^2}{m_s^4}+1\right) ^{-\,n}\right] , \end{aligned}$$
(36)

and

$$\begin{aligned} f_H(R) = R-\frac{m^2 c_1 \left( \frac{R}{m^2}\right) ^\mu }{c_2 \left( \frac{R}{m^2}\right) ^\mu +1}. \end{aligned}$$
(37)

The de Sitter solution for the Starobinsky model is

$$\begin{aligned} s_1= \frac{x_s \left( x_s^2+1\right) ^{n+1}}{2 \left( x_s^2 \left( x_s^2+1\right) ^n+\left( x_s^2+1\right) ^n-n x_s^2-x_s^2-1\right) }, \end{aligned}$$
(38)

where \(x_s = R_0/m_s^2\). Similarly, for the Hu–Sawicki model, using \(\mu = 1\) the de Sitter solution is found to be,

$$\begin{aligned} c_2 = \frac{c_1+\sqrt{(c_1-1) c_1}-1}{x_h}. \end{aligned}$$
(39)

The contour plots of the parameters \(s_1\) and \(c_2\) respectively for the Starobinsky model and the Hu–Sawicki model are shown in Fig. 9. From the contours it can be seen that for very small values of n and \(x_s\) in the Starobinsky model, we do not have de Sitter solutions. The parameter \(s_1\) rises very rapidly for the increase of the parameters n and \(x_s\). For the Hu–Sawicki model, we see that, there is no de Sitter solutions present for \(c_1<1\) and the parameter \(c_2\) rises very slowly for increments of \(c_1\) beyond 1. On the other hand, for our toy model, we see that \(\alpha \) and x have a wider region of de Sitter solutions. The other parameter \(\beta \) has a higher value than \(\alpha \) in the de Sitter solution space towards higher values of x.

Fig. 9
figure 9

Figure on left side shows the de Sitter solutions of the Starobinsky model and figure on right side shows the de Sitter solutions for the Hu–Sawicki model

The de Sitter space defined by the Eq. (35) is stable if it satisfies the condition (3). Our model (1) in Eq. (3) leads to the inequality:

$$\begin{aligned}&\left[ 2 \alpha e^x \left( 3 x^4-1\right) +\pi \beta \left( x^4+1\right) ^2\right] \nonumber \\&\qquad \times \left[ \pi \beta (x+1) \left( x^4+1\right) ^2+e^x \lbrace 8 \alpha x^5-\pi \left( x^4+1\right) ^2\rbrace \right] \nonumber \\&\quad <0. \end{aligned}$$
(40)

By using Eq. (35) in the above inequality, we can further reduce it to the following form:

$$\begin{aligned}&\left( x-2 e^x+2\right) ^2 \nonumber \\&\qquad \times \left[ \pi x \left( x^4+1\right) ^2-2 \alpha \left( x^4+1\right) ^2 \right. \nonumber \\&\qquad \times \left. \cot ^{-1}\left( \frac{1}{x^2}\right) +2 \alpha A \right] \nonumber \\&\qquad \times \left[ 2 \alpha (x+1) \left( x^4+1\right) ^2 \cot ^{-1}\left( \frac{1}{x^2}\right) \right. \nonumber \\&\qquad \left. +\pi \left( -x^2-2 x+2 e^x-2\right) \left( x^4+1\right) ^2-2 \alpha B x^2\right] \nonumber \\&\quad >0, \end{aligned}$$
(41)

where \(A = \left( x^6-3 x^5-6 x^4+e^x \left( 6 x^4-2\right) +x^2+x+2\right) \) and \(B = \left( x^5-3 x^4+8 \left( e^x-1\right) x^3+x+1\right) \). When the model satisfies this Eq. (41), it can have stable de Sitter solutions. The stability region of the model in parameter space is shown in Fig.10. Again, in order to get oscillatory behaviour around the de Sitter space, the following condition needs to be satisfied [20]:

$$\begin{aligned} \frac{f'(R_0)}{f''(R_0)}>\dfrac{25 R_0}{16}. \end{aligned}$$
(42)

Using our model in this inequality, we get,

$$\begin{aligned}&\frac{\left( x^4+1\right) \lbrace e^x \left( \pi x^4-2 \alpha x+\pi \right) -\pi \beta \left( x^4+1\right) \rbrace }{2 \alpha e^x \left( 3 x^4-1\right) +\pi \beta \left( x^4+1\right) ^2} >\frac{25 x}{16}\nonumber \\ \end{aligned}$$
(43)

Now, eliminating \(\beta \) by using Eq. (35), we may rewrite the above expression as

$$\begin{aligned} -\,\frac{2 \left( x^4+1\right) \lbrace -\pi \left( -x+e^x-1\right) \left( x^4+1\right) -\alpha \left( x^4+1\right) \cot ^{-1}\left( \frac{1}{x^2}\right) +2 \alpha \left( e^x-1\right) x\rbrace }{\pi x \left( x^4+1\right) ^2-2 \alpha \left( x^4+1\right) ^2 \cot ^{-1}\left( \frac{1}{x^2}\right) +2 \alpha \lbrace x^6-3 x^5-6 x^4+e^x \left( 6 x^4-2\right) +x^2+x+2\rbrace }>\frac{25 x}{16}. \end{aligned}$$
(44)

If the toy model satisfies the above condition, it will have a stable de Sitter solution as well as oscillatory behaviour around the de Sitter solution. Similarly, for the Starobinsky model, the stability condition (3) takes the form:

$$\begin{aligned}&s_1 n \lbrace (2 n+1) x_s^2-1\rbrace \lbrace 4 s_1 n (n+1) x_s^3-\left( x_s^2+1\right) ^{n+2}\rbrace \nonumber \\&\quad <0. \end{aligned}$$
(45)

Using Eq. (38) in the above inequality, we obtain,

$$\begin{aligned}&C \left[ x^4 \lbrace -2 n^2+\left( x_s^2+1\right) ^n-3 n-1\rbrace +\left( x_s^2+1\right) ^n \right. \nonumber \\&\qquad \left. +x_s^2 \lbrace 2 \left( x_s^2+1\right) ^n-n-2\rbrace -1\right] >0, \end{aligned}$$
(46)

where \(C = n x_s \left( x_s^2+1\right) ^{2 n+2} \lbrace (2 n+1) x_s^2-1\rbrace \left[ \left( x_s^2+1\right) ^n+x_s^2 \lbrace \left( x_s^2+1\right) ^n-n-1\rbrace -1\right] ^2\). The Starobinsky model gives stable de Sitter solutions for any parameter set satisfying the above inequality. For this model, the condition (42) gives,

$$\begin{aligned}&C \left[ x_s^4 \lbrace -50 n^2+16 \left( x_s^2+1\right) ^n-16-57 n\rbrace \right. \nonumber \\&\quad +x_s^2 \lbrace 32 \left( x_s^2+1\right) ^n-32-7 n\rbrace \nonumber \\&\quad \left. +16 \left( x_s^2+1\right) ^n-16\right] >0. \end{aligned}$$
(47)

Any parameter sets in the Starobinsky model satisfying this condition will give oscillations around de Sitter solutions in de Sitter space. Finally for the Hu–Sawicki model, condition (3) gives:

$$\begin{aligned}&c_1 \mu x_h^{\mu +1} \lbrace c_2 (\mu +1) x_h^{\mu }-\mu +1\rbrace \nonumber \\&\qquad \times \left[ x_h \left( c_2 x_h^{\mu }+1\right) {}^3-c_1 \mu x_h^{\mu } \lbrace c_2 (\mu +2) x_h^{\mu }-\mu +2\rbrace \right] \nonumber \\&\quad >0. \end{aligned}$$
(48)

Now using Eq. (39) in the above expression and considering \(\mu = 1\), we get,

$$\begin{aligned}&\left( c_1-1\right) x_h \lbrace \sqrt{\left( c_1-1\right) c_1 x_h^2}+\left( c_1-1\right) x_h\rbrace \nonumber \\&\quad \lbrace 2 \sqrt{\left( c_1-1\right) c_1 x_h^2}+2 c_1 x_h-x_h\rbrace >0. \end{aligned}$$
(49)

Similarly, for the Hu–Sawicki model, the condition for existence of oscillations around stable de Sitter solutions gives:

$$\begin{aligned}&c_1^2 x_h \lbrace \sqrt{\left( c_1-1\right) c_1 x_h^2}+\left( c_1-1\right) x_h\rbrace \lbrace \left( 32 c_1-41\right) \nonumber \\&\quad \sqrt{\left( c_1-1\right) c_1 x_h^2}+\left( 32 c_1^2-57 c_1+25\right) x_h\rbrace >0. \end{aligned}$$
(50)

We have plotted these inequalities to check the stability region of the models in parameter space in Figs. 11 and 12.

Fig. 10
figure 10

Stability region of the model (1). Figure on the left side shows the stability region allowed by Eq. (41) and figure on the right side shows the stability region allowed by Eq. (44)

Fig. 11
figure 11

Stability region of the Starobinsky model. Figure on the left side shows the stability region allowed by Eq. (46) and figure on the right side shows the stability region allowed by Eq. (47)

Fig. 12
figure 12

Stability region of the Hu–Sawicki model. Figure on the left side shows the stability region allowed by Eq. (49) and figure on the right side shows the stability region allowed by Eq. (50)

From the Fig. 10, it is seen that the toy model has a small region of instability for lower values of x and comparatively higher values of \(\alpha \) (greater than 0.4 approximately). For higher values of x, the model has a continuous stability region in the parameter space. In case of the Starobinsky model, from Fig. 11 it is seen that there is a small region of instability in the parameter space. But in case of Hu–Sawicki model, the model is stable only for the values of \(c_1> 1\) (Fig. 12). This suggests that, the toy model has a wider range of stability region in the parameter space. Since, the model also has a comparatively larger region admitting Eq. (42), it can be easily concluded that the toy model with a selected set of parameters can pass the stability conditions as well as the condition for having oscillatory solutions in the de Sitter space.

Now, we would like to compare the bahviour of the toy model with the Starobinsky and Hu–Sawicki models using the functions used in the expressions (32), (33) and (34). These expressions ensure the viability of the models in local systems. Although, in the previous section, we have chosen the parameters freely to check the solar system viabilities, now we shall study the same and compare with the other two models in de Sitter sitter stability regions of the parameter spaces. However, it should be noted that most of the parameter sets used in that sections are found to belong automatically in the de Sitter sitter stability region of the model. The function used in relation (32) takes the following respective form in the toy model, Starobinsky model and Hu–Sawicki model in the de Sitter solution regime,

$$\begin{aligned}&\dfrac{f(R_0) - R_0}{R_0} \nonumber \\&\quad = \frac{\alpha \left( x^4+1\right) \cot ^{-1}\left( \frac{1}{x^2}\right) -\left( e^x-1\right) \left( \pi x^4+2 \alpha x+\pi \right) }{\pi \left( -x+2 e^x-2\right) \left( x^4+1\right) }, \end{aligned}$$
(51)
$$\begin{aligned}&\dfrac{f_S(R_0) - R_0}{R_0} \nonumber \\&\quad = -\frac{\left( x_s^2+1\right) \left( \left( x_s^2+1\right) {}^n-1\right) }{2 \left( \left( x_s^2+1\right) {}^n+x_s^2 \left( \left( x_s^2+1\right) {}^n-n-1\right) -1\right) }\nonumber \\ \end{aligned}$$
(52)

and

$$\begin{aligned}&\dfrac{f_H(R_0) - R_0}{R_0} \nonumber \\&\quad = -\frac{c_1 \left( x_h\right) _h^{\mu +1}}{\sqrt{\left( c_1-1\right) c_1 \left( x_h\right) _h^2} \left( x_h\right) _h^{\mu }+\left( c_1-1\right) \left( x_h\right) _h^{\mu +1}+\left( x_h\right) _h^2}.\nonumber \\ \end{aligned}$$
(53)

The second function in (33) gives,

$$\begin{aligned}&f'(R_0) - 1 \nonumber \\&\quad = \frac{2 \alpha \left( x^4+1\right) \cot ^{-1}\left( \frac{1}{x^2}\right) -x \left( \pi \left( x^4+1\right) +4 \alpha \left( e^x-1\right) \right) }{\pi \left( -x+2 e^x-2\right) \left( x^4+1\right) }, \end{aligned}$$
(54)
$$\begin{aligned}&f_S'(R_0) - 1 \nonumber \\&\quad = \frac{n x_s^2}{-\left( x_s^2+1\right) {}^n+x_s^2 \left( -\left( x_s^2+1\right) {}^n+n+1\right) +1}\nonumber \\ \end{aligned}$$
(55)

and

$$\begin{aligned}&f_H'(R_0) - 1 \nonumber \\&\quad = -\frac{c_1 \mu x_h^{\mu +3}}{\left( \sqrt{\left( c_1-1\right) c_1 x_h^2} x_h^{\mu }+\left( c_1-1\right) x_h^{\mu +1}+x_h^2\right) {}^2}.\nonumber \\ \end{aligned}$$
(56)

The third function for the respective models are calculated as:

$$\begin{aligned}&R_0 f''(R_0) \nonumber \\&\quad = \frac{2 \alpha x \left( 3 x^4-1\right) }{\pi \left( x^4+1\right) ^2} \nonumber \\&\qquad -\frac{x \left( -\pi x^5-2 \alpha x^2+2 \alpha \cot ^{-1}\left( \frac{1}{x^2}\right) +2 \alpha x^4 \cot ^{-1}\left( \frac{1}{x^2}\right) -\pi x\right) }{\pi \left( -x+2 e^x-2\right) \left( x^4+1\right) }, \end{aligned}$$
(57)
$$\begin{aligned}&R_0 f_S''(R_0) \nonumber \\&\quad = \frac{n x_s^2 \left( (2 n+1) x_s^2-1\right) }{\left( x_s^2+1\right) \left( \left( x_s^2+1\right) {}^n+x_s^2 \left( \left( x_s^2+1\right) {}^n-n-1\right) -1\right) }\nonumber \\ \end{aligned}$$
(58)

and

$$\begin{aligned}&R_0 f_H''(R_0) \nonumber \\&\quad = \frac{c_1 \mu x_h^{\mu {+}3} \left( (\mu {+}1) \sqrt{\left( c_1{-}1\right) c_1 x_h^2} x_h^{\mu }{+}\left( c_1{-}1\right) (\mu {+}1) x_h^{\mu {+}1}{-}(\mu {-}1) x_h^2\right) }{\left( \sqrt{\left( c_1-1\right) c_1 x_h^2} x_h^{\mu }+\left( c_1-1\right) x_h^{\mu +1}+x_h^2\right) {}^3}.\nonumber \\ \end{aligned}$$
(59)

These functions for the said models are compared in Fig. 13. It is seen that the models are capable of passing the solar system tests in the de Sitter stable regime for higher values of x. The Hu–Sawicki model in this regime shows a constant behaviour. However, the Starobinsky model and the toy model give the higher values of the test functions for lower x values. In this case, the behaviour of the toy model still indicates the ability to pass the solar system tests. One interesting point to note that for the toy model and the Starobinsky model, the asymptotic behaviours of the test functions are almost same. These results suggest that the behaviour of the toy model is closer to the Starobinsky model. However, to make a clear conclusion, a detailed study of the models is required and this is beyond the scope of this manuscript.

Fig. 13
figure 13

Solar system test functions with respect to x for the toy model (with \(\alpha = 0.3\)), Starobinsky model (with n = 2 at the upper panel and \(n=1\) at the lower panel) and Hu–Sawicki model (with \(\mu = 1\) and \(c_1 = 1.5\) at the upper panel and \(c_1 = 1.1\) at the lower panel)

4 Constraints on the model

For the viability of a model in f(R) gravity it is utmost necessary to impose constraints on the model on the basis of different available observational data. A model which passes such constraints are considered as a viable model in f(R) gravity. Starobinsky model and Hu Sawicki model are examples of two viable models in this context as mentioned earlier. There are several ways to constrain an f(R) gravity model [26,27,28,29,30,31,32]. A constrained model is helpful to study different implications of the model. In this section, we will try to constrain our toy model using the results published in [26, 27] and [29]. In Ref. [26], authors carried out a Markov chain Monte Carlo (MCMC) analysis for GWs from Hu Sawicki model using the data sets of cosmic microwave background (CMB) and baryon acoustic oscillations (BAO) together with the independent constraints on the relationship between the matter clustering amplitude \(\sigma _8\) and the matter mass-energy density \(\Omega _m\) from Planck Sunyaev–Zeldovich (PSZ) cluster number counts and also from the CFHTLens weak lensing tomography measurements. Combining CMB, BAO and \(\sigma _8-\Omega _m\) relationship from the PSZ catalog [33], they obtained a bound which is still better than the bounds obtained from the GW event GW170817 [29]. The bound on the parameter \(f'(R)\) reported by them is

$$\begin{aligned} -3.7 \times 10^{-6}< f'(R) - 1 < 3.7 \times 10^{-6} , \end{aligned}$$
(60)

with \(95\%\) confidence level at upper bound [26]. On the other hand in Ref. [27] a constraint was introduced on the Compton wavelength \(\lambda _{g}\) of the graviton. From their study, we have a constraint on \(\lambda _{g}^{-1}\) as given by,

$$\begin{aligned} 0 ~\text {m}^{-1}< \lambda _{g}^{-1}<1.098901099 \times 10^{-23} ~\text {m}^{-1} \end{aligned}$$
(61)

with \(90\%\) confidence level on upper bound [27].

Now, we have computed the values of \(f'(R) - 1\) and \(\lambda _{g}^{-1}\) for our model by taking into consideration of above cited respective upper bounds with the corresponding confidence levels to constraint our model parameters \(R_c, \alpha \) and \(\beta \). The results of this computation along with the contour plots are shown in Fig. 14. Here we have not considered the constraint on the model parameters coming from the fact that at higher curvatures it goes near to the cosmological constant. However, the parameters are chosen from the stability region of the parameter space. It is seen that the model can be a viable one within a proper range of variables. The figure shows the contours with \(95\%\) confidence level for \(f'(R) - 1\) and \(90\%\) confidence level for \(\lambda _{g}^{-1}\) (the larger contour) and with \(68\%\) confidence level for the both (the smaller contour). The central point denotes the boundary value for both the parameters \(f'(R) - 1\) and \(\lambda _{g}^{-1}\) and any value lower than the boundary value is viable. This point corresponds to the galaxy cluster Abell 1689 data [27]. In the plots we have considered three sets of the parameters and we see that the smaller values of \(\alpha \) allow the model to pass the constraints easily. In the Fig. 14, we have also shown 3 other points corresponding to galaxy clusters Abell 262, Abell 1991 and Abell 383 data from the Ref. [28]. All these points lie within the confidence level contours.

Fig. 14
figure 14

Contours with \(95\%\) confidence level on the upper bounds of \(f'(R) - 1\) [26] and \(90\%\) confidence level on the upper bounds of \(\lambda _{g}^{-1}\) [27] (the larger contour) and with \(68\%\) confidence level on the both (the smaller contour). The central red dot denotes the \(\lambda _g\) corresponding to galaxy cluster Abell 1689 [27], blue dot corresponds to Abell 262, black one corresponds to Abell 1991 and yellow one corresponds to Abell 383 [28] data

The model can be constrained by using the GWs event GW170817 also. In a recent study [29], f(R) gravity was constrained by using the GW170817. They provided a bound on \(f'(R)\), which is

$$\begin{aligned} -3 \times 10^{-3}< f'(R)-1 < 3 \times 10^{-3}. \end{aligned}$$
(62)

Using our toy model in this expression, we find,

$$\begin{aligned} -\,\frac{3}{1000}<\frac{2 \alpha x}{\pi x^4+\pi }+\beta e^{-x}<\frac{3}{1000}. \end{aligned}$$
(63)

Now, using Eq. (35) in the above expression, we can have,

$$\begin{aligned} -\,\frac{3}{1000}< & {} \frac{\pi x \left( x^4+1\right) -2 \alpha \left( x^4+1\right) \cot ^{-1}\left( \frac{1}{x^2}\right) +4 \alpha \left( e^x-1\right) x}{\pi \left( -x+2 e^x-2\right) \left( x^4+1\right) } \nonumber \\< & {} \frac{3}{1000}. \end{aligned}$$
(64)

Choosing \(x = 7.5\) (an arbitrary point in the stability region from Fig. 10) and considering \(\alpha \) to be a positive quantity, the above equation reduces to,

$$\begin{aligned} 0\le \alpha <0.743783. \end{aligned}$$
(65)
Fig. 15
figure 15

Plot on left shows the variation of \(\lambda _{g}\) with respect to \(R_0\), plot in middle shows the variation of \(f'(R)-1\) with respect to \(R_0\) and plot on right shows variation of \(m_{g}\) with respect to \(f'(R)-1\) for (i) \( \alpha = 0.150, \beta = 0.500, R_c = 30 \times 10^{-50} ~\text {m}^{-2}\); (ii) \(\alpha = 0.045, \beta = 0.500, R_c = 70 \times 10^{-50} ~\text {m}^{-2}\); (iii) \(\alpha = 0.300, \beta = 0.850, R_c = 10 \times 10^{-50} ~\text {m}^{-2}.\) These are the constrained set of parameters of our model that is done on the basis of upper bounds on \(f'(R)-1\) [26] and \(\lambda _g\) [27]

In Fig. 15, we have shown the variations of \(\lambda _{g}\) with respect to \(R_0\), \(f'(R)-1\) with respect to \(R_0\) and \(m_{g}\) with respect to \(f'(R)-1\) for the values of the model parameters used in the contour plots in Fig. 14. The hump in the mass of the scalar field encountered in the Fig. 1 and in the Fig. 6 are also present in the \(\lambda _{g}\) vs. \(R_0\) curves. However, as mentioned earlier, this hump vanishes when \((\beta -\alpha ) \gg 0\). For higher curvatures, \(\lambda _{g}\) rapidly moves towards zero. The function \(f'(R)-1\) also has significantly higher values near the Minkowski spacetime and as soon as the background Ricci curvature increases, this model dependent function decreases rapidly. This nature of the model is suitable for overcoming the local system constraints. Again as seen from the figure, with the increase of the function \(f'(R)-1\) mass of the scalar field decreases initially at a faster rate and then becomes almost constant at later stage.

5 Polarization modes of GWs in the model

In this section, we wish to check the polarization modes of GWs in the model. In presence of massive polarization mode, it would be easy to constraint the model using the experimental results. To explore the polarization modes of GWs in the model, at first we’ll introduce the perturbation to the field equation.

5.1 Perturbation to the field equation

If there are propagating GWs in spacetime, then they perturbs the metric around its background value. Considering the background metric as \({\bar{g}}_{\mu \nu }\) we may express the spacetime metric to the first order of perturbation value \(h_{\mu \nu }\), which is usually usually very small, as

$$\begin{aligned} g_{\mu \nu }={\bar{g}}_{\mu \nu } + h_{\mu \nu },\;\; \text{ where }\; \left| \, h_{\mu \nu } \,\right| \ll \left| \, {\bar{g}}_{\mu \nu }\, \right| . \end{aligned}$$
(66)

Now, expanding the Ricci tensor and the Ricci scalar upto the first order of \(h_{\mu \nu }\), we may write:

$$\begin{aligned} R_{\mu \nu }&\simeq {\tilde{R}}_{\mu \nu } + \delta R_{\mu \nu } + {\mathcal {O}}(h^2) \nonumber \\&= {\tilde{R}}_{\mu \nu } - \dfrac{1}{2} (\nabla _\mu \nabla _\nu h - \nabla _\mu \nabla ^{\lambda } h_{\lambda \nu }- \nabla _\nu \nabla ^{\lambda } h_{\mu \lambda } + \square h_{\mu \nu }) \nonumber \\&\quad + {\mathcal {O}}(h^2) \end{aligned}$$
(67)

and

$$\begin{aligned} R&\simeq {\tilde{R}} + \delta R + {\mathcal {O}}(h^2) \nonumber \\&={\tilde{R}} - \square h + \nabla ^\mu \nabla ^\nu h_{\mu \nu } -{\tilde{R}}_{\mu \nu } h^{\mu \nu } + {\mathcal {O}}(h^2). \end{aligned}$$
(68)

Similarly, we may write for the f(R) and \(f'(R)\) as

$$\begin{aligned} f(R)&\simeq f({\tilde{R}}) + f'({\tilde{R}})\, \delta R + {\mathcal {O}}(h^4), \end{aligned}$$
(69)
$$\begin{aligned} f'(R)&\simeq f'({\tilde{R}}) + f''({\tilde{R}})\, \delta R + {\mathcal {O}}(h^4), \end{aligned}$$
(70)

where \({\tilde{R}}\) is some constant curvature. Thus, due to the perturbation in spacetime the trace equation (9) can be rewritten as

$$\begin{aligned} 3 f''({\tilde{R}})\, \square \delta R + \left[ f''({\tilde{R}}){\tilde{R}} - f'({\tilde{R}})\right] \delta R = 0, \end{aligned}$$
(71)

where we have used \(T_{\mu \nu } = 0\) for the empty space or far away from the source. Fixing the gauge to be harmonic gauge with \( \nabla _\mu h^\mu _\nu = \dfrac{1}{2}\,\nabla _\nu h\), which after operating by \(\nabla ^\nu \) we find,

$$\begin{aligned} \nabla ^\mu \nabla ^\nu h_{\mu \nu } = \dfrac{1}{2}\, \square h. \end{aligned}$$
(72)

An important point to be mentioned here is that, the Eq. (8) is also satisfied by another solution: \(R_{\mu \nu } = \Lambda g_{\mu \nu } ={\tilde{R}}_{\mu \nu } \), giving

$$\begin{aligned} 2f'({\tilde{R}}) {\tilde{R}}_{\mu \nu } -g_{\mu \nu } f({\tilde{R}}) = \kappa ^2\, T_{\mu \nu }. \end{aligned}$$
(73)

This equation actually corresponds to the Eq. (15), which is the stationary condition used earlier in the Sect. 2. In empty space, this equation has the form:

$$\begin{aligned} 2f'({\tilde{R}}) {\tilde{R}}_{\mu \nu } -g_{\mu \nu } f({\tilde{R}}) = 0 \end{aligned}$$
(74)

and it leads to have the equation,

$$\begin{aligned} 2f({\tilde{R}}) - {\tilde{R}} f'({\tilde{R}}) = 0. \end{aligned}$$
(75)

This is the stationary condition (15) of the scalar field potential in the empty space corresponding to the constant curvature \({\tilde{R}}\) of spacetime. Using the Eqs. (68), (72) and (75) in Eq. (71), we get

$$\begin{aligned}&3 f''({\tilde{R}})\, \square ^2 h + \left( \frac{5 f({\tilde{R}}) f''({\tilde{R}})}{f'({\tilde{R}})}-f'({\tilde{R}})\right) \square h \nonumber \\&\quad + \left( \frac{2 f({\tilde{R}})^2 f''({\tilde{R}})}{f'({\tilde{R}})^2}-f({\tilde{R}}) \right) h=0. \end{aligned}$$
(76)

Now, we would like to define \(\square h = \text {m}^2 h\), where m is the mass of the associated scalar field. Using this definition in the above equation we obtain,

$$\begin{aligned}&3 f''({\tilde{R}}) \, \text {m}^4 + \left( \frac{5 f({\tilde{R}}) f''({\tilde{R}})}{f'({\tilde{R}})}-f'({\tilde{R}})\right) \text {m}^2 \nonumber \\&\quad + \left( \frac{2 f({\tilde{R}})^2 f''({\tilde{R}})}{f'({\tilde{R}})^2}-f({\tilde{R}}) \right) =0. \end{aligned}$$
(77)

This is a quadratic equation in \(\text {m}^2\) and solution for \(\text {m}^2\) gives,

$$\begin{aligned} \text {m}^2 = \frac{f'({\tilde{R}})}{3 f''({\tilde{R}})}-\frac{2 f({\tilde{R}})}{3 f'({\tilde{R}})} = \dfrac{1}{3} \left[ \dfrac{f'({\tilde{R}})}{f''({\tilde{R}})} - {\tilde{R}} \right] , \end{aligned}$$
(78)

and

$$\begin{aligned} \text {m}^2 = -\,\frac{f({\tilde{R}})}{f'({\tilde{R}})} = -\,\dfrac{{\tilde{R}}}{2}. \end{aligned}$$
(79)

We see that the second solution corresponds to tachyonic scalar field which becomes zero in the Minkowski spacetime or at far distance away from the source. The first solution is identical to Eq. (16). Thus the term \(\text {m}^2\) given by Eq. (78) is exactly same as the scalar field mass square term \(m_\phi ^2\) given in Eq. (17) for our model, when \({\tilde{R}} = R_0\). Therefore this solution suggests that there exists a massive scalar mode of polarization of GWs in the theory apart from the massless tensor modes.

At very far distance away from the source, we can consider \({\bar{g}}_{\mu \nu } = \eta _{\mu \nu }\), i.e. the Minkowski metric and the background curvature \({\tilde{R}} = 0\). In this case, the Ricci scalar slowly varies near zero, i.e. \(R \simeq 0 + \delta R\). Hence, for the Minkowski space the Eq. (66) can be written as

$$\begin{aligned} g_{\mu \nu }=\eta _{\mu \nu } + h_{\mu \nu }. \end{aligned}$$
(80)

And to the first order of \(h_{\mu \nu }\), we get

$$\begin{aligned} R_{\mu \nu }= & {} \frac{1}{2}\left( \partial _\mu \partial _\rho h^\rho _\nu +\partial _\nu \partial _\rho h^\rho _\mu -\partial _\mu \partial _\nu h-\square h_{\mu \nu }\right) , \end{aligned}$$
(81)
$$\begin{aligned} R= & {} \partial _\mu \partial _\rho h^{\rho \mu }-\Box h, \end{aligned}$$
(82)

where \(h=\eta ^{\mu \nu }h_{\mu \nu }\). So for our model, to the first order of perturbation, the Eq. (8) becomes

$$\begin{aligned} R_{\mu \nu }-\frac{1}{2}\eta _{\mu \nu }R - \frac{ (2 \alpha - \pi \beta ) }{ \pi (\beta -1) R_c}\left( \partial _\mu \partial _\nu R-\eta _{\mu \nu }\,\square R\right) =0.\nonumber \\ \end{aligned}$$
(83)

Taking the trace of this Eq. (83), we get

$$\begin{aligned} (\square -m_0^2)R=0, \end{aligned}$$
(84)

where

$$\begin{aligned} m_0^2=\frac{\pi (\beta -1) R_c}{6 \alpha -3 \pi \beta } \end{aligned}$$

with \(\alpha > 0\) and \(\beta > 0\). This is also exactly the same mass square term \(m_\phi ^2\big |_{R_0\,=\,0}\) in Minkowski space given by the Eq. (18) for our model. Next, we introduce a variable

$$\begin{aligned} {\bar{h}}_{\mu \nu }=h_{\mu \nu }-\frac{1}{2}\eta _{\mu \nu }h- \frac{ (2 \alpha - \pi \beta ) }{ \pi (\beta -1) R_c}\, \eta _{\mu \nu } R. \end{aligned}$$
(85)

The trace of this variable is

$$\begin{aligned} {\bar{h}}=\eta ^{\mu \nu }{\bar{h}}_{\mu \nu }=-\,h-4 \frac{ (2 \alpha - \pi \beta ) }{ \pi (\beta -1) R_c} R. \end{aligned}$$
(86)

Using this Eq. (86) in the variable (85) we find,

$$\begin{aligned} h_{\mu \nu }={\bar{h}}_{\mu \nu }-\frac{1}{2}\eta _{\mu \nu }{\bar{h}}-\frac{ (2 \alpha - \pi \beta )}{ \pi (\beta -1) R_c}\, \eta _{\mu \nu } R. \end{aligned}$$
(87)

From Eq. (85) and Eq. (87), one can easily see that both \(h_{\mu \nu }\) and \({\bar{h}}_{\mu \nu }\) are interchangeable, i.e. replacing \({\bar{h}}_{\mu \nu }\) by \(h_{\mu \nu }\) and vice-versa in Eq. (85) gives Eq. (87). Again, under an infinitesimal coordinate transformation, \(x^\mu \rightarrow x^{\mu \prime }=x^\mu +\varsigma ^\mu \), we have

$$\begin{aligned} h_{\mu \nu }'=h_{\mu \nu }-\partial _\mu \varsigma _\nu -\partial _\nu \varsigma _\mu . \end{aligned}$$
(88)

The trace of this equation is

$$\begin{aligned} h'=h-2\,\partial _\mu \varsigma ^\mu . \end{aligned}$$
(89)

And

$$\begin{aligned} {\bar{h}}_{\mu \nu }'={\bar{h}}_{\mu \nu }-\partial _\mu \varsigma _\nu -\partial _\nu \varsigma _\mu +\eta _{\mu \nu }\,\partial _\rho \varsigma ^\rho . \end{aligned}$$
(90)

The trace trace of this equation gives,

$$\begin{aligned} {\bar{h}}'={\bar{h}}+2\,\partial _\rho \varsigma ^\rho . \end{aligned}$$
(91)

Here we raise or lower the indices with the help of \(\eta _{\mu \nu }\), i.e. with the Minkowski metric. The Lorentz gauge condition \(\partial ^\mu {\bar{h}}_{\mu \nu }'=0\) can be obtained if \(\varsigma _\mu \) satisfies \(\square \varsigma _\nu =\partial ^\mu {\bar{h}}_{\mu \nu }\). The Lorentz gauge condition does not constrain the gauge freedom and there is always a possibility to choose the transverse and traceless conditions, i.e. \(\partial ^\mu {\bar{h}}_{\mu \nu }=0\) and \( {\bar{h}}=\eta ^{\mu \nu }{\bar{h}}_{\mu \nu }=0\) [14, 34,35,36]. By using the transverse traceless gauge condition and substituting the Eq. (87) into Eq. (81), we get

$$\begin{aligned} R_{\mu \nu }= & {} \frac{1}{2}\left[ -\,\square {\bar{h}}_{\mu \nu }+2\, \frac{ (2 \alpha - \pi \beta )}{ \pi (\beta -1) R_c}\,\partial _\mu \partial _\nu R \right. \nonumber \\&\quad \left. +\frac{(2 \alpha - \pi \beta )}{ \pi (\beta -1) R_c}\,\eta _{\mu \nu }\,\square R\right] . \end{aligned}$$
(92)

Plugging Eq. (92) into Eq. (83), we obtain

$$\begin{aligned} 3\,\frac{2 \alpha -\pi \beta }{2 \pi (\beta -1) R_c}\,\eta _{\mu \nu }(\square -m_0^2)R-\frac{1}{2}\,\square {\bar{h}}_{\mu \nu }=0. \end{aligned}$$
(93)

Combining Eqs. (84) and (93), we get

$$\begin{aligned} \square {\bar{h}}_{\mu \nu }=0, \end{aligned}$$
(94)

which is the wave equation of the massless tensor field. The solution to this Eq. (94) is [34, 35]

$$\begin{aligned} {\bar{h}}_{\mu \nu }=e_{\mu \nu }\exp (iq_\mu x^\mu )+\text {c.c.}, \end{aligned}$$
(95)

where \(\eta _{\mu \nu }q^\mu q^\nu =0\) and \(q^\mu e_{\mu \nu }=0\). Whereas the solution to the massive scalar field Eq. (84) is given by [34, 35],

$$\begin{aligned} R= \psi =\psi _0 \exp (ip_\mu x^\mu )+\text {c.c.}, \end{aligned}$$
(96)

where \(\eta _{\mu \nu }p^\mu p^\nu =-\,\text {m}^2_0\). Assuming the GWs propagation direction along z, the general solution can be written as [14]

$$\begin{aligned} h_{\mu \nu } = {\bar{h}}_{\mu \nu }(t-z) + \frac{2 \alpha -\pi \beta }{\pi (1-\beta )R_c}\, \eta _{\mu \nu }\, \psi (vt-z), \end{aligned}$$
(97)

where \({\bar{h}}_{\mu \nu } \) is transverse and traceless and it represents the standard spin-2 graviton, and \(\psi \) represents the scalar field which is massive in nature and travels with a speed less than c. The solution of the scalar part along z axis, i.e. \(\psi (vt-z)\) can be expressed as

$$\begin{aligned} \psi = \psi _0\, e^{-i\omega t\,+\,ikz} \end{aligned}$$
(98)

and hence the mass of the field in terms of k and \(\omega \) is

$$\begin{aligned} m_0 = \sqrt{\omega ^2 - k^2}. \end{aligned}$$
(99)

5.2 Calculation of exact polarization amplitudes and Newman–Penrose quantities of the model

In 1973, a powerful method was introduced in Ref. [37] which deals with the study of the properties of GWs in any metric theory of gravity. This method involves analysing all the relevant components of Riemann tensor, which results relative acceleration between two test particles. They used a null-tetrad basis in order to calculate the Newman–Penrose quantities [38]. In the Newman–Penrose formalism, there are ten \(\Psi \)’s, nine \(\Phi \)’s, and a \(\Lambda \), which are algebraically independent and represent the irreducible parts of the Riemann tensor \(R_{\lambda \mu \kappa \nu }\). They are known as Newman–Penrose quantities. But in case of plane waves or nearly plane waves, the differential and symmetry properties of \(R_{\lambda \mu \kappa \nu }\) reduce the number of independent, nonvanishing components, to six. Hence, in this formalism, the set \(\{\Psi _2,\Psi _3,\Psi _4,\Phi _{22}\}\) is used to describe the six independent components of GWs in the metric theory. In the tetrad basis, the Newman–Penrose quantities of the Riemann tensor are [37]:

$$\begin{aligned} \Psi _2&= -\,\frac{1}{6}\, R_{lklk}, \end{aligned}$$
(100)
$$\begin{aligned} \Psi _3&= -\,\frac{1}{2}\, R_{lkl{\overline{m}}},\end{aligned}$$
(101)
$$\begin{aligned} \Psi _4&= -\, R_{l{\overline{m}}l{\overline{m}}},\end{aligned}$$
(102)
$$\begin{aligned} \Phi _{22}&= -\, R_{lml{\overline{m}}}. \end{aligned}$$
(103)

It should be noted that, \(\Psi _3\) and \(\Psi _4\) are complex. Therefore, each one of them is capable of describing two independent polarizations. One polarization mode for the real part and one for the imaginary part. Thus total number of polarization modes is 6.

The tetrad components of Ricci tensors can be expressed as [37],

$$\begin{aligned} R_{lk}&= R_{lklk},\end{aligned}$$
(104)
$$\begin{aligned} R_{ll}&= 2\, R_{lml{\overline{m}}},\end{aligned}$$
(105)
$$\begin{aligned} R_{lm}&= R_{lklm},\end{aligned}$$
(106)
$$\begin{aligned} R_{l{\overline{m}}}&= R_{lkl{\overline{m}}}, \end{aligned}$$
(107)

and the Ricci scalar is

$$\begin{aligned} R = -\, 2\, R_{lk} = -\, 2\, R_{lklk}. \end{aligned}$$
(108)

In normal coordinate system [37],

$$\begin{aligned} \Psi _2&= -\,\dfrac{1}{6}\, R_{ztzt}, \\ \Psi _3&= -\,\dfrac{1}{2}\,R_{xtzt}+\dfrac{i}{2}\, R_{ytzt}, \\ \Psi _4&= -\,R_{xtxt} + R_{ytyt}+ 2i\, R_{xtyt}, \\ \Phi _{22}&= -\, R_{xtxt} - R_{ytyt}. \end{aligned}$$

Although the amplitudes \(\{\Psi _2,\Psi _3,\Psi _4,\Phi _{22}\}\) of a wave depend on the observer [37], there are certain invariant statements about them that hold true for all the standard observers if they hold true for any one. These statements characterize the invariant E(2) classes of waves. For a standard observer, under the assumptions that (a) the wave travels in the \(+z\) direction, and (b) the same frequency for a monochromatic wave is observed, the E(2) classes are:

  • Class \(II_6\): \(\Psi _2 \ne 0\). Standard observers measure the same non-vanishing amplitude in the \(\Psi _2\) mode. Presence or absence of all other modes is observer-dependent;

  • Class \(III_5\): \(\Psi _2 = 0,~\Psi _3 \ne 0\). Standard observers measure the absence of \(\Psi _2\) and the presence of \(\Psi _3\). Presence or absence of \(\Psi _4\) and \(\Phi _{22}\) is observer-dependent;

  • Class \(N_3\): \(\Psi _2 = \Psi _3 = 0,~\Psi _4 \ne 0, \Phi _{22} \ne 0\). Presence or absence of all modes is observer-independent;

  • Class \(N_2\): \(\Psi _2 = \Psi _3 = \Phi _{22} = 0;~\Psi _4 \ne 0\). Observer-independent;

  • Class \(O_1\): \(\Psi _2 = \Psi _3 = \Psi _4 = 0;~\Phi _{22} \ne 0\). Observer-independent;

  • Class \(O_0\): \(\Psi _2 = \Psi _3 = \Psi _4 = \Phi _{22} = 0\). Observer-independent. All standard observers measure no wave.

In f(R) gravity, the field equation derived from the Lagrangian in metric formalism results dynamical expressions for Ricci tensor and Ricci scalar, as we have already seen. Using this method, the expressions for Ricci tensor and scalar are calculated in weak field limit, i.e. far from the GWs source considering that the GW is propagating along z axis. Thus, the Ricci tensor components corresponding to directions other than z and t will vanish. But from earlier sections, we see that a massive scalar mode of polarization is present in this model and so it is not possible to use this Newman–Penrose formalism formalism which is developed for null waves [14]. For waves with massive propagation mode, modified Newman–Penrose formalism has to be applied [39]. In modified Newman–Penrose formalism, the polarization amplitudes as well as Newman–Penrose quantities are calculated for a massive wave subject to proper gauge condition. Considering the monochromatic wave solution of the form:

$$\begin{aligned} h_{\mu \nu }=C_{\mu \nu }e^{-i\omega t+ikz} \, , \end{aligned}$$
(109)

where \(\omega \) is the frequency and k is the wave number. However, we have noticed that, the modified Newman–Penrose scalars and polarization amplitudes introduced in [39] using Lorentz gauge condition can not distinguish breathing mode when the scalar field becomes massless. It is due to the fact that, they have used the transverse traceless condition to make the non-tensor modes vanish when \(\omega = k\). But \(\omega = k\) can’t demand \(p_{6}^{(b)}=0\) because, the breathing mode is massless in nature [15]. Breathing modes satisfy the transverse condition but not the traceless condition. It implies that a model having \(\omega = k\) can have massless breathing mode of polarization [15]. Keeping this fact in mind, we have modified the polarization amplitudes and Newman–Penrose quantities. According to our calculations, modified polarization amplitudes are expressed as:

$$\begin{aligned} p_{1}^{(l)}\,\,&= \frac{1}{2} \left( \frac{ \omega ^{2}-k^{2} }{\omega ^{2}+k^{2}} \right) \omega ^{2}(h_{tt}+h_{zz})-\frac{1}{2}\left( \omega ^{2}-k^{2} \right) h_{tt}\,,\nonumber \\ p_{2}^{(x)}\,&= \frac{1}{2}\left( \omega ^{2}-k^{2}\right) h_{xz}\,,\nonumber \\ p_{3}^{(y)}\,&= \frac{1}{2}\left( \omega ^{2}-k^{2}\right) h_{yz}\, , \nonumber \\ p_{4}^{(+)}&= \frac{1}{2}\,\omega ^{2}\left( h_{xx}-h_{yy} \right) \,,\nonumber \\ p_{5}^{(\times )}&= \frac{1}{2}\,\omega ^{2}h_{xy},\nonumber \\ p_{6}^{(b)}\,&= \frac{1}{2}\,\omega ^{2}(h_{xx}+h_{yy})\, . \end{aligned}$$
(110)

Here, we have not applied the traceless condition to the breathing mode. These are the exact polarization amplitudes of the wave. These expressions are valid for any metric theory. The modified Newman–Penrose quantities now can be expressed as:

$$\begin{aligned} \Psi _{2}&=-\,\frac{1}{24}\left( \frac{\omega ^{2}-k^{2}}{\omega ^{2}+k^{2}} \right) \left[ (3k^{2}-\omega ^{2})h_{tt}+(k^{2}-3\omega ^{2})h_{zz}\right] \,,\nonumber \\ \Psi _{3}&=\frac{1}{8}\,\frac{(\omega -k)(\omega +k)^{2}}{\omega }\, (h_{xz}-ih_{yz})\,,\nonumber \\ \Psi _{4}&=\frac{1}{8}\,(\omega +k)^2(h_{xx}+h_{yy})-\frac{1}{4 }(\omega +k)^{2}(h_{yy}+ih_{xy})\,,\nonumber \\ \Phi _{22}&=\frac{1}{8}\,(\omega +k)^2(h_{xx}+h_{yy})\,. \end{aligned}$$
(111)

These equations differ from those in Ref. [39].

Now, for our model the general wave solution is given by Eq. (97). Using this Eq. (97) in the above set of Eqs. (110), we found the polarization amplitudes for our model as

$$\begin{aligned} p_{1}^{(l)}\,\,&= \dfrac{1}{2}\,\text {m}^2_0\, C_1 \psi \,,\nonumber \\ p_{2}^{(x)}\,&= 0\,,\nonumber \\ p_{3}^{(y)}\,&= 0\, , \nonumber \\ p_{4}^{(+)}&= -\, \dfrac{1}{2}\,(\ddot{{\bar{h}}}_{xx} - \ddot{{\bar{h}}}_{yy})\, , \nonumber \\ p_{5}^{(\times )}&= -\, \frac{1}{2}\, \ddot{{\bar{h}}}_{xy}\,,\nonumber \\ p_{6}^{(b)}\,&= \omega ^2 C_1 \psi \,, \end{aligned}$$
(112)

where \(\text {m}^2_0\) is given by Eq. (99) and \(C_1 = \frac{2 \alpha -\pi \beta }{\pi (1-\beta )R_c}\) (see Eq. (97)). From the above expressions we can calculate the Newman–Penrose quantities. Note that, above results suggest, there are 4 non-zero polarization amplitudes in the theory. Using Eqs. (111), we’ve calculated the Newman–Penrose quantities for the model as

$$\begin{aligned} \Psi _{2}&=\dfrac{1}{12}\, \text {m}^2_0\, C_1 \psi \,,\nonumber \\ \Psi _{3}&=0\,,\nonumber \\ \Psi _{4}&=(\ddot{{\bar{h}}}_{yy}+i\ddot{{\bar{h}}}_{xy})\,,\nonumber \\ \Phi _{22}&=\dfrac{1}{4}\,(\omega +k)^2\, C_1 \psi \,. \end{aligned}$$
(113)

Thus the E(2) classification of the model is \(II_6\). The model exhibits non-zero Newman–Penrose quantities for 4 polarization modes viz., tensor plus, tensor cross, scalar transverse massless breathing mode and scalar longitudinal massive mode of polarization. However, the degrees of freedom associated with the theory is 3. This suggests that the breathing mode and the longitudinal mode exist in a mixed state to give rise to a single polarization mode. If \(m_0 =0\), the massive longitudinal mode will vanish, giving \(\Psi _{2}=0\). Note that \(m_0 =0\) is not a sufficient condition to imply the absence of scalar degrees of freedom in the theory. It is because, in f(R) theory there exists massless breathing mode which is transverse but not traceless. Absence of scalar degrees of freedom requires both \(\Psi _{2}=0\) and \(\Phi _{22} = 0\). When both \(m_0\) and \(\psi (vt-z)\) vanish, the theory reduces to GR giving only tensor modes of polarizations.

6 Detection of polarization modes of GWs—a review

Experimental detection of polarization modes of GWs is very important to know the exact nature of GWs and hence in checking the viabilities of modified gravity theories. In this section we discuss the Pulsar Timing Arrays (PTAs) as a tool to distinguish between different polarization modes. Moreover, we include a discussion on the results based on our model.

PTAs play a significant role in the indirect detection of GWs. They are also used for numerous astrophysical applications. In 1968, Counselman and Shapiro explained that the observations of pulsars could be used to test GR [40]. Later in 1982, the first millisecond pulsar was discovered [41]. Till now, a pretty good number of millisecond pulsars has been discovered. The advantage of these pulsars over the normal pulsars is that they are very stable. Their arrival times can be measured and predicted with a good accuracy. This allows to use these pulsars as a probe to search for GWs. In 2004, the Parkes Pulsar Timing Array (PPTA) project began with the Parkes 64 m telescope [42, 43]. After three years, the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) in North America was founded [44, 45]. NANOGrav uses the Arecibo and Green Bank telescopes to observe around 36 pulsars. In the same year, the European Pulsar Timing Array (EPTA) project was also founded [46]. Using the Sardinian, Effelsberg, Nancay, Westerbork and Jodrell Bank telescopes, EPTA observes around 42 pulsars. Later, by combining these three projects, the International Pulsar Timing Array (IPTA) was formed [47, 48]. In this study, we have used some selected data from PPTA [42] and IPTA [47, 48].

Fig. 16
figure 16

Variation of correlation functions \((\zeta (\theta ))\) with respect to \(\theta \). Plot on the left shows the correlation function for tensor modes and massless breathing mode of polarization of GWs. Plot on the right shows the correlation function for the longitudinal mode of polarization for different values of the mass of the mode as predicted by the model (1) along with correlation functions for some selected pulsars obtained from PPTA [42] and IPTA [47, 48] data

From the Ref. [49], we can have the correlation functions for different polarization modes. The calculation of correlation functions for the tensor and breathing modes are model independent, but in the case of massive longitudinal mode, the correlation function is model dependent as it depends on the mass of the scaler graviton. The correlation function for tensor modes is [49]

$$\begin{aligned} C^{+,\times }(\theta ) = \xi ^{GR}(\theta )\int _0^\infty \dfrac{|h_c^{+,\times }|^2}{24\pi ^2 f^3}\,df, \end{aligned}$$
(114)

where

$$\begin{aligned} \xi ^{GR}(\theta )= & {} \dfrac{3\,(1-\cos \theta )}{4}\log \left( \dfrac{1-\cos \theta }{2}\right) \\&+ \dfrac{1}{2} - \dfrac{1-\cos \theta }{8} + \dfrac{\delta (\theta )}{2}, \end{aligned}$$

and \(\theta \) is the angular separation between two pulsars. For the scalar modes it is [49]

$$\begin{aligned} C^{b}(\theta ) = \xi ^{b}(\theta )\int _0^\infty \dfrac{|h_c^{b}|^2}{12\pi ^2 f^3}\,df, \end{aligned}$$
(115)

where

$$\begin{aligned} \xi ^{b}(\theta ) = \dfrac{1}{8}\big [\cos \theta + 3 + 4\,\delta (\theta )\big ]. \end{aligned}$$

The normalized correlation function in general is given by:

$$\begin{aligned} \zeta (\theta ) = \dfrac{C(\theta )}{C(0)}. \end{aligned}$$

These are the correlation functions for tensor modes and massless breathing mode of GW polarization. But in our model, there exists a massive longitudinal mode. Thus to see the effect of massive mode, we follow the Refs. [50,51,52] in which the timing residual induced by GWs is expressed as

$$\begin{aligned} R=-\,\frac{1}{{{\mathcal {S}}}}\, A^{ij} H_{ij}, \end{aligned}$$
(116)

here \({{\mathcal {S}}}=2\left( 1+({c}/{{\omega _{g}}}){{{\mathbf {k}}_{g}}} \cdot {\hat{\mathbf{n }}}\right) \) gives the dispersion relation of the GWs, the terms \(\omega _{g}\) and \(k_{g}\) connect the mass of the longitudinal mode of polarization \(m_g\) by the relation \(m_g^2 = \omega _{g}^2 - k_{g}^2\), \(A^{ij} \equiv {\hat{\mathbf{n }}}^{i} {\hat{\mathbf{n }}}^{j}\) and \(H_{ij}= \int _{0}^{\tau }h_{ij}(\tau ,0)-h_{ij}(\tau -| \mathbf{D }|/c, \mathbf{D } )\,d\tau \). \(\mathbf{D }\) is the displacement vector from the observer to the pulsar and \({\hat{\mathbf{n }}}^{i}\) and \({\hat{\mathbf{n }}}^{j}\) are two unit vectors pointing to two pulsars. The correlation coefficient C between two different pulsars is given by

$$\begin{aligned} C_{1,2}(\theta )=\langle R_{1} R_{2}\rangle ={A}_1{A}_2 \langle \mathcal{S}_1 {{\mathcal {S}}}_2 {H}_1 {H}_2\rangle , \end{aligned}$$

where the sub-scripts are indices for the pulsars. With these assumptions and following Ref. [49], the correlation functions are calculated numerically for different values of \(m_g\) (see Fig. 16). In terms of our model,

$$\begin{aligned} m_g^2\equiv & {} \text {m}^2_{\phi }=\left[ \frac{R_c e^{R/R_c} \left( \pi \left( R^4+R_c^4\right) ^2-8 \alpha R^5 R_c^3\right) -\pi \beta (R+R_c) \left( R^4+R_c^4\right) ^2}{3 \pi \beta \left( R^4+R_c^4\right) ^2-6 \alpha R_c^4 e^{R/R_c} \left( R_c^4-3 R^4\right) }\right] _{R\, =\, R_0}. \end{aligned}$$

All these numerically calculated correlation functions are plotted with respect to \(\theta \) in Fig. 16. In this figure we have used three values of \(m_g\) for the following sets of parameters:

  1. (i)

    \(m_g = 0\) for \(R = 0 ~\text {m}^{-2}, R_c = 0 ~\text {m}^{-2}, ~\alpha , \beta \in [0, 1]\),

  2. (ii)

    \(m_g = 10^{-24}\) eV for \( R=0 ~\text {m}^{-2}, R_c= 10^{-50} ~\text {m}^{-2}, \alpha = 0.05, \beta = 0.031831\) and

  3. (iii)

    \(m_g = 10^{-25}\) eV for \(R= 4.44\times 10^{-52} ~\text {m}^{-2}, R_c = 5\times 10^{-51} ~\text {m}^{-2}, \alpha = 0.0314565, \beta = 0.17268558\).

To check the experimental viability of our model, we have calculated the correlation functions for GWs using some selected data from IPTA and PPTA data set [42, 47, 48, 53, 54] as mentioned above, which are also plotted in Fig. 16. Although we do not have a clear conclusion, which requires more observation period as well as data, we can still see that these experimental data could not directly rule out the existence of extra polarization modes of GWs. However, to distinguish between polarization modes, we need to wait for more PTA data with GW events.

7 Summary and conclusion

In this work, we have introduced a new f(R) gravity toy model and studied the polarization modes of GWs in it. The study shows that, in metric formalism, there exists 3 polarization modes of GWs viz., tensor plus mode, tensor cross mode and scalar mode. The scalar mode is a mixed state of massless breathing mode and massive longitudinal mode. The tensor modes of polarization are transverse, traceless and massless in nature. The scalar breathing mode is transverse but exists with non vanishing trace and massless in nature. On the other hand, the scalar longitudinal mode is massive in nature and hence propagates with speed less than that of tensor modes. When the scalar field becomes massless, the longitudinal mode vanishes and only the massless scalar breathing mode exists in the scalar degrees of freedom. As an experimental correspondence of our model prediction on the modes of GWs, we have compared the correlation function of massive longitudinal mode as predicted by the model with that of the some selected PTAs data of PPTA and IPTA. The result is found to be quite encouraging.

We have also shown that a wisely selected set of the parameters easily allows the model to pass the solar system tests, which is a very important requirement for the viability of a model. Further, the model has been constrained using combined CMB, BAO and \(\sigma _8-\Omega _m\) relationship from the PSZ catalog and Abell 1689 galaxy cluster data. The model can be easily constrained with the help of the parameters \(R_c\), \(\alpha \) and \(\beta \). Further, we have also constrained the model using the GW event GW170817. It is seen that, the model can withstand the constraints put by GW170817 and hence can be included as a post GW170817 viable model.

In the present work, we have studied only few properties of the model. However, for a proper understanding of the model characteristics, a detailed study is required. As such, in future the model can be checked also for various stabilities and constraints as well as in different cosmological and astrophysical contexts, which will give us more information about the viability of the model. Moreover, it is to be noted that in the Palatini formalism the polarization modes of GWs in f(R) gravity is model independent. So, this formalism can not distinguish our model from other f(R) gravity models in this context. Nevertheless, there may be some cosmological variations and stellar structure differences of the model in Palatini formalism, which might be useful to study the generation of GWs in such situations.