1 Introduction

General relativity is a real scientific theory of gravity developed by Albert Einstein in 1915. Einstein’s theory of general relativity (GR) is one of the most successful theories in physics, with a set of simple and beautiful field equations. It is highly consistent with cosmological observations and has created new insight into space-time concepts [1]. The mathematical framework of this geometric theory is based on Riemannian geometry, which describes the characteristics of the gravitational field using the space-time curvature tensor. One of the basic equations in this theory is the geodesic deviation equation (GDE), which provides the relationship between the Riemann curvature tensor and the relative acceleration between two nearby test particles. This equation describes the relative motion of free-falling particles to bend toward or away from each other under a gravitational field. The GDE provides a very elegant way to understand the properties of space-time and describe the nature of gravitational forces [2, 3]. In GR, particle motion is described by the curvature of space-time, and the curvature is described by the Riemann curvature tensor. The GDE acts as a force equation; in other words, the concept of force is replaced by geometry, and the path of particles is determined by geodesics instead of by the force equation. In 1933, the GDE was investigated for the first time by Synge, who used the GDE for the geometrical interpretation of Riemann curvature and also to explore the properties of Riemannian spaces and the properties of space-time with constant curvature [4].

Even though ordinary GR is a powerful gravitational theory, it is not the final answer to all cosmological and gravitational issues [1]. Alternative theories have been constructed to generalize the standard cosmology, including modified gravity models [5, 6]. In the last 10 years, f(R) theories have been studied using the Palatini approach, where the metric and the connection are treated as independent fields; see for example [7]. The metric formalism in f(R) gravity as described in [8, 9], in which we vary the action with respect to the metric \(g_{\mu \nu }\), can be promoted to the Palatini approach in which we vary the action concerning the metric and the connection [10,11,12,13,14,15]. This continues to form a novel modification of general relativity wherein an \(f({\mathcal {R}})\) term is added to the metric Einstein–Hilbert Lagrangian [16], and the authors can also go further with a modification like \(f\left( R,{\mathcal {R}}\right) \), where the gravitational action depends on a general function of both the metric and Palatini curvature scalars that is called generalized hybrid metric-Palatini gravity [17]. Of note, it has been reported that using the dynamically equivalent scalar–tensor representation causes the theory to pass solar system observational constraints [18]. Cosmological studies of this hybrid metric-Palatine gravitational theory were also conducted in [19]. The authors of [20] explored the Einstein static universe in this theory as well. In the present work, motivated by the fact that the GDE has been studied in several gravitational theories [21, 22], we aim to explore the GDE in the context of the generalized hybrid metric-Palatini theory. In addition, the generalized GDE has been studied in various papers. For example, in the context of modified gravity theories, it has been considered in arbitrary curvature–matter coupling theories, i.e., \(f(R, L_m)\) gravity [23], and has also been studied in f(R, T) gravity [24], f(Q, T) gravity [25], Brans–Dicke theory [26], f(Q) gravity [27], and the chameleon scalar field model [28]. In [29], the authors considered the generalized GDE in the brane world. In [30], the GDE was considered in Saez–Ballester theory.

The main target of this work is to systematically use the GDE to consider the geometry of the standard Friedmann–Lemaître–Robertson–Walker (FLRW) universe in the context of generalized hybrid metric-Palatini theory. In this regard, by considering the GDE for time-like, null, and space-like geodesic congruences in FLRW geometries and also obtaining the Raychaudhuri equation, we aim to determine the cosmological time evolution of these models. Also, we consider the generalized GDE for fundamental observers besides the modified Pirani equation. We study GDE for null vector fields to extract the null GDE equation and investigate the focusing condition for this model, in which the geodesics experience convergence besides the modified Mattig relation. Moreover, we propose a particular case \(f(R,{\mathcal {R}})=R+{{\mathcal {R}}}\) in order to study the numerical behavior of the deviation vector \(\eta (z)\) and the observer area–distance \(r_{0}(z)\) as a function of redshift. The existence of a maximum point for \(\eta (z)\) and \(r_{0}(z)\) at a certain redshift indicates that there were maximum values for the deviation vector and observer area–distance in the past when our universe was experiencing the inflationary regime. After that, the universe exited the inflationary regime, and the deviation vector gradually decreased with the increase in z. Therefore, our study characterizes the main geometrical and physical properties of the FLRW space-time using the generalized GDE, thereby demonstrating the utility of this equation in obtaining all the basic geometrical and dynamical results of modified standard cosmology in a unified way.

The paper is organized as follows: In Sect. 2, we review field equations in hybrid metric-Palatine gravity and its cosmological equations, and we also study the GDE for fundamental observers and null vector fields in \(f(R,{{\mathcal {R}}})=R+f({{\mathcal {R}}})\) gravity. In Sect. 3, we study the GDE in the scalar–tensor representation of \(f(R,{{\mathcal {R}}}) \) gravity. Finally, we close the paper with conclusions in Sect. 4.

2 Field equations in hybrid metric-Palatini gravity

Generalized gravity models attempt to provide a suitable alternative for dark energy by using the generalization of gravitational equations. That is, instead of Einstein’s equations of general relativity, alternative equations are obtained. So, by solving these equations, and without the need for cosmologists to introduce dark energy, accelerated dynamics for the universe can be obtained. In the cosmological context, f(R) gravity, as an alternative to dark energy, has been introduced to explain the recent acceleration of the universe. As mentioned in the introduction, the modified GR theory has two approaches to obtaining field equations: the metric approach and the Palatini approach. In metric formalisms, the field equations are obtained by the variation of the action with respect to the metric, and in this case, the affine connections are the functions of the metric. In the Palatini approach, metric and affine connections are considered as two independent variables. In the metric and Palatini formulation, symmetrical connections are assumed. In the metric-Palatini formulation, in addition to the independence of the metric and connection, the condition of symmetry in connections is absent. In this section, to obtain the field equations, we take the following action [19].

$$\begin{aligned} S=\frac{1}{2\kappa ^{2}}\int d^4{x}\sqrt{-g}[R+{f({{\mathcal {R}}})}]+S_{m}. \end{aligned}$$
(1)

In Eq. (1), R is the Ricci curvature scalar formed, \( \Gamma ^{\lambda }_{\mu \nu } \) is the Levi–Civita connection, and \({\mathcal {R}}\) is the Palatini curvature of an independent torsionless connection \( \hat{\Gamma }^{\lambda }_{\mu \nu } \), in analogy with the Palatini approach and also \(\kappa ^{2}=8\pi G\). Here, G is the Newtonian gravitational constant. Variation of the Eq. (1) with respect to the metric yields

$$\begin{aligned} G_{\mu \nu }+F({{\mathcal {R}}} ){{\mathcal {R}}} _{\mu \nu }-\frac{1}{2}f({\mathcal {R}})g_{\mu \nu }=\kappa ^{2}T_{\mu \nu }, \end{aligned}$$
(2)

with the usual definition of the matter stress–energy tensor

$$\begin{aligned} T_{\mu \nu }=-\frac{2}{\sqrt{-g}}\frac{\delta (\sqrt{-g}~L_{m})}{\delta (g_{\mu \nu })}, \end{aligned}$$
(3)

where \(L_{m}=L_{m}\left( g^{\mu \nu },\Psi \right) \) is the matter Lagrangian including the minimally coupled matter fields \(\psi \) to the metric \(g_{\mu \nu }\). Tracing the field equation gives us

$$\begin{aligned} 2f({{\mathcal {R}}} )-F({{\mathcal {R}}} ){{\mathcal {R}}}=-\kappa ^{2}T-R. \end{aligned}$$
(4)

Note that we have

$$\begin{aligned} F({{\mathcal {R}}}) =\frac{df({{\mathcal {R}}})}{d{{\mathcal {R}}}}. \end{aligned}$$
(5)

By rewriting Eq. (2), the Ricci tensor can be expressed as

$$\begin{aligned} R_{\mu \nu }=\kappa ^{2}T_{\mu \nu }+\frac{1}{2}g_{\mu \nu }(F({\mathcal {R}})+R)-F({{\mathcal {R}}}){{\mathcal {R}}}_{\mu \nu }. \end{aligned}$$
(6)

Using Eqs. (2), (4), and (6), we obtain the hybrid Ricci tensor as

$$\begin{aligned} {{\mathcal {R}}}_{\mu \nu }= & {} \frac{1}{1+F({\mathcal {R}})}\bigg [\frac{1}{2}g_{\mu \nu }\bigg (R+f({\mathcal {R}})-\frac{\Box {F({{\mathcal {R}}})}}{F({{\mathcal {R}}} )}\bigg )\nonumber \\{} & {} +\kappa ^{2}T_{\mu \nu } \!+\!\frac{3}{2}\frac{1}{F^{2}({{\mathcal {R}}})}F({\mathcal {R}})_{,\mu }F({{\mathcal {R}}})_{,\nu }\!-\!\frac{\nabla _{\mu }F({\mathcal {R}})_{,\nu }}{F({{\mathcal {R}}})}\bigg ],\nonumber \\ \end{aligned}$$
(7)

and from (4), we get

$$\begin{aligned} R =-\kappa ^{2}T+F({{\mathcal {R}}}){{\mathcal {R}}}-2f({{\mathcal {R}}}). \end{aligned}$$
(8)

Thus far, we have extracted some relations by which we will find the basic feature of the GDE in \(f(R,{{\mathcal {R}}})=R+f({{\mathcal {R}}})\) gravity. In the following section, similar to the previous works [21, 22], we try to assemble the general form of the right-hand side of the GDE in the context of the generalized hybrid metric-Palatini gravity.

2.1 Geodesic deviation equation in \(f(R,{{\mathcal {R}}})=R+f({{\mathcal {R}}})\) gravity

The GDE is one of the basic equations in the theory of general relativity, and provides the relationship between the Riemann curvature tensor and the relative acceleration between two test particles.

$$\begin{aligned} \frac{D^{2}\eta ^{\alpha }}{D\nu ^{2}}=-R^{\alpha }_{\,\beta \gamma \delta }V^{\beta }\eta ^{\gamma }V^{\delta }. \end{aligned}$$
(9)

This equation describes the relative motion of free-falling particles to bend toward or away from each other under a gravitational field. If we describe the geodesic as \( x^{\alpha }=(\nu ,s) \), \(R _{\alpha \beta \gamma \delta } \) is the Riemann curvature tensor, and \(V^{\alpha }=\frac{dx^{\alpha }}{d\nu }\) is the normalized tangent vector that belongs to the geodesics. In the above equation, \( \eta ^{\alpha }=\frac{dx^{\alpha }}{ds} \) denotes the deviation vector of these two adjacent geodesics.

Note that in several classes of modified gravity theories, some new terms appear on the right-hand side of Eq. (9), mainly due to the presence of couplings between different fields and geometric quantities. This may lead to non-conservation of the energy–momentum tensor of matter and thus to the appearance of an extra force; see [23]. However, in the hybrid metric-Palatini gravity, basically, there is no coupling between the matter fields and geometric quantities, and conservation of the energy–momentum tensor is preserved; thus the standard form of the GDE is satisfied. Furthermore, as we know that the universe is isotropic and homogeneous, only the time derivatives of the scalar fields appear, and in the comoving frame, one has \(\eta ^{0} = 0\); thus, in the scalar tensor framework of this model that is studied in the following, we again have the standard form of the GDE.

In general, the Riemann tensor can be decomposed as follows [2, 3]:

$$\begin{aligned} R _{\alpha \beta \gamma \delta }= & {} \frac{1}{2}\bigg (g_{\alpha \gamma }R _{\delta \beta }-g_{\alpha \delta }R _{\gamma \beta }+g_{\beta \delta }R _{\gamma \alpha }-g_{\beta \gamma }R _{\delta \alpha }\bigg )\nonumber \\{} & {} -\frac{R}{6}\bigg (g_{\alpha \gamma }g_{\delta \beta }-g_{\alpha \delta }g_{\gamma \beta }\bigg )+C_{\alpha \beta \gamma \delta }, \end{aligned}$$
(10)

where \(C_{\alpha \beta \gamma \delta }\) is the Weyl tensor.

In continuation of our study, we take the standard cosmology model line element, the FLRW universe, as

$$\begin{aligned} \textrm{d}s^{2}=-\textrm{d}t^{2}+a(t)^{2}\left[ \frac{\textrm{d}r^{2}}{1-Kr^{2}}+r^{2}\textrm{d}\theta ^{2}+r^{2}sin\theta ^{2}\textrm{d}\phi ^{2}\right] , \end{aligned}$$
(11)

where a(t) is the scale factor, and K denotes the three-dimensional spatial curvature with values \(-1\), 0, and 1. The energy momentum tensor can be written in the form of a perfect fluid as

$$\begin{aligned} T_{\alpha \beta }=(\rho +P)u_{\alpha }u_{\beta }+P~g_{\alpha \beta }, \end{aligned}$$
(12)

where \( \rho \) and P are the energy density and pressure, respectively. The trace of \(T_{\alpha \beta }\) is given by

$$\begin{aligned} T=-\rho +3P. \end{aligned}$$
(13)

We know that by redefining the cosmic time, t, to the conformal time \(\tau \) by \(\textrm{d}t = a(t)\textrm{d}\tau \), the FLRW metric (11) can be rewritten as a form of a conformally flat metric, and according to the conformal invariance property of the Weyl tensor, in the homogeneous and isotropic space-time we can set \(C_{\alpha \beta \gamma \delta }=0\). Thus, by using Eqs. (6), (8), and (10), the hybrid Riemann tensor will be in the following form:

$$\begin{aligned} R _{\alpha \beta \gamma \delta }=\,&\frac{1}{2}\bigg [ \left( \frac{2R}{3}+f({{\mathcal {R}}})\right) (g_{\alpha \gamma }g_{\delta \beta }-g_{\alpha \delta }g_{\gamma \beta })\nonumber \\&+ \kappa ^{2}(T_{\delta \beta }g_{\alpha \gamma }-T_{\gamma \beta }g_{\alpha \delta }+T_{\alpha \gamma }g_{\beta \delta } -T_{\alpha \delta }g_{\beta \gamma })\nonumber \\&+F({{\mathcal {R}}})[-g_{\alpha \gamma } {{\mathcal {R}}}_{\delta \beta }+g_{\alpha \delta }{{\mathcal {R}}}_{\gamma \beta } -g_{\beta \delta }{{\mathcal {R}}}_{\gamma \alpha } +g_{\beta \gamma }{{\mathcal {R}}}_{\delta \alpha }] \bigg ]. \end{aligned}$$
(14)

By contracting \(R ^{\alpha }_{\beta \gamma \delta }\) with \( V^{\beta }\eta ^{\gamma }V^{\delta } \), Eq. (14) can be written as follows:

$$\begin{aligned}&R^{\alpha }_{\,\beta \gamma \delta }V^{\beta }\eta ^{\gamma }V^{\delta }=\frac{1}{2(1+F({\mathcal {R}} ))}\bigg [ \kappa ^{2}(-\delta ^{\alpha }_{\delta }T_{\gamma \beta }+\delta ^{\alpha }_{\gamma }T_{\delta \beta }\nonumber \\&\quad +T^{\alpha }_{\gamma }g_{\delta \beta }-T^{\alpha }_{\delta }g_{\gamma \beta })+\left[ \left( \frac{2-F({\mathcal {R}} )}{3}R\right) \right. \nonumber \\&\quad \left. + f({{\mathcal {R}}} )+\square F({{\mathcal {R}}} )\right] (\delta ^{\alpha }_{\gamma }g_{\delta \beta }\nonumber \\&\quad -\delta ^{\alpha }_{\delta }g_{\beta \gamma })-\frac{3}{2F({\mathcal {R}} )}(\delta ^{\alpha }_{\gamma }F({{\mathcal {R}}} )_{,\delta }F({{\mathcal {R}}} )_{,\beta }\nonumber \\&\quad -\delta ^{\alpha }_{\delta }F({{\mathcal {R}}} )_{,\gamma }F({{\mathcal {R}}} )_{,\beta }+g_{\delta \beta }F({{\mathcal {R}}} )^{,\alpha }F({{\mathcal {R}}} )_{,\gamma }\nonumber \\&\quad -g_{\beta \gamma }F({{\mathcal {R}}} )^{,\alpha }F({{\mathcal {R}}} )_{,\delta })+\delta ^{\alpha }_{\gamma }\nabla _{\delta }F({{\mathcal {R}}} )_{,\beta }-\delta ^{\alpha }_{\delta }\nabla _{\gamma }F({{\mathcal {R}}} )_{,\beta }\nonumber \\&\quad +g_{\delta \beta }\nabla _{\gamma }F({{\mathcal {R}}} )^{,\alpha }-g_{\beta \gamma }\nabla _{\delta }F({{\mathcal {R}}} )^{,\alpha }\bigg ]V^{\beta }\eta ^{\gamma }V^{\delta }. \end{aligned}$$
(15)

The four-velocity is \(u^{\alpha }=\left( 1,0,0,0\right) \); therefore, from the orthogonality conditions, we have \(E=-V_{\alpha }u^{\alpha }=-V_{0}\), \(\eta _{\alpha }u^{\alpha }=\eta _{0}u^{0}=0 \), which means that the deviation vector just has non-vanishing spatial components \(\eta ^{0}=0\). Moreover, we have \(\eta _{\alpha }V^{\alpha }=\eta _{i}V^{i}\). Note that the Ricci scalar R in the FLRW space-time is only a function of time; thus, by taking Eqs. (11), (12), and (13), we can write the following terms as

$$\begin{aligned}{} & {} \delta ^{\alpha }_{\gamma }T_{\delta \beta } V^{\beta }\eta ^{\gamma }V^{\delta }=\left( \epsilon P+\left( \rho +P\right) E^{2}\right) \eta ^{\alpha }, \end{aligned}$$
(16)
$$\begin{aligned}{} & {} \delta ^{\alpha }_{\gamma } F({{\mathcal {R}}})_{,\delta } F({{\mathcal {R}}})_{,\beta } V^{\beta } \eta ^{\gamma } V^{\delta }=f^{''2}\dot{{{\mathcal {R}}}}^{2}E^{2}\eta ^{\alpha }, \end{aligned}$$
(17)
$$\begin{aligned}{} & {} g_{\beta \delta }\nabla _{\gamma }F({{\mathcal {R}}})^{,\alpha } V^{\beta } \eta ^{\gamma } V^{\delta }=-\epsilon H F^{'} \dot{{{\mathcal {R}}}} \eta ^{\alpha }, \end{aligned}$$
(18)
$$\begin{aligned}{} & {} \delta ^{\alpha }_{\gamma } \nabla _{\delta } F({{\mathcal {R}}})^{,\beta } V^{\beta } \eta ^{\gamma } V^{\delta }=\left( E^{2}\left( F^{'}\ddot{{{\mathcal {R}}}}+ F^{''}\dot{{\mathcal {R}}}^{2}\right. \right. \nonumber \\{} & {} \qquad \left. \left. -H F^{'}\dot{{{\mathcal {R}}}}\right) - \epsilon H F^{'} \dot{{\mathcal {R}}}\right) \eta ^{\alpha }, \end{aligned}$$
(19)
$$\begin{aligned}{} & {} \square F({\mathcal {R}})=-F^{'} \ddot{{{\mathcal {R}}}} - F^{''} \dot{{{\mathcal {R}}}}^{2}-3H F^{'} \ddot{{{\mathcal {R}}}}, \end{aligned}$$
(20)

where \(\square = \nabla _{\sigma } \nabla ^{\sigma }\), and \(F({\mathcal {R}})=df({{\mathcal {R}}})/d{{\mathcal {R}}}\). Consequently, the right-hand side of the GDE reduces to

$$\begin{aligned}&R^{\alpha }_{\,\beta \gamma \delta }V^{\beta }\eta ^{\gamma }V^{\delta }=\frac{\eta ^{\alpha }}{2(1+F({\mathcal {R}}))}\bigg [\epsilon \bigg (\frac{2-F({{\mathcal {R}}})}{3}R+2\kappa ^{2}P\nonumber \\&\qquad + f({{\mathcal {R}}})- F^{\prime \prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}^{2}- F^{\prime }({{\mathcal {R}}})\ddot{{{\mathcal {R}}}}-5H F^{\prime }({\mathcal {R}})\dot{{{\mathcal {R}}} }\bigg ) \nonumber \\&\qquad +E^{2}\bigg (\kappa ^{2}(\rho +P)-\frac{3}{2F({\mathcal {R}})}F^{\prime }({{\mathcal {R}}})^{2}\dot{{{\mathcal {R}}}}^{2}+F^{\prime }({\mathcal {R}})\ddot{{{\mathcal {R}}}}\nonumber \\&\qquad +F^{\prime \prime }({{\mathcal {R}}})\dot{{\mathcal {R}}}^{2}-HF^{\prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}\bigg )\bigg ]. \end{aligned}$$
(21)

We define the following terms

$$\begin{aligned} \rho _{\text {eff}}= & {} \kappa ^{2}\rho +\frac{F({\mathcal {R}})-2}{6}R-\frac{f({{\mathcal {R}}})}{2}-\frac{3F^{'}({\mathcal {R}})^{2}\dot{{{\mathcal {R}}}}^{2}}{2F({{\mathcal {R}}})}\nonumber \\{} & {} + \frac{3F^{''}({{\mathcal {R}}})\dot{{\mathcal {R}}}^{2}}{2}+\frac{3}{2}HF^{'}({{\mathcal {R}}})\dot{{\mathcal {R}}}+\frac{3}{2}F^{'}({{\mathcal {R}}})\ddot{{{\mathcal {R}}}} \end{aligned}$$
(22)

and

$$\begin{aligned} P_{\text {eff}}= & {} \kappa ^{2}P+\frac{-F({{\mathcal {R}}})+2}{6}R+\frac{f({\mathcal {R}})}{2}- \frac{F^{''}({{\mathcal {R}}})\dot{{\mathcal {R}}}^{2}}{2}\nonumber \\{} & {} -\frac{5}{2}HF^{'}({{\mathcal {R}}})\dot{{{\mathcal {R}}}}-\frac{1}{2}F^{'}({\mathcal {R}})\ddot{{{\mathcal {R}}}}. \end{aligned}$$
(23)

As a result, Eq. (21) is reformed into a more compact structure

$$\begin{aligned} R^{\alpha }_{\,\beta \gamma \delta }V^{\beta }\eta ^{\gamma }V^{\delta }=\frac{\eta ^{\alpha }}{2(1+F({\mathcal {R}}))}\left[ \epsilon \left( 2P_{\text {eff}}\right) +E^{2}\left( \rho _{\text {eff}}+P_{\text {eff}}\right) \right] , \end{aligned}$$
(24)

which is the modified Pirani equation. Eventually, the generalized GDE in \(f(R, {{\mathcal {R}}})\) gravity is

$$\begin{aligned} \frac{D^{2}\eta ^{\alpha }}{D\nu ^{2}}=-\frac{\eta ^{\alpha }}{2(1+F({\mathcal {R}}))}\left[ \epsilon \left( 2P_{\text {eff}}\right) +E^{2}\left( \rho _{\text {eff}}+P_{\text {eff}}\right) \right] . \end{aligned}$$
(25)

In addition, in a particular GR case, i.e., with \(f\left( R, {{\mathcal {R}}}\right) =R\), we can obtain the original Pirani equation and the GDE in GR as

$$\begin{aligned} \frac{D^{2}\eta ^{\alpha }}{D\nu ^{2}}=-R^{\alpha }_{\,\beta \gamma \delta }V^{\beta }\eta ^{\gamma }V^{\delta }=-\left[ \frac{\kappa ^{2}\rho }{3}\epsilon +\frac{\kappa ^{2}}{2}\left( \rho +P\right) E^{2}\right] \eta ^{\alpha }. \end{aligned}$$
(26)

This reduction confirms the correctness of our calculations. In the next section, we study the details of the GDE for fundamental observers.

2.2 GDE for fundamental observers

Here, we consider \( V^{\alpha } \) to be \( u^{\alpha } \) as the four-velocity, and the affine parameter \( \nu \) is interpreted as the time coordinate \( \nu =t \) which satisfies

$$\begin{aligned} u^{\alpha } u_{\alpha }=\epsilon \quad \frac{Du^{\alpha }}{Dt}=u^{\beta }\nabla _{\beta }u^{\alpha }=0. \end{aligned}$$
(27)

In this case we get \(\epsilon =-1\). In addition, we set the vector field normalization with \(E=1\). As a result, the generalized Pirani equation reduces to

$$\begin{aligned} R^{\alpha }_{\,\beta \gamma \delta }u^{\beta }\eta ^{\gamma }u^{\delta }=&\, \frac{\eta ^{\alpha }}{2(1+F({\mathcal {R}}))}\bigg [\kappa ^{2}(\rho - P)+\frac{F({{\mathcal {R}}})-2}{3}R\nonumber \\&-f({{\mathcal {R}}})+2F^{\prime \prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}^{2}\nonumber \\&+2F^{\prime }({{\mathcal {R}}})\ddot{{{\mathcal {R}}}}+4HF^{\prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}-\frac{3}{2F({{\mathcal {R}}})}F^{'}({\mathcal {R}})^{2}\dot{{{\mathcal {R}}}}^{2}\bigg ]\nonumber \\ =&\, \frac{\eta ^{\alpha }}{2(1+F({\mathcal {R}}))}\left[ \rho _{\text {eff}}-P_{\text {eff}}\right] . \end{aligned}$$
(28)

By putting \( \eta _{\alpha }=le_{\alpha } \) (where \(e_{\alpha }\) is propagated parallelly along the cosmic time), we find

$$\begin{aligned} \frac{De^{\alpha }}{Dt}=0 \end{aligned}$$
(29)

and

$$\begin{aligned} \frac{D^{2}\eta ^{\alpha }}{Dt^{2}}=\frac{D^{2}l}{Dt^{2}}e^{\alpha }. \end{aligned}$$
(30)

Using Eqs. (30), (9), and (28), we will have

$$\begin{aligned} \frac{D^{2}l^{\alpha }}{Dt^{2}}=&\frac{-1}{1+F({\mathcal {R}})}\bigg [\frac{F({{\mathcal {R}}} )R}{6}-\frac{R}{3}- \frac{f({\mathcal {R}})}{2}\nonumber \\&+F^{\prime \prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}^{2} +F^{\prime }({\mathcal {R}})\ddot{{{\mathcal {R}}} }+2HF^{\prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}\nonumber \\&+\frac{\kappa ^{2}(\rho - P)}{2}-\frac{3}{4F({\mathcal {R}})}F^{\prime }({{\mathcal {R}}})^{2}\dot{{{\mathcal {R}}}}^{2}\bigg ]l\nonumber \\ =&\frac{-1}{1+F({{\mathcal {R}}})}\left[ \rho _{\text {eff}}-P_{\text {eff}}\right] . \end{aligned}$$
(31)

If we put \( l=a(t) \), we have

$$\begin{aligned} \frac{\ddot{a}}{a}=&\, \frac{1}{1+F({{\mathcal {R}}})}\bigg [\frac{-F({{\mathcal {R}}} )R}{6}+\frac{R}{3}+\frac{f({{\mathcal {R}}})}{2}\nonumber \\&-F^{\prime \prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}} }^{2} -F^{\prime }({\mathcal {R}})\ddot{{{\mathcal {R}}}}-2HF^{\prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}\nonumber \\&+\frac{\kappa ^{2}\left( -\rho + P\right) }{2} +\frac{3}{4F({\mathcal {R}})}F^{\prime }({{\mathcal {R}}})^{2}\dot{{\mathcal {R}}}^{2}\bigg ]\nonumber \\ =&\, \frac{1}{1+F({{\mathcal {R}}})}\left[ P_{\text {eff}}-\rho _{\text {eff}}\right] . \end{aligned}$$
(32)

In order to check the correctness of our result, we should compare the above relation with the result found by building the standard modified Friedmann equations in [31]. Thus, we insert Eq. (8) into Eq. (32), and instead of the second term of the above Eq. (32), we will have

$$\begin{aligned} \frac{\ddot{a}}{a}=&\, \frac{1}{1+F({{\mathcal {R}}} )}\bigg [\frac{-F({\mathcal {R}})R}{6}+\frac{F({{\mathcal {R}}}){{\mathcal {R}}}}{3}-\frac{f({\mathcal {R}})}{6}\nonumber \\&-F^{\prime \prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}^{2} -F^{\prime }({{\mathcal {R}}})\ddot{{{\mathcal {R}}}}-2HF^{\prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}\nonumber \\&-\kappa ^{2}(\frac{P}{2}+\frac{\rho }{6})+\frac{3}{4F({\mathcal {R}})}F^{\prime }({{\mathcal {R}}})^{2}\dot{{{\mathcal {R}}} }^{2}\bigg ]. \end{aligned}$$
(33)

Using (7) to omit \({{\mathcal {R}}}\), and with some simplification, we can find the following expression:

$$\begin{aligned} \frac{\ddot{a}}{a}= & {} \frac{1}{1+F({{\mathcal {R}}})}\bigg [\frac{F({\mathcal {R}})R}{6}-\frac{f({{\mathcal {R}}})}{6}+HF^{\prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}\nonumber \\{} & {} -\kappa ^{2}\left( \frac{P}{2}+\frac{\rho }{6}\right) +\frac{F^{\prime }({\mathcal {R}})^{2}\dot{{{\mathcal {R}}}}^{2}}{4F({{\mathcal {R}}})}\bigg ], \end{aligned}$$
(34)

which is consistent with the final result for the Raychaudhuri equation in \(f(R, {{\mathcal {R}}})\) gravity by means of the standard form of the modified Friedmann equations in [31].

2.3 GDE for null vector fields

In this subsection, we calculate the GDE for past-directed null vector fields where we have \( V^{\alpha }=k^{\alpha }, k_{\alpha }k^{\alpha }=0 \), so Eq. (24) reduces to

$$\begin{aligned} R^{\alpha }_{\,\beta \gamma \delta }\kappa ^{\beta }\eta ^{\gamma }k^{\delta }=&\,\frac{\eta ^{\alpha }E^{2}}{2\left( 1+F({\mathcal {R}})\right) }\bigg [\kappa ^{2}(\rho + P)\nonumber \\&-\frac{3}{2F({{\mathcal {R}}})}F^{\prime }({{\mathcal {R}}})^{2}\dot{{\mathcal {R}}}^{2}+F^{\prime }({{\mathcal {R}}})\ddot{{{\mathcal {R}}}} \nonumber \\&+F^{\prime \prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}^{2}-HF^{\prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}\bigg ]. \end{aligned}$$
(35)

If we consider \(\eta ^{\alpha } \) as \( \eta ^{\alpha }=\eta e^{\alpha }, e_{\alpha }e^{\alpha }=1,e_{\alpha } u^{\alpha }=e_{\alpha } k^{\alpha } =0\), and \( \frac{D e^{\alpha }}{D\nu }=0 \), Eq. (25) reduces to

$$\begin{aligned} \frac{D^{2}\eta ^{\alpha }}{D\nu ^{2}}=&\, -\frac{\eta ^{\alpha }E^{2}}{2\left( 1+F({\mathcal {R}})\right) }\bigg [\kappa ^{2}\left( \rho + P\right) -\frac{3}{2F({\mathcal {R}})}F^{\prime }({{\mathcal {R}}})^{2}\dot{{{\mathcal {R}}}}^{2}\nonumber \\&+F^{\prime }({{\mathcal {R}}})\ddot{{{\mathcal {R}}}} +F^{\prime \prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}^{2}-HF^{\prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}\bigg ]. \end{aligned}$$
(36)

In the case of GR discussed in [32], all null geodesics experience convergence, provided that \( \kappa (\rho +P)>0 \), and thus the focusing condition for \( f(R,{{\mathcal {R}}}) \) gravity, is

$$\begin{aligned}&\frac{1}{\left( 1+F({{\mathcal {R}}})\right) }\bigg [\kappa ^{2}(\rho + P)-\frac{3}{2F({{\mathcal {R}}})}F^{\prime }({{\mathcal {R}}})^{2}\dot{{\mathcal {R}}}^{2}\nonumber \\&\quad +F^{\prime }({{\mathcal {R}}})\ddot{{{\mathcal {R}}}}+F^{\prime \prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}^{2}-HF^{\prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}\bigg ] >0. \end{aligned}$$
(37)

In order to compare with cosmological observations, we write Eq. (36) as a function of the redshift parameter z. Differential operators can be used as follows:

$$\begin{aligned} \frac{\textrm{d}}{\textrm{d}\nu }&=\frac{\textrm{d}z}{\textrm{d}\nu }\frac{\textrm{d}}{\textrm{d}z}, \end{aligned}$$
(38)
$$\begin{aligned} \frac{\textrm{d}^{2}}{\textrm{d}\nu ^{2}}&=\frac{\textrm{d}z}{\textrm{d}\nu }\frac{d}{dz}\left( \frac{\textrm{d}}{\textrm{d}\nu }\right) \nonumber \\&=\left( \frac{\textrm{d}\nu }{\textrm{d}z}\right) ^{-2}\bigg [-\left( \frac{\textrm{d}\nu }{\textrm{d}z}\right) ^{-1}\frac{\textrm{d}^{2}\nu }{\textrm{d}z^{2}}\frac{\textrm{d}}{\textrm{d}z}+\frac{\textrm{d}^{2}}{\textrm{d}z^{2}}\bigg ]. \end{aligned}$$
(39)

For null geodesics, we have

$$\begin{aligned} (1+z)=\frac{a_{0}}{a}=\frac{E_{0}}{E}\longrightarrow \frac{\textrm{d}z}{1+z}=-\frac{\textrm{d}a}{a}, \end{aligned}$$
(40)

and we know that \( \frac{\textrm{d}t}{\textrm{d}\nu }=E_{0}(1+z) \), so we can get

$$\begin{aligned} \frac{\textrm{d}\nu }{\textrm{d}z}=\frac{1}{E_{0}H(1+z)^{2}} \end{aligned}$$
(41)

and

$$\begin{aligned} \frac{\textrm{d}^{2}\nu }{\textrm{d}z^{2}}=-\frac{1}{E_{0}H(1+z)^{3}}\bigg [\frac{1}{H}(1+z)\frac{\textrm{d}H}{\textrm{d}z}+2\bigg ]. \end{aligned}$$
(42)

First, we get \( \frac{\textrm{d}H}{\textrm{d}z} \) and then put it into Eq. 42)

$$\begin{aligned} \frac{\textrm{d}H}{\textrm{d}z}=- \frac{1}{H(1+z)}\frac{\textrm{d}H}{\textrm{d}t}. \end{aligned}$$
(43)

Defining the Hubble parameter \(H=\frac{\dot{a}}{a}\)

$$\begin{aligned} \dot{H}= \frac{\textrm{d}H}{\textrm{d}t}=\frac{\textrm{d}}{\textrm{d}t}\frac{\dot{a}}{a}=\frac{\ddot{a}}{a}-H^{2}. \end{aligned}$$
(44)

From Eqs. (34) and (44), we can write

$$\begin{aligned} \dot{H}=&\frac{1}{1+F({{\mathcal {R}}})}\bigg [\frac{F({\mathcal {R}})R}{6}+\frac{F^{\prime }({{\mathcal {R}}})^{2}\dot{{{\mathcal {R}}}}^{2}}{4F({\mathcal {R}})}\nonumber \\&+HF^{\prime }({{\mathcal {R}}})\dot{{\mathcal {R}}}-\frac{P}{2}-\frac{\rho }{6}-\frac{f({{\mathcal {R}}})}{6}\bigg ]-H^{2}. \end{aligned}$$
(45)

Finally, Eq. (42) is written as

$$\begin{aligned} \frac{\textrm{d}^{2}\nu }{\textrm{d}z^{2}}=&-\frac{1}{E_{0}H(1+z)^{3}}\bigg [\frac{-1}{H^{2}(1+F({\mathcal {R}}))}\bigg (\frac{F({{\mathcal {R}}})R}{6}\nonumber \\&+\frac{F^{\prime }({{\mathcal {R}}})^{2}\dot{{{\mathcal {R}}}}^{2}}{4F({\mathcal {R}})}+HF^{\prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}-\frac{P}{2}\nonumber \\&-\frac{\rho }{6}-\frac{f({{\mathcal {R}}})}{6}\bigg )+3\bigg ]. \end{aligned}$$
(46)

Therefore,

$$\begin{aligned} \frac{\textrm{d}^{2}\eta }{\textrm{d}\nu ^{2}}&=(EH(1+z))^{2}\bigg [\frac{-1}{1+z}\bigg (\frac{1}{H^{2}(1+F({\mathcal {R}}))}\bigg (\frac{F({{\mathcal {R}}})R}{6}\nonumber \\&\quad +\frac{F^{\prime }({{\mathcal {R}}})^{2}\dot{{{\mathcal {R}}}}^{2}}{4F({{\mathcal {R}}})} +HF^{\prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}\nonumber \\&\quad -\frac{P}{2}-\frac{\rho }{6}-\frac{f({\mathcal {R}})}{6}\bigg )+3\bigg )\frac{\textrm{d}\eta }{\textrm{d}z}+\frac{\textrm{d}^{2}\eta }{\textrm{d}z^{2}}\bigg ]. \end{aligned}$$
(47)

According to the calculations, Eq. (36) is written as

$$\begin{aligned}&\frac{\textrm{d}^{2}\eta }{\textrm{d}z^{2}}+\frac{3}{1+z}\bigg [1+\frac{1}{3H^{2}(1+F({\mathcal {R}}))}\bigg (-\frac{F({{\mathcal {R}}})R}{6}\nonumber \\&\quad -\frac{F^{\prime }({{\mathcal {R}}} )^{2}\dot{{{\mathcal {R}}}}^{2}}{4F({{\mathcal {R}}})} -HF^{\prime }({\mathcal {R}})\dot{{\mathcal {R}}}+\kappa ^{2}\left( \frac{\rho }{6}+\frac{P}{2}\right) +\frac{f({\mathcal {R}})}{6}\bigg )\bigg ]\frac{\textrm{d}\eta }{\textrm{d}z}\nonumber \\&\quad +\frac{\eta }{2H^{2}(1+z)^{2}(1+F({\mathcal {R}}))}\bigg [\kappa ^{2}(\rho +P)-\frac{3F^{\prime }({\mathcal {R}})^{2}\dot{{{\mathcal {R}}}}^{2}}{2F({{\mathcal {R}}})}\nonumber \\&\quad + F^{\prime \prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}^{2}+F^{\prime }({{\mathcal {R}}})\ddot{{\mathcal {R}}}-HF^{\prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}\bigg ]=0. \end{aligned}$$
(48)

The expression of energy density and pressure can be considered from [3, 4] as

$$\begin{aligned} \kappa ^{2}\rho&=3H_{0}^{2}\left( \Omega _{m_{0}}(1+z)^{3}+\Omega _{r_{0}}(1+z)^{4}\right) , \nonumber \\ \kappa ^{2}P&=H_{0}^{2}\Omega _{r_{0}}(1+z)^{4}, \end{aligned}$$
(49)

and

$$\begin{aligned} \Omega _{K_{0}}=-\frac{K}{H_{0}^{2}a_{0}^{2}}, \end{aligned}$$
(50)

where \(\Omega _{m_{0}}\) and \(\Omega _{r_{0}}\) represent the dimensionless cosmological density parameters, and the labels m and r refer to the matter and radiation, respectively. By using the above equations, the null GDE Eq. (48) reads

$$\begin{aligned}&\frac{\textrm{d}^{2}\eta }{\textrm{d}z^{2}}+{{\mathcal {P}}}(H,R,z,{\mathcal {R}})\frac{\textrm{d}\eta }{\textrm{d}z}+{{\mathcal {Q}}}(H,R,z,{{\mathcal {R}}})\eta =0, \end{aligned}$$
(51)

with

$$\begin{aligned}&{{\mathcal {P}}}(H,R,z,{\mathcal {R}})=\frac{1}{1+z}\nonumber \\&\quad \bigg [\frac{4\Omega _{r_{0}}(1+z)^{4}+\frac{7}{2}\Omega _{m_{0}}(1+z)^{3}+3(1+F({\mathcal {R}}))\Omega _{K_{0}}(1+z)^{2}}{(1+F({\mathcal {R}}))\Omega _{K_{0}}(1+z)^{2}+\Omega _{r_{0}}(1+z)^{4}+\Omega _{m_{0}}(1+z)^{3}+\Omega _{DE}}\bigg ] \nonumber \\&\quad +\frac{1}{1+z}\nonumber \\&\quad \bigg [\frac{2\Omega _{DE}-\frac{3F^{\prime }({\mathcal {R}})^{2}\dot{{{\mathcal {R}}}}^{2}}{4F({{\mathcal {R}}})H_{0}^{2}} -\frac{HF^{\prime }({{\mathcal {R}}})\dot{{\mathcal {R}}}}{2H_{0}^{2}}+\frac{F^{\prime \prime }({{\mathcal {R}}})\dot{{\mathcal {R}}}^{2}}{2H_{0}^{2}}}{(1+F({\mathcal {R}}))\Omega _{K_{0}}(1+z)^{2}+\Omega _{r_{0}}(1+z)^{4}+\Omega _{m_{0}}(1+z)^{3}+\Omega _{DE}}\bigg ], \nonumber \\ \end{aligned}$$
(52)
$$\begin{aligned}&{{\mathcal {Q}}}(H,R,z,{\mathcal {R}})=\frac{1}{2(1+z)^{2}}\nonumber \\&\quad \bigg [\frac{4\Omega _{r_{0}}(1+z)^{4}+3\Omega _{m_{0}}(1+z)^{3}}{(1+F({\mathcal {R}}))\Omega _{K_{0}}(1+z)^{2}+\Omega _{r_{0}}(1+z)^{4}+\Omega _{m_{0}}(1+z)^{3}+\Omega _{DE}}\bigg ]\nonumber \\&\quad +\frac{1}{2(1+z)^{2}}\nonumber \\&\quad \bigg [\frac{2\Omega _{DE}-\frac{HF^{\prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}}{H_{0}^{2}}+\frac{f({\mathcal {R}})}{3H_{0}^{2}}-\frac{F({\mathcal {R}})R}{3H_{0}^{2}}+\frac{F^{\prime }({{\mathcal {R}}})^{2}\dot{{\mathcal {R}}}^{2}}{F({{\mathcal {R}}})H_{0}^{2}}}{(1+F({\mathcal {R}}))\Omega _{K_{0}}(1+z)^{2}+\Omega _{r_{0}}(1+z)^{4}+\Omega _{m_{0}}(1+z)^{3}+\Omega _{DE}}\bigg ], \nonumber \\ \end{aligned}$$
(53)

in which

$$\begin{aligned} \Omega _{DE}=&\frac{1}{H_{0}^{2}}\bigg [\frac{F({{\mathcal {R}}} )R}{6}-\frac{f({{\mathcal {R}}})}{6}-\frac{F^{\prime }({\mathcal {R}})^{2}\dot{{{\mathcal {R}}}}^{2}}{4F({\mathcal {R}})}\nonumber \\&+\frac{1}{2}\bigg (F^{\prime \prime }({{\mathcal {R}}})\dot{{\mathcal {R}}}^{2}+F^{\prime }({{\mathcal {R}}})\ddot{{{\mathcal {R}}}}+HF^{\prime }({\mathcal {R}})\dot{{{\mathcal {R}}}} \bigg )\bigg ]. \end{aligned}$$
(54)

From [31], we redefine

$$\begin{aligned} {{\mathcal {H}}}=H+\frac{F^{\prime }({{\mathcal {R}}})\dot{{{\mathcal {R}}}}}{2F({{\mathcal {R}}})}. \end{aligned}$$
(55)

As a result, the modified first Friedmann equation is obtained as

$$\begin{aligned} H^{2}=-\frac{K}{a^{2}}(1+F({{\mathcal {R}}}))-F({{\mathcal {R}}}){\mathcal {H}}^{2}+\frac{1}{6}(F({{\mathcal {R}}}){{\mathcal {R}}} -f({\mathcal {R}}))+\frac{\kappa ^{2}\rho }{3}. \end{aligned}$$
(56)

As before, contracting \({{\mathcal {R}}}_{\mu \nu }\) with \( g^{\mu \nu } \) in (7) leads to the following relation [19]:

$$\begin{aligned} {{\mathcal {R}}}= & {} R-\frac{3F^{\prime }({{\mathcal {R}}})^{2}\dot{{\mathcal {R}}}^{2}}{2F({{\mathcal {R}}})^{2}}+\frac{3}{F({\mathcal {R}})}\nonumber \\{} & {} \bigg (F^{\prime \prime }({{\mathcal {R}}})\dot{{\mathcal {R}}}^{2}+F^{\prime }({{\mathcal {R}}})\ddot{{{\mathcal {R}}}}+3HF^{\prime }({\mathcal {R}})\dot{{{\mathcal {R}}}}\bigg ). \end{aligned}$$
(57)

From (54)–(57), it is possible to obtain the first modified Friedmann equation in \(R+f\left( {{\mathcal {R}}}\right) \) as

$$\begin{aligned} H^{2}=&\frac{H_{0}^{2}}{1+F({\mathcal {R}})}\bigg [\Omega _{K_{0}}(1+z)^{2}\left( 1+F({\mathcal {R}})\right) \nonumber \\&+\Omega _{m_{0}}(1+z)^{3}+\Omega _{r_{0}}(1+z)^{4}+\Omega _{DE}\bigg ]. \end{aligned}$$
(58)

As a particular model, let us consider the case \( f({{\mathcal {R}}})={\mathcal {R}}-2\Lambda \), where \(\Lambda \) is a cosmological constant parameter whereby \(\Omega _{\Lambda }=\frac{\Lambda }{3H_{0}^{2}}\). Therefore, we have

$$\begin{aligned} F({{\mathcal {R}}})=1 \quad F^{\prime }({{\mathcal {R}}})=0, \end{aligned}$$
(59)

so

$$\begin{aligned} \Omega _{DE}=\frac{1}{6H_{0}^{2}}\left[ R-\left( {\mathcal {R}}-2\Lambda \right) \right] . \end{aligned}$$
(60)

Equations (52) and (53) are reduced as follows:

$$\begin{aligned}&{{\mathcal {P}}}(H,R,z,{\mathcal {R}})=\frac{1}{1+z} \bigg [\frac{4\Omega _{r_{0}}(1+z)^{4}+\frac{7}{2} \Omega _{m_{0}}(1+z)^{3}+6\Omega _{K_{0}}(1+z)^{2}+ 2\Omega _{DE}}{2\Omega _{K_{0}}(1+z)^{2}+\Omega _{r_{0}}(1+z)^{4}+\Omega _{m_{0}}(1+z)^{3}+\Omega _{DE}}\bigg ], \end{aligned}$$
(61)
$$\begin{aligned}&{{\mathcal {Q}}}(H,R,z,{\mathcal {R}})=\frac{1}{\left( 1+z\right) ^{2}} \left[ \frac{2\Omega _{r_{0}}(1+z)^{2}+\frac{3}{2}\Omega _{m_{0}}(1+z)^{3} +\frac{\Omega _{DE}}{2}-\frac{\Omega _{\Lambda }}{2}}{\Omega _{r_{0}}(1+z)^{2} \left( z^{2}+2z-1\right) +\Omega _{m_{0}}(1+z)^{2} (z-1)+2(1+z)^{2}+\Omega _{DE}}\right] . \end{aligned}$$
(62)

For the particular choices \( \Omega _{\Lambda }=0 \) and \( \Omega _{K_{0}}=1- \Omega _{m_{0}}- \Omega _{r_{0}} \), we can find the modified Mattig relation. Thus,

$$\begin{aligned}&{\tilde{\mathcal {P}}}(z)=\frac{1}{1+z}\bigg [\frac{4\Omega _{r_{0}}(1+z)^{2}\left( z^{2}+2z-\frac{1}{2}\right) +\frac{7z-5}{2}\Omega _{m_{0}}(1+z)^{2}+ 6(1+z)^{2}+2\Omega _{DE}}{2(1+z)^{2}+ \Omega _{r_{0}}(1+z)^{2}\left( z^{2}+2z-1\right) +\Omega _{m_{0}}(1+z)^{2}(z-1)+\Omega _{DE}}\bigg ], \end{aligned}$$
(63)
$$\begin{aligned}&{\tilde{\mathcal {Q}}}(z)=\frac{4\Omega _{r_{0}}(1+z)^{2}+3\Omega _{m_{0}}(1+z)^{3}+\Omega _{DE}}{2(1+z)^{2} \bigg [2(1+z)^{2}+ \Omega _{r_{0}}(1+z)^{2}(z^{2}+2z-1)+\Omega _{m_{0}}\left( 1+z\right) ^{2} \left( z-1\right) +\Omega _{DE}\bigg ]}. \end{aligned}$$
(64)

Finally, we find the modified Mattig relation in \(f\left( R,{{\mathcal {R}}}\right) \) as follows:

$$\begin{aligned} \frac{\textrm{d}^{2}\eta }{\textrm{d}z^{2}}+{\tilde{\mathcal {P}}}(z) \frac{\textrm{d}\eta }{\textrm{d}z} +{\tilde{\mathcal {Q}}}(z)\eta =0. \end{aligned}$$
(65)

It is worth noting that for a spherically symmetric space-time similar to the FLRW universe, the deviation vector magnitude \(\eta \) is proportional to the proper area \(\textrm{d}A\) of a source with a redshift z as \(\textrm{d}\eta \propto \sqrt{\textrm{d}A}\), which leads to the definition of the observer area–distance \( r_0(z)\) with the expression

$$\begin{aligned} r_0(z)=\sqrt{\left| \frac{\textrm{d}A_0(z)}{\textrm{d}\Omega }\right| }=\left| \frac{\eta (z')|_{z'=z}}{\textrm{d}\eta (z')/\textrm{d}l|_{z'=0}}\right| , \end{aligned}$$
(66)

where \(A_0\) represents the area of the object, and \(\Omega _0\) is the solid angle [32, 34]. Thus, by implying the relation \(|\textrm{d}/\textrm{d}l|=E^{-1}_0 (1+ z)^{-1}\textrm{d}/\textrm{d}\nu =H (1 + z)\textrm{d}/\textrm{d}z\), where \(\textrm{d}l= a(t)\textrm{d}r\), while assuming the deviation vector to be zero at \(z=0\), Eq. (66) can be written as

$$\begin{aligned} r_0(z)=\left| \frac{\eta (z)}{H_0\textrm{d}\eta (z')/\textrm{d}z'|_{z'=0}}\right| . \end{aligned}$$
(67)

This equation denotes the observed area–distance \(r_0(z)\) as a function of z in units of the present-day Hubble radius.

Fig. 1
figure 1

The deviation vector \(\eta (z)\) evolution plot (left) and the observer area–distance \(r_{0}(z)\) evolution plot (right), with the numerical value consideration \(\Omega _{m_{0}}=0.3\), \(\Omega _{r_{0}}=0\), \(\Omega _{\Lambda }=0\), \(\Omega _{K_{0}}=0.001\), \(\Omega _{DE}=0.7\), and \(\frac{d\eta (z)}{dz}|_{z=0}=0.1\)

Full size image

2.4 Numerical solutions of the GDE for \(f(R,{{\mathcal {R}}})\) gravity

Clearly, to find the solutions of (51) (the null GDE), we are supposed to consider \(f\left( R,{{\mathcal {R}}}\right) \) forms. The standard form is the case \(f\left( R,{{\mathcal {R}}}\right) =R-2\Lambda \). In this case, we obtain the trivial solution, i.e., the \(\Lambda \)CDM model. Another functional form is the case \(f\left( R,{\mathcal {R}}\right) =f(R)\), which was considered in [21, 22]. In order to discover the new properties of \(f\left( R,{{\mathcal {R}}}\right) \) gravity, we should consider the cases with \({{\mathcal {R}}}\ne 0\).

In order to examine our study, we consider the numerical solutions of the GDE by taking the hybrid metric-Palatini function as \(f\left( R,{{\mathcal {R}}}\right) =R+{{\mathcal {R}}}\); thus, Eq. (51) is reduced to

$$\begin{aligned}&\frac{\textrm{d}^{2}\eta }{\textrm{d}z^{2}}+{{\mathcal {P}}}_{0}(H,R,z,{\mathcal {R}})\frac{\textrm{d}\eta }{\textrm{d}z}+{{\mathcal {Q}}}_{0}(H,R,z,{{\mathcal {R}}})\eta =0, \end{aligned}$$
(68)

where we define

$$\begin{aligned}&{{\mathcal {P}}}_{0}(H,R,z,{\mathcal {R}})=\frac{1}{1+z}\nonumber \\&\quad \bigg [\frac{4\Omega _{r_{0}}(1+z)^{4}+\frac{7}{2}\Omega _{m_{0}}(1+z)^{3}+ 6\Omega _{K_{0}}(1+z)^{2}+2\Omega _{DE}}{ 2\Omega _{K_{0}}(1+z)^{2}+\Omega _{r_{0}}(1+z)^{4}+\Omega _{m_{0}}(1+z)^{3}+\Omega _{DE}}\bigg ], \end{aligned}$$
(69)

and

$$\begin{aligned}&{{\mathcal {Q}}}_{0}\left( H,R,z,{\mathcal {R}}\right) =\frac{1}{\left( 1+z\right) ^{2}}\nonumber \\&\quad \bigg [\frac{2\Omega _{r_{0}}(1+z)^{4}+\frac{3}{2}\Omega _{m_{0}}(1+z)^{3}+2\Omega _{DE}-\Omega _{\Lambda }}{ 2\Omega _{K_{0}}(1+z)^{2}+\Omega _{r_{0}}(1+z)^{4}+\Omega _{m_{0}}(1+z)^{3}+\Omega _{DE}}\bigg ]. \end{aligned}$$
(70)

Now we can solve Eq. (68) numerically to find the evolution of \(\eta (z)\) and \(r_{0}(z)\) as functions of z, and the result is plotted in Fig. 1.

3 Scalar–tensor representation of \( f(R,{{\mathcal {R}}}) \) gravity

We start from the following action [33], in which we have a general function with two variables, metric and Palatini curvature scalars. In this section, we take a look at how this generalization can be considered dynamically equivalent to a gravitational theory with two scalar fields. According to [33], the general form of metric-Palatini action can be written as

$$\begin{aligned} S=\frac{1}{2\kappa ^{2}}\int \textrm{d}^4{x}\sqrt{-g}f\left( R,{\mathcal {R}}\right) +S_{M}. \end{aligned}$$
(71)

By varying the Eq. (71) with respect to the metric and connection, the field equations can be respectively written as follows:

$$\begin{aligned}&\frac{\partial f}{\partial R}R_{\mu \nu }-\frac{1}{2}g_{\mu \nu }f-\left( \nabla _{\mu }\nabla _{\nu }-g_{\mu \nu }\square \right) \frac{\partial f}{\partial R}+\frac{\partial f}{\partial {{\mathcal {R}}}}{\mathcal {R}}_{\mu \nu } \nonumber \\&\quad =\kappa ^{2} T_{\mu \nu }, \end{aligned}$$
(72)
$$\begin{aligned}&\hat{\nabla }_{\lambda }\bigg (\sqrt{-g}\frac{\partial f}{\partial {{\mathcal {R}}}}g^{\mu \nu }\bigg )=0, \end{aligned}$$
(73)

where the covariant derivative \(\hat{\nabla }\) is related to the metric \( h_{\mu \nu }=\frac{\partial f}{\partial {{\mathcal {R}}}}g_{\mu \nu } \). We can take the action with two scalar fields \(\alpha \) and \(\beta \) as

$$\begin{aligned} S= & {} \frac{1}{2\kappa ^{2}}\int d^4{x}\sqrt{-g}\bigg [f\left( \alpha ,\beta \right) +\frac{\partial f\left( \alpha ,\beta \right) }{\partial \alpha }\left( R-\alpha \right) \nonumber \\{} & {} +\frac{\partial f\left( \alpha ,\beta \right) }{\partial \beta }\left( {{\mathcal {R}}}-\beta \right) \bigg ]+S_{M}. \end{aligned}$$
(74)

Then, it is possible to obtain the field equations by variation with respect to \( \alpha \) and \( \beta \) from the equation of (74). We define the two new scalar fields as

$$\begin{aligned} \chi =\frac{\partial f(\alpha ,\beta )}{\partial \alpha } \hspace{4mm} \text {and} \hspace{4mm} \xi =-\frac{\partial f(\alpha ,\beta )}{\partial \beta }. \end{aligned}$$
(75)

According to [33], the Eq. (74) can be rewritten as

$$\begin{aligned} S=\frac{1}{2\kappa ^{2}}\int d^4{x}\sqrt{-g}\bigg [\left( \chi -\xi \right) R-\frac{3}{2\xi }(\partial \xi )^{2}-V(\chi ,\xi )\bigg ]+S_{M}, \end{aligned}$$
(76)

given that \( V(\chi ,\xi ) \) is considered as

$$\begin{aligned} V(\chi ,\xi )=-f\left( \alpha (\chi ),\beta (\xi )\right) +\chi \alpha (\chi )-\xi \beta (\xi ). \end{aligned}$$
(77)

We define a new scalar field as \( \phi =\chi -\xi \), and we can perform a conformal transformation to exchange from Jordan’s framework to Einstein’s as follows:

$$\begin{aligned} g_{\mu \nu }\rightarrow \tilde{g}_{\mu \nu }=\phi g_{\mu \nu }. \end{aligned}$$
(78)

Therefore, we have

$$\begin{aligned} S{} & {} =\frac{1}{2\kappa ^{2}}\int \textrm{d}^4{x}\sqrt{-\tilde{g}}\nonumber \\{} & {} \quad \bigg [\tilde{R}-\frac{3}{2\phi ^{2}}(\partial \phi )^{2}-\frac{3}{2\phi \xi }(\partial \xi )^{2}- \frac{W(\phi ,\xi )}{\phi ^{2}}\bigg ]+S_{M}. \nonumber \\ \end{aligned}$$
(79)

Now, we redefine two new scalar fields as

$$\begin{aligned} \tilde{\phi }=\sqrt{\frac{3}{2}}\frac{\ln \phi }{k}\hspace{4mm}\text {and}\hspace{4mm}\tilde{\xi }=\frac{2\sqrt{2}}{k}\sqrt{\xi }. \end{aligned}$$
(80)

Finally, we have

$$\begin{aligned}{} & {} S=\int \textrm{d}^4{x}\sqrt{-\tilde{g}}\bigg [\frac{1}{2\kappa ^{2}}\tilde{R}-\frac{1}{2}(\tilde{\nabla }\tilde{\phi })^{2}- \frac{1}{2}e^{\frac{-\sqrt{2}\kappa \tilde{\phi }}{\sqrt{3}}}(\tilde{\nabla }\tilde{\xi })^{2}\nonumber \\{} & {} \qquad -\tilde{W}(\tilde{\phi },\tilde{\xi })\bigg ] +\tilde{S}_{M}, \end{aligned}$$
(81)
$$\begin{aligned}{} & {} \tilde{S}_{M}=e^{\frac{-\sqrt{2}\kappa \tilde{\phi }}{3}}S_{M}, \end{aligned}$$
(82)

where

$$\begin{aligned} \tilde{W}(\tilde{\phi },\tilde{\xi })= \frac{1}{2\kappa ^{2}}e^{-\frac{\sqrt{2}\kappa \tilde{\phi }}{\sqrt{3}}}W\left( e^{\frac{\sqrt{2}\kappa \tilde{\phi }}{\sqrt{3}}},\kappa ^{2}\frac{\tilde{\xi }^{2}}{8}\right) , \end{aligned}$$
(83)

which is the new potential.

It should be noted that the Brans–Dicke context introduces the scalar field \(\phi \) as the Brans–Dicke field and \(\xi \) as the inflation. In fact, the Eq. (76) is usually more extended than the form we considered, which means that it includes a kinetic term for \(\phi \) or a more general coupling term between \(\phi \) and R [35,36,37,38,39,40,41]. For the sake of simplicity, from now on we will omit the tildes in Eq. (81). The field equations can be obtained by varying Eq. (81) with respect to \( g_{\mu \nu } \) as

$$\begin{aligned} G_{\mu \nu }=\kappa ^{2}\bigg (T^{(\phi )}_{\mu \nu }+e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( T^{(\xi )}_{\mu \nu }+T_{\mu \nu }\right) - g_{\mu \nu }W \bigg ), \end{aligned}$$
(84)

by considering

$$\begin{aligned} T^{(\phi )}_{\mu \nu }= & {} \triangledown _{\mu }\phi \triangledown _{\nu }\phi -\frac{1}{2}g_{\mu \nu }(\nabla \phi )^{2}, \end{aligned}$$
(85)
$$\begin{aligned} T^{(\xi )}_{\mu \nu }= & {} \triangledown _{\mu }\xi \triangledown _{\nu }\xi -\frac{1}{2}g_{\mu \nu }(\nabla \xi )^{2}. \end{aligned}$$
(86)

In order to extract the GDE, we first need to calculate \( R_{\mu \nu } \) from the modified Einstein equation (84):

$$\begin{aligned} R_{\mu \nu }=\kappa ^{2}T^{(\text {tot})}_{\mu \nu }+\frac{1}{2}g_{\mu \nu }\left( R-2\kappa ^{2}W\right) , \end{aligned}$$
(87)

and

$$\begin{aligned} R=\kappa ^{2}\left( 4W-T^{(\text {tot})}\right) , \end{aligned}$$
(88)

while

$$\begin{aligned} T^{(\text {tot})}_{\mu \nu }=T^{(\phi )}_{\mu \nu }+e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( T^{(\xi )}_{\mu \nu }+T_{\mu \nu }\right) . \end{aligned}$$
(89)

To complete our investigation to obtain the GDE in the context of the scalar–tensor theory of \(f\left( R,{{\mathcal {R}}}\right) \) gravity, we should carry out the extraction of the product of the Riemann tensor contraction with respect to the normalized tangent vectors and the geodesic deviation vector in this modified theory.

3.1 GDE in the context of scalar–tensor theory of \( f(R,{{\mathcal {R}}}) \)

To continue, we will investigate the GDE for the Eq. (81). First, we calculate the Riemann tensor through Eqs. (10), (87), and (88), written in the following form:

$$\begin{aligned} R^{\alpha }_{\,\beta \gamma \delta }= & {} \frac{\kappa ^{2}}{2}\bigg [\delta _{\gamma }^{\alpha } T_{\delta \beta }^{(\text {tot})}-\delta _{\delta }^{\alpha }T_{\gamma \beta }^{(\text {tot})}+ g_{\beta \delta }T_{\gamma }^{(\text {tot})\alpha }-g_{\beta \gamma } T_{\delta }^{(\text {tot})\alpha }\bigg ]\nonumber \\{} & {} +\frac{\kappa ^{2}}{3}\bigg [\left( \delta _{\gamma }^{\alpha }g_{\delta \beta }-\delta _{\delta }^{\alpha } g_{\gamma \beta }\right) \left( W-T^{(\text {tot})}\right) \bigg ]. \end{aligned}$$
(90)

Contracting the Riemann tensor with the \(V^{\beta }\eta ^{\gamma }V^{\delta }\) term, the following result is obtained.

$$\begin{aligned} R^{\alpha }_{\,\beta \gamma \delta }V^{\beta }\eta ^{\gamma }V^{\delta }&= \kappa ^{2}\bigg [E^{2}\left( \frac{\dot{\phi }^{2}}{2}+\frac{e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}}{2}\left( \dot{\xi }^{2}+\rho \ +P\right) \right) \nonumber \\&\quad +\epsilon \left( \frac{\dot{\phi }^{2}}{2}+e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( \frac{\dot{\xi }^{2}}{2}+p\right) +\frac{\left( W-T^{(\text {tot})}\right) }{3}\right) \bigg ]\eta ^{\alpha }. \end{aligned}$$
(91)

Again, it is useful to define \(\tilde{\rho }_{\text {eff}}\) and \(\tilde{P}_{\text {eff}}\) to reduce the GDE in a comprehensive form. As a result, we can write

$$\begin{aligned}&\tilde{\rho }_{\text {eff}}=\frac{\dot{\phi }^{2}}{2}+e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( \frac{\dot{\xi }^{2}}{2} +\rho \right) -\frac{\left( W-T^{(\text {tot})}\right) }{3}, \end{aligned}$$
(92)
$$\begin{aligned}&\tilde{P}_{\text {eff}}=\frac{\dot{\phi }^{2}}{2}+e^{-\frac{\sqrt{2}k{\phi }}{\sqrt{3}}}\left( \frac{\dot{\xi }^{2}}{2}+P\right) + \frac{\left( W-T^{(\text {tot})}\right) }{3}. \end{aligned}$$
(93)

Hence, the modified Pirani equation is obtained as

$$\begin{aligned} R^{\alpha }_{\,\beta \gamma \delta }V^{\beta }\eta ^{\gamma }V^{\delta }= \frac{\kappa ^{2}}{2}\left[ E^{2}\left( \tilde{\rho }_{\text {eff}}+\tilde{P}_{\text {eff}}\right) +\epsilon \left( 2\tilde{P}_{\text {eff}}\right) \right] \eta ^{\alpha }, \end{aligned}$$
(94)

by which the GDE becomes

$$\begin{aligned} \frac{D^{2}\eta }{D\nu ^{2}}=-\frac{\kappa ^{2}}{2}\left[ E^{2}\left( \tilde{\rho }_{\text {eff}}+\tilde{P}_{\text {eff}}\right) +\epsilon \left( 2\tilde{P}_{\text {eff}}\right) \right] \eta ^{\alpha }. \end{aligned}$$
(95)

As before, in the next step we are supposed to find the GDE for fundamental observers with the condition \(E^{2}=1\) and \(\epsilon =-1\).

3.2 GDE for fundamental observers

Now we are going to find the GDE for fundamental observers in the scalar–tensor theory of \(f\left( R,{{\mathcal {R}}}\right) \) gravity by exerting the condition \(E^{2}=1\) and \(\epsilon =-1\). Subsequently, in this case,

$$\begin{aligned} R^{\alpha }_{\,\beta \gamma \delta }u^{\beta }\eta ^{\gamma }u^{\delta } =\frac{\kappa ^{2}}{2}\left[ \tilde{\rho }_{\text {eff}}-\tilde{P}_{\text {eff}}\right] \eta ^{\alpha }. \end{aligned}$$
(96)

According to the method mentioned in Sect. 2, we will have

$$\begin{aligned} \frac{\ddot{a}}{a}=\frac{\kappa ^{2}}{2}\left[ \tilde{P}_{\text {eff}}-\tilde{\rho }_{\text {eff}}\right] . \end{aligned}$$
(97)

From Eq. (89) we can calculate \(T^{(\text {tot})} \) as

$$\begin{aligned} T^{(\text {tot})}=T^{(\phi )}+e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( T^{(\xi )}+T\right) . \end{aligned}$$
(98)

Thus, the above equation can be rewritten as

$$\begin{aligned} T^{(\text {tot})}=\dot{\phi }^{2}+e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( \dot{\xi }^{2}+3p-\rho \right) . \end{aligned}$$
(99)

Hence, the modified Raychaudhuri equation (97) can be written as

$$\begin{aligned} \frac{\ddot{a}}{a}=\kappa ^{2}\bigg [\frac{W}{3}-\frac{\dot{\phi }^{2}}{3}-e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( \frac{\dot{\xi }^{2}}{3}+ \frac{\rho }{6}+\frac{p}{2}\right) \bigg ]. \end{aligned}$$
(100)

Moreover, we are going to check the correctness of the result (100) using the first and second modified Friedmann equations described in [33]. The cosmological equations are in the following order:

$$\begin{aligned}&\frac{K}{a^{2}}+H^{2}=\frac{\kappa ^{2}}{3}\left( e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( \frac{\dot{\xi }^{2}}{2}-\rho \right) + \frac{\dot{\phi }^{2}}{2}+W\right) , \end{aligned}$$
(101)
$$\begin{aligned}&\frac{K}{a^{2}}+2\dot{H}+3H^{2}=-\kappa ^{2}\left( e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( \frac{\dot{\xi }^{2}}{2}+p\right) + \frac{\dot{\phi }^{2}}{2}-W\right) . \end{aligned}$$
(102)

On the other hand, we know that \(\frac{K}{a^{2}}=\frac{R}{6}-H^{2}-\frac{\ddot{a}}{a}\) and \(\frac{\ddot{a}}{a}=\dot{H}+H^{2}\). Besides the above two cosmological Eqs. (101) and (102), we can derive the relation in Eq. (100), which shows that the result (100) is correct.

Fig. 2
figure 2

The deviation vector \(\eta (z)\) evolution plot (left) and the observer area–distance \(r_{0}(z)\) evolution plot (right), with the numerical value consideration \(\Omega _{m_{0}}=0.3\), \(\Omega _{r_{0}}=0\), \(\Omega _{\Lambda }=0\), \(\Omega _{K_{0}}=0.001\), \(\Omega _{DE}=0.7\), and \(\frac{\textrm{d}\eta (z)}{\textrm{d}z}|_{z=0}=0.1\), assuming \(\kappa \phi _0=\kappa ^{2}W=1\)

Full size image

3.3 GDE for null vector fields

In the following, we calculate the GDE for null vector fields in the scalar–tensor theory of \(f\left( R,{{\mathcal {R}}}\right) \) gravity. As in Sect. 2, we have \( V^{\alpha }=k^{\alpha } \), which means that \(\epsilon =0\); therefore, Eq. (94) reduces to

$$\begin{aligned} R^{\alpha }_{\,\beta \gamma \delta }k^{\beta }\eta ^{\gamma }k^{\delta } =\frac{\kappa ^{2}E^{2}}{2}\left[ \tilde{\rho }_{\text {eff}}+\tilde{P}_{\text {eff}}\right] \eta ^{\alpha }, \end{aligned}$$
(103)

resulting in

$$\begin{aligned} \frac{\textrm{d}^{2}\eta }{d\nu ^{2}}=-\frac{\kappa ^{2}E^{2}}{2}\left[ \tilde{\rho }_{\text {eff}}+\tilde{P}_{\text {eff}}\right] \eta ^{\alpha }. \end{aligned}$$
(104)

In addition, in this case the past-directed null geodesics experience focusing if the null energy condition is satisfied as

$$\begin{aligned} \tilde{\rho }_{\text {eff}}+\tilde{P}_{\text {eff}}>0. \end{aligned}$$
(105)

Equivalently, we can say

$$\begin{aligned} \rho +P>-e^{\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\dot{\phi }^{2}+\dot{\xi }^{2}. \end{aligned}$$
(106)

Here, similar to the approach that it represented in Sect. 2, we obtain the GDE for null vector fields in the framework of the scalar–tensor representation of the model. To continue, we first obtain \(\dot{H}\) in this model as

$$\begin{aligned} \dot{H}=\kappa ^{2}\bigg [\frac{e^{\frac{-\sqrt{2}\kappa \phi }{\sqrt{3}}}\left( p-\rho \right) }{2} +\frac{W-T^{\text {tot}}}{3}\bigg ]-H^{2}. \end{aligned}$$
(107)

Then we have

$$\begin{aligned} \frac{\textrm{d}^{2}\nu }{\textrm{d}z^{2}}&=\frac{-3}{EH\left( 1+z\right) ^{2}}\nonumber \\&\quad \left[ 1+\frac{\kappa ^{2}}{3H^{2}}\left( \frac{e^{\frac{\sqrt{2}\kappa \phi }{\sqrt{3}}}\left( \rho -P\right) }{2}+\frac{T^{(\text {tot})}-W}{3}\right) \right] . \end{aligned}$$
(108)

Finally, by using (42) and (41), \(\frac{\textrm{d}^{2}\eta }{\textrm{d}\nu ^{2}} \) can be obtained as follows:

$$\begin{aligned} \frac{\textrm{d}^{2}\eta }{\textrm{d}\nu ^{2}}=&\left( EH(1+z)\right) ^{2}\bigg [\frac{3}{1+z}\bigg (1-\frac{\kappa ^{2}}{3H^{2}} \left( \frac{e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}(p-\rho )}{2}\right. \nonumber \\&\left. +\frac{W-T^{\text {tot}}}{3}\right) \bigg )\frac{\textrm{d}\eta }{\textrm{d}z}+\frac{\textrm{d}^{2}\eta }{\textrm{d}z^{2}}\bigg ]. \end{aligned}$$
(109)

Using (104) we can imply

$$\begin{aligned}&\frac{\textrm{d}^{2}\eta }{\textrm{d}z^{2}}+\frac{3}{1+z}\bigg [1-\frac{\kappa ^{2}}{6H^{2}}\bigg (\tilde{P}_{\text {eff}}- \tilde{\rho }_{\text {eff}}\bigg )\bigg ]\frac{\textrm{d}\eta }{\textrm{d}z}\nonumber \\&\qquad +\frac{\kappa ^{2}}{2H^{2}(1+z)^{2}}\left[ \tilde{\rho }_{\text {eff}}+ \tilde{P}_{\text {eff}}\right] \eta =0, \end{aligned}$$
(110)
$$\begin{aligned}&\frac{\textrm{d}^{2}\eta }{\textrm{d}z^{2}}+{{\mathcal {P}}}(H,R,z)\frac{\textrm{d}\eta }{\textrm{d}z}+{\mathcal {Q}}(H,R,z,)\eta =0, \end{aligned}$$
(111)

with

$$\begin{aligned}&{\mathcal {P}}(H,R,z)=\frac{1}{1+z}\bigg [\frac{\left( -2\Omega _{r_{0}}(1+z)^{4} -\frac{5}{2}\Omega _{m_{0}}(1+z)^{3}\right) e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}+ 3\Omega _{K_{0}}(1+z)^{2}+5\Omega _{DE}-\frac{\kappa ^{2}W}{H^{2}_{0}}}{-e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( \Omega _{r_{0}}(1+z)^{4}+ \Omega _{m_{0}}(1+z)^{3}\right) +\Omega _{K_{0}}(1+z)^{2}+\Omega _{DE}}\bigg ], \end{aligned}$$
(112)
$$\begin{aligned}&{\mathcal {Q}}(H,R,z,)=\frac{1}{\left( 1+z\right) ^{2}} \left[ \frac{e^{\frac{-\sqrt{2}\kappa \phi }{\sqrt{3}}}\left( \frac{3}{4}\Omega _{m_{0}}\left( 1+z\right) ^{3}+ \Omega _{r_{0}}\left( 1+z\right) ^{4}\right) +\frac{3}{2}\Omega _{DE}-\frac{\kappa ^{2}W}{2H_{0}^{2}}}{-e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}} \left( \Omega _{r_{0}}(1+z)^{4}+ \Omega _{m_{0}}(1+z)^{3}\right) +\Omega _{K_{0}}(1+z)^{2}+\Omega _{DE}}\right] , \end{aligned}$$
(113)

and \( H^{2} \) given in Eq. (101) is rewritten as

$$\begin{aligned} H^{2}=&H^{2}_{0}\bigg [-e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\left( \Omega _{r_{0}}(1+z)^{4} +\Omega _{m_{0}}(1+z)^{3}\right) \nonumber \\&+ \Omega _{K_{0}}(1+z)^{2}+\Omega _{DE}\bigg ], \end{aligned}$$
(114)

where

$$\begin{aligned} \Omega _{DE}=\frac{\kappa ^{2}}{H_{0}^{2}}\bigg [\dot{\phi }^{2}+2W+e^{-\frac{\sqrt{2}\kappa {\phi }}{\sqrt{3}}}\dot{\xi }^{2}\bigg ]. \end{aligned}$$
(115)

For the generalized case \(\Omega _{DE}\ne 0\), \(\Omega (m_{0})\ne 0\), \(\Omega (K_{0})\ne 0\), and \(\Omega (r_{0})\ne 0\), we can investigate the solution for Eq. (111) by applying numerical analysis similar to Sect. 2. Thus, we have plotted the deviation vector and the observer area–distance evolution in terms of the redshift z depicted in Fig. 2.

4 Conclusions

In this paper, we have investigated the GDE as a basic equation in hybrid metric-Palatini gravity \(f\left( R,{\mathcal {R}}\right) \) and the scalar–tensor representation of \(f\left( R,{\mathcal {R}}\right) \) gravity in order to study the relationship between the Riemann curvature tensor and the relative acceleration between two nearby test particles. First, we studied the field equations in \(f\left( R,{\mathcal {R}}\right) \) gravity considering an action including a general function \(f({\mathcal {R}})\) besides the Einstein–Hilbert action one in the form of \(R+f({\mathcal {R}})\). We then obtained the GDE generalized expression for \(R+f({\mathcal {R}})\) in the context of the FLRW universe with a perfect fluid energy–momentum tensor, in which the effective energy density and pressure are given by \(\rho _{\text {eff}}\) and \(P_{\text {eff}}\), respectively. In the next part, we set \(f\left( R,{\mathcal {R}}\right) =R-2\Lambda \) and checked the correctness of our result by analogy with the GR scenario. In Sect. 2.2, we found the generalized GDE for fundamental observers besides the modified Pirani equation and the Raychaudhuri equation. We also studied the GDE for null vector fields to extract the null GDE, and investigated the focusing condition for this model in which the geodesics experience convergence, in addition to the modified Mattig relation. Moreover, we proposed a particular case \(f(R,{\mathcal {R}})=R+{{\mathcal {R}}}\) in order to study the numerical behavior of the deviation vector \(\eta (z)\) and the observer area–distance \(r_{0}(z)\) as a function of redshift, and the results are plotted in Fig. 1. The appearance of a peak point for \(\eta (z)\) and \(r_{0}(z)\) in a specified redshift implies that there existed maximum values for \(\eta (z)\) and \(r_{0}(z)\) in the past when our universe was experiencing the inflation regime. After this, the universe exited the inflation regime, and the deviation vector gradually decreased with the increase in z. In Sect. 3 we reviewed a dynamically equivalent approach to \(f\left( R,{\mathcal {R}}\right) \) gravity with two scalar fields, and we found the GDE in the scalar–tensor representation of the model. We repeated the main approach of this study for this modification, and the deviation vector \(\eta (z)\) and the observer area–distance \(r_{0}(z)\) versus redshift are plotted in Fig. 2.

To summarize our results, in this work, we have studied the observed area–distance of the hybrid metric-Palatini gravity through the GDE of the null vector fields. Furthermore, the obtained results indicate that the general performances of the observer area–distance and the null deviation vector fields in the hybrid metric-Palatine gravity for a matter-dominated universe are almost similar to other corresponding modified gravity theories. We can summarize that the behavior of the deviation vector in the modified gravity theories at a low redshift regime is similar to the \(\Lambda \)CDM model, according to the principle of correspondence. This means that our results in these theories fluctuate around GR with small corrections like a cosmological constant. To use the applications of this study, we can say that the equation of the area–distance (66) and (67) can be applied to compute the angular size versus redshift based on the Sunyaev–Zel’dovich effect [42, 43], and to compact the radio sources as cosmic rulers [44, 45]. Additionally, by using the relation between the area and the luminosity distance [46], there are possibilities to extend studies of the GDE in hybrid metric-Palatini gravity with the data obtained from the observations of SNIa [47, 48].