1 Introduction

In the past two decades many researchers have put tremendous efforts to develop and improve the plethora of theoretical models that explain the accelerated expansion of our Universe. Astrophysical measurements that reveal such a phenomenon put into the quest to give convincing theoretical explanations from various possible directions [1,2,3,4,5,6,7,8,9,10,11]. The dark energy model is one such proposed model that attributes the expansion phenomenon to an energy component with negative pressure so called dark energy which dominates the universe at late time. The simplest type of dark energy is the cosmological constant [12]. In this context of accelerated expansion the theory of general relativity (GR) modified by a cosmological constant term \(\Lambda \), which is known as the famous \(\Lambda \) CDM model is one of the most popular one [13]. But, despite its fine agreement with the observation data, there are two major issues that have driven our young minds to focus sharply on some modifications to the assumed \(\Lambda \) CDM model, namely, “the cosmological constant problem” which deals with the discrepancy between theoretical and expected values of the cosmological constant [14,15,16]; and “the cosmic coincidence problem” [17]. To mend up these issues, running \(\Lambda \) cosmological models have been developed.

In the year 2002, Shapiro and Solá made the first development regarding the scaling evolution of the cosmological constant [18]. Various possible cosmological and astrophysical implications have been presented on variable cosmological constant as Planck scale effect and on running G as well as at low energies from the side of physics [19, 20]. Among the running cosmological constant models that have been proposed, it is worthy enough to mention the time dependent cosmological constant motivated by quantum field theory [21,22,23], \(\Lambda (t)\) cosmology induced by a slowly varying Elko field [24], a running vacuum in the context of supergravity [25], etc. The first paper where an action functional is associated to running vacuum was presented in [26]. The models of the type \(\Lambda (H)\) have important implications both on the theoretical arena as well as phenomenologically to smooth out some of the existing tensions of the \(\Lambda CDM\) model. The most important implication on the theoretical side is that they can help to solve the cosmological constant problem [27, 28]. Some of the related papers which can be of interest on these matters which are worthy of mentioning are [29,30,31,32]. A detailed theoretical and phenomenological explanation about these models can be found in [33, 34] also. In Newtonian gravity, without any requirement of further constraints to be satisfied we can explicitly write the time variation of G. But in GR there are other constraints to be satisfied. For instance if we assume that the ordinary energy–momentum conservation law holds then there should not be any variation in the gravitational coupling with respect to the space time or otherwise the ordinary energy–momentum conservation law will be violated [35, 36]. A direct application of these ideas on running vacuum energy to study the variation of the fundamental constants of nature is given in [37, 38]. In the light of Dirac’s idea [39,40,41] which propose that some of the fundamental constants cannot remain constant forever, it is essential to do some modifications in GR field equations [42,43,44] if we are to consider this running cosmological constant term. In this regard, studying the cosmic scenario with varying G needs modified field equations as well as modified conservation laws. We can mention Brans-Dicke theory where there are modifications of GR with a varying G without violating the ordinary energy–momentum conservation law [45,46,47]. There are many other models that employ varying G theories that give a better understanding of the Universe regarding its late time behavior and nature [47,48,49,50,51,52,53,54,55,56,57,58,59,60]. As there are no rigorous proves that indicate whether the cosmological constant is running or not [60], one can study the cosmological implications of different possible theoretical assumptions of \(\Lambda \) term. Motivated by the quantum field theory [19, 20, 61] and some theoretical motivations about the varying \(\Lambda \) form [21, 22], Aleksander Stachowski and Marek Szydtowski have also studied the dynamics of cosmological models with various forms of \(\Lambda (t)\) [62].

In this paper, we consider a running vacuum model which evolves in power series of H. Our aim is to set up dynamical systems out of the cosmological field equations by introducing new set of variables and study the stability of the systems in the light of cosmological implications of the system. Based on the possibilities of the gravitational constant G and the vacuum energy density \(\rho _{\Lambda }\), we develop different dynamical system for three cases and analyse the stability through different approaches by finding respective fixed points. The cosmological scenario associated with each fixed point has been discussed in detail. We arrange the paper in the following ways. In Sect. 2, we give preliminaries that provides a brief introduction on dynamical systems approach to cosmology with some definitions and theorems which will be required to understand the subsequent analysis in the paper. In Sect. 3, we have three cases. In case I of Sect. 3 we show the setting up of cosmological equations and dynamical system analysis where both G and \(\rho _{\Lambda }\) are taken to be constants which is the case of standard \(\Lambda \) CDM cosmology. Under Case I we have three subsections based on analysis using spectral radius, perturbation function and stability at infinity using Poincaré sphere. We present, in Case II, the model dynamics where G = constant and \(\rho _{\Lambda } \ne \) constant. Under case II, we have two subsections based on analysis through spectral radius and using perturbation along each of the axes with respect to increase in cosmic time. In Case III we have dynamical system analysis where \(G \ne \) constant and \(\rho _{\Lambda } \ne \) constant. Under Case III we present three subsections on the basis of analysing stability by the use of perturbation function, Center Manifold Theory and Poincaré sphere. In Sect. 4 we give conclusion of our study. Stability analysis for each of the cases at the respective fixed points is presented and their corresponding cosmological implications along with the evaluation of various cosmological parameters at the respective fixed points are also obtained.

2 Preliminaries

Dynamical system is a mathematical system that describes the time dependence of the position of a point in the space that surrounds it, termed as ambient space. Here, we are approaching towards the system through an autonomous system of ordinary differential equations, (ASODE). ASODE is a system of ordinary differential equations which does not depend explicitly on time. Surendra and Chingtham [63] have also used this approach to study cosmological models in the presence of a scalar field using different forms of potential. From [63] we can also notice that in three dimensional dynamical system we can analyse stability by analysing the nature of perturbation along each of the axes. A dynamical system is generally written in the form of the following [64]:

$$\begin{aligned} \dot{x}=f(x), \end{aligned}$$
(1)

where \(x=(x_{1},x_{2},\ldots x_{n})\) is an element of state space \(X\subseteq {\mathbb {R}}^{n}\) and the function \(f:X \rightarrow X\).

The overhead dot denotes the derivative with respect to cosmic time, t. The function f(x) is such that \(f(x)=(f_{1}(x), f_{2}(x),\ldots f_{n}(x))\) which can be viewed as a vector field in \({\mathbb {R}}^{n}\).

Definition 2.1

[65] Fixed Point: The point \(x=x_{o}\) of the state space \(X \subset {\mathbb {R}}^{n}\) is said to be a fixed point of the autonomous equation \(\dot{x}=f(x)\) if and only if \(f(x_{o})=0\).

Definition 2.2

[65] Stable Fixed Point: A fixed point \(x_{o} \) of a dynamical system represented by \(\dot{x}=f(x)\) is called a stable fixed point if for every \(\epsilon >0\) there exist \(\delta \) such that if \(\psi (t)\) is any solution of the system satisfying \(\parallel \psi (t_{o})-x_{o} \parallel < \delta \), then the solution \(\psi (t)\) exists for all \(t\ge t_{o}\) and it satisfies \( \parallel \psi (t)-x_{o} \parallel < \epsilon \) for all \(t\ge t_{o}\).

Definition 2.3

[66] Local Stability: Let \(g:I \rightarrow I\) be a map and \(x_{o}\) be a fixed point of g, where I is an interval of real numbers. Then

  1. (i)

    the fixed point \(x_{o}\) is said to be locally stable if, for any \(\epsilon >0\), there exists \(\delta >0\) such that, for all \(x\in I\) with \(\mid x-x_{o}\mid <\delta \), we have \(\mid g^{n}(x)-x_{o}\mid <\epsilon \), for all \(n \in {\mathbb {N}}\). Otherwise, the fixed point \(x_{o}\) will be called unstable;

  2. (ii)

    the fixed point \(x_{o}\) is said to be attracting if there exists \(\zeta >0\) such that \(\mid x-x_{o}\mid < \zeta \) implies \(lim_{n \rightarrow \infty }g^{n}(x)=x_{o}\);

  3. (iii)

    the fixed point \(x_{o}\) is said to be locally asymptotically stable if it is both stable and attracting. If in the previous item \(\zeta =\infty \), then \(x_{o}\) is said to be globally asymptotically stable.

Definition 2.4

Hyperbolic point: A fixed point \(x=x_{o} \in X \subset {\mathbb {R}}^{n}\) of the system \(\dot{x}=f(x)\) is said to be a hyperbolic fixed point if none of the eigenvalues of the Jacobian matrix at \(x_{o}\) ,\(J(x_{o})\) have zero real part, otherwise the point is called non-hyperbolic.

Definition 2.5

Jacobian matrix of dynamical system at a fixed point: The Jacobian matrix of the dynamical system given in (1) at a fixed point \(x_{o}\) is given by

$$\begin{aligned} J_{x_{o}} = \begin{bmatrix} \frac{\delta f_{1}}{\delta x_{1}}&{} \frac{\delta f_{1}}{\delta x_{2}}&{} . &{} . &{} . &{}\frac{\delta f_{1}}{\delta x_{n}} \\ . &{}~~~~~~~.&{}&{}&{}&{}.\\ .&{}&{}.&{}&{}&{} .\\ . &{}&{} &{}.&{} &{} .\\ \frac{\delta f_{n}}{\delta x_{1}} &{}\frac{\delta f_{n}}{\delta x_{2}}&{}. &{} .&{} .&{} \frac{\delta f_{n}}{\delta x_{n}} \end{bmatrix} \end{aligned}$$

where \(\frac{\delta f_{i}}{\delta x_{i}}\), \(i=1,2,\ldots ,n\) denotes the first partial derivative of \(f_{i}\) with respect to the \(i^{th}\) component \(x_{i}\) of the element \(x=(x_{1},x_{2},\ldots x_{n})\in X \subseteq {\mathbb {R}}^{n}\) .

The concept of Linear stability theory is one of the simplest ways used to understand the dynamics of a system near a fixed point. In Linear stability theory, the function f is assumed to be sufficiently regular so that we can linearise the system around its fixed point. The eigenvalues of the Jacobian matrix at a fixed point play an important role in studying the stability of the fixed point.

For hyperbolic fixed points if all the eigenvalues of \(J_{x_{o}}\) have positive real parts, then \(x_{o}\) acts as a repeller and it is unstable as all the trajectories closed enough to it are repelled from it. \(x_{o}\) is stable when all the eigenvalues of \(J_{x_{o}}\) have negative real parts. Here \(x_{o}\) is called as attractor and it attracts all nearby trajectories towards it. If at least two eigenvalues have real parts with opposite sign then, \(x_{o}\) behaves as a saddle fixed point which attracts trajectories in some directions and repels along other directions.

If at least one of the eigenvalues of the Jacobian matrix at a fixed point \(x_{o}\) have zero real part then we can not do stability analysis by using eigenvalues of the Jacobian matrix. Such a fixed point is referred to as non-hyperbolic fixed point. To analyse stability of such fixed points we need a better approach other than the linear stability analysis like Center manifold theory, perturbation function, Lyapunov stability. Center manifold theory is the most popular method which reduces the dimensionality of the system and determines the stability of the critical points of the parent system according as the stability of the reduced system. Wiggins [64] and Carr [67] have discussed the Center manifold theory in detail.

The eigenvalues of the Jacobian matrix J with order \(n\times n\) given in Definition 2.5 will have n eigenvalues. The eigenvectors of J associated to the eigenvalues with negative real part spans a vector space called stable space, \(J^{s}\) and the eigenvectors associated with positive real part spans a vector space called the unstable space, \(J^{u}\). Similarly \(J^{c}\) represents the vector space spanned by the eigenvectors associated with zero real part. Here, the superscript suc denote the dimensions of the respective vector spaces. Also the spaces \(J^{s},J^{u}\) and \(J^{c}\) are the subspaces of \({\mathbb {R}}^{n}\). The space \({\mathbb {R}}^{n}\) can be written as the direct sum of these three subspaces, that is, \({\mathbb {R}}^{n}=J^{s}\oplus J^{u}\oplus J^{c}\). These results have been detailed in Carr’s book [67], Elaydi’s book [68] and Zhang’s book [69]. If at least one eigenvalue of J at a fixed point \(x_{o}\) has positive real part then \(x_{o}\) will be unstable whether it is hyperbolic or not. But if \(x_{o}\) is non-hyperbolic and no eigenvalues has positive real part, then we can use Center manifold theory to determine stability of the fixed point.

Let us consider a two dimensional dynamical system. Using a suitable coordinate transformation we can rewrite any system of the form (1) as follows:

$$\begin{aligned} \left. \begin{array}{c} \dot{{\textbf {x}}}=A{\textbf {x}} + f({\textbf {x,y}}), \\ \dot{{\textbf {y}}}=B{\textbf {y}} + g({\textbf {x,y}}), \end{array}\right\} \end{aligned}$$
(2)

where A is a \(c\times c\) matrix having eigenvalues with zero real parts, B is an \(s \times s\) matrix having eigenvalues with negative real parts and \(({\textbf {x,y}})\in J^{c} \times J^{s}\). The functions f and g satisfy the following:

$$\begin{aligned} f(0,0)= & {} 0 \end{aligned}$$
(3)
$$\begin{aligned} g(0,0)= & {} 0 \end{aligned}$$
(4)
$$\begin{aligned} \nabla f(0,0)= & {} 0 \end{aligned}$$
(5)
$$\begin{aligned} \nabla g(0,0)= & {} 0. \end{aligned}$$
(6)

Definition 2.6

[67] Center Manifold: A geometrical space \(M^{c}(0)\) is a center manifold for (2) if it can be locally represented as

$$\begin{aligned} M^{c}(0)= & {} \{({\textbf {x,y}})\in J^{c}\times J^{s} | {\textbf {y}}=h({\textbf {x}}),|{\textbf {x}}|<\delta , h(0)\nonumber \\= & {} 0,\nabla h(0)=0\}, \end{aligned}$$
(7)

for a sufficiently regular function \(h({\textbf {x}})\) on \(J^{s}\) and \(\delta \) however small it may be. The proofs of the existence of the center manifold for the system (2) is also provided in [67] and he has given the dynamics of the system (2) restricted to the center manifold as follows:

$$\begin{aligned} \dot{{\textbf {v}}}=A{\textbf {v}}+f({\textbf {v}},h({\textbf {v}})), \end{aligned}$$
(8)

for sufficiently small \({\textbf {v}} \in {\mathbb {R}}^{c}\).

Theorem 2.1

[70] Consider a flow defined by a dynamical system on \({\mathbb {R}}^{2}\)

$$\begin{aligned} \left. \begin{array}{c} \dot{x} = P_{1}(x,y), \\ \dot{y} = P_{2}(x,y), \end{array}\right\} \end{aligned}$$
(9)

where \(P_{1}\) and \(P_{2}\) are polynomial functions of x and y. Let \(P_{1m}\) and \(P_{2m}\) denote the \(m^{th}\) degree term in \(P_{1}\) and \(P_{2}\) respectively. Then, the critical points at infinity for the \(m^{th}\) degree polynomial system (9) occur at the points (XY, 0) on the equator of the Poincaré sphere where

$$\begin{aligned} X^{2}+Y^{2}=1 \text {and} XP_{2m}(X,Y)-YP_{1m}(X,Y)=0, \end{aligned}$$

or equivalently at the polar angle \(\theta _{j}\) and \(\theta _{j}+\pi \) satisfying

$$\begin{aligned} G_{m+1}(\theta )\equiv & {} cos\theta Q_{m}(cos\theta , sin\theta )-sin\theta P_{m}(cos\theta , sin\theta )\\= & {} 0 \end{aligned}$$

This equation has at most \(m+1\) pairs of roots \(\theta _{j}\) and \(\theta _{j}+\pi \) unless \(G_{m+1}(\theta )\) is identically zero. If \(G_{m+1}(\theta )\) is not identically zero, then the flow on the equator of the Poincaré sphere is counter-clockwise at points corresponding to polar angles \(\theta \) where \(G_{m+1}(\theta )>0\) and it is clockwise at points corresponding to polar angles \(\theta \) where \(G_{m+1}(\theta )<0\).

Theorem 2.2

[70] The flow defined by (9) in a neighborhood of any critical point of (9) on the equator of \(S^{2}\), except the points \((0, \pm 1, 0)\), is topologically equivalent to the flow defined by the following system

$$\begin{aligned}&\pm \, \dot{y}=yz^{m}P_{1}\bigg (\frac{1}{z}, \frac{y}{z}\bigg )-z^{m}P_{2}\bigg (\frac{1}{z}, \frac{y}{z}\bigg ), \\&\pm \, \dot{z}=z^{m+1}P_{1}\bigg (\frac{1}{z}, \frac{y}{z}\bigg ), \end{aligned}$$

the signs being determined by the flow on the equator of \(S^{2}\) as determined in Theorem 2.1.

Theorem 2.3

[70] Let us consider a flow in \({\mathbb {R}}^{3}\) defined by

$$\begin{aligned} \left. \begin{array}{c} \dot{x} = P_{1}(x,y,z), \\ \dot{y} = P_{2}(x,y,z), \\ \dot{y} = P_{3}(x,y,z), \end{array}\right\} \end{aligned}$$
(10)

where \(P_{1}\), \(P_{2}\) and \(P_{3}\) are polynomial functions of x, y, z of maximum degree m.

The critical points at infinity for the \(m^{th}\) degree polynomial system (10) occur at the points (XYZ, 0) on the equator of the Poincaré sphere \(S^{3}\) where \(X^{2}+Y^{2}+Z^{2}=1\) and

$$\begin{aligned} XP_{2m}(X,Y,Z)-YP_{1m}(X,Y,Z)= & {} 0, \\ XP_{3m}(X,Y,Z)-ZP_{1m}(X,Y,Z)= & {} 0, \\ YP_{3m}(X,Y,Z)-ZP_{2m}(X,Y,Z)= & {} 0, \end{aligned}$$

where \(P_{1m}\), \(P_{2m}\) and \(P_{3m}\) denote the \(m^{th}\) degree terms in \(P_{1}\), \(P_{2}\) and \(P_{3}\) respectively.

Theorem 2.4

[70] The flow defined by the system (10) in a neighborhood of \((\pm 1, 0, 0, 0) \in S^{3}\) is topologically equivalent to the flow defined by the system:

$$\begin{aligned}&\pm \,\dot{y}=yw^{m}P_{1}\bigg (\frac{1}{w},\frac{y}{w},\frac{z}{w}\bigg )-w^{m}P_{2}\bigg (\frac{1}{w},\frac{y}{w},\frac{z}{w}\bigg ), \\&\pm \,\dot{z}=zw^{m}P_{1}\bigg (\frac{1}{w},\frac{y}{w},\frac{z}{w}\bigg )-w^{m}P_{3}\bigg (\frac{1}{w},\frac{y}{w},\frac{z}{w}\bigg ), \\&\pm \,\dot{w}=w^{m+1}P_{1}\bigg (\frac{1}{w},\frac{y}{w},\frac{z}{w}\bigg ). \end{aligned}$$

3 Dynamical system analysis for different possibilities of G and \(\rho _{\Lambda }\)

In this section we present the dynamical system analysis when \(G=\)constant and \(\rho _{\Lambda }\)=constant. This is a standard model and we present it as case I of our analysis.

Case I: Dynamical system analysis when \(G=\) constant and \(\rho _{\Lambda }\) = constant

The Einstein field equations in the presence of cosmological constant \(\Lambda \) are given by

$$\begin{aligned} \left. \begin{array}{c} R_{\mu \nu }-\frac{1}{2}g_{\mu \nu }R = 8\pi G( T_{\mu \nu }+g_{\mu \nu }\rho _{\Lambda }), \\ R_{\mu \nu }-\frac{1}{2}g_{\mu \nu }R = 8\pi G {\textbf {T}}_{\mu \nu }, \end{array}\right\} \end{aligned}$$
(11)

where \(T_{\mu \nu }\) is the ordinary energy–momentum tensor, \( {\textbf {T}}_{\mu \nu }\equiv T_{\mu \nu }+g_{\mu \nu }\rho _{\Lambda }\) is the modified energy–momentum tensor and \(\rho _{\Lambda }=\frac{\Lambda }{8\pi G}\) is the vacuum energy density in the presence of \(\Lambda \).

We assume that the universe is filled with a perfect fluid with velocity four-vector field \(V_{\mu }\). With this consideration we have \(T_{\mu \nu }=-p_{m}g_{\mu \nu }+(\rho _{m}+p_{m})U_{\mu }U_{\nu }\), where \(\rho _{m}\) is the density of matter-radiation and \(p_{m}=(\gamma -1)\rho _{m}\) is the corresponding pressure. In the similar way, the modified energy–momentum tensor can be expressed as

$$\begin{aligned} {\textbf {T}}=-p_{tt}g_{\mu \nu }+(\rho _{tt}+p_{tt})U_{\mu }U_{\nu }, \end{aligned}$$
(12)

where \(p_{tt}=p_{m}+p_{\Lambda }\), \(\rho _{tt}=\rho _{m}+\rho _{\Lambda }\) and \(p_{\Lambda }=\omega _{de} \rho _{\Lambda }\) is the associated pressure in the presence of \(\Lambda \). With this substitution in the above expression we have

$$\begin{aligned} {\textbf {T}}=(\rho _{\Lambda }-p_{m})g_{\mu \nu }+(\rho _{m}+p_{m})U_{\mu }U_{\nu } \end{aligned}$$
(13)

By assuming a spatially flat Friedmann–Lemaître–Robertson–Walker(FLRW) metric along with the above modified energy–momentum tensor [71,72,73,74], we have the following gravitational field equations:

$$\begin{aligned} 8\pi G \rho _{tt}\equiv 8\pi G \rho _{m} + \Lambda= & {} 3H^{2}, \end{aligned}$$
(14)
$$\begin{aligned} 8\pi G p_{tt}\equiv 8\pi G p_{m} - \Lambda= & {} -2\dot{H}-3H^{2}, \end{aligned}$$
(15)

where the overhead dot denotes the derivative with respect to the cosmic time t.

With the help of FLRW metric and the Bianchi identities by respecting the Cosmological Principle embodied in the FLRW metric we have the following generalized local conservation law:

$$\begin{aligned} \dot{\rho _{m}}+\dot{\rho _{\Lambda }}+3H(\rho _{m}+p_{m}+\rho _{\Lambda }+p_{\Lambda })=0. \end{aligned}$$
(16)

If we put \(p_{m}=(\gamma -1)\rho _{m}\) and \(p_{\Lambda }=\omega _{de} \rho _{\Lambda }\), where we assume that \(\omega _{de} \rightarrow -1\) in the above equation we have the following balanced conservation equation:

$$\begin{aligned} \dot{\rho _{m}} + 3\gamma H\rho _{m}=-\dot{\rho _{\Lambda }}. \end{aligned}$$
(17)

Since \(\rho _{\Lambda }\) is taken to be constant the right hand side of the above equation vanishes to give the following equation:

$$\begin{aligned} \dot{\rho _{m}} + 3\gamma H\rho _{m}=0. \end{aligned}$$
(18)

Motivated by the work of Aleksander Stachowski et al. [62], let us consider that the cosmological constant \(\Lambda \) evolves with time through the hubble parameter H with \(\Lambda (H)\) given in the form of Taylor series with respect to H.

$$\begin{aligned} \Lambda (H)= & {} \sum _{n=0}^{\infty }\frac{1}{2n!} \frac{d^{2n}}{dH^{2n}} \Lambda (H)|_{0}H^{2n} \nonumber \\ \Rightarrow \Lambda (H)= & {} \Lambda _{o}+\alpha _{2}H^{2}+\alpha _{4}H^{4}+\cdots \end{aligned}$$
(19)

where \(\Lambda _{0}=\Lambda (H)|_{0}\) and \(\alpha _{2n}~'s, n=1,2,\ldots \) are the coefficients in the Taylor series expansion of \(\Lambda (H)\) given by \(\alpha _{2n}=\frac{1}{2n!}\frac{d^{2n}\Lambda (H)}{dH^{2n}}|_{0}, n=1,2,\ldots \)

The constant \(\Lambda _{o}\) in (19) denotes the dominant term when H is near the current value \(H_{o}\). The reason why only even powers of H appear in the running cosmological constant is due to the general covariance of the effective action of quantum field theories and there are no covariant structures which are odd in metric derivatives [75]. This was also explained in [18] and in more detail in [76]. The contribution of only the even powers of Hubbble parameter to the time varying \(\Lambda (t)\) has also been presented in detail in [21]. In the year 2020, the particle production and the corresponding entropy increase in running vacuum model was studied and explanations about the cosmic history from the very early Universe upto the current Universe and further into the final de Sitter era was also given in [77]. The implication from these even powers of H for the early Universe and for the current Universe is clearly explained in [77]. At early stage of the Universe the presence of \(H^{4}\) in the vacuum energy density triggers inflation while \(H^{4}\) term becomes irrelevant in the late epochs of the current Universe [77]. The inflationary models triggered by the power \(H^{4}\) were first proposed phenomenologically in [78] and later \(H^{n}\) situation is also studied in [77, 79]. In the subsequent studies, they were first supported from string theory [75, 80]. Recent analysis has been made on the gravitational and chiral anomalies in the running vacuum Universe and the study of the matter–antimatter asymmetry is also given in [80]. Also, the study of the stringy-running-vacuum-model inflation from primordial gravitational waves and stiff axion matter to dynamical dark energy is done in [75]. In addition, a detailed analysis of the vacuum energy density of quantum field theory in FLRW spacetime is also presented in [28]. Also they have proven from first principles that a series of even powers of H emerges, hence predicting a generic mechanism of inflation fully based on quantum field theory in curved space time [28]. Thus it is no longer an ad hoc assumption but, it is a result of a calculation in quantum field theory in curved backgrounds.

Now, using (19) in (15), we obtain the following relations:

$$\begin{aligned} -2\dot{H}-3H^{2}= & {} 8\pi G(\gamma -1)\rho _{m}-\Lambda _{0}-\alpha _{2}H^{2}-\alpha _{4}H^{4} -\cdots ,\nonumber \\ \text {that is,} \ 2\dot{H}= & {} \Lambda _{0}+(\alpha _{2}-3)H^{2}+\alpha _{4}H^{4}+\cdots \nonumber \\&-8\pi G (\gamma -1) \rho _{m}, \end{aligned}$$
(20)

To set up the dynamical system we consider a new variable \(x=(\frac{H}{8\pi G})^{2}\) and substitute \(y=\rho _{m}\) for our convenience in the subsequent analysis. With this substitution we can express (20) in terms of x and y as follows:

$$\begin{aligned} 2\dot{H}= & {} \frac{8\pi G\dot{ x}}{\sqrt{x}}=8\pi G[C_{0}+\beta _{1}x+\beta _{2}x^{2}\nonumber \\&+\cdots -(\gamma -1) y], \end{aligned}$$
(21)

where

$$\begin{aligned} C_{o}= & {} \frac{\Lambda _{o}}{8\pi G}; \beta _{i}\\= & {} \left\{ \begin{array}{ll} (\alpha _{2}-3)8\pi G, &{} i=1, \\ \alpha _{2j}(8\pi G)^{(2i-1)}, &{} i \ge 1, \end{array} \right. i=1,2,\ldots \text {and } j\ge 2. \end{aligned}$$

Since we want to do qualitative analysis on the current Universe and its behaviour in the late time, we consider only a few powers of H beyond the term \(C_{o}\) so as to ensure a better \(\Lambda \)CDM limit. All the other terms involving higher powers of H beyond \(H^{2}\) term are neglected as their contribution is completely negligible at present [37, 77, 78].

Using (21) and the newly introduced variables in the above field equations, we obtain the following set of ordinary differential equations which will represent the required dynamical system:

$$\begin{aligned} x'=\frac{dx}{d\Theta }= \frac{dx}{dt}\frac{dt}{d\Theta }, \end{aligned}$$

where \(\Theta =\ln a\) denotes the logarithmic time with respect to the scale factor a. The overhead dash denotes the derivative with respect to \(\Theta \) while the overhead dot denotes the derivative with respect to cosmic time t.

$$\begin{aligned} x'= & {} \frac{1}{8\pi G}(C_{0}+\beta _{1}x-(\gamma -1) y).\\ y'= & {} \frac{dy}{d\Theta }=\frac{dy}{dt}.\frac{dt}{d\Theta },\nonumber \end{aligned}$$
(22)

that is,

$$\begin{aligned} y'=-3\gamma y. \end{aligned}$$
(23)

To analyse stability, firstly we need to find the fixed points of the system. For this we equate \(x'=0\), \(y'=0\), that is,

$$\begin{aligned} x'=\frac{1}{8\pi G}(C_{0}+\beta _{1}x-(\gamma -1)y)=0. \end{aligned}$$

This implies

$$\begin{aligned} x=\frac{(\gamma -1)y-C_{o}}{\beta _{1}}, \end{aligned}$$

where \(\beta _{1}=(\alpha _{2}-3)8\pi G\) and \(y'=-3\gamma y=0\).

This implies either \(y=0\) or \(\gamma =0\). We can also have \(y\rightarrow 0\) in evaluating the fixed point. We need to observe both the possibilities and their implications to the evolving cosmological scenario. When \(y=0\) in the expression of x above we get \(x=\frac{-C_{o}}{(\alpha _{2}-3)8\pi G}\). So the first fixed point we have obtained is \(F_{1}=(\frac{-C_{o}}{(\alpha _{2}-3)8\pi G},0)\). Again when \(\gamma =0\) then from (18) we see that \(\rho _{m}=\) constant. Let us suppose that \(\rho _{m}=\xi \), that is, \(y = \xi \). Then the second fixed point we have obtained for the case of \(\gamma =0\) is \(F_{2}=(\frac{-C_{o}-\xi }{(\alpha _{2}-3)8\pi G},y=\xi )\). When we consider \(y\rightarrow 0\) we will obtain a special case of non-hyperbolic fixed points called a normally hyperbolic fixed point which is actually a set of non-isolated fixed points. For normally hyperbolic fixed points stability is decided by the sign of real part of the remaining eigenvalue even if one of the eigenvalues of the Jacobian matrix vanishes. So when we choose \(y \rightarrow 0\) then we can write the fixed point as \(F_{3}=(\frac{-C_{o}}{(\alpha _{2}-3)8\pi G},y\rightarrow 0)\). Now let us evaluate the Jacobian matrices \(J_{F_{1}}\), \(J_{F_{2}}\) and \(J_{F_{3}}\) at the respective fixed points to study the stability of the system.

Let \(f(x,y)=\frac{1}{8\pi G}(C_{0}+\beta _{1}x-(\gamma -1) y)\), \(g(x,y)=-3\gamma y\).

The Jacobian matrix at the respective fixed points are given by

$$\begin{aligned} J_{F_{1}}= & {} J_{F_{3}} =\left( \begin{array}{cc} f_{x} &{} f_{y} \\ g_{x} &{} g_{y} \\ \end{array} \right) =\left( \begin{array}{cc} \frac{\beta _{1}}{8\pi G} &{} \frac{-(\gamma -1)}{8\pi G} \\ 0 &{} -3\gamma \\ \end{array} \right) ,\\ J_{F_{2}}= & {} \left( \begin{array}{cc} \frac{\beta _{1}}{8\pi G} &{} \frac{1}{8\pi G} \\ 0 &{} 0 \\ \end{array} \right) , \end{aligned}$$

where \(\beta _{1}=(\alpha _{2}-3)8\pi G\).

The above matrices are upper triangular matrices. We all know that the eigenvalues of the Jacobian matrices are given by the diagonal entries. So, the eigenvalues of \(J_{F_{1}}=J_{F_{3}}\) are \(EV^{J_{1}}_{1}=EV^{J_{3}}_{1}=\frac{\beta _{1}}{8\pi G}=(\alpha _{2}-3)\), \(EV^{J_{1}}_{2}=EV^{J_{3}}_{2}=-3\gamma \) and those of \(J_{F_{2}}\) are \(EV^{J_{2}}_{1}=EV^{J_{1}}_{1}=\frac{\beta _{1}}{8\pi G}=(\alpha _{2}-3)\), \(EV^{J_{2}}_{2}=0\). The fixed points \(F_{1}\) and \(F_{3}\) are hyperbolic for \(\gamma \ne 0\) as none of the eigenvalues vanishes. When \(\gamma \ne 0\), \(EV^{J_{1}}_{1}\), \(EV^{J_{3}}_{1}<0\) for \(\alpha _{2}<3\) and \(EV^{J_{3}}_{1}\),\(EV^{J_{1}}_{2}<0\) for all \(\gamma \in (0,2]\). As all the eigenvalues of \(J_{F_{1}}\) and \(J_{F_{3}}\) possess negative values for \(\gamma \ne 0\), \(\alpha _{2}<3\), \(F_{1}\) and \(F_{3}\) are stable fixed points. If \(y \rightarrow 0\) is considered, though \(EV^{J_{3}}_{2}=0\) \(F_{3}\) is still stable as the remaining eigenvalue \((\alpha _{2}-3)\) is negative for \(\alpha _{2}<3\). The fixed points \(F_{1}\) and \(F_{3}\) are stable and behaves as an attractor for \(\alpha _{2}<3\). Figures 1 and  2 shows the phase plot of \(F_{1}\) for \(\gamma =\frac{4}{3}\) and \(\gamma =2\) respectively with \(\alpha _{2}=2<3\) where all the nearby trajectories are attracted towards it. When \(\alpha _{2}>3\), the eigenvalues of \(J_{F_{1}}\) possess opposite signs which shows that \(F_{1}\) behaves as a saddle fixed point. Figure 3 shows the phase plot of the system for \(\alpha _{2}=4>3\) where trajectories in some directions are attracted towards \(F_{1}\) while trajectories along some other directions are repelled away from it. For the fixed point \(F_{2}\) we see that \(J_{F_{2}}\) is non-hyperbolic as one of the eigenvalues, namely, \(EV^{J_{2}}_{2}=0\). For non-hyperbolic fixed point \(F_{2}\) we can not analyse stability using the above linear stability theory. Since it is a two dimensional dynamical system we can use the notion of perturbation function and spectral radius of the Jacobian matrix for the non-hyperbolic fixed point \(F_{2}\) to analyse the stability. In the subsequent paragraph we will show the stability analysis using these methods.

Fig. 1
figure 1

The phase plot for \(F_{1}\) at \(\alpha _{2}=2<3\), \(\gamma = \frac{4}{3}\), stable attractor

Fig. 2
figure 2

The phase plot for stable \(F_{1}\) at \(\gamma =2\), \(\alpha _{2}<3\)

Fig. 3
figure 3

The phase plot for \(F_{1}\) at \(\alpha _{2}=4>3\) representing saddle point

A. Stability analysis for \(F_{2}\) using the concept of Spectral radius:

Let’s rewrite the Jacobian matrix at the fixed point \(F_{2}\) as follows:

$$\begin{aligned} J_{F_{2}}=\left( \begin{array}{cc} (\alpha _{2}-3) &{} \frac{1}{8\pi G} \\ 0 &{} 0 \\ \end{array} \right) . \end{aligned}$$

Trace of \(J_{F_{2}}\), \(tr(J_{F_{2}})\) = sum of eigenvalues = \( EV^{J_{2}}_{1} +EV^{J_{2}}_{2} \) =\((\alpha _{2}-3)\).

Determinant of \(J_{F_{2}}\), \(det(J_{F_{2}})\) = product of eigenvalues = \(EV^{J_{2}}_{1} \times EV^{J_{2}}_{2} =0\) .

The spectral radius of a matrix is the maximum of the absolute values of all the eigenvalues of the matrix. The stability of a fixed point (xy) of a dynamical system can be determined by the value of spectral radius of its Jacobian matrix evaluated at the fixed point. The notion of spectral radius in discussing stability of a fixed point has been given in detail in [68].

The spectral radius of the above Jacobian matrix is given by

$$\begin{aligned} \sigma _{J_{F_{2}}}= & {} max \{|\lambda | : \lambda ~is~ the~ eigenvalue \}, \nonumber \\= & {} max\{ |\alpha _{2}-3|,0 \}, \nonumber \\= & {} \bigg \{ \begin{array}{cc} \alpha _{2}-3, &{} \alpha _{2}>3, \\ -(\alpha _{2}-3), &{} \alpha _{2}<3. \end{array} \end{aligned}$$
(24)

By theorem [68], page 221], \(F_{2}\) will be locally asymptotically stable if \(\sigma _{J_{F_{2}}}<1\). And we can not determine stability when \(\sigma _{J_{F_{2}}}=1\) and hence Center manifold theory is very useful in analysing stability. With reference to [68], page 200], spectral radius will be less than unity if and only if

$$\begin{aligned} |tr(J_{F_{2}})|-1<det(J_{F_{2}})<1. \end{aligned}$$

From the above arguments, \(F_{2}\) is locally asymptotically stable for \(3< \alpha _{2}<4\) or \(2<\alpha _{2}<3\). It can be noted that we have assume \(\alpha _{2}\ne 3\) here so that we can study our system with fixed points in finite phase plane.

B. Stability analysis for \(F_{2}\) using the concept of Perturbation function:

To analyse stability in a simpler way we find perturbation function along each axis as a function of logarithmic time \(\Theta \). It is noted that while studying perturbation along x-axis we assume \(y=0\) as we are analysing only along x-axis. We can make the interval where \(\alpha _{2}\) lies finer by analysing the stability from this side of perturbation function. Now to find the perturbation function we perturb the system by a small amount, that is, \(x= \frac{-c_{o}-\xi }{(\alpha _{2}-3)8\pi G}+\eta _{x}\) and \(y=\xi + \eta _{y}\), where \(\eta _{x}\) and \(\eta _{y}\) represent small perturbations along x and y axes respectively. With these perturbed system, (22) and (23) takes the following form:

$$\begin{aligned} \eta _{x}' = \frac{1}{8\pi G}\bigg (C_{o}+\beta _{1}\bigg (\frac{-C_{o}-\xi }{(\alpha _{2}-3)8\pi G}\bigg )+\eta _{x}\bigg ). \end{aligned}$$

Solving the above differential equation we obtain \(\eta _{x}\) as a function of logarithmic time, \(\Theta \) as follows:

$$\begin{aligned} \eta _{x} = e^{(\alpha _{2}-3)\Theta } + \frac{\xi }{(\alpha _{2}-3)8\pi G}. \end{aligned}$$
(25)

Similarly,

$$\begin{aligned} \eta _{y} = \frac{C-\xi e^{3\gamma \Theta }}{e^{3\gamma \Theta }}. \end{aligned}$$
(26)

When \(\alpha _{2}<3\), as \(\Theta \) tends to infinity the perturbation along x-axis, \(\eta _{x}\) evolves to a constant value which is \(\frac{\xi }{(\alpha _{2}-3)8\pi G}\). In the above expression of \(\eta _{y}\) if we consider \(\Theta \rightarrow \infty \), we get \(\frac{\infty }{\infty }\) form. So we can apply L Hospital’s rule of finding limit in the expression of \(\eta _{y}\) to obtain its limiting value as \(-\xi \) for any value of \(\gamma \) . We can also directly put \(\gamma =0\) in (23) to get \(\eta _{y}'=0\) and obtain \(\eta _{y}=\)constant. But by doing so we won’t be able to show the nature of \(\eta _{y}\) in terms of \(\Theta \) and further with (26) we can achieve the constant value towards which \(\eta _{y}\) evolves in a finer way. As perturbation along both the axes evolve to a constant value when \(\alpha _{2}<3\), we conclude that \(F_{2}\) is stable for \(\alpha _{2}<3\) and it is locally asymptotically stable for \(2<\alpha <3\). If \(\Phi =\{\alpha _{2}: \eta _{x}\rightarrow 0\) or a constant \(\wedge \) \(\eta _{y}\rightarrow 0\) or a constant}, then \(F_{2}\) is stable for any \(\alpha _{2} \in \Phi \) where \(\Phi =(-\infty ,3)\). The perturbation plots shown in Fig. 4 shows the variation of perturbation function along y axis with respect to \(\Theta \) for \(F_{2}\). From Fig. 4 we see that when \(\gamma =0\), \(\eta _{y}\) becomes a constant function, but if \(\gamma \ne 0\) then as \(\Theta \rightarrow \infty \), \(\eta _{y}\) takes \(\frac{\infty }{\infty }\) form. So by applying L Hospital’s rule as \(\Theta \rightarrow \infty \), \(\eta _{y}\) tends to \(-\xi \) which is a constant value. Figure 5 shows that the perturbation along x-axis tends to a constant value, namely, \(\frac{\xi }{(\alpha _{2}-3)8\pi G}\) when \(\alpha _{2}<3\). In the plot shown in Fig. 5 we take \(\xi =1\), \(8\pi G=1\) and \(\alpha _{2}=2.5<3\) to show that \(\eta _{y}\) tends to \(\frac{\xi }{(\alpha _{2}-3)8\pi G}=-2\) here.

Fig. 4
figure 4

Variation of \(\eta _{y}\) with respect to \(\Theta \) for \(F_{2}\)

Fig. 5
figure 5

The variation of \(\eta _{x}\) with respect to \(\Theta \) for \(F_{2}\)

Table 1 Table for Case I (G=constant, \(\rho _{\Lambda }\)=constant)

In terms of the variables x and y we obtain the value of effective equation of state \(\omega _{eff}\) and total energy density \(\Omega _{tt}\) as follows:

$$\begin{aligned} \omega _{eff}= & {} \frac{p_{tt}}{\rho _{tt}} ,\nonumber \\= & {} -1-\frac{\alpha _{2}-3}{3}-\frac{C_{o}}{24\pi G x}+\frac{(\gamma -1)y}{24\pi G x}, \end{aligned}$$
(27)

where \(\rho _{tt}=24\pi G x\) and \(p_{tt}=(-24\pi Gx-\beta _{1}x-C_{o}+ (\gamma -1)y\);

$$\begin{aligned} \Omega _{tt}= \frac{\Lambda _{o}}{3(8\pi G)^{2}x}+\frac{\alpha _{2}}{3}+ \frac{y}{24\pi G x}, \end{aligned}$$
(28)

where vacuum energy density, \(\Omega _{\Lambda }=\frac{\Lambda _{o}}{3(8\pi G)^{2}x}+\frac{\alpha _{2}}{3}\) and matter density, \(\Omega _{m}=\frac{y}{24\pi G x}\).

With the redshift function \(a(t)=\frac{1}{1+z_{r}}\) we express the EoS for dark energy sector \(\omega _{de}\), the Hubble parameter H and the deceleration parameter q in terms of redshift \(z_{r}\) as follows:

$$\begin{aligned} \omega _{de}= & {} \frac{y}{3\rho _{\Lambda }^{2}}(-\alpha _{2}\rho _{\Lambda }-\gamma +1)-\frac{\alpha _{2}}{3} \nonumber \\&-\frac{C_{o}}{24\pi Gx}+\frac{y}{24\pi Gx\rho _{\Lambda }}((\gamma -1)(y+\rho _{\Lambda })-C_{o}) ,\nonumber \\ \end{aligned}$$
(29)

, where \(\omega _{de}\) is the EoS for dark energy sector.

This implies,

$$\begin{aligned} \omega _{de}(z_{r})= & {} \frac{E_{1m}}{\rho _{\Lambda }}\bigg (\frac{2}{9\ln \frac{(1+z_{r})}{E_{1}}}-\gamma \bigg ) e^{\gamma (1+\frac{\rho _{\Lambda }}{\xi })}\nonumber \\&+\frac{2}{9\ln (\frac{1+z_{r}}{E_{1}})}-1, \end{aligned}$$
(30)
$$\begin{aligned} H(z_{r})= & {} \bigg (\frac{9}{2}\bigg (\frac{3-\alpha _{2}}{3}+\frac{(1+z_{r})^{3\gamma }}{9}-40.309\bigg )\nonumber \\&\times \bigg (\ln \frac{(1+z_{r})}{b_{1}}\bigg )^{\frac{1}{2}}, \end{aligned}$$
(31)
$$\begin{aligned} q(z_{r})= & {} \frac{-1}{18}\bigg (\frac{3\gamma (1+z_{r})^{3\gamma }}{-38.97+\frac{(1+z_{r})^{3\gamma }}{9}}\bigg )\nonumber \\&-\frac{1}{2\ln \frac{(1+z_{r})}{B_{1}}}-1, \end{aligned}$$
(32)

where \(E_{1m}\), \(E_{1}\), \(b_{1}\), \(B_{1}\) are arbitrary constants.

Also we have the transition redshift, \(z_{rt}\) as follows:

$$\begin{aligned} z_{rt}=\{z_{r}:q_{z_{r}}=0\}. \end{aligned}$$
(33)

At \(F_{1}\) the value of EoS parameter \(\omega _{de}\) is calculated as \(-1\) which assures the presence of negative pressure in the existing cosmological scenario with the numerical value of \(\Omega _{tt}\) as \(\Omega _{tt} \approx 1\). Thus the presence of this late time attractor contributes to our model with an accelerated expansion phase of the Universe with \(\omega _{de}=-1\) and \(\Omega _{tt}=0.99\approx 1\) which is in agreement with the observational data in [81]. Also when we evaluate the above cosmological parameters at the fixed point \(F_{2}\), we obtained \(\omega _{de}=-1-\frac{(\gamma -1)}{3}\) where we have taken \(\xi =\rho _{\Lambda }^{2}\). So, \(\omega _{de}=\frac{-2}{3}=-0.66>-1\) when \(\gamma =0\) and \(\omega _{de}=-1.11<-1\) when \(\gamma =\frac{4}{3}\). The relative energy density at \(F_{2}\) is found to be \(\Omega _{tt}= 1\). These results have been tabulated in Table 1. To analyse the phantom-like or quintessence like behavior, lets us discuss \(\omega _{de}\) by expressing it in terms of redshift \(z_{r}\).

The expression given by (30) is very useful to study the behaviour of the EoS at late time when the cosmic time t tends to infinity. We can analyse the phantom-like or quintessence-like behaviour when the redshift parameter \(z_{r}\) approaches the fixed point value near \(-1\). Figure 6 shows the plot for \(\omega _{de}\) with respect to \(z_{r}\). In this plot we see that the Universe is in the phantom-like phase at the present epoch (assuming the present value of scale factor to be unity) with the present value of EoS obtained as \(\omega _{de} \simeq -1.029<-1\). As \(z\rightarrow -1\) at late time it is observed that the curve approaches the phantom region. As we know that the deceleration parameter determines the accelerating or decelerating behaviour while the Hubble parameter decides the rate of expansion of the Universe, it is also important to analyse the behavior of these parameters in redshift. From the plots shown in Figs. 7 and  8 respectively, we see that the Hubble parameter increases with the increase in time while the deceleration parameter decreases monotonically with deceleration–acceleration transition taking place at \(z_{rt}\simeq 0.71\). This value of \(z_{rt}\) represents the the value of the transition point from early decelerated regime \(q(z_{r})>0\)(corresponding to \((z_{r}>z_{rt})\), into current accelerated one \(q(z_{r}<0)\) (corresponding to \((z_{r}<z_{rt})\)) and it remains close to the \(\Lambda CDM\) [61]. Also the present values of the above parameters are tabulated in Table 4 and these results are in agreement with those reported in [61, 82,83,84,85].

Fig. 6
figure 6

The plot for \(\omega _{de}\) versus redshift \(z_{r}\) for case I at \(E_{1m}=0.000097\rho _{\Lambda }\), \(\gamma =\frac{4}{3}\), \(E_{1}=200\), \(\xi =1300\rho _{\Lambda }\)

Fig. 7
figure 7

The graphical behaviour of \(H(z_{r})\) in redshift with \(b_{1}=10^{13}\), \(\alpha _{2}=2<3\), \(\gamma =\frac{4}{3}\)

Fig. 8
figure 8

The graphical behaviour of \(q(z_{r})\) in redshift with \(B_{1}=2.82\), \(\gamma =\frac{4}{3}\)

C. Stability at infinity and Poincaré sphere:

The detail explanation of Poincaré sphere and behavior at infinity is given in [70]. By using stereographic projection we can study the behavior of trajectories far from origin by considering the so-called Poincaré sphere where we project from the center of the unit sphere \(S^{2}=\{ (X,Y,Z)\in {\mathbb {R}}^{3}:X^{2}+Y^{2}+Z^{2}=1\}\) onto the (xy)-plane tangent to \(S^{2}\) at the north pole [70] by using the transformation of coordinates given by

$$\begin{aligned} x=\frac{X}{Z}, ~ y=\frac{Y}{Z}. \end{aligned}$$
(34)

The equations defining (XYZ) in terms of (xyz) are given by

$$\begin{aligned}&X=\frac{x}{\sqrt{1+x^{2}+y^{2}}}, Y=\frac{y}{\sqrt{1+x^{2}+y^{2}}},\\&Z=\frac{1}{\sqrt{1+x^{2}+y^{2}}}. \end{aligned}$$

The critical points at infinity are mapped on the equator of the Poincaré sphere. We consider the following flow in \({\mathbb {R}}^{2}\):

$$\begin{aligned} x'= & {} \frac{1}{8\pi G}(C_{o}+\beta _{1}x-(\gamma -1)y), \end{aligned}$$
(35)
$$\begin{aligned} y'= & {} -3\gamma y. \end{aligned}$$
(36)

Let \(f(x,y)=\frac{1}{8\pi G}(C_{o}+\beta _{1}x-(\gamma -1)y)\), \(g(x,y)=-3\gamma y\). The degree of this polynomial system is one and let \(f_{1}\) and \(g_{1}\) denotes the homogeneous polynomials in f and g of first degree, that is, \(f_{1}=\frac{1}{8\pi G}(\beta _{1}x-(\gamma -1)y)\), \(g_{1}=-3\gamma y\). In terms of the polar coordinates r, \(\theta \) with \(x=r\cos \theta \), \(y=r\sin \theta \), we can express the above equations as

$$\begin{aligned} r'= & {} \frac{C_{o}cos\theta }{8\pi G}+r\bigg (\bigg (3\gamma +\frac{\beta _{1}}{8\pi G}\bigg )cos^{2}\theta \nonumber \\&-3\gamma +\frac{(\gamma -1)sin2\theta }{16\pi G}\bigg ), \end{aligned}$$
(37)
$$\begin{aligned} \theta '= & {} \frac{-C_{o}sin\theta }{8\pi G}\frac{1}{r}-\frac{\bigg (3\gamma +\frac{\beta _{1}}{8\pi G}\bigg )sin2\theta }{2}\nonumber \\&+\frac{(\gamma -1)sin^{2}\theta }{8\pi G}. \end{aligned}$$
(38)

Order of r in (37) as \(r\rightarrow \infty \) is \(\bar{i}=1\) and that of (38) is \(\bar{j}=0\). Let us denote \(\bar{k}=\bar{i}-\bar{j}=1-0=1\). And using (37), we have

$$\begin{aligned} \lim _{r\rightarrow \infty }r'=\lim _{r\rightarrow \infty }\frac{dr}{d\Theta }=\infty \ne 0. \end{aligned}$$

Then using Theorem 2.1 we find \(G_{2}(\theta )\) which is also equal to the highest power term in r of the \(\Theta '\) expression [86].

$$\begin{aligned} G_{2}(\theta )=\frac{-3\gamma sin2\theta }{2}-\frac{\beta _{1}sin2\theta }{16\pi G}+\frac{(\gamma -1)sin^{2}\theta }{8\pi G}. \end{aligned}$$
(39)

Solving \(\theta \) for which \(G_{2}(\theta )=0\) we get \(\theta =n\pi \), where \(n=0,\pm 1,\pm 2,\ldots \) So we can conclude that \(G_{2}(\theta )\) is not identically equal to zero but it becomes zero in those directions where \(\theta \) takes the value \(n\pi \). Since \(G_{2}(\theta )\) has at most 2 pairs of roots \(\theta \) and \(\theta +\pi \), the equator of the Poincaré sphere has finite number of fixed points located at \(\theta \) such that \(G_{2}(\theta )=0\), that is, at \(\theta =0,\pi ,\pi , 2\pi \) or equivalently \(\theta =0, \pi \). At \(\gamma =0, \frac{4}{3}\) and 2, \(G_{2}(\theta )\) takes the following form:

$$\begin{aligned} G_{2}(\theta )=\left\{ \begin{array}{ll} \frac{-(\alpha _{2}-3)sin2\theta }{2}-\frac{sin^{2}\theta }{8\pi G}, &{}\gamma =0; \\ \frac{-(1+\alpha _{2})sin2\theta }{2}+\frac{sin^{2}\theta }{24\pi G}, &{} \gamma =\frac{4}{3}; \\ \frac{-(3+\alpha _{2})sin2\theta }{2}+\frac{sin^{2}\theta }{8\pi G}, &{} \gamma =2. \end{array} \right. \end{aligned}$$
(40)

The flow on the equator of the Poincaré sphere is counterclockwise at points corresponding to polar angles \(\{\theta : \theta <\tan ^{-1}[(3-\alpha _{2})8\pi G]\}\) where \(G_{2}(\theta )>0\), for example \(\theta =(2n\pi +\frac{\pi }{4}), n=0,\pm 1,\pm 2,\ldots \) with \(\alpha _{2}<3-\frac{1}{8\pi G}\). The flow is clockwise at points corresponding to polar angles \(\{\theta : \theta >\tan ^{-1}((3-\alpha _{2})8\pi G)\}\) where \(G_{2}(\theta )<0\), for example \(\theta =(2n+1)\frac{\pi }{2}\). For \(\gamma =\frac{4}{3}\) the flow on the equator of the Poincaré sphere is counterclockwise at points corresponding to polar angles \(\{\theta : \theta >\tan ^{-1}[(1+\alpha _{2})24\pi G])\}\) where \(G_{2}(\theta )>0\) and the flow is clockwise at points corresponding to polar angles \(\{\theta : \theta <\tan ^{-1}[(1+\alpha _{2})24\pi G]\}\) where \(G_{2}(\theta )<0\). For \(\gamma =2\) the flow is counterclockwise at points corresponding to polar angles \(\{\theta : \theta >\tan ^{-1}((3+\alpha _{2})8\pi G)\}\) where \(G_{2}(\theta )>0\) and the flow is clockwise at those points corresponding to polar angles \(\{\theta : \theta <\tan ^{-1}((3+\alpha _{2})8\pi G)\}\) where \(G_{2}(\theta )<0\).

By Theorem 2.1, the critical points at infinity for the system occur at the points (XY, 0) on the equator of the Poincaré sphere where \(X^{2}+Y^{2}=1\) and

$$\begin{aligned} Xg_{1}(X,Y)-Yf_{1}(X,Y)=0, \end{aligned}$$

where \(f_{1}(x,y)=(\alpha _{2}-3)x-\frac{(\gamma -1)y}{8\pi G}\) and \(g_{1}(x,y)=-3\gamma y\). Using (34), the above equation becomes

$$\begin{aligned} -3\gamma XY-(\alpha _{2}-3)XY+\frac{(\gamma -1)Y^{2}}{8\pi G}=0. \end{aligned}$$
(41)

Solving for X and Y from the above equations, we find that fixed point occurs at \((\pm 1,0,0)\). Also we see from the expression in (41) that for \(\gamma =0\) the flow on the equator of \(S^{2}\) is clockwise for \(XY>0\) and counterclockwise for \(XY<0\). For \(\gamma =\frac{4}{3}\), the flow on the equator of \(S^{2}\) is clockwise for \(XY>0\) and \(-(1+\alpha _{2})XY>\frac{Y^{2}}{24\pi G}\) ; and the flow is counterclockwise for \(XY<0\). For \(\gamma =2\), the flow on the equator of \(S^{2}\) is clockwise for \(XY>0\) and \(-(3+\alpha _{2})XY>\frac{Y^{2}}{8\pi G}\) ; and the flow is counterclockwise for \(XY<0\). Using Theorem 2.2, the behavior in the neighbourhood of the critical point (1, 0, 0) is topologically equivalent to the behavior of the following system,

$$\begin{aligned} y'= & {} yzf\bigg (\frac{1}{z},\frac{y}{z}\bigg )-zg\bigg (\frac{1}{z},\frac{y}{z}\bigg ), \end{aligned}$$
(42)
$$\begin{aligned} z'= & {} z^{2}f\bigg (\frac{1}{z},\frac{y}{z}\bigg ), \end{aligned}$$
(43)

where the overhead dash denotes the derivative with respect to logarithmic time \(\Theta \).

Putting the expressions of fg in (42) and (43) we get

$$\begin{aligned} y'= & {} yz\frac{C_{o}}{8\pi G}+((\alpha _{2}-3)+3\gamma )y-\frac{(\gamma -1)y^{2}}{8\pi G}, \end{aligned}$$
(44)
$$\begin{aligned} z'= & {} \frac{C_{o}z^{2}}{8\pi G}+(\alpha _{2}-3)z-\frac{(\gamma -1)yz}{8\pi G}. \end{aligned}$$
(45)

The Jacobian matrix of the above system is

$$\begin{aligned} J_{inf}(0,0)=\left( \begin{array}{cc} (\alpha _{2}-3)+3\gamma &{} 0 \\ 0 &{} \alpha _{2}-3 \\ \end{array} \right) \end{aligned}$$

This is a diagonal matrix. So the eigenvalues are given by the main diagonal entries, that is, \(m_{1}=(\alpha _{2}-3)+3\gamma \) and \(m_{2}=\alpha _{2}-3\). Both \(m_{1}\) and \(m_{2}\) are negative for \(\alpha _{2}<3\) and \(\gamma <|(\frac{\alpha _{2}-3}{3})|\). So the critical point (1, 0, 0) behaves as a stable attractor which represents the late time accelerated expansion phase of the Universe for \(\alpha _{2}<3\), \(\gamma <|(\frac{\alpha _{2}-3}{3})|\). As we can see when \(\alpha _{2}<3\) and \(\gamma =\frac{4}{3}\), the critical point (1, 0, 0) behaves as a saddle point with \(m_{1}>0\) and \(m_{2}<0\) which is unstable representing the matter dominated phase of the evolving Universe. For \(\alpha _{2}>3\), both \(m_{1}\) and \(m_{2}\) are positive for any \(\gamma \) and the critical point (1, 0, 0) behaves as an unstable repeller representing the inflationary epoch of the evolving Universe. Figures 9 and  10 shows the phase plot of stable attractor as well as the unstable repeller respectively for \(\gamma =0\). Figures 11 and  12 show the phase plot for unstable saddle point and repeller for respectively for \(\gamma =\frac{4}{3}\). Since the degree of f(xy) and g(xy) is odd, the behavior at the antipodal point \((-1,0,0)\) is exactly the same as the behavior at (1, 0, 0).

What is interesting in this model is that in infinite spacetime when we study fixed points at infinity the presence of the critical point (1, 0, 0) indicates the possibility of inflationary phase and it is represented also in the phase plots shown in Figs. 10 and  11 where all the nearby trajectories are repelled away from it.

Fig. 9
figure 9

The phase plot of stable attractor (0, 0) for analysing stability at infinity for case I when \(\gamma =0\), \(\alpha _{2}<3\) taking \(C_{o}=8\pi G=1\)

Fig. 10
figure 10

The phase plot of unstable repeller (0, 0) for analysing stability at infinity for case I when \(\gamma =0\), \(\alpha _{2}>3\) taking \(C_{o}=8\pi G=1\)

Fig. 11
figure 11

The phase plot of unstable saddle point (0, 0) for analysing stability at infinity for case I when \(\gamma =\frac{4}{3}\), \(\alpha _{2}<3\) taking \(C_{o}=8\pi G=1\)

Fig. 12
figure 12

The phase plot of unstable repeller (0, 0) for analysing stability at infinity for case I when \(\gamma =\frac{4}{3}\), \(\alpha _{2}>3\) taking \(C_{o}=8\pi G=1\)

Case II: Dynamical system analysis for \(\dot{G}\ne 0\) and \(\rho _{\Lambda }\) = constant

Let us consider the following Einstein field equations:

$$\begin{aligned} G_{\mu \nu }-g_{\mu \nu }\Lambda =8\pi G, \end{aligned}$$

where \(G_{\mu \nu }=R_{\mu \nu }-\frac{1}{2}g_{\mu \nu }R\) denotes the Einstein tensor.

With general Bianchi identity \(\nabla ^{\mu }G_{\mu \nu }=0\), the above field equation gives the following relation:

$$\begin{aligned} \nabla ^{\mu }({\textbf {T}}_{\mu \nu })=\nabla ^{\mu }[G(T_{\mu \nu }+g_{\mu \nu }\rho _{\Lambda })]=0. \end{aligned}$$

This implies that the local conservation law takes the following form which we named it mixed local conservation law:

$$\begin{aligned} \frac{d}{dt}[G(\rho _{m}+\rho _{\Lambda })]+3GH(\rho _{m}+p_{m})=0. \end{aligned}$$
(46)

If we assume that \(\dot{G}\ne 0 \) and \(\rho _{\Lambda }=\)constant, then the above relation leads to the following equation which indicates a non-conservation of matter as G does not remain constant here:

$$\begin{aligned} \dot{G}(\rho _{m}+\rho _{\Lambda })+G[\dot{\rho _{m}}+3H(\rho _{m}+p_{m})]=0. \end{aligned}$$
(47)

Since we are inclined to qualitative study of the dynamics of the Universe, we set up a dynamical system for case-II by introducing a new variable: \(x=\frac{8\pi G}{3H^{2}}\) and putting \(y=\rho _{m}\).

With these new variables the field equations can be rewritten as

$$\begin{aligned}&~~~ 8\pi G \rho _{tt}\equiv 8\pi G \rho _{m} + \Lambda = 3H^{2} \nonumber \\&\Rightarrow 8\pi G (\rho _{m}+\rho _{\Lambda }) = 3H^{2} \nonumber \\&\Rightarrow \frac{8\pi G}{3H^{2}}(\rho _{m}+\rho _{\Lambda }) =1 \nonumber \\&\Rightarrow x(y+\rho _{\Lambda }) = 1 \nonumber \\&\Rightarrow \frac{1}{x} =y+\rho _{\Lambda }. \end{aligned}$$
(48)

Again using the Taylor series form of \(\Lambda (H)\) in the field equation \(8\pi G \rho _{m} + \Lambda =3H^{2}\), we get

$$\begin{aligned}&~~~~ 8\pi G \rho _{m} + \Lambda = 3H^{2} \nonumber \\&\Rightarrow 8\pi G \rho _{m}+ \Lambda _{o}+\alpha _{2}H^{2} = 3H^{2} \nonumber \\&\Rightarrow \frac{8\pi G\rho _{m}}{3H^{2}}+\frac{\Lambda _{o}}{3H^{2}}+\frac{(\alpha _{2}-3)}{3} =0 \nonumber \\&\Rightarrow \frac{\Lambda _{o}}{3H^{2}}=\frac{(3-\alpha _{2})}{3}-xy. \end{aligned}$$
(49)

Now the dynamical system is represented by the following system of ordinary differential equations:

$$\begin{aligned} x'= & {} \frac{dx}{dt}\frac{dt}{d\Theta } \nonumber \\= & {} \frac{8\pi \dot{G}}{3H^{3}}-\frac{2\dot{H}(8\pi G)}{3H^{4}}. \end{aligned}$$
(50)

Using the expression of \(\dot{G}\),\(\dot{H}\) and \(\frac{\Lambda _{o}}{3H^{2}}\) we have found above, we get

$$\begin{aligned} x'= & {} \frac{-x\Lambda _{o}}{H^{2}}-x(\alpha _{2}-3)+3x^{2}(\gamma -1)y, \nonumber \\= & {} 3\gamma x^{2}y. \end{aligned}$$
(51)

and

$$\begin{aligned} y'= & {} \frac{dy}{d\Theta }\frac{d\Theta }{dt}, \nonumber \\= & {} -3\gamma y. \end{aligned}$$
(52)

In order to find the fixed points we equate \(x'=0\) and \(y'=0\). If \(x'=0\), then either \(y=0\) or \(\gamma =0\) as \(x\ne 0\) otherwise if \(x=0\), then (48) will be violated. Again if \(\gamma =0\) is considered then we get \(y=b\) where b is a real constant and \(x=a\) where \(a,b \in {\mathbb {R}}\) satisfies \(a(b+\rho _{\Lambda })=1\). So the first fixed point we have obtained here is \(P=(a,b)\) where \( a(b+\rho _{\Lambda })=1; a,b \in {\mathbb {R}}\). Now consider \(y=0\) when \(\gamma \ne 0\) then \(x=\frac{1}{\rho _{\Lambda }}\), that is, \(Q=(\frac{1}{\rho _{\Lambda }},0)\) is the second fixed point. In studying the stability of the fixed points, Jacobian matrix of the system plays a leading role. The Jacobian matrix \(J_{2}\) of the system is as follows:

$$\begin{aligned} J_{2}=\left( \begin{array}{cc} 6\gamma xy &{} 3\gamma x^{2} \\ 0 &{} -3\gamma \\ \end{array} \right) . \end{aligned}$$

At the fixed points P, Q, \(J_{2}\) takes the following form respectively:

$$\begin{aligned} J_{P}=\left( \begin{array}{cc} 6\gamma ab &{} 3\gamma (\frac{1}{b+\rho _{\Lambda }})^{2} \\ 0 &{} 0 \\ \end{array} \right) . \end{aligned}$$

Since P is obtained when \(\gamma =0\), \(J_{P}\) becomes a null matrix and hence the eigenvalues of \(J_{P}\) are \(m_{1}=0\), \(m_{2}=0\). The eigenvalues of \(J_{Q}\) are \(m_{3}=0\), \(m_{4}=-3\gamma \). We see that at least one of the eigenvalues vanish at both the fixed points and hence both P and Q are non-hyperbolic. So we need to use the concept of perturbation function as it is easy to analyse the behaviour of the system from the nature of perturbation function expressed in terms of \(\Theta \). As \(\Theta \) tends to \(\infty \), if the perturbation along each of the axes grows then the fixed point is unstable whereas if the perturbation along each of the axes decays to zero or evolves to a constant value, then the fixed point is stable. We shall not employ Center manifold theory for two dimensional problems as it is simpler to use the method of perturbation function, but for higher dimensional problems as using Center manifold theory is one of the prominent tools to study stability of a system, we have also shown in the later part, namely, Case III of this section how the dynamics of the center manifold determines the dynamics of the entire system.

A. Stability analysis using the concept of Spectral radius of the Jacobian matrix at the respective fixed points:

The spectral radius of \(J_{P}\) and \(J_{Q}\) are given by

$$\begin{aligned} \sigma _{P}=0<1, \sigma _{Q}=max\{|-3\gamma |,0\}=\bigg \{\begin{array}{cc} 3\gamma &{} , \gamma >0, \\ 0 &{} ,\gamma =0. \end{array}. \end{aligned}$$

Since \(\sigma _{P}<1\), all the eigenvalues of \(J_{P}\) lie inside a unit disc. So P is stable. When \(\gamma >0\), \(\sigma _{Q}<1\) if \(\gamma <\frac{1}{3}\) and \(\sigma _{Q}=1\) if \(\gamma =\frac{1}{3}\). So, Q is stable for \(0\le \gamma <\frac{1}{3}\) and we can’t say whether Q is stable or not if \(\gamma =\frac{1}{3}\). In addition when \(\gamma =\frac{1}{3}\) one eigenvalue of \(J_{Q}\),namely, \(-3\gamma \), has absolute value equal to one, the other eigenvalue, that is, zero has absolute value less than one. In this case a bifurcation may occur where a small change in the parameter values of the system leads to a sudden qualitative change in terms of topological behavior of the system. We need to further our study from the concept of perturbations along each of the axes and study the behaviour of perturbations when \(\Theta \rightarrow \infty \).

B. Stability analysis using the concept of Perturbation function:

Let \(x=x_{P}+\eta _{x}\) and \(y=y_{P}+\eta _{y}\), where \(x_{P}\), \(y_{P}\) are the values of xy at P and \(\eta _{x}, \eta _{y}\) are small perturbations along x-axis and y-axis respectively. Putting the perturbed value of x and y in the dynamical system equations (51) and (52) leads to the following relations:

$$\begin{aligned} \eta _{x}= & {} c_{1},\\ \eta _{y}= & {} c_{1}e^{-3\gamma \Theta }-b, \end{aligned}$$

where \(c_{1}\) is an arbitrary constant. Similarly, at fixed point Q we get

$$\begin{aligned} \eta _{x}= & {} c_{2}, \\ \eta _{y}= & {} c_{2}e^{-3\gamma \Theta }, \end{aligned}$$

where \(c_{2}\) is an arbitrary constant.

As \(\Theta \) increases and tends to \(\infty \), \(\eta _{y}\) for P evolves to a constant value for all \(\gamma \in [0,2]\) and \(\eta _{y}\) for Q also converges to zero for all \(\gamma \in [0,2]\). Since the perturbation along each axis does not grow with the increase in \(\Theta \), P is stable for all \(\gamma \in [0,2]\), in particular for \(\gamma =0\). When \(\gamma \ne 0\) \(\eta _{y}\rightarrow -b\) as \(\Theta \rightarrow \infty \) but if we directly put \(\gamma =0\) in the expression of \(\eta _{y}\) above, \(\eta _{y}\) becomes a constant function, \(\eta _{y}=c_{1}-b\). Figure 13 shows the variation of perturbation along y-axis , \(\eta _{y}\) with respect to \(\Theta \) as \(\gamma \rightarrow 0^{+}\) for the fixed point P. From Fig. 13 we see that as \(\gamma \rightarrow 0\) from the right the curves gradually tends to \(\eta _{y}=c_{1}-b\). Figure 14 shows that \(\eta _{y}\) decreases exponentially as \(\Theta \) increases and ultimately decays to zero as \(\Theta \) tends to \(\infty \) for Q for any positive value of \(\gamma \). So it is obvious that \(\eta _{y}\rightarrow 0\) as \(\Theta \rightarrow \infty \) for \(\gamma <\frac{1}{3}\) as determined from the concept of spectral radius. So Q is also no doubt stable for all \(0<\gamma < \frac{1}{3}\). The relative energy density \(\Omega _{tt}=\Omega _{m}+\Omega _{\Lambda }\), where \(\Omega _{m}=xy\), \(\Omega _{\Lambda }=\frac{\Lambda _{o}}{3H^{2}}+\frac{\alpha _{2}}{3}=1-xy\). We have calculated the value of effective equation of state parameter \(\omega _{eff}\) as follows:

$$\begin{aligned} \omega _{eff}=\frac{p_{tt}}{\rho _{tt}}=-1-\gamma xy. \end{aligned}$$
(53)
Fig. 13
figure 13

Variation of \(\eta _{y}\) with respect to \(\Theta \) for fixed point P as \(\gamma \rightarrow 0^{+}\)

Fig. 14
figure 14

The variation of \(\eta _{y}\) with respect to \(\Theta \) for Q at \(\gamma =\frac{1}{4}<\frac{1}{3}\)

Table 2 Table for Case II (\(\dot{G}\ne 0\), \(\rho _{\Lambda }\)=constant)

As a function of redshift parameter \(z_{r}\), we can express \(\omega _{de}\), Hubble parameter \(H(z_{r})\) and deceleration parameter \(q(z_{r})\) respectively as follows:

$$\begin{aligned} \omega _{de}=-1-\gamma xy\bigg (\frac{y}{\rho _{\Lambda }}+1\bigg )-\frac{\gamma y}{\rho _{\Lambda }}. \end{aligned}$$
(54)

This implies,

$$\begin{aligned} \omega _{de}(z_{r})= & {} \left\{ \begin{array}{ll} -1, &{} \gamma =0; \\ -1-\frac{2E_{22}\gamma (1+z_{r})^{3\gamma }}{8\rho _{\Lambda }}, &{} \gamma \ne 0, \end{array} \right. \end{aligned}$$
(55)
$$\begin{aligned} H(z_{r})= & {} \left( \frac{8\pi \gamma }{3E_{21}}\bigg (1+\frac{\rho _{\Lambda }}{E_{22}\gamma (1+z_{r})^{3\gamma }}\bigg )e^{\frac{(1+z_{r})^{\ln { E_{24}-3\gamma }}}{E_{23}E_{22}e^{(\ln (1+z_{r})^{\sqrt{\frac{3\gamma (3-\alpha _{2})}{2}}}})^{2}}}\right) ^{\frac{1}{2}}, \end{aligned}$$
(56)
$$\begin{aligned} q(z_{r})= & {} -1-\frac{3\gamma }{2(\gamma (1+z_{r})^{3\gamma }+1)}+ \frac{(1+z_{r})^{(\ln E_{26}-3\gamma -1)}}{2E_{25}e^{(\ln (1+z_{r})^{\sqrt{\frac{3\gamma (3-\alpha _{2})}{2}}}})^{2}}(\ln E_{26}-3\gamma -3\gamma (\alpha _{2}-3)\ln (1+z_{r})), \end{aligned}$$
(57)

where \(E_{i}'s\) are arbitrary constants.

The expression for transition redshift is obtained as

$$\begin{aligned} z_{rt}=\{z_{r}: q(z_{r})=0\} \end{aligned}$$
(58)

At the fixed point P, we get \(\omega _{de}=-1-\frac{2\gamma b}{\rho _{\Lambda }}\). When \(\gamma =0\) \(\omega _{de}\) at P becomes \(-1\) and at Q \(\omega _{de}=-1\) with \(\Omega _{tt}=1\) for both P and Q which is in agreement with the observational data in [81]. Since \(\omega _{de}\) is found to be negative unity, the presence of the stable fixed point P indicates the presence of negative pressure in the developed cosmological model which contributes to our model with an accelerated expansion phase of the Universe. We tabulate the results in Table 2.

The plot for \(\omega _{de}\) versus \(z_{r}\) is shown in Fig. 15. It is observed that the present epoch is dominated by phantom dark energy as \(\omega _{de}<-1\) at present. Then as \(z_{r}\) approaches the fixed point value \(-1\), \(\omega _{de}\) tends to a fixed point value \(\omega _{de}\simeq -1\) for any value of \(\gamma \). As \(\omega _{de}\simeq -1\), this indicates the complete dominance of the Universe in late time with purely cosmological-constant type dark energy which drives the Universe to expand with acceleration and continue expanding as cosmic time tends to infinity. This phenomena is again supported from Figs. 16 and  17 where we observe that the Hubble parameter increases with the increase in time while the deceleration parameter decreases with time and ultimately takes negative value. Since \(q(z_{r})=0\) indicates the position of the transition point from early decelerated expansion to current accelerated expansion in the Universe, using (58) we obtain the value of transition redshift \(z_{rt}\simeq 0.72\) and this is supported from Fig. 17 also. The present values of these parameters are tabulated in Table 4 and these results are in agreement with the cosmological observations.

Case III: Dynamical system analysis for \(\dot{G}\ne 0\) and \(\dot{\rho _{\Lambda }}\ne 0\)

In this case both G and \(\rho _{\Lambda }\) are no longer constants, that is, \(\dot{G}\ne 0\) and \(\dot{\rho _{\Lambda }}\ne 0\). There are many possibilities here. We consider the simplest one by assuming the standard local covariant conservation of matter-radiation, that is, \(\dot{\rho _{m}}+3H(\rho _{m}+p_{m})=0\) [37], the relation in (46) leads to the following:

$$\begin{aligned} \dot{G}(\rho _{m}+\rho _{\Lambda })+G\dot{\rho _{\Lambda }}=0. \end{aligned}$$
(59)
Fig. 15
figure 15

The plot for EoS, \(\omega _{de}\) versus redshift, \(z_{r}\) for the cosmological model in case II for the considered value of \(E_{2}=\rho _{\Lambda }\) and \(\gamma =\frac{1}{4}<\frac{1}{3}\)

Fig. 16
figure 16

The graphical behaviour of \(H(z_{r})\) over \(z_{r}\) with \(\alpha _{2}=2\), \(\gamma =\frac{1}{4}<\frac{1}{3}\), \(E_{21}=0.14\), \(E_{22}=\rho _{\Lambda }\), \(E_{23}=\frac{239}{\rho _{\Lambda }}\), \(E_{24}=9\) and \(\alpha _{2}=3.000005\)

Fig. 17
figure 17

The graphical behaviour of \(q(z_{r})\) over \(z_{r}\) with \(\gamma =\frac{1}{4}<\frac{1}{3}\), \(E_{25}=0.69\), \(E_{26}=2.42E_{24}\) and \(\alpha _{2}=3.000005\)

We introduce the following new variables xyz to set up the corresponding dynamical system such that \(x=\frac{8\pi G}{3H^{2}}\), \(y=\rho _{m}\), \(z=\rho _{\Lambda }\). We take derivative of the newly introduced variables with respect to logarithmic time, \(\Theta \) and obtain the following relations:

$$\begin{aligned} x'= & {} \dot{x}\frac{dt}{d\Theta }, \\= & {} \frac{1}{H}\frac{d}{dt}\bigg (\frac{8\pi G}{3H^{2}}\bigg ), \\= & {} \frac{8\pi \dot{G}}{3H^{3}}-\frac{2(8\pi G \dot{H})}{3H^{4}}. \end{aligned}$$

Using (59) in the above equation and the necessary substitutions we get

$$\begin{aligned} x'= & {} x^{2}y'+3(3\gamma -1)x^{2}y-(\alpha _{2}-3)x, \end{aligned}$$
(60)
$$\begin{aligned} y'= & {} \dot{\rho _{m}}\frac{dt}{d\Theta }, \nonumber \\= & {} \frac{1}{H}\dot{\rho _{m}}, \nonumber \\= & {} \frac{1}{H}(\dot{-\rho _{\Lambda }}-3\gamma H\rho _{m}), \nonumber \\= & {} -z'-3\gamma y, \end{aligned}$$
(61)
$$\begin{aligned} z'= & {} \dot{\rho _{\Lambda }}\frac{dt}{d\Theta }, \nonumber \\= & {} \bigg (6-16\pi \gamma +\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}\bigg )y+ \nonumber \\&\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}-\frac{16\pi }{3}\bigg )z-\frac{3y}{x^{2}}. \end{aligned}$$
(62)

Putting the above expression of \(z'\) in (61), we get the expression of \(y'\) as follows:

$$\begin{aligned} y'= & {} \bigg (-6+16\pi \gamma -3\gamma -\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}\bigg )y \nonumber \\&-\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}\bigg )-\frac{16\pi }{3}\bigg )z+\frac{3y}{x^{2}}. \end{aligned}$$
(63)

Finally putting the value of \(y'\) above in (60), we get the expression of \(x'\) as follows:

$$\begin{aligned} x'= & {} -(\alpha _{2}-3)x+3y+\bigg (-9+16\pi \gamma +6\gamma \nonumber \\&+\bigg (2-\frac{8\pi }{3}\bigg )\alpha _{2}\bigg )x^{2}y \nonumber \\&-\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}-\frac{16\pi }{3}\bigg )x^{2}z. \end{aligned}$$
(64)

We equate \(x'=0\), \(y'=0\), \(z'=0\) using (64), (63) and (62) to obtain the fixed points. As \(y\rightarrow 0\), \(z\rightarrow 0\), then since xyz holds the relation \(\frac{1}{y+z}=x\), x must tend to infinity. If we view from the sequential approach of real analysis, any real sequence of the form \(\frac{1}{n}\) converges to zero as \(n\rightarrow \infty \) but never equals to zero. For every \(\epsilon >0\) there exist a positive integer m such that \(|\frac{1}{n}-0|<\epsilon \) for all \(n\ge m\), that is, in every neighbourhood of zero there contains infinite members of the sequence \(\frac{1}{n}\). Similarly when \(n\rightarrow 0\), \(\frac{1}{n}\rightarrow \infty \). So as \(y\rightarrow 0\), \(z\rightarrow 0\) x must tends to infinity. To ensure that the fixed points obtained are physically feasible with the developed system, \(\alpha _{2}\) must be equal to 3 and with this consideration we can analyse our fixed points in the finite phase plane. Let us consider \(x'=0\), \(y'=0\), \(z'=0\) at \(\alpha _{2}=3\), then as \(y\rightarrow 0.0009\), \(z\rightarrow 0\), x must also tends to a number, \(l=\frac{1}{(0.0009+0)}=1111\). Let this fixed point be denoted by \(S=(x\rightarrow l,y\rightarrow 0.0009,z\rightarrow 0)\).

Stability of the above fixed points is determined by the eigenvalues of the Jacobian matrix \(J_{3}\) of the above dynamical system which is obtained as follows:

$$\begin{aligned} J_{3} = \begin{pmatrix} 2\bigg (-9+16\pi \gamma +6\gamma +\bigg (2-\frac{8\pi }{3}\bigg )\alpha _{2}\bigg )xy &{} 3+\bigg (-9+16\pi \gamma +6\gamma &{} -\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2} \\ -\alpha _{2}+3-2\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}-\frac{16\pi }{3}\bigg )xz &{} +\bigg (2-\frac{8\pi }{3}\bigg )\alpha _{2}\bigg )x^{2} &{} -\frac{16\pi }{3}\bigg )x^{2}\\ &{} &{} \\ -6\frac{y}{x} &{} \bigg (-6+(16\pi -3)\gamma -\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}\bigg ) &{} -\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}-\frac{16\pi }{3}\bigg ) \\ &{} +\frac{3}{x^{2}} &{} \\ &{} &{} \\ 6\frac{y}{x} &{} \bigg (6-16\pi \gamma +\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}\bigg )-\frac{3}{x^{2}} &{} \bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}-\frac{16\pi }{3}\bigg ) \end{pmatrix} \end{aligned}$$

The above matrix is a \(3\times 3\) matrix. The eigenvalues of \(J_{3}\) at the fixed point determines the stability of the fixed point. At S when \(\gamma =0\), \(J_{3}\) takes the following form:

$$\begin{aligned}&J_{3}(S)\\&\quad =\left( \begin{array}{ccc} -(\alpha _{2}-3) &{} 3+(-9-6.37\alpha _{2})l^{2} &{} -(-10.74+6.37\alpha _{2})l^{2} \\ 0 &{} (-6-6.37\alpha _{2})+\frac{3}{l^{2}}&{} -(-10.74+6.37\alpha _{2}) \\ 0 &{} (6+6.37\alpha _{2})-\frac{3}{l^{2}} &{} (-10.74+6.37\alpha _{2}) \\ \end{array} \right) \end{aligned}$$

The above matrix is a \(3\times 3\) matrix with eigenvalues 0, \(-16.74\), \(-(\alpha _{2}-3)=0\). Since some of the eigenvalues becomes zero, S is a non-hyperbolic fixed point. We analyse stability through perturbation function and Center manifold theory as it is a three dimensional problem with the fixed point as non-hyperbolic one and using these methods are more suitable.

A. Stability analysis for S using the concept of Perturbation function :

We perturb the system by a small amount putting \(x=x_{F}+\eta _{x}, y=y_{F}+\eta _{y}, z=z_{F}+\eta _{z}\) where \(x_{F},y_{F},z_{F}\) represent the values of xyz at the fixed point to be analysed for stability and \(\eta _{x},\eta _{y},\eta _{z}\) denote the perturbations along xyz axes respectively. With these perturbed values in the dynamical system equations (64), (63) and (62) and necessary substitutions, we obtain the following perturbations as a function of logarithmic time \(\Theta \):

$$\begin{aligned} \eta _{x}= & {} \left\{ \begin{array}{ll} C_{1}e^{-(\alpha _{2}-3)\Theta }-1, &{} \hbox {for any } \gamma ; \\ C_{1}-1, &{} \hbox {for any } \gamma \text { and } ~ \alpha _{2}=3. \end{array} \right. \end{aligned}$$
(65)
$$\begin{aligned} \eta _{y}= & {} \left\{ \begin{array}{ll} C_{2}e^{-(6+6.4\alpha _{2})\Theta }, &{} \gamma =0; \\ C_{2}e^{(57-6.4\alpha _{2})\Theta }, &{} \gamma =\frac{4}{3}; \\ C_{2}e^{(88.5-6.4\alpha _{2})\Theta }, &{} \gamma =2. \end{array} \right. \end{aligned}$$
(66)
$$\begin{aligned} \eta _{z}= & {} \left\{ \begin{array}{ll} C_{3}e^{(-10.7+6.4\alpha _{2})\Theta }, &{} \hbox {for any } \gamma ; \\ C_{3}e^{8.5\Theta }, &{} \hbox {for any} \gamma \text { and } \alpha _{2}=3. \end{array} \right. \end{aligned}$$
(67)

where \(C_{i}, i\in \kappa \) are arbitrary constants and \(\kappa \) is the index set.

Let \(\Phi =\{\alpha _{2}: \eta _{x} \rightarrow 0\) or c, \(\eta _{y}\rightarrow 0\) or c, \(\eta _{z}\rightarrow 0\) or c as \(\Theta \rightarrow \infty \), where \(c \in {\mathbb {R}}\) is any real constant }. If we consider only the expression of \(\eta _{x}\) obtained as a function of \(\Theta \) regardless of restricting the value of \(\alpha _{2}\), then we can see that when \(\Theta \rightarrow \infty \), \(\eta _{x}\rightarrow C_{1}-l\) for \(\alpha _{2}=3\), \(\eta _{x}\rightarrow -l\) for \(\alpha _{2}>3\), \(\eta _{y}\rightarrow C_{2}\) for any positive value of \(\alpha _{2}\). Similarly it is seen that \(\eta _{z}\) exponentially increases for \(\alpha _{2}>1.67\). So we fail to obtain such value of \(\alpha _{2}\) where all of these \(\eta _{x},\eta _{y},\eta _{z}\) decay or evolve to a constant value as \(\Theta \) tends to infinity. So \(\Phi \) is an empty set. Only when all of these \(\eta _{x}\), \(\eta _{y}\) and \(\eta _{z}\) decay to zero or tends to a constant value when \(\Theta \rightarrow \infty \), we can conclude that the fixed point is stable otherwise unstable if at least one of them go on increasing as \(\Theta \rightarrow \infty \). For S to be stable \(\Phi \) should not be an empty set. Figures 18,  19 and  20 show the perturbation plots for S at \(\gamma =0\). From Fig. 18, as \(\alpha _{2}\rightarrow 3^{-}\), the slope of the curve gradually decreases and as \(\alpha _{2}\) becomes exactly equal to 3, the slope of the curve equals zero and then as \(\alpha _{2}\) becomes just greater than 3, \(\eta _{x}\) becomes an exponentially decreasing function of \(\Theta \). So when \(\alpha _{2}>3\) as \(\Theta \rightarrow \infty \), \(\eta _{x}\) exponentially decreases and evolves to a constant value, namely, \(-l\). Figure 19 shows that \(\eta _{y} \rightarrow 0\) as \(\Theta \rightarrow \infty \) for \(\gamma =0\) and any value of \(\alpha _{2}\). But from Fig. 20 it is clear that when \(\alpha _{2}\ge 3\), \(\eta _{z}\) exponentially increases as \(\Theta \) increases and continue to grow as \(\Theta \rightarrow \infty \). So S is unstable for any value of \(\alpha _{2}\). Hence, S is unstable for \(\alpha _{2}=3\) also. In this case III, we have already presumed \(\alpha _{2}\) to be equal to 3 in order to ensure that the fixed point S obtained above is in the finite phase plane and remain physically feasible with respect to the dynamical system we have set up. So using the above arguments we conclude that S is unstable from the side of perturbation function. We will also show the use of Center manifold theory in determining the stability of the fixed point S. Using Center manifold theory is one of the most rigorous mathematical technique to determine stability for non-hyperbolic fixed points as the nature of orbits on a center manifold reflects the nature of the system in the neighbourhood of the fixed point. To use Center manifold theory we need to transform the dynamical system equations into the standard form to study Center manifold theory. We know that \(S(x\rightarrow l,y\rightarrow 0.0009,z\rightarrow 0)\) is a non-hyperbolic fixed point. Now using a suitable coordinate transformation we need to transformed the system in the required standard form to apply Center manifold theory for it will not change the nature of the fixed point. We present how to analyse stability using the Center manifold theory in the following section.

Fig. 18
figure 18

Variation of \(\eta _{x}\) with respect to \(\Theta \) for S

Fig. 19
figure 19

Variation of \(\eta _{y}\) with respect to \(\Theta \) for S at \(\gamma =0\)

Fig. 20
figure 20

The variation of \(\eta _{z}\) with respect to \(\Theta \) for S at \(\gamma =0\)

Fig. 21
figure 21

Variation of \(\eta _{y}\) and \(\eta _{z}\) at \(\gamma =\frac{4}{3}\) and \(\alpha _{2}=3\)

Fig. 22
figure 22

Variation of \(\eta _{x}\) with respect to \(\Theta \) for S when \(\gamma =\frac{4}{3}\)

Fig. 23
figure 23

The variation of \(\eta _{z}\) with respect to \(\Theta \) at S at \(\gamma =2\)

B. Stability analysis for S using Center manifold theory:

Firstly, we need to transform the dynamical system equations into the form required to use Center manifold theory. For this we need to shift the fixed point to origin (0, 0, 0) by doing suitable coordinate transformation as follows:

$$\begin{aligned} X=x-l, Y=y-0.0009, Z=z; \end{aligned}$$

In terms of this new coordinates our dynamical system equations (64), (63)and (62) with \(\alpha _{2}=3\) can be written as follows:

$$\begin{aligned}&\left( \begin{array}{c} X' \\ Y' \\ Z' \\ \end{array} \right) = \left( \begin{array}{ccc} -0.05 l &{} (3-28.11 l^{2}) &{} -8.37 l^{2} \\ 0 &{} -25.11 &{} -8.3 \\ &{} 25.11 &{} 8.3 \\ \end{array} \right) \left( \begin{array}{c} X \\ Y \\ Z \\ \end{array} \right) \\&+ \left( \begin{array}{c} f(X,Y,Z). \\ g_{1}(X,Y,Z).\\ g_{2}(X,Y,Z).\\ \end{array} \right) , \end{aligned}$$

where

$$\begin{aligned}&f(X,Y,Z)=-0.025X^{2}-28.11X^{2}Y-8.37X^{2}Z\\&-(56.22 l)XY-(16.74 l)XZ, \\&g_{1}(X,Y,Z)=\frac{3(Y+0.0009)}{(X+l)^{2}}, \\&g_{2}(X,Y,Z)=-\frac{3(Y+0.0009)}{(X+l)^{2}}. \end{aligned}$$

The Jacobian matrix of the above system at origin is

$$\begin{aligned} J_{(X=0,Y=0,Z=0)}= \left( \begin{array}{ccc} -0.05 l &{} 3-28.11 l^{2} &{} -8.3 l^{2} \\ -\frac{0.0054}{4l^{2}} &{} -25.11 &{} -8.3 \\ \frac{0.0054}{4l^{2}} &{} 25.11 &{} 8.3 \\ \end{array} \right) \end{aligned}$$

The above Jacobian matrix has zero determinant which means at least one of the eigenvalues has become zero. To find the eigenvalues say \(m_{i}\) we solve the characteristic equation \(det(J_{X=0,Y=0,Z=0}-mI)=0\) and obtain \(m_{1}=0, m_{2}=-0.05 l, m_{3}=-16.81\). The minimal polynomial that annihilates \(J_{X=0,Y=0,Z=0}\) is given by \(m(m+0.05 l)(m+16.81)\). As the linear factors occur exactly once in the minimal polynomial, \(J_{(X=0,Y=0,Z=0)}\) is diagonalisable. To diagonalise \(J_{(X=0,Y=0,Z=0)}\) to obtain the required form to use Center manifold theory, we need to find the stable subspace \(E^{s}\) generated by the eigenbasis associated with the negative eigenvalues, the center subspace \(E^{c}\) generated by the eigenbasis associated with the zero eigenvalue of above Jacobian matrix. The eigenspace associated with zero eigenvalue can be found out by solving for \(x_{1},x_{2},x_{3}\) in the following matrix equation:

$$\begin{aligned} (J-(0)I_{3\times 3})\left( \begin{array}{c} x_{1} \\ x_{2} \\ x_{3} \\ \end{array} \right) =O_{3\times 3}, \end{aligned}$$

where \(I_{3\times 3}\) and \(O_{3\times 3}\) represents the identity matrix and null matrix respectively. Solving the above equations we get the eigenbasis as

$$\begin{aligned} E^{c}=\left\{ \left( \begin{array}{c} -728 l\\ -1 \\ 1 \\ \end{array} \right) \right\} \end{aligned}$$

Similarly we find the eigenbasis associated with the eigenvalues \(-0.05 l\) and \(-\) 16.81 so that we can write stable subspace \((E^{s})\) as follows:

$$\begin{aligned} E^{s}=\left\{ \left( \begin{array}{c} -592 l^{2} \\ -0.3 \\ 1 \\ \end{array} \right) , \left( \begin{array}{c} 196 l \\ -1 \\ 1 \\ \end{array} \right) \right\} \end{aligned}$$

Both \(E^{c}\) and \(E^{s}\) are the subspaces of \({\mathbb {R}}\times {\mathbb {R}} \times {\mathbb {R}}\). Let us define a matrix P whose column vectors are formed by the above eigenbases as follows:

$$\begin{aligned} P=\left( \begin{array}{ccc} -728 l &{} -592 l^{2} &{}196 l \\ -1 &{} -0.3 &{} -1 \\ 1 &{} 1 &{} 1 \\ \end{array} \right) \end{aligned}$$

P is a non-singular matrix with \(det(P)=-646.8 l\). So P is invertible matrix with \(P^{-1}\) as \(P^{-1}=\frac{1}{det(P)}Adj(P)\), where Adj(P) denotes the adjoint of P. Therefore

$$\begin{aligned} P^{-1}=\left( \begin{array}{ccc} \frac{-0.7}{646.8 l} &{} -(0.9 l+0.3) &{} 0.53 \\ 0 &{} 1.4 &{} 1.4 \\ \frac{0.7}{646.8 l} &{} 0.9 l &{} 0.9 l \\ \end{array} \right) . \end{aligned}$$

We again define a new co-ordinate transformation as:

$$\begin{aligned} P\left( \begin{array}{c} U \\ V \\ W \\ \end{array} \right) =\left( \begin{array}{c} X \\ Y\\ Z \\ \end{array} \right) , \end{aligned}$$

that is,

$$\begin{aligned} P^{-1}\left( \begin{array}{c} X \\ Y \\ Z \\ \end{array} \right) =\left( \begin{array}{c} U \\ V\\ W \\ \end{array} \right) . \end{aligned}$$

In terms of the new coordinates U, V, W; X, Y and Z can be expressed as follows:

\(X=-728 l U-592 l^{2}V+196 l W\), \(Y=-U-0.3V-W\), \(Z=U+V+W.\)

The definition of center manifold allows us to take \(h_{1}\) and \(h_{2}\) in Taylor’s series form as \(V=h_{1}(U)=a_{1}U^{2}+a_{2}U^{3}\) and \(W=h_{2}(U)=b_{1}U^{2}+b_{2}U^{3}\) so that \(h_{1}(0)=h_{1}(0)=0\) and \(Dh_{1}(0)=Dh_{2}(0)=0\), where \(D=\frac{d}{dU}\).

We then obtain the required standard form to apply Center manifold theory as follows:

\( \left( \begin{array}{c} U' \\ V' \\ W' \\ \end{array} \right) =\left( \begin{array}{ccc} 0 &{} 0 &{} 0 \\ 0 &{} -0.05 l &{} 0 \\ 0 &{} 0 &{} -16.81 \\ \end{array} \right) \left( \begin{array}{c} U \\ V \\ W \\ \end{array} \right) \qquad \qquad \quad +P^{-1}\left( \begin{array}{c} f(U,V,W) \\ g_{1}(U,V,W) \\ g_{2}(U,V,W) \end{array} \right) \),

where

$$\begin{aligned} f(U,V,W)= & {} (-13249 l-40928 l+12186 l^{2})U^{2}\\&+\bigg (-21548 l^{3}a_{1} +7134l^{2}b_{1}+14897850l^{2}\\&-12278a_{1} -40928lb_{1}-33282l^{3}a_{1}\\&+11019l^{2}b_{1} -4435966l^{2}+12186l^{2}a_{1}\\&+12186b_{1} +9910l^{3}a_{1}-3281l^{2}a_{1}\bigg )U^{3},\\ g_{1}(U,V,W)= & {} -3\frac{-U-0.3V-W}{(-728l U-592l^{2}V+196lW)^{2}}, \\ g_{2}(U,V,W)= & {} 3\frac{U+0.3V+W}{(-728l U-592l^{2}V+196lW)^{2}}. \end{aligned}$$

Now computing the above equations we obtain the following relations:

$$\begin{aligned} U'= & {} \frac{-0.7}{646.8l}\bigg \{(-13249 l-40928 l +12186 l^{2})U^{2}\nonumber \\&+\bigg (-21548 l^{3}a_{1} +7134 l^{2}b_{1}+14897850 l^{2}\nonumber \\&-12278a_{1}-40928lb_{1} -33282l^{3}a_{1}+11019l^{2}b_{1} \nonumber \\&-4435966l^{2}+12186l^{2}a_{1}+12186b_{1}\nonumber \\&+9910l^{3}a_{1}-3281l^{2}a_{1}\bigg )U^{3}\bigg \}, \end{aligned}$$
(68)
$$\begin{aligned} V'= & {} -0.05la_{1}U^{2}-0.05la_{2}U^{3}, \end{aligned}$$
(69)
$$\begin{aligned} W'= & {} -16.81(b_{1}U^{2}+b_{2}U^{3})\nonumber \\&+\frac{0.7}{646.8l}\bigg \{(-13249 l-40928 l+12186 l^{2})U^{2} \nonumber \\&+\bigg (-21548 l^{3}a_{1}+7134 l^{2}b_{1}+14897850l^{2}\nonumber \\&-12278a_{1}-40928lb_{1}-33282l^{3}a_{1} \nonumber \\&+11019l^{2}b_{1}-4435966l^{2}+12186l^{2}a_{1}\nonumber \\&+12186b_{1}+9910l^{3}a_{1}-3281l^{2}a_{1}\bigg )U^{3}\bigg \}. \nonumber \\ \end{aligned}$$
(70)

The dynamics of the center manifold is given by:

$$\begin{aligned} U'=AU+f(U,h_{1}(U),h_{2}(U)), \end{aligned}$$

where \(A=0\), \(V=h_{1}(U),W=h_{2}(U)\).

The tangency condition requires that

$$\begin{aligned} V'-\frac{dh_{1}}{dU}U'= & {} 0, \end{aligned}$$
(71)
$$\begin{aligned} W'-\frac{dh_{2}}{dU}U'= & {} 0. \end{aligned}$$
(72)

By equating the coefficients of \(U^{2}\) and \(U^{3}\) in the tangency conditions (71) and (72), we can find the constants \(a_{1},a_{2}\) and \(b_{1}, b_{2}\) where we unconsider all the powers of U higher than \(U^{3}\). Equating the coefficients of \(U^{2}\) and \(U^{3}\) in the tangency condition of V, we get \(a_{1}=a_{2}=0\) and from the tangency conditions of W comparing the coefficient of \(U^{2}\), we get

$$\begin{aligned}&~~~ -16.81b_{1}+\bigg (\frac{0.7}{646.8}\bigg )(-54177+12186 l) = 0 \\&\Rightarrow b_{1} = \frac{1}{-16.81}(58.6-13.2 l) \\&\Rightarrow b_{1} = -3.5+0.8 l. \end{aligned}$$

Since l is a very large number, \(b_{1}\sim 0.8l\) and comparing the coefficient of \(U^{3}\) we get

$$\begin{aligned}&~~~ -26.4 b_{1} l = -16.81b_{2}+\frac{0.7}{646.8 l}(18153l^{2}\\&\quad -40928l+12186)b_{1} \\&\Rightarrow 16.81 b_{2} = 36.8l^{2}-35.4l+10.6 \\&\Rightarrow b_{2} = 2l^{2}-2l+0.6. \end{aligned}$$

Putting the values of \(a_{1},a_{2},b_{1},b_{2}\) in the dynamics of center manifold we get

$$\begin{aligned} U'=j_{1}U^{2}+j_{2}U^{3}+{\mathbb {O}}(U^{4}), \end{aligned}$$
(73)

where \(j_{1}=(-54177l+12186l^{2})\) and \(j_{2}=(14522l^{2}-32742l+9748)(2l^{2}-2l+0.6)\).

Since the first term of \(U'\) is in even power of U, we deduce instability. If suppose \(j_{1}=0\) then we will consider the next term which is in the odd power of U. Here if \(j_{2}\) is negative then, it is stable otherwise if it is positive then we again achieve instability. But in our case \(j_{1}\) never equals zero. So from the side of Center manifold theory we conclude that the fixed point S is unstable.

Now when we take \(\gamma =\frac{4}{3}\) then, (64), (63) and (62) becomes

$$\begin{aligned} x'= & {} -(\alpha _{2}-3)x+3y+(6-6.4\alpha _{2})x^{2}y\\&-(-10.7+6.4\alpha _{2})x^{2}z,\\ y'= & {} (57-6.4\alpha _{2})y-(-10.7+6.4\alpha _{2})z+3y(y+z)^{2}, \\ z'= & {} (-61+6.4\alpha _{2})y+(-10.7+6.4\alpha _{2})z-3y(y+z)^{2}. \end{aligned}$$

Now when \(\gamma =\frac{4}{3}\) we have the Jacobian matrix at S as follows:

$$\begin{aligned} J_{3}(S)=\left( \begin{array}{ccc} -(\alpha _{2}-3) &{} 3-13.2 l^{2} &{} 0 \\ 0 &{} (57-6.4\alpha _{2})&{} -(-10.7+6.4\alpha _{2}) \\ 0 &{} (-61+6.4\alpha _{2}) &{} (-10.7+6.37\alpha _{2}) \\ \end{array} \right) \end{aligned}$$

Since we obtain S when \(\alpha _{2}=3\), we get the eigenvalues as \(m_{1}=(3-\alpha _{2})=0\), \(m_{2}=12.8(1.8-\sqrt{3+0.15\alpha _{2}})=-0.7\) and \(m_{3}=12.8(1.8+\sqrt{3+0.15\alpha _{2}})=46.8>0\). So for \(\gamma =\frac{4}{3}\) at \(\alpha _{2}=3\), S becomes non hyperbolic fixed point. We need to analyse stability through perturbation function and Center manifold theory. However stability analysis using Center manifold theory is similar to the above shown. So we will only analyse through perturbation function. From (65),(66) and (67), we see that for \(\alpha _{2}=3\) \(\eta _{x}\) tends to a constant, namely, \((C_{1}-l)\) as \(\Theta \rightarrow \infty \) but \(\eta _{y}\) exponentially increases as \(\Theta \rightarrow \infty \). \(\eta _{z}\) is also an exponentially increasing function of \(\Theta \) and hence it fails to decay or evolve to a constant value as \(\Theta \rightarrow \infty \). Figure 21 shows the exponential increasing nature of \(\eta _{y}\) and \(\eta _{z}\) at \(\gamma =\frac{4}{3}\), \(\alpha _{2}=3\). Figure 22 shows the perturbation plot for \(\eta _{x}\) as \(\Theta \) tends to infinity. So S is unstable at \(\alpha _{2}=3\) and \(\gamma =\frac{4}{3}\). As the perturbation along each of the axes fail to decay or evolve to a constant value we conclude that S is also unstable for \(\gamma =\frac{4}{3}\). For \(\gamma =2\) also we can see from (67) that the perturbation along z axis, namely, \(\eta _{z}\) is an exponentially increasing function of \(\theta \). So S is unstable for any value of \(\alpha _{2}\) for \(\gamma =2\) and this is shown in Fig. 23 also.

C. Stability at infinity and Poincaré sphere:

Any polynomial system in rectangular coordinates can be extended to the Poincaré sphere [86]. So the idea of projective geometry done in the case of \({\mathbb {R}}^{2}\) can be extended to higher dimensions for flows in \({\mathbb {R}}^{3}\) also. Here, the upper hemisphere of \(S^{3}\) can be projected onto \({\textbf {R}}^{3}\) using the transformation of coordinates given by \(x=\frac{X}{W}\), \(y=\frac{Y}{W}\), \(z=\frac{Z}{W}\) and \(X=\frac{x}{\sqrt{1+|x|^{2}}}\), \(Y=\frac{y}{\sqrt{1+|x|^{2}}}\), \(Z=\frac{z}{\sqrt{1+|x|^{2}}}\) and \(W=\frac{1}{\sqrt{1+|x|^{2}}}\) for \({\textbf {X}}=(X,Y,Z,W)\in S^{3}\) with \(|{\textbf {X}}|=1\) and for \({\textbf {x}}=(x,y,z)\in {\mathbb {R}}^{3}\). Now we consider the dynamical system equations (64), (63) and (62) in the following way:

$$\begin{aligned} \left. \begin{array}{c} x' =P_{1}(x,y,z), \\ y' =P_{2}(x,y,z), \\ z' =P_{3}(x,y,z), \end{array}\right\} \end{aligned}$$
(74)

where

$$\begin{aligned} P_{1}(x,y,z)= & {} -(\alpha _{2}-3)x+3y+\bigg (-9+16\pi \gamma +6\gamma \\&+\bigg (2-\frac{8\pi }{3}\bigg )\alpha _{2}\bigg )x^{2}y\\&-\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}-\frac{16\pi }{3}\bigg )x^{2}z, \\ P_{2}(x,y,z)= & {} \bigg (-6+16\pi \gamma -3\gamma -\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2})y-\\&\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}\bigg )-\frac{16\pi }{3}\bigg )z+3y(y+z)^{2},\\ P_{3}(x,y,z)= & {} \bigg (6-16\pi \gamma +\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}\bigg )y+\\&\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}-\frac{16\pi }{3}\bigg )z-3y(y+z)^{2}. \end{aligned}$$

We have used the relation \(\frac{1}{x}=(y+z)\) in (63) and (62) above for our convenience with polynomial functions of x, y, z with maximum degree 3 on the right side of (74). Let us denote the maximum degree terms in \(P_{1}\), \(P_{2}\) and \(P_{3}\) by \(\bar{P}_{1}\), \(\bar{P}_{2}\) and \(\bar{P}_{3}\) respectively. Then we have,

$$\begin{aligned} \left. \begin{array}{c} \bar{P}_{1}(x,y,z) =\bigg (-9+16\pi \gamma +6\gamma +\bigg (2-\frac{8\pi }{3}\bigg )\alpha _{2}\bigg )x^{2}y \\ -\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}-\frac{16\pi }{3}\bigg )x^{2}z,\\ \bar{P}_{2}(x,y,z) = 3y(y+z)^{2}, \\ \bar{P}_{3}(x,y,z) =-3y(y+z)^{2}. \end{array}\right\} \nonumber \\ \end{aligned}$$
(75)

In terms of X, Y, Z we express the above polynomials as follows:

$$\begin{aligned} \left. \begin{array}{c} \bar{P}_{1}(X,Y,Z) =\bigg (-9+16\pi \gamma +6\gamma +\bigg (2-\frac{8\pi }{3}\bigg )\alpha _{2}\bigg )X^{2}YW^{-3} \\ -\bigg (6+\bigg (\frac{8\pi }{3}-2\bigg )\alpha _{2}-\frac{16\pi }{3}\bigg )X^{2}ZW^{-3}, \\ \bar{P}_{2}(X,Y,Z) = 3Y(Y+Z)^{2}W^{-3}, \\ \bar{P}_{3}(X,Y,Z) =-3Y(Y+Z)^{2}W^{-3}. \end{array}\right\} \nonumber \\ \end{aligned}$$
(76)

Theorem 2.3 determines the location of the critical points at infinity for the above polynomial system by considering the following equations:

$$\begin{aligned}&~~~ X \bar{P}_{2}(X,Y,Z)-Y\bar{P}_{1}(X,Y,Z) = 0 \nonumber \\&\Rightarrow 3(Y+Z)^{2}-(-9+56.24\gamma -6.4\alpha _{2})XY\nonumber \\&+(6.4\alpha _{2}-10.74)XZ=0. \end{aligned}$$
(77)
$$\begin{aligned}&~~~ X \bar{P}_{3}(X,Y,Z)-Z\bar{P}_{1}(X,Y,Z) = 0\nonumber \\&\Rightarrow -3Y(Y+Z)^{2}-(-9+56.24\gamma -6.4\alpha _{2})XYZ\nonumber \\&+(6.4\alpha _{2}-10.74)XZ^{2}=0. \end{aligned}$$
(78)
$$\begin{aligned}&~~~ Y \bar{P}_{3}(X,Y,Z)-Z\bar{P}_{2}(X,Y,Z) = 0 \nonumber \\&\Rightarrow 3Y(Y+Z)^{2}(-Y-Z)= 0 \nonumber \\&\Rightarrow \text{ either }~~ Y=0~~\text{ or }~~Y=-Z. \end{aligned}$$
(79)

If \(Y=0\) then from (78) we get either \(X=0\) or \(Z=0\). If \(Y=0\) and \(X=0\) is considered then from (77) we see that \(Z=0\). But since \(X^{2}+Y^{2}+Z^{2}=1\) must hold, the condition \(X=0\) is neglected. If \(Z=0\) when \(Y=0\) in \(X^{2}+Y^{2}+Z^{2}=1\), we get \(X=\pm 1\). So \((\pm 1,0,0,0)\) is a fixed point. Also from (79) if we consider \(Y=-Z\), then from (78) we get either \(X=0\) or \(Z=0\). If \(X=0\) then \(X^{2}+Y^{2}+Z^{2}=1\) does not hold. So when \(Y=-Z\) and \(Z=0\) then \((\pm 1,0,0,0)\) is a fixed point. Using Theorem 4 we obtain that the flow defined by (74) in a neighbourhood of \((\pm 1,0,0,0)\in S^{3}\) is topologically equivalent to the flow defined by the following system:

$$\begin{aligned} \left. \begin{array}{c} \pm y' = (\alpha _{2}-3)yw^{2}+3y^{2}w^{2}-(-6+(16\pi -3)\gamma -6.4\alpha _{2})yw^{2}\\ +\bigg (6+6.4\alpha _{2}-\frac{16\pi }{3}\bigg )zw^{2}+(-9+56.24\gamma -6.4\alpha _{2})y^{2}\\ -(-10.74+6.4\alpha _{2})zy-3y(y+z)^{2},\\ \pm z' = (\alpha _{2}-3)zw^{2}+3yzw^{2}+(-9+56.24\gamma -6.4\alpha _{2})yz\\ -(-10.74+6.4\alpha _{2})z^{2}-(6-16\pi \gamma +6.4\alpha _{2})yw^{2} \\ -(-10.74+6.4\alpha _{2})zw^{2}+3y(y+z)^{2},\\ \pm w' = (\alpha _{2}-3)w^{3}+3yw^{3}+(-9+56.24\gamma -6.4\alpha _{2})yw-(-10.74+6.4\alpha _{2})zw.\\ \end{array}\right\} \end{aligned}$$
(80)

The Jacobian matrix of the above system at the fixed point (0, 0, 0) is a null matrix which has all the eigenvalues as zero. So it is a non-hyperbolic fixed point. We will analyse the stability by finding perturbation functions along each of the axes as a function of logarithmic time \(\Theta \) by perturbing the system (80) by a small amount. If the system comes back to the fixed point following the perturbation then the system is stable otherwise if the perturbation grows to make the system moves away from the fixed point then the system is unstable. Nandan Roy and Narayan Banerjee [87] has also used the concept of perturbation function to analyse stability for non-hyperbolic fixed points for three dimensional systems where linear stability fails. Now firstly consider the expression of (80) corresponding to \(+y\), \(+z\) and \(+w\) respectively. Then we perturb our system (80) by taking \(y=\eta _{y}\), \(z=\eta _{z}\) and \(w=\eta _{w}\).

$$\begin{aligned}&~~~ \frac{d\eta _{y}}{d\Theta } = (-9+56.24\gamma -6.4\alpha _{2})\eta _{y}^{2}-3\eta _{y}^{3}, \\&\Rightarrow d\Theta = \frac{d\eta _{y}}{(-9+56.24\gamma -6.4\alpha _{2})\eta _{y}^{2}-3\eta _{y}^{3}}, \\&\Rightarrow d\Theta =\frac{Ad\eta _{y}}{\eta _{y}}+\frac{Bd\eta _{y}}{\eta _{y}^{2}}+\\&~~~~~~~~~~~~~ \frac{Cd\eta _{y}}{(-9+56.24\gamma -6.4\alpha _{2})-3\eta _{y}}, \end{aligned}$$

where

$$\begin{aligned}&A=\frac{3}{(-9+56.24\gamma -6.4\alpha _{2})^{2}}, \\&B=\frac{1}{(-9+56.24\gamma -6.4\alpha _{2})}, \\&C=\frac{9}{(-9+56.24\gamma -6.4\alpha _{2})^{2}}. \end{aligned}$$

Integrating both sides of the above differential equation, we get

$$\begin{aligned} \Theta =f(\eta _{y})=\frac{C}{3} \ln \bigg (\frac{\eta _{y}}{(k-3\eta _{y})}\bigg )-\frac{B}{\eta _{y}}, \end{aligned}$$
(81)

where \(k=(-9+56.24\gamma -6.4\alpha _{2})\). The domain of definition \(D_{\Theta }\) of the above function at \(\gamma =0\) is

$$\begin{aligned} D_{\Theta _{0}}=\bigg (-\infty ,\frac{k}{3}\bigg ); k=-9-6.4\alpha _{2}, \alpha _{2} \in {\mathbb {R}}^{+}. \end{aligned}$$

The domain of definition \(D_{\Theta }\) of the above function at \(\gamma =\frac{4}{3}\) and \(\gamma =2\) respectively are as follows:

$$\begin{aligned} D_{\Theta _{\frac{4}{3}}}= & {} \left\{ \begin{array}{ll} (0,\frac{k}{3}), &{} \alpha _{2}<10.29, k=65.9-6.4\alpha _{2}>0;\\ (-\infty ,\frac{k}{3}), &{} \alpha _{2}>10.29, k=65.9-6.4\alpha _{2}<0 \end{array} \right. \\ D_{\Theta _{2}}= & {} \left\{ \begin{array}{ll} (0,\frac{k}{3}), &{} \alpha _{2}<16.15,k=103.4-6.4\alpha _{2}>0;\\ (-\infty ,\frac{k}{3}), &{} \alpha _{2}>16.15, k=103.4-6.4\alpha _{2}<0 \end{array} \right. \end{aligned}$$

With the above domain and the choice of \(+y\) on the left side of (80), we cannot analyse our system for \(\Theta \rightarrow \infty \) as \(\Theta \) becomes bounded above and unbounded below as \(\eta _{y}\) tends to 0, that is, when \(\Theta \rightarrow -\infty \), \(\eta _{y}\rightarrow 0\). Since we want to analyse the late time behaviour of the Universe as logarithmic time \(\Theta \rightarrow \infty \) we only consider the expressions of (80) corresponding to \(-y\), \(-z\) and \(-w\) on the left sides of (80) as follows:

$$\begin{aligned} -y'= & {} (\alpha _{2}-3)yw^{2}+3y^{2}w^{2}-(-6+(16\pi -3)\gamma \\&-6.4\alpha _{2})yw^{2}+\bigg (6+6.4\alpha _{2}-\frac{16\pi }{3}\bigg )zw^{2}\\&+(-9+56.24\gamma -6.4\alpha _{2})y^{2}\\&-(-10.74+6.4\alpha _{2})zy-3y(y+z)^{2},\\ -z'= & {} (\alpha _{2}-3)zw^{2}+3yzw^{2}+(-9+56.24\gamma -6.4\alpha _{2})yz\nonumber \\&-(-10.74+6.4\alpha _{2})z^{2}-(6-16\pi \gamma +6.4\alpha _{2})yw^{2}\nonumber \\&-(-10.74+6.4\alpha _{2})zw^{2}+3y(y+z)^{2},\\ -w'&=(\alpha _{2}-3)w^{3}+3yw^{3}+(-9+56.24\gamma -6.4\alpha _{2})yw\\&-(-10.74+6.4\alpha _{2})zw. \end{aligned}$$

With this consideration we get the expression of \(\Theta \) as a function of \(\eta _{y}\) as follows:

$$\begin{aligned} \Theta =f((\eta _{y})=\frac{C}{3}\ln \frac{k-3\eta _{y}}{\eta _{y}}+\frac{B}{\eta _{y}}. \end{aligned}$$

When \(\Theta \rightarrow \infty \), \(f(\eta _{y}) \rightarrow \infty \) which implies \(\eta _{y}\rightarrow 0\). So as \(\Theta \rightarrow \infty \) the perturbation along y-axis decays to zero. For analysing the perturbation along z and w axes we consider the expression for \(+z\) and \(+w\) from (80) and find out the expressions of \(\eta _{z}\) and \(\eta _{w}\) as follows:

$$\begin{aligned} \eta _{z}= & {} \frac{1}{(-10.74+6.4\alpha _{2})\Theta + c_{1}} ; \alpha _{2}\ne 1.6, \\ \eta _{w}= & {} \pm \frac{1}{\sqrt{2(3-\alpha _{2})\Theta +2c_{2}}}, \end{aligned}$$

where \(c_{1}\) and \(c_{2}\) are arbitrary constants of integration. As \(\Theta \) tends to infinity both \(\eta _{z}\) and \(\eta _{w}\) tends to zero. Figures 24,  25 and  26 show the projection of perturbations along y, z and w axes respectively for system (80). Since all of \(\eta _{y}\), \(\eta _{z}\) and \(\eta _{w}\) decay to zero as \(\Theta \) tends to infinity, we conclude that the fixed point \((\pm 1,0,0,0)\) is a stable critical point.

Fig. 24
figure 24

The variation of \(\Theta \) with respect to \(\eta _{y}\) for analysing stability at infinity for case III

Fig. 25
figure 25

The variation of \(\eta _{z}\) with respect to \(\Theta \) for analysing stability at infinity for case III

Fig. 26
figure 26

The variation of \(\eta _{w}\) with respect to \(\Theta \) for analysing stability at infinity for case III

Fig. 27
figure 27

The plot for \(\omega _{de}\) with respect to z taking \(\alpha _{2}=3.05>3\) and \(\gamma =\frac{4}{3}\) for \(E_{3}=\frac{1}{2.8}<\frac{1}{e}\), \(e=2.7\) for cosmological model associated with case III

Fig. 28
figure 28

The graphical nature of \(H(z_{r})\) over \(z_{r}\) for the cosmological model associated with case III. The free parameters \(E_{5}=1.45\), \(\gamma =\frac{4}{3}\), \(\alpha _{2}=4>3\) and \(c_{1}=0.0085\), \(E_{4}=0.0005\) and \(E_{6}=3.04\)

pagination

Fig. 29
figure 29

The graphical nature of \(q(z_{r})\) over \(z_{r}\) for the cosmological model associated with case III. The free parameters \(E_{5}=1.45\), \(\gamma =\frac{4}{3}\), \(\alpha _{2}=4>3\) and \(c_{1}=0.0085\), \(E_{4}=0.0005\) and \(E_{6}=3.04\)

The expressions of total energy density \(\Omega _{tt}\) and effective equation of state \(\omega _{eff}\) in terms of the variables xyz are as follows:

$$\begin{aligned} \Omega _{tt}= & {} xy+\frac{z}{y+z}, \end{aligned}$$
(82)
$$\begin{aligned} \omega _{eff}= & {} \frac{p_{tt}}{\rho _{tt}}, \end{aligned}$$
(83)
$$\begin{aligned}= & {} \frac{(\gamma -1)y-z}{y+z}. \end{aligned}$$
(84)

The EoS parameter for dark energy sector \(\omega _{de}\), the Hubble parameter \(H(z_{r})\) and the deceleration parameter, \(q(z_{r})\) expressed as functions of redshift \(z_{r}\) are as follows:

$$\begin{aligned} \omega _{de} = \gamma y\bigg (x\bigg (1+\frac{y}{z}\bigg )-\frac{1}{z}\bigg )-1, \end{aligned}$$
(85)

This implies,

$$\begin{aligned} \omega _{de}= & {} \left\{ \begin{array}{ll} -1.006, &{} \alpha _{2}=3; \\ \ln E_{3}(1+z_{r})^{-\gamma (\alpha _{2}-3)}\bigg (\frac{\alpha _{2}-3}{6\gamma +6-\alpha _{2}}+1\bigg ), &{} \alpha _{2}\ne 3. \end{array} \right. \end{aligned}$$
(86)
$$\begin{aligned} H(z_{r})= & {} \bigg (\frac{E_{5}^{\frac{1}{\gamma }}e^{\bigg (\bigg (1-\frac{1}{\gamma }\bigg )-\ln 8\pi C_{1}-\ln E_{4}-\ln E_{6}\ln (1+z_{r})+\ln (1+z_{r})^{\sqrt{\frac{3\gamma (\alpha _{2}-3)}{2}}}}\bigg )}{3(1+z_{r})^{(\alpha _{2}-3)}}\bigg )^{\frac{1}{2}}, \end{aligned}$$
(87)
$$\begin{aligned} q(z_{r})= & {} \frac{(1+z_{r})^{-\alpha _{2}+3}}{6}((3\gamma \ln (1+z_{r})-1)(\alpha _{2}-3)-3\ln E_{5}), \end{aligned}$$
(88)

where \(E_{i}'s\) are arbitrary constants.

The transition redshift can be obtained by using the following relations:

$$\begin{aligned} z_{rt}=\{z_{r}: q(z_{r})=0\} \end{aligned}$$
(89)

The graphical natures of cosmological parameters \(\omega _{de}\), \(H(z_{r})\) and \(q(z_{r})\) are shown in Figs. 27,  28 and  29 respectively. From the plot, we find that the curve for \(\omega _{de}\) evolves from the phantom phase and remains in the phantom region at present where \(\omega _{de}<-1\) and then it approaches the purely cosmological constant type dark energy dominated epoch where \(\omega _{de}=-1\) and finally in the late time the Universe is dominated by quintessence where \(\omega _{de}>-1\). At present time when \(z_{r}=0\) assuming the present value of scale factor to be unity, we find the present value of the EoS parameter as \(\omega _{de}\simeq -1.061<-1\) which clearly depicts the phantom-like behaviour at present. When \(\alpha _{2}=3\), the value of \(\omega _{de}=-1\) at present time as well as at late time, this indicates that the Universe is associated with purely cosmological constant type dark energy and hence the model behaves as a cosmological constant model. Also the curve for \(H(z_{r})\) monotonically increases as time increases while the curve for \(q(z_{r})\) decreases as time increases with the transition redshift \(z_{rt}\) found to be \(z_{rt}\simeq 0.70\). This means as \(z_{r}\) passes this point \(z_{rt}\) the Universe begins to undergo accelerated expansion with deceleration parameter value being negative. The present value of deceleration parameter \(q(z_{ro})\) is obtained as -0.35 and the present values of these parameters, namely, \(\omega _{de}(z_{ro})\), \(H(z_{ro})\), \(q(z_{ro})\) are tabulated in Table 4 and they all are in agreement with the cosmological observations. Hence the present model associated with case III supports the fact that the Universe is currently in accelerated expansion epoch and will continue to expand in the late time.

Table 3 Table for Case III (\(\dot{G}\ne 0\), \(\dot{\rho _{\Lambda }}\ne 0\))
Table 4 Table for present values of cosmological parameters

4 Conclusion

In this work we have presented three accelerating cosmological models through dynamical systems approach in the presence of a time varying cosmological constant term under three different cases based on the possibilities of G and \(\rho _{\Lambda }\). We use dynamical systems approach and find the fixed points to analyse stability as well as late time behavior of the evolving Universe. The cosmological parameters, namely, \(\omega _{de}(z_{r}), H(z_{r}), q(z_{r}), z_{rt}\) are expressed in terms of redshift \(z_{r}\) and their graphical behaviour with the change in redshift have been analysed. The cosmological models associated with case I, case II and case III show phantom behaviour at the present time with \(\omega _{de}<-1\). It is found that both analytical and geometrical findings agree with the fact that the Universe is in the accelerated expansion phase with deceleration–acceleration transition taking place at values of \(z_{r}\simeq 0.72\) which are also tabulated in Table 4 for all of the three cases and they are quite consistent with the spatially flat \(\Lambda CDM\) model, the standard model of cosmology where the cosmological constant \(\Lambda \) is the dark energy. Along with the dynamical system analysis in the finite phase plane what is more interesting in this work is the stability analysis of the systems for critical points at infinity by using the concept of Poincaré sphere where we use stereographic projection to study the behavior of trajectories far from origin.

In case I where both G and \(\rho _{\Lambda }\) are taken to be constants, we observe that the time varying cosmological constant model can be represented by a two dimensional dynamical system. It has three fixed points \(F_{1}, F_{2}, F_{3}\) in the finite phase plane out of which \(F_{1}\) behaves as a late time attractor as shown in Fig. 1 when \(\alpha _{2}<3\) with \(\omega _{de}=-1\) and \(\Omega _{tt}=1\). The presence of this stable attractor in the cosmological model associated with case I assures the presence of negative pressure representing the accelerated expansion phase of the Universe.Also more interestingly, when we analyse stability for the critical points at infinity using poincarè sphere, the critical point (1, 0, 0) behaves as an unstable repeller when \(\alpha _{2}>3\) for any value of \(\gamma \). This repelling behaviour is also shown in the phase plots of Figs. 10 and  12. The presence of this repeller adds up to this model with the inflationary phase of the evolving Universe. The EoS parameter for dark energy sector remains in the phantom region at present with \(\omega _{de}(z_{ro})\simeq -1.029<-1\) as seen from Fig. 6 while \(H(z_{r})\) increases monotonically with the increase in time with \(H(z_{ro})\simeq 73.5\) (shown in Fig. 7). As seen from Fig. 8, the deceleration parameter \(q(z_{r})\) decreases monotonically and vanishes at \(z_{rt}\simeq 0.71\) and remains negative after this transition point. It is at this point where the transition from early decelerated expansion to current accelerated expansion in the Universe takes place and this shows that the model associated with this case remains close to \(\Lambda CDM\). The present values of these parameters are tabulated in Table 4 and they all are in agreement with the cosmological observations. Thus, the behaviours of these cosmological parameters depict the accelerated expansion epoch of the model Universe behaving as phantom-like at present time.

In case II when G no longer remains constant, we obtain two non-hyperbolic fixed points P and Q both of which are stable attractors for \(0\le \gamma <\frac{1}{3}\) with the numerical values of \(\omega _{de}\) and \(\Omega _{tt}\) obtained as − 1 and 1 respectively. From the expression of \(\omega _{de}\) as a function of redshift \(z_{r}\) along with the graphical interpretation shown in Fig. 15, it is observed that as the redshift parameter \(z_{r}\) approaches − 1, \(\omega _{de}\) tends to a fixed point value \(\sim -1\). So this particular cosmological model associated with case II shows the dominance of a purely cosmological-constant type dark energy [80] in the late time. Presently, this model shows a phantom-like phase with present value of EoS found to be \(\omega _{de}(z_{ro})\simeq -1.06<-1\). The Hubble parameter also shows a monotonically increasing nature with the increase in cosmic time with the present value of Hubble parameter noted to be \(H(z_{ro})\simeq 75\). In this model the transition from deceleration regime to acceleration regime takes place at \(z_{rt}\simeq 0.72\). After this transition point the value of \(q(z_{r})\) remains negative which depicts that the model Universe will continue to expand with acceleration in the late time also.

However in case III, when both G and \(\rho _{\Lambda }\) are taken to be non-constants, we find that we can extend the system to a three dimensional dynamical system problem. Here we obtain a fixed point S in the finite phase plane which is physically feasible with the developed dynamical system with the value of \(\alpha _{2}\) presumed to be equal to 3. From the side of perturbation function approach as well as Center manifold theory, we get S to be unstable fixed point for any value of \(\gamma \). But the main thing that distinguishes this approach is the presence of the stable fixed point \((\pm 1,0,0,0)\) at infinity lying on the northpole of the Poincaré sphere \(S^{3}\) with the use of projective geometry being carried over to higher dimension in \({\mathbb {R}}^{3}\). This fixed point behaves as a stable attractor in infinite spacetime. The model associated with this case evolves from the phantom-phase and gradually approaches the quintessence region in late time (shown in Fig. 27). At present it behaves as phantom-like with \(\omega _{de}\simeq -1.061<-1\). This model also supports the fact that the Universe is in acceleration era as we can see from Figs. 28 and  29 where \(H(z_{r})\) tends to infinity as \(z_{r}\) tends to \(-1\) while \(q_(z_{r})\) transits from positive to negative at \(z_{r}\simeq 0.70\). This point represents the transition redshift \(z_{rt}\) where the transition from early decelerated expansion to present accelerated expansion of the Universe takes place. Throughout the entire work we find that the developed cosmological models associated with case I, case II and case III strongly support the accelerated expansion phenomena thereby depicting that the Universe is in the phase of expansion with acceleration and will continue to expand in late time also.