1 Introduction

The renormalization structure in a curved space-time is well known at both general and perturbative levels. In particular, it is well known that any theory which is renormalizable in flat space remains renormalizable in curved space [1] (see also [2] for a recent review). The necessary elements of the consistent quantum theory in a curved space-time are the purely gravitational vacuum action, which consists of the Einstein–Hilbert term with cosmological constant and also of the four fourth-derivative terms. On top of that, if the theory under discussion has scalar fields \(\varphi _i\), new non-minimal terms of the form \(\xi _{ij} R\varphi _i \varphi _j\) have to be included in the action. The renormalization group (RG) in curved space-time was introduced in [3–6] (see also [1]) as a useful tool to explore the scaling properties of the theory.

The renormalization and RG in curved space follow some important hierarchy, which means that: (i) The RG equations for the matter fields couplings and masses do not depend on \(\xi _{ij}\) and on the parameters of the vacuum action. More general, these equations are not affected by the presence of external gravitational field.Footnote 1 (ii) The RG equations for \(\xi _{ij}\) depend on the matter fields couplings (but not on the masses of the fields, in the case of the minimal subtraction scheme of renormalization), but they do not depend on the parameters of the vacuum action.

(iii) The RG equations for the parameters of the vacuum action may depend on the couplings (beyond one-loop approximation) and on \(\xi _{ij}\).

One has to note that the running of \(\xi _{ij}\) may have some important applications, especially to inflationary models such as Higgs inflation [10], because this running is closely related to the effective potential of the Higgs field in curved space [1] (see also [9]). The same concerns also other inflationary models, including the ones based on inflation, and Starobinsky inflation [11] and especially its modified version [12]. Therefore, it would be quite useful to know whether the non-minimal parameter can experience a strong running at some moment of the history of the universe. One of the possibilities to observe an intensive running of \(\xi \) is related to the non-perturbative effects in the framework of the functional RG (FRG) approach, developed by Polchinski and Wetterich [13–15] (see also [16] for a similar original derivation and [17–20] and [21] for reviews and an introduction to the subject). In the present Letter we present the FRG equations in curved space, in a background-independent covariant way similar to what has been done before for the perturbative RG in curved space. The FRG approach on a fixed de Sitter background has been previously considered in [22–24] and [25].Footnote 2 The present paper is essentially restricted to the case of a single scalar and hence to the equation for a single parameter \(\xi \). We consider first the local potential approximation (LPA), dealing with the most simple theory with unbroken symmetry, and then we explore the more complicated case with the broken symmetry and wave-function renormalization. In fact, the extension of the RG flow for the broken phase is especially interesting, because the running of the non-minimal parameter in this case was not sufficiently well explored even in the perturbative approach. Some potentially interesting consequences of the RG flow for the non-minimal parameter are related to the scale dependence of the non-local parts of the induced gravitational action, which emerge due to the curvature dependence of the vacuum expectation value of the scalar field [26].

In the parallel work [27] we will also consider generalizations like the theory with a more general form \(f(\phi )R\) of non-minimal interaction at quantum level and in different dimensions.

The paper is organized as follows. In Sect. 2 we describe the general scheme of FRG in a curved space-time and especially derivation of the RG equation for \(\xi \). Section 3 is devoted to the numerical analysis of the equation for \(\xi \). Section 4 describes the FRG in the scalar theory with broken symmetry. Finally, in Sect. 5 we draw our conclusions.

2 FRG for scalar field with non-minimal coupling

The renormalizable theory of a single scalar \(\phi \) in a curved space starts from the classical action of the form

$$\begin{aligned} S =\int \limits _x \,\left[ -\frac{1}{2}\phi \triangle _g\phi +\frac{\xi }{2}R\phi ^2+V(\phi )\right] +S^\mathrm {grav}[g], \end{aligned}$$
(1)

where we assumed Euclidean signature and use the notation \(\int _x \equiv \int \mathrm{d}^4 x\sqrt{g(x)}\). Furthermore, \(S^\mathrm {grav}[g]\) corresponds to the vacuum action as described in the Introduction. We expect to discuss the FRG flow for the vacuum part in a separate article, so it will not be seriously dealt with in the present Letter or in [27]. \(V(\phi )\) is a classical potential, which may be restricted to the form \((1/4!)\lambda \phi ^4\) in the case we intend to remain within the scope of perturbatively renormalizable theories.

As usual in the RG approach, at quantum level all quantities start to depend on the scale which we identify as k. The practical use of the FRG approach implies the choice of the truncation scheme, which we choose in a most simple way, assuming that the effective average action is

$$\begin{aligned} \varGamma _k =\int \limits _x \left[ -\frac{Z_k}{2}\phi \triangle _g\phi +\frac{\xi _k}{2}R\phi ^2+u_k(\phi )\right] + \varGamma _k^\mathrm {grav}[g]. \end{aligned}$$
(2)

This truncation includes a scale-dependent effective potential \(u_k\), a wave-function renormalization \(Z_k\) and the running non-minimal parameter \(\xi _k\), which does not depend on the momenta or on the field \(\phi \). The invariant cutoff action has the form

$$\begin{aligned} \varDelta S_k = \frac{1}{2} \int \limits _x \,\phi R_k(-\triangle _g)\phi , \end{aligned}$$
(3)

where

$$\begin{aligned} R_k(-\triangle _g)=Z_k r_k(-\triangle _g). \end{aligned}$$

\(R_k\) is assumed to have the well-known properties of a cutoff function [17–20]. The anomalous dimension is defined as

$$\begin{aligned} \eta _k = -\frac{k\partial _k Z_k}{Z_k}=-\frac{\partial _t Z_k}{Z_k}, \quad \mathrm{where}\ t=\log \frac{k}{\mu }. \end{aligned}$$
(4)

When the scale k runs from the UV-cutoff \(\varLambda \) to the IR, the dimensionless scale parameter t runs from \(\log (\varLambda /\mu )\) to \(-\infty \).

The Wetterich equation for the scale dependent effective average action reads [14–16]

$$\begin{aligned} \partial _t \varGamma _k[\phi ]=\frac{1}{2}\,\text{ Tr }\,\left( \frac{\partial _t R_k}{\varGamma ^{(2)}_k[\phi ]+R_k}\right) , \end{aligned}$$
(5)

where \(\varGamma ^{(2)}_k\) indicates a second variational derivative with respect to the scalar field and \(\,\text{ Tr }\,\) includes the coincidence limit and covariant integration over the space-time variables. For the truncation (2) the l.h.s. becomes

$$\begin{aligned} \partial _t\varGamma _k= & {} \int \limits _x \left[ -\frac{\eta _k Z_k}{2}\phi \triangle _g\phi +\frac{\partial _t\xi _k}{2}R\phi ^2+\partial _t u_k(\phi )\right] \nonumber \\&+\,\partial _t\varGamma _k^\mathrm {grav}[g]. \end{aligned}$$
(6)

In order to derive the r.h.s. of (5), we need

$$\begin{aligned} \varGamma ^{(2)}_k =-Z_k\triangle _g+\xi _k R+u''_k(\phi ), \end{aligned}$$
(7)

where prime means simple derivative with respect to scalar field. The variation of the cutoff function can be cast into the form

$$\begin{aligned} \partial _t R_k=Z_k\left( \partial _t r_k-\eta _k r_k\right) . \end{aligned}$$
(8)

Then the r.h.s. of the flow equation (5) takes the form

$$\begin{aligned}&\frac{1}{2}\!\! \,\text{ Tr }\,\left( \frac{\partial _t R_k}{\varGamma ^{(2)}_k[\phi ]+R_k}\right) \nonumber \\&\quad =\frac{1}{2}\!\!\,\text{ Tr }\,\left[ \frac{(\partial _t-\eta _k)r_k(-\triangle _g)}{(-\triangle _g+r_k(-\triangle _g)) + \xi _k Z_k^{-1} R + Z_k^{-1} u''_k(\phi )}\right] .\nonumber \\ \end{aligned}$$
(9)

Equation (5) with (6) and (9) represents the covariant flow equation corresponding to the truncation (2). It can be improved by including higher derivative terms into (6), but then the calculations of (9) should also be evaluated up to the corresponding higher order of approximation.

2.1 Elaborating the Wetterich equation

It proves useful to define

$$\begin{aligned} u_k(\phi )=\frac{m_k^2}{2}\phi ^2+w_k(\phi ), \end{aligned}$$
(10)

such that

$$\begin{aligned} u_k''(\phi )=m_k^2+w''_k(\phi ). \end{aligned}$$
(11)

Thus we arrive at the following form of Eq. (9):

$$\begin{aligned} \frac{1}{2}\,\text{ Tr }\,\left( \frac{\partial _t R_k}{\varGamma ^{(2)}_k[\phi ]+R_k}\right) =\frac{1}{2}\,\text{ Tr }\,\left( \frac{B_k(-\triangle _g)}{P_k(-\triangle _g)+\varSigma _k}\right) , \end{aligned}$$
(12)

where we introduced the abbreviations

$$\begin{aligned} B_k(-\triangle _g)= & {} \left( \partial _t - \eta _k \right) r_k(-\triangle _g), \end{aligned}$$
(13)
$$\begin{aligned} P_k(-\triangle _g)= & {} -\triangle _g+r_k(-\triangle _g)+\frac{m_k^2}{Z_k}, \end{aligned}$$
(14)
$$\begin{aligned} \varSigma _k(\phi ,R)= & {} \frac{\xi _k}{Z_k} R+\frac{1}{Z_k}w''_k(\phi ). \end{aligned}$$
(15)

In order to analyze the flow equation in the truncation (2), we need to evaluate the expression (12) up to the first order in scalar curvature, while the terms with derivatives of curvature and higher powers of the curvature tensor can be disregarded. This means that we can effectively consider an approximation with constant R.

It is easy to note that the operators \(B_k\) and \(P_k\) commute. But for an inhomogeneous field and curvature the space-time-dependent \(\varSigma _k\) does not commute with \(B_k\) and \(P_k\). But they commute in the constant curvature and constant \(\phi \) approximation, the latter corresponds to the local potential approximation (LPA).

To simplify notations in what follows we skip the arguments of \(B_k,P_k\), and \(\varSigma _k\). Then the expansion of the r.h.s. of (12) into a power series in \(\varSigma _k\) gives

$$\begin{aligned} \,\text{ Tr }\,\left( \frac{B_k}{P_k+\varSigma _k}\right)= & {} \,\text{ Tr }\,\left( \frac{B_k}{P_k(1+P_k^{-1}\varSigma _k)}\right) \nonumber \\= & {} \,\text{ Tr }\,\left( B_kP^{-1}_k\frac{1}{1+P_k^{-1}\varSigma _k}\right) \nonumber \\= & {} \,\text{ Tr }\,Q_{k,1} -\,\text{ Tr }\,\big (Q_{k,2}\varSigma _k\big ) \nonumber \\&+\,\text{ Tr }\,\left( Q_{k,2}\varSigma _k\frac{1}{P_k}\varSigma _k\right) +O(\varSigma _k^3), \end{aligned}$$
(16)

where \(Q_{k,m}=\frac{B_k}{P_k^m}.\)

The first term on the r.h.s. is \(\phi \)-independent and contributes only to the running in the vacuum sector \(\varGamma _k^\mathrm {grav}\). Until the cutoff action is specified, \(B_k\) and \(P_k\) are some unknown functions of \(-\triangle _g\), which should be expanded to first order in the curvature tensor. To this end we shall apply the useful off-diagonal heat-kernel method, based on Laplace and Mellin transforms, such that the operators in (16) can be derived from the heat kernel of the covariant Laplacian. The method is described in detail in [29–32] (see also [27]), so here we only sketch the main points of the derivation.

The functions \(Q_{k,m}\) of the covariant Laplacian in the Neumann series (16) admit representations in terms of the inverse Laplace transform,

$$\begin{aligned} Q_{k,m}(-\triangle _g)=\int _0^\infty \mathrm {d}t\,\mathcal {L}^{-1} [Q_{k,m}](t)\,\mathrm {e}^{t\triangle _g}, \end{aligned}$$
(17)

where

$$\begin{aligned} \mathcal {L}[f](s)=\int _0^\infty \mathrm {d}t\, \mathrm {e}^{-st}f(t). \end{aligned}$$
(18)

In what follows we shall apply the useful formula [29–32]

$$\begin{aligned}&\frac{1}{(4\pi )^{d/2}}\int _0^\infty \mathrm {d}t\,t^{-p}\mathcal {L}^{-1}[f](t) \nonumber \\&\qquad =\frac{1}{(4\pi )^{d/2}\varGamma (p)}\int _0^\infty \mathrm{d}s\, s^{p-1} f(s). \end{aligned}$$
(19)

To evaluate the effective action in the given truncation (discussed below) we need the coincidence limit of the matrix elements \(\langle x\vert Q_{k,m}\vert x^\prime \rangle \). According to (17) we may use the heat-kernel expansion for small t [33],

$$\begin{aligned} \langle x \vert \mathrm{e}^{t\triangle _g} \vert x \rangle =\frac{1}{(4\pi t)^{d/2}} \big [a_0(x) + ta_1(x)+t^2 a_2(x)+\cdots \big ]\nonumber \\ \end{aligned}$$
(20)

to find a series expansion of \(\langle x\vert Q_{k,m}\vert x\rangle \) in invariant powers of curvatures and their covariant derivatives. The first Schwinger–DeWitt coefficients \(a_0,a_1,a_2,\ldots \) have the form

$$\begin{aligned} a_0= & {} 1,\quad a_1=\frac{1}{6}R,\nonumber \\ a_2= & {} \frac{1}{180}\left( R_{\mu \nu \alpha \beta }R^{\mu \nu \alpha \beta } -R_{\mu \nu }R^{\mu \nu }+6\triangle _g R+\frac{5}{2}R^2\right) .\nonumber \\ \end{aligned}$$
(21)

In a given truncation scheme we need only \(a_0\) and \(a_1\), but it is not difficult to keep also the next terms. Using the regulator function [34]

$$\begin{aligned} r_k(s) = (k^2-s)\theta (k^2-s), \end{aligned}$$
(22)

one can arrive at the explicit expression

$$\begin{aligned} \begin{aligned}&Q_{k,m}(s) = \frac{2k^2-(k^2-s)\eta _k}{M_k^m} \,\theta (k^2-s),\\&M_k=k^2+\frac{m_k^2}{Z_k}. \end{aligned} \end{aligned}$$
(23)

Let us note that the general review of the functions \(Q_n\) similar to the ones defined above can be found in Appendix A of the review paper on quantum gravity of Ref. [35].

Inserting the expansion (20) into (17), and using (19) we obtain

$$\begin{aligned}&\langle x\vert Q_{k,m}(-\triangle _g)\vert x\rangle \nonumber \\&\quad =\sum _{n=0}^\infty \int _0^\infty \mathrm {d}t\,\mathcal {L}^{-1}[Q_{k,m}](t) \,\frac{1}{(4\pi t)^{d/2}}\,a_n(x) t^n \nonumber \\&\quad =\frac{1}{(4\pi )^{d/2}} \sum _{n=0}^\infty \frac{a_n(x)}{\varGamma (d/2-n)} \int _0^\infty \mathrm {d}s\,Q_{k,m}(s)\, s^{d/2-n-1}, \nonumber \\ \end{aligned}$$
(24)

where the identification \(p=d/2-n\) has been used in Eq. (19) already.

In order to evaluate the integrals over s in Eq. (24), one can note that \(Q_{k,m}\) in (19) are nonzero only on the interval \([0,k^2]\), which gives

$$\begin{aligned}&\int _0^\infty \mathrm {d}s\,Q_{k,m}(s)s^{d/2-n-1} \nonumber \\&\quad = \left( 1-\frac{\eta _k}{d-2n+2}\right) \, \frac{1}{M_k^m}\,\frac{2 k^{d-2n+2}}{d/2-n}. \end{aligned}$$
(25)

After integration we arrive at the result

$$\begin{aligned}&\langle x\vert Q_{k,m}(-\triangle _g)\vert x\rangle = \frac{2}{(4\pi )^{d/2}}\,\frac{1}{M_k^m}\,\nonumber \\&\quad \times \sum _n \left( 1-\frac{\eta _k}{d-2n+2}\right) \,\frac{a_n(x)\,k^{d-2n+2}}{\varGamma (d/2-n+1)}. \end{aligned}$$
(26)

As we have already mentioned, for our purposes it is sufficient to consider the \(n=0,1\) terms in the last series. Expanding the r.h.s. of the flow equation (16) in powers of \(\phi \) and the curvature up to the first order, in four dimensions we meet

$$\begin{aligned}&\frac{1}{2}\,\text{ Tr }\,Q_{k,1}(-\triangle _g)\nonumber \\&\quad = \frac{1}{32\pi ^2\,M_k} \,\int \limits _x \Big [ k^6 \left( 1-\frac{\eta _k}{6}\right) +\frac{k^4}{3}\Big (1-\frac{\eta _k}{4}\Big )\,R +\dots \Big ].\nonumber \\ \end{aligned}$$
(27)

It is easy to see that these terms contribute only to the purely gravitational terms and hence are irrelevant for the running of \(\xi \).

The second-order contribution is

$$\begin{aligned} \frac{1}{2}\,\text{ Tr }\,\big [ Q_{k,2}(-\triangle _g)\varSigma _k \big ]= & {} \frac{1}{32\pi ^2\,M_k^2} \int \limits _x \left[ k^6\left( 1-\frac{\eta _k}{6}\right) \varSigma _k\right. \nonumber \\&\quad \left. +\frac{k^4}{3}\left( 1-\frac{\eta _k}{4}\right) R \varSigma _k+\cdots \right] ,\nonumber \\ \end{aligned}$$
(28)

where we again disregarded higher powers of the curvature.

The derivation of the third term in the expansion (16) requires some commutations of \(\varSigma _k\) with \(P_k^{-1}\), e.g.,

$$\begin{aligned}&\frac{1}{2}\,\text{ Tr }\,\left( Q_{k,2}\varSigma _k\frac{1}{P_k}\varSigma _k\right) \nonumber \\&\quad =\frac{1}{2}\,\text{ Tr }\,\left( Q_{k,2}\varSigma _k\left( \varSigma _k-\frac{1}{P_k} \big [P_k,\varSigma _k\big ]\right) \frac{1}{P_k}\right) \nonumber \\&\quad =\frac{1}{2}\,\text{ Tr }\,\left( Q_{k,3}\varSigma _k^2\right) -\frac{1}{2}\,\text{ Tr }\,\left( Q_{k,3}\varSigma _k\frac{1}{P_k}\big [P_k,\varSigma _k\big ] \right) . \end{aligned}$$
(29)

The last term containing the commutator of \(P_k\) and \(\varSigma _k\) gives rise to a running of the wave-function renormalization and will be dealt with in Sect. 4. It does not contribute to the running of \(u_k\) and \(\xi _k\) and thus can be neglected for the moment being. Thus we arrive at

$$\begin{aligned}&\frac{1}{2}\,\text{ Tr }\,\left( Q_{k,3}(-\triangle _g)\varSigma _k^2\right) \nonumber \\&\quad = \frac{1}{16\pi ^2}\frac{1}{M_k^3}\sum _n \frac{k^{6-2n}}{\varGamma (3-n)} \left( 1-\frac{\eta _k}{6-2n}\right) \,\text{ tr }\,(a_n\varSigma _k^2) \nonumber \\&\quad =\frac{1}{32\pi ^2\,M_k^3} \int \limits _x \Bigg [ k^6\left( 1-\frac{\eta _k}{6}\right) \varSigma _k^2 \nonumber \\&\qquad +\frac{k^4}{3}\left( 1-\frac{\eta _k}{4}\right) \,R\,\varSigma ^2_k+\cdots \Bigg ]. \end{aligned}$$
(30)

In the last expression we omitted most purely gravitational contributions (not all, since for example \(R\varSigma _k\) contains still a term \(\propto R^2\)) and terms beyond the truncation scheme (2). Similarly, one obtains for constant field and curvature

$$\begin{aligned}&\frac{1}{2}\,\text{ Tr }\,\left( Q_{k,2}\varSigma _k\left( P_k^{-1}\varSigma _k\right) ^{m-1}\right) = \frac{1}{2}\,\text{ Tr }\,\left( Q_{k,m+1}\varSigma _k^m\right) \nonumber \\&\quad =\frac{1}{32\pi ^2\,M_k^{1+m}} \int \limits _x \left[ k^6\left( 1-\frac{\eta _k}{6}\right) \varSigma _k^m\right. \end{aligned}$$
(31)
$$\begin{aligned}&\qquad \left. +\frac{k^4}{3}\left( 1-\frac{\eta _k}{4}\right) \,R\,\varSigma ^m_k+\cdots \right] . \end{aligned}$$
(32)

In what follows we ignore the purely gravitational contribution \(\varGamma _k^\mathrm {grav}[g]\) in Eq. (6) and related RG flows. We expect to consider this subject in a separate article.

2.2 RG flow for couplings and non-minimal parameter

Inserting (27)–(31) into (16) and using the definition (15) yields

$$\begin{aligned}&\frac{1}{2}\,\text{ Tr }\,\left( \frac{\partial _t R_k}{\varGamma ^{(2)}_k[\phi ]+R_k}\right) \nonumber \\&\quad = \frac{1}{32\pi ^2M_k} \,\int \limits _x \left[ k^6 \left( 1-\frac{\eta _k}{6}\right) + \frac{k^4}{3}\left( 1-\frac{\eta _k}{4}\right) \,R \right] \nonumber \\&\qquad \times \left[ 1-\frac{\xi _k R + w_k''}{Z_k\,M_k}+ \left( \frac{\xi _k R + w_k''}{Z_k M_k}\right) ^2\right. \nonumber \\&\qquad \left. -\left( \frac{\xi _k R + w_k''}{Z_k M_k}\right) ^3 + \cdots \right] \nonumber \\&\qquad -\frac{1}{2Z^2_k}\,\text{ Tr }\,\left( Q_{k,3}w''_k(\phi )\frac{1}{P_k}\Big [P_k,w''_k(\phi )\Big ]\right) +\cdots \!, \end{aligned}$$
(33)

where terms containing \(R^2\) and \(R^3\) go beyond the truncation scheme and must be omitted.

One can assume that the classical potential for the scalar field at the cutoff is an even function. It follows from the flow equation (33) that the scale dependent effective potential remains even at all scales. Let us further assume that the \(\mathbb { Z}_2\) symmetry is not spontaneously broken. Then the minimum of the effective potential is at \(\phi =0\) and we may expand \(w_k\) in (10) as

$$\begin{aligned} w_k (\phi ) \!=\! \sum _{n=2}^{\infty }\frac{1}{(2n)!} \, \lambda _{(2n)k}\, \phi ^{2n} \!=\! \frac{1}{24} \lambda _{4k} \phi ^4 \!+ \frac{1}{720} \lambda _{6k} \phi ^6 \!+\! \cdots ,\nonumber \\ \end{aligned}$$
(34)

where \(\lambda _{(2n)k}\) are scale-dependent coefficients. Their running is determined by the FRG equations, which will be now derived, along with the one for \(\xi _k\).

For \(w_k\) in (34) the last term in (33) has the form \(\phi ^2(\partial ^2\phi ^2+\dots )\) and is beyond the truncation (2). As a result the wave function is not renormalized. Note that for a non-even cut-off potential with a \(\phi ^3\) term, or for the flow in the broken phase, in which the even potential \(u_k\) is expanded about a nonzero mean field, there is a wave-function renormalization, as we shall see in Sect. 4. One may note that this situation does not depend on the presence of the curved background and can be already observed in scalar field theories in flat space-time [17–21].

One can compare the \(\phi \)-dependent terms in the expression (33) with Eqs. (6) and (34). In this way we arrive at the truncated flow equation for the scalar field theory non-minimally coupled to gravity,

$$\begin{aligned}&-\frac{\eta _k \, Z_k}{2}\phi \triangle _g\phi + \frac{\partial _t\xi _k}{2}R\phi ^2 + \frac{\phi ^2}{2}\,\partial _t m_k^2\nonumber \\&\quad +\frac{\phi ^4}{24}\, \partial _t \lambda _{4k} + \frac{\phi ^6}{720}\, \partial _t \lambda _{6k} = -\frac{1}{M_k^2} \frac{1}{32 \pi ^2 Z_k} \left[ k^6 \left( 1-\frac{\eta _k}{6}\right) \right. \nonumber \\&\quad \left. \times \left( \frac{1}{2} \lambda _{4k} \, \phi ^2 + \frac{1}{24} \lambda _{6k} \, \phi ^4\right) + \frac{k^4}{3}\left( 1-\frac{\eta _k}{4}\right) R \, \lambda _{4k} \, \frac{\phi ^2}{2}\right] \nonumber \\&\quad +\frac{1}{M_k^3} \frac{1}{32 \pi ^2 Z^2_k} \left\{ k^6 \left( 1-\frac{\eta _k}{6}\right) \left[ \xi _k R \, \lambda _{4k} \, \phi ^2 + \frac{1}{4} (\lambda _{4k})^2 \, \phi ^4\right. \right. \nonumber \\&\quad \left. \left. +\frac{1}{24}\lambda _{4k}\lambda _{6k}\, \phi ^6 \right] \right\} - \frac{1}{M_k^4} \frac{1}{32 \pi ^2 Z_k^3}\left\{ k^6\left( 1-\frac{\eta _k}{6}\right) \frac{\lambda _{4k}^3}{8}\, \phi ^6\right\} \nonumber \\&\quad +\cdots . \end{aligned}$$
(35)

Now, we can compare coefficients on both sides of this equation:

  • For the kinetic term \(\phi \triangle _g\phi \) we observe that \(Z_k= \text{ constant }\) or \(\eta _k = 0\), hence there is no wave-function renormalization for an even potential in the symmetric phase. One can fix then \(Z_k=Z_\varLambda =1\) at all scales k. Then, in particular, \(M_k=k^2+m_k^2\), and we denote \(\,D_k=\big (k^2+m^2_k\big )^{-1}\) for the sake of convenience.

  • For the mass term the RG equation can easily be obtained from (35),

    $$\begin{aligned} \partial _t m_k^2 = - \frac{1}{32\pi ^2} \, k^6\, D_k^2\,\lambda _{4k}. \end{aligned}$$
    (36)

  • For the first two interaction terms we have, with \(Z_k\equiv 1\),

    $$\begin{aligned} \partial _t \lambda _{4k}= & {} - \,\frac{k^6}{32 \pi ^2}\, D_k^2\, \Big (\lambda _{6k} - 6D_k\lambda _{4k}^2\Big ),\end{aligned}$$
    (37)
    $$\begin{aligned} \partial _t\lambda _{6k}= & {} -\frac{k^6\,D_k^2}{32\pi ^2}\, \left( \lambda _{8k}-30 D_k\lambda _{4k}\lambda _{6k} + 90 D_k^2\lambda ^3_{4k}\right) .\nonumber \\ \end{aligned}$$
    (38)

    In order to keep our consideration simple, we shall truncate the Taylor expansion for the potential by disregarding all coefficients starting from \(\lambda _{8,k}\), including setting \(\lambda _{8,k}=0\) in Eq. (38).

  • Finally, the non-minimal term \(\phi ^2R\) yields, for \(Z_k\equiv 1\),

    $$\begin{aligned} \partial _t\xi _k =\frac{k^6 D_k^2}{16\pi ^2}\, \left( D_k\,\xi _k - \frac{1}{6 k^2}\right) \,\lambda _{4k}. \end{aligned}$$
    (39)

In a perfect agreement with the general features of the perturbative RG in a curved space-time (as described in the Introduction), the equations for the couplings \(\lambda _{4k}\) and \(\lambda _{6k}\) do not depend on \(\xi _k\), while the FRG equation for \(\xi _k\) depends on the couplings. At the same time, different from the minimal-subtraction based RG, here the \(\beta \)-functions for both couplings and non-minimal parameter do depend on the running mass of field \(m_k\). In this respect the FRG equations resemble the physical, momentum-subtraction-based RG, developed for the interacting scalar field in a curved space-time in [36], but the mass dependence in the present FRG equations is much stronger.

3 FRG flow for couplings, mass and \(\xi \)

The RG equations (36)–(39) should be explored numerically. To this end we introduce the dimensionless quantities

$$\begin{aligned}&m_t = \frac{m_k}{k},\quad D_t = \frac{1}{1 + m_{t}^2},\nonumber \\&\lambda _{4t} = \lambda _{4k},\quad \lambda _{6t} = k^2 \lambda _{6k},\quad \lambda _{8t} = k^4 \lambda _{8k},\ldots \end{aligned}$$
(40)

which are supposed to depend on the dimensionless parameter \(t=\log \ (k/\mu )\), defined in (4). Then the equations become

$$\begin{aligned} \partial _t m^2_t= & {} - 2m^2_t-\frac{1}{32 \pi ^2}\, D_t^2 \lambda _{4t}, \end{aligned}$$
(41)
$$\begin{aligned} \partial _t \lambda _{4t}= & {} - \frac{1}{32 \pi ^2}\,D_t^2\,\big (\lambda _{6t} - 6 D_t \lambda _{4t}^2\big ), \end{aligned}$$
(42)
$$\begin{aligned} \partial _t \lambda _{6t}= & {} 2 \lambda _{6t}-\frac{1}{32\pi ^2}\,D_t^2\, \left( \lambda _{8t} - 30 D_t \lambda _{4t}\lambda _{6t} + 90 D_t^2\lambda ^3_{4t}\right) .\nonumber \\ \end{aligned}$$
(43)

Furthermore, the FRG equation for \(\xi (t)\) in terms of the new variable has the form

$$\begin{aligned} \partial _t \xi _t=\frac{1}{16 \pi ^2}\, D_t^3\,\left( \xi _t - \frac{1}{6\,D_t}\right) \, \lambda _{4t}. \end{aligned}$$
(44)

It is easy to see that in the massless limit \(D_t \rightarrow 1\) this equation reproduces the main features of the one-loop RG equation in the minimal-subtraction scheme, as is well known from [1, 3].

The numerical analysis of these equations shows that the RG flow can be pretty different from the one for the perturbative one-loop RG running, mainly due to the mass dependence. As one can see from the plots presented at Fig. 1, the (dimensionful) mass grows quickly when one flows from the cutoff scale in the UV at \(t=5\) (corresponding to a cutoff value \(\varLambda =e^5\mu \)) to the IR at \(t=0\), which corresponds to \(k=\mu \). As we have already noted above, in the non-perturbative FRG approach there is a sixth-power IR decoupling, which is very strong compared to the usual quadratic decoupling in the perturbative Appelquist and Carazzone theorem [37]. As a result, in the case under discussion one can observe that the running for all couplings and in particular \(\xi _t\) actually freezes at values \(t\lesssim 1\) or equivalently at scales \(k \lesssim \mu \cdot e\).

Fig. 1
figure 1

Flow of the non-minimal parameter \(\xi _t\) and couplings \(m^2_{k},\,\lambda _{4k},\,\lambda _{6k}\) (in units of \(\mu \)) with \(t=\log (k/\mu )\) in the unbroken phase. The initial data at the cutoff \(\varLambda =\mu e^5\) are \(m^2=\mu ^2,\;\lambda _{4} = 0.5\), \(\lambda _{6} = 0\), and \(\xi = 1/6\)

Full size image

Figure 2 shows the running of the non-minimal parameter \(\xi _t\) for the same initial potential as in Fig. 1, but for varying initial \(\xi \)-values of at the UV-cutoff. Depending on the initial value the parameter may increase or decrease during the flow toward the infrared. But, in all considered cases, the values in the UV and IR are not very different. In the flows investigated (only some are displayed in the figure) the relative change was only about one percent.

Fig. 2
figure 2

Running of the non-minimal parameter \(\xi _t\) with \(t=\log (k/\mu )\) for the same initial couplings as in Fig. 1, but with different initial values \(\xi \) in the vicinity of the conformal coupling \(\xi _c=1/6\). Clockwise from top \(\xi _c-2\delta ,\,\xi _c-\delta ,\,\xi _c+\delta \), and \(\xi _c+2\delta \) with \(\delta =1/24\)

Full size image

4 Broken symmetry and FRG flow for anomalous dimension

As we already know, in the LPA truncation (2) there is no RG running for the anomalous dimension \(\eta _k\) for even potentials. At the same time, such running is present in scalar theory beyond one loop and it would be interesting to observe it within the FRG approach. One of the possibilities is related to theories with broken symmetries. This means we shall introduce the negative mass-squared in the classical action (1) and implement this information into the effective average action (2) by imposing the corresponding boundary condition at the cutoff scale \(k=\varLambda \). Then the effective potential of the scalar field is not convex at intermediate scales and this must be taken into account, for in this case one has to consider oscillations near the non-symmetric minima of this potential.

In curved space the spontaneous symmetry breaking meets serious complications, because the position of such a minimum is not constant for a non-constant curvature. The situation was explored in detail in [26] and it was shown that the non-localities emerge in such a theory even at low orders in curvature. However, since our intention here is to consider relatively simple cases, let us consider the zeroth-order approximation and, correspondingly, assume that the position of the minimum of the potential is homogeneous and curvature-independent, denoted by \(\phi _{0,k}\), such that \(u''_k(\phi _{0,k})=m_k^2\ge 0\) is the (physical) mass in the broken phase. Then one has to expand the effective potential as

$$\begin{aligned} u_k=\lambda _{0k}+\sum _{n\ge 2}\frac{\lambda _{nk}}{n!}(\phi -\phi _{0k})^n, \end{aligned}$$
(45)

with small \(\phi -\phi _{0k}\) and scale-dependent minimum \(\phi _{0k}\). Then

$$\begin{aligned} u''_k= & {} \lambda _{2,k}+\sum _{n\ge 3} \frac{\lambda _{n,k}}{(n-2)!}(\phi -\phi _{0k})^{n-2}\nonumber \\\equiv & {} m^2_k+w_k''. \end{aligned}$$
(46)

One can easily note that these definitions of \(m_k^2\) and \(w_t\) are different from the previous ones, because the expansion is performed in the spontaneously broken phase. However, in the new notations Eqs. (13) and (15) have almost the same form as before in the old notations. But in the broken phase odd powers of \(\phi -\phi _{0k}\) appear such that many more terms arise in the power series expansion of the rhs of the flow equation. When one calculates the lhs of the flow equation one must take into account that the minimum \(\phi _{0k}\) of the scale dependent potential flows.

Thus, we continue by inserting the expansions (45) and (46) into the FRG equation with scale-dependent wave-function renormalization, in pretty much the same way as we did before. Finally, changing over to dimensionless quantities (40) and to dimensionless fields according to

$$\begin{aligned} \chi =k^{-1}Z_k^{1/2}\phi ,\quad \chi _{0k}=k^{-1}Z_k^{1/2}\phi _{0k} \end{aligned}$$
(47)

one can arrive at the FRG equations in the broken phase. To simplify the notation we use the following abbreviations:

$$\begin{aligned} A_t=\frac{1}{32\pi ^2}\left( 1-\frac{\eta _t}{6}\right) ,\quad G_n=\frac{\lambda _n}{1+m_t^2}. \end{aligned}$$
(48)

The running of the (cosmological) constant \(\lambda _{0t}\) is given by

$$\begin{aligned} \partial _t\lambda _{0t}+4\lambda _{0t}=A_tD_t, \end{aligned}$$
(49)

and since it does not feed back into the running of the remaining couplings it will be discarded. Comparing terms linear in \(\chi -\chi _{0t}\) yields the running of the mean field,

$$\begin{aligned} \partial _t\chi _{0t} +\left( 1+\frac{\eta _t}{2}\right) \chi _{0t}=A_tD_t\frac{G_3}{m_t^2}. \end{aligned}$$
(50)

This flow equation ensures that \(\chi _{0t}\) remains a minimum of the scale dependent potential at all scales. In writing the flow equations for the dimensionless couplings \(m_t^2,\lambda _{3t},\dots ,\lambda _{6t}\) in an expansion up to order 6 we use the flow equation (50) to simplify the resulting expressions. This way one arrives at

$$\begin{aligned}&\big (\partial _t+2-\eta _t\big )m^2_t = A_tD_t \left( \frac{\lambda _{3t}}{m_t^2}G_3 -G_{4}+2G_3^2\right) ,\end{aligned}$$
(51)
$$\begin{aligned}&\left( \partial _t+1-\frac{3}{2}\eta _t\right) \lambda _{3t}\nonumber \\&\quad =A_tD_t\left( \frac{\lambda _{3t}}{m_t^2}G_4- G_{5}+6G_3 G_4-6G_3^3\right) ,\end{aligned}$$
(52)
$$\begin{aligned}&\big (\partial _t-2\eta _t\big )\lambda _{4t}=A_tD_t \left( \frac{\lambda _{3t}}{m_t^2}G_5 -G_{6}+6G_4^2\right. \nonumber \\&\quad \left. +8G_3G_5-36G_3^2 G_4+24G_3^4\right) ,\end{aligned}$$
(53)
$$\begin{aligned}&\left( \partial _t\!-\!1\!-\!\frac{5}{2}\eta _t\right) \lambda _{5t} \!=\! A_tD_t\left( \frac{\lambda _{3t}}{m_t^2}G_6 +20G_4 G_5 \!+\!10G_3G_6 \right. \nonumber \\&\quad \left. -90G_3 G_4^2-60G_3^2G_5 +240 G_3^3G_4 -120 G_3^5\right) ,\end{aligned}$$
(54)
$$\begin{aligned}&\big (\partial _t-2-3\eta _t\big )\lambda _{6t} = A_tD_t \Big ( 20G_5^2 +30G_4 G_6-90G_4^3\nonumber \\&\quad -360G_3G_4G_5-90 G_3^2 G_6 +1080 G_3^2G_4^2 +480 G_3^3G_5\nonumber \\&\quad -1800G_3^4G_4 +720G_3^6\Big ). \end{aligned}$$
(55)

In the two last flow equations the terms containing \(\lambda _{7t}\) and \(\lambda _{8t}\) are omitted in the sixth-order polynomial approximation.

On top of that, in curved space one meets the new equation for the non-minimal parameter,

$$\begin{aligned} \big (\partial _t - \eta _t \big ) \xi _{t}= & {} \frac{D_t}{16\pi ^2} \, \left[ \left( 1-\frac{\eta _t}{6}\right) \, \big (G_4 - 3 G^2_{3}\big )\,D_t\xi _t\right. \nonumber \\&\left. -\frac{1}{6}\,\left( 1-\frac{\eta _t}{4}\right) \, \big (G_{4} - 2G^2_{3}\big )\right] . \end{aligned}$$
(56)

In order to explore the system of equations (51)–(56) we need an additional equation which defines the scale dependence of \(\eta _t\). In order to obtain this dependence – in our truncation it is induced by the last term in (29) or equivalently the last term in (33) – one has to remember that the running of all couplings and of \(Z_k\) (which defines \(\eta _t\)) does not depend on the presence of the curved space background. As a result we can use the well-known flat-space result for \(\eta _t\) derived in [21, 38] and recently explored in [39]. In terms of dimensionless quantities the result for the anomalous dimension reads

$$\begin{aligned} \eta _t= & {} \frac{1}{32\pi ^2}\, \,\frac{\big [ {u_t'''}(\chi _{0t})\big ]^2}{\big [1 + u''_t(\chi _{0t})\big ]^4} =\frac{1}{32\pi ^2}\frac{\lambda _{3t}^2}{(1+m^2_t)^4} \,, \end{aligned}$$
(57)
Fig. 3
figure 3

Flow of the (dimensionful) minimum of the effective potential as a function of \(t=\log (t/\mu )\). For \(\phi _{0\varLambda }\approx 3.5\mu \) (and the same initial parameters as in Fig. 4) the system remains in the broken phase for all scales k

Full size image
Fig. 4
figure 4

Running of the non-minimal parameter \(\xi _t\) and couplings \(m^2_{k},\lambda _{3k},\dots ,\lambda _{6k}\) (in units of \(\mu \)) as functions of \(t=\log (k/\mu )\). The initial data at the cutoff scale \(k=\varLambda =\mu e^{5}\) are \(m^2=\mu ^2,\;\lambda _{4} = 0.5,\;\lambda _3=\mu ,\; \lambda _5=\lambda _{6} = 0\), and \(\xi = 1/6\)

Full size image

where \(\chi _{0t}\) is the scale-dependent position of the minimum of the potential. Clearly, in the truncation scheme considered, the renormalization of the wave function only happens in the broken phase with nonzero coefficient \(\lambda _{3t}\). It follows from (50) that symmetry can only be broken if \(\lambda _{3t}\) in the UV is nonzero, and this must be taken into account for by a numerical analysis of the system of equations (51)–(56).

The numerical results of such an analysis are presented at Figs. 3 and 4. The first plot of Fig. 3 shows the scale dependence of the minimizing value of the field \(\phi _{0k}\) in the broken phase. As expected, the minimum of the potential are driven closer to the origin by the quantum fluctuations. For the chosen initial parameters at the cutoff \(\varLambda =\mu e^{5}\)

$$\begin{aligned}&\phi _{0}\approx 3.5\mu ,\quad m^2=\mu ^2,\quad \lambda _3=\mu ,\nonumber \\&\lambda _4=0.5,\quad \lambda _5=\lambda _6=0. \end{aligned}$$
(58)

the system stays in the broken phase for all scales. This can also be seen from the running of the couplings shown in Fig. 4. The coupling \(\lambda _{3k}\) decreases rapidly when one moves from the UV to the IR. At the same time the higher couplings \(\lambda _{5k}\) and \(\lambda _{6k}\) acquire nonzero values, although they remain small in the IR.

We solved the flow equations with vanishing \(\eta _t\). Of course, one should choose the anomalous dimension self-consistently. But since at the scale \(k=\mu \) we have \(\lambda _{3}\approx 0.0189\) and \(m^2\approx 15.2\), the first guess for the anomalous dimension

$$\begin{aligned} \eta _{k=\mu }\approx \frac{1}{32\pi ^2}\frac{\lambda _{3\mu }^2}{(1+m_\mu ^2)^4} \approx 1.6\times 10^{-11} \end{aligned}$$
(59)

yields a tiny value in the infrared. Thus, we assume that \(\eta _t=0\) is a very good approximation.

In order to induce more dramatic qualitative changes in the flow of the non-minimal parameter, we considered other sets of initial parameters in the UV. For initial parameters which can be integrated to the infrared we did not observe a strong running of \(\xi \). Thus we conclude that the qualitative form of the flow of \(\xi _t\) is not very sensitive to the initial parameters.

5 Conclusions

We have constructed and explored the FRG equations for a real scalar field in a curved space-time background, including the non-minimal parameter \(\xi \) of the non-minimal interaction between scalar field and curvature. The \(\beta \)-functions obtained within the very simple truncation scheme (2) reproduce several important features of the standard perturbative renormalization group, including what one can prove to be the non-perturbative universal properties of the RG flows. First of all, the FRG equations follow the well-known hierarchy of renormalization in curved space, as described by the points (i)–(iii) in the Introduction. Furthermore, the FRG trajectory for \(\xi \) corresponds to the equation which is linear in \(\xi \), exactly as it should be at both perturbative and non-perturbative levels. In the massless case, the \(\beta \)-function for \(\xi \) has the conformal fixed point \(\xi =1/6\), which is typical for the one-loop case [1]. In our opinion, this may be the result of the restricted form of the truncation (2), because higher-loop corrections involve powers of \(\log (\phi )\) in both potential and kinetic sectors, which are beyond the given approximation. It would be interesting to explore the inclusion of such terms in an FRG analysis; we expect to do this in a future work.

The most remarkable aspect of the RG flow for a massive theory is a strong sixth-power decoupling in the IR. As a result the running of all couplings and \(\xi \) actually stops very soon on the way from the cutoff scale in the UV down to the IR. We conclude that the desirable strong running of \(\xi \) cannot be achieved in the framework of scalar theory. At the same time, there are chances to achieve such an effect in the mixed theory with different mass scales, especially through the quantum effects of relatively light or massless particles.

An interesting extension of the RG flow in the theory with the non-minimal parameter is related to the broken phase, when one can also observe the wave-function renormalization and its effect on the RG trajectories for couplings and \(\xi \). We have found that, regardless of the more complicated form of the RG flow, the qualitative form of the flow for \(\xi \) remains the same, in the sense that the numerical effect of the scale dependence is quite small in the scalar theory. One of the consequences is that the scale dependence in the non-local part of the induced action of gravity in the theory with spontaneous symmetry breaking will be also small. However, as discussed in [2], any form of scale dependence may have a significant impact on the induced cosmological constant term and especially on its non-local extensions. Therefore, the problem of the running of \(\xi \) from UV to IR deserves further detailed studies, especially in more general theories, which involve several mass scales.