Graviton non-gaussianity in $\alpha$-vacuum

We compute the leading order non-Gaussianity, i.e., the bispectrum, of the tensor perturbation in the general $\alpha$-vacuum on de Sitter space in general relativity. In addition to the well-known Bunch-Davies (BD) vacuum, there exits an infinite number of de Sitter invariant vacua represented by a real parameter $\alpha$ and a phase $\phi$, with $\alpha=0$ being the BD vacuum. They are called $\alpha$-vacua. In the standard slow-roll inflation, as de Sitter invariance no longer applies, the $\alpha$-vacua lose its relevance in the rigorous sense. Nevertheless, if we assume that the parameter $\alpha$ is only weakly dependent on the wavenumber with an appropriate UV cutoff, we may consider pseudo-$\alpha$-vacua. In the case of false vacuum inflation where the background spacetime is pure de Sitter, a non-trivial (non-BD) $\alpha$-vacuum could indeed be realized. We find an intriguing result that the bispectrum may be exponentially enhanced to be detectable by observation even if the spectrum is too small to be detected.


Introduction
Triggered by the discovery of gravitational waves (GWs) from a merging black hole binary by LIGO [1] and subsequent discoveries of many merging compact binaries, GWs have become one of the hot topics in astrophysics and cosmology. In particular, a lot of attention has been paid to possible GW signatures from the early universe in the last few years. Among them, one of the important sources of GWs is the one that comes from vacuum fluctuations during inflation [2,3].
Conventionally the tensor mode (graviton) vacuum fluctuations are considered to be in the adiabatic vacuum as in the case of the scalar mode vacuum fluctuations. In this case, the amplitude is uniquely determined by the Hubble parameter H during inflation, namely, it is characterized by the tensor power spectrum proportional to H 2 (t k ) where t k is the time at which the mode with comoving wavenumber k left the Hubble horizon, and the bispectum is known to be small.
However, there have been several proposals that may alter this prediction. One of such is the case of modified gravity, e.g., Horndeski theory, massive gravity, etc. Furthermore, it has been suggested that the quantum nature of the graviton state may be observationally detected if gravitons were not in the adiabatic vacuum [4,5]. This motivates us to study possible non-Gaussianities for non-adiabatic initial states. In this paper, without specifying a particular model, we consider a vacuum state that deviates from the adiabatic one. More precisely, assuming that the inflationary stage can be approximated by a pure de Sitter space (or it can indeed be pure de Sitter in the case of false vacuum inflation [6] or the false vacuum stage prior to vacuum tunneling in open inflation [7,8]), we consider the general α-vacuum for the tensor perturbation, and compute the spectrum and bispectrum without assuming that the deviation from the BD vacuum is small.
The α-vacuum is known to be the general de Sitter invariant vacuum, specified by a real parameter α and a phase φ [9]. The case α = 0 is called the Bunch-Davies (BD) vacuum and it corresponds to the adiabatic vacuum in the general slow-roll inflation. A non-zero value of α means a deviation from the adiabatic vacuum, independent of the phase φ.
The 3-point correlation function for primordial tensor fluctuations in the BD vacuum is first calculated in [10], and an extensive discussion on the possible forms of tensor non-Gaussianities due to the cubic Weyl terms are given in [11,12,13]. The tensor bispectrum in the α-vacuum are calculated for squeezed configurations in [14]. In [15], both squeezed and folded configurations of the bispectrum are considered with the assumption that there was a transition from the BD vacuum to an α-vacuum at an early epoch. In the case of the scalar curvature perturbation, the 3-point function for the general α-vacuum was derived in [16].
In the case of the conventional slow-roll inflation, deviations from the adiabatic vacuum give rise to extra energy density carried by the field, and the condition that this extra energy density be smaller than the vacuum energy density responsible for inflation places a severe constraint on these deviations [17]. Thus, in particular, in many of the previous studies on the tensor perturbation in the α-vacuum, only small deviations from the BD vacuum were discussed [18]. However, thus obtained constraints from the backreation arguments depend not only on the energy scale of inflation but also on the wavenumbers under consideration. Furthermore, in the case of false vacuum inflation when the spacetime is pure de Sitter, it seems there is no strong reason why the state cannot be in a non-trivial (α = 0) α-vacuum, as it respects the full de Sitter invariance. At the least, it is worth presenting the explicit form of the tensor bispectrum for the general α-vacuum without any approximations. The paper is organized as follows. In section 2, We briefly review the quantization of the tensor perturbation in de Sitter space, describe the α-vacuum, and present the action up through cubic self-interactions. For the cubic self-interactions, we focus on the case of general relativity as its presence seems robust independent of models of interest. In section 3, we compute the 2-point and 3-point functions in momentum space, which correspond to the spectrum and bispectrum, respectively, for the general α-vacuum. We find that the resulting bispectrum can become arbitrarily large, while the power spectrum is kept small, or even significantly suppressed for a particular choice of the phase φ. As particular cases of interest, we then take the squeezed and folded limits. The result for the squeezed configuration is in agreement with [14]. Section 4 is devoted to conclusions and discussion. Some technical details are deferred to Appendix A.

Graviton in de Sitter Space
The graviton in a spatially flat expanding background may be represented by the tensor mode perturbation in the three-dimensional metric, where η is the conformal time and the metric perturbation γ ij satisfies the transvers traceless Here and in what follows the spatial indices i, j, k, · · · are raised and lowered by δ ij and δ kℓ .
We focus on general relativity. The Einstein-Hilbert action up through the third order in h ij ≡ M pl 2 γ ij is given by where M 2 pl = 1/(8πG) and Here, a prime denotes the derivative with respect to the conformal time.

Free graviton in de Sitter space
At quadratic order, we expand h ij (η, x i ) in the Fourier modes, where e A ij (k) is the polarization tensor for the k mode, normalized as e ijA e B ij = δ AB with A, B = +, ×. Then the quadratic action, (2.3), is rewritten as where k = |k|. The tensor mode h A k (η) satisfies Quantizing the above, the Fourier mode h A k (η) is promoted to an operator, and it may be expressed in terms of the creation and annihilation operators as , and the canonical commutation relation implies the Klein-Gordon normalization condition on the mode function u k (η) called the positive frequency function, In the case of de Sitter space, the scale factor is given by a(η) = −1/(Hη) where −∞ < η < 0. The de Sitter space is symmetric under SO(4,1) transformations. It is known that there are infinitely many vacua that respect the de Sitter symmetry, called the α-vacua.
Among them, a unique vacuum that corresponds to the adiabatic vacuum in an expanding universe is the BD vacuum. Assuming b A k is the operator that annihilates the BD vacuum, b A k |0 k BD = 0, the corresponding mode function u k is given by For an α-vacuum, we consider another expansion of h A k (η), The mode functions and the creation and annihilation operators for the α-vacuum are related to those for the BD vacuum by the Bogoliubov transformation, where and below we assume 0 ≤ α < ∞ and 0 ≤ φ < 2π without loss of generality.
The α-vacuum is a squeezed state from the point of view of the BD vacuum, and it may also be regarded as an entangled state. Applying Eq.
This leads to the expression of the α-vacuum in terms of the BD vacuum as 15) where N k is a normalization constant. If we expand the exponential function in Taylor series, we find This is a two-mode squeezed state which consists of an infinite number of entangled particles. In particular, in the highly squeezing limit α → ∞, the α-vacuum becomes the maximally entangled state from the point of view of the BD vacuum. Before closing this subsection, let us mention the issue of backreaction. It has been argued that any state that substantially deviates from the adiabatic vacuum would cause a backreaction problem because there would be a large number of excited particles with respect to the adiabatic vacuum (BD vacuum in the case of de Sitter space) as given in (2.16), whose energy density would dominate the universe and nullify inflation [17]. If we apply this picture to our case, any non-trivial α-vacuum would not be allowed as the parameter α is independent of the comoving momentum k.
In this paper, as we mentioned briefly in the introduction, we take a more flexible point of view. In the case of conventional slow-roll inflation, the adiabatic vacuum is indeed the unique, physically most motivated vacuum. Hence we consider the k-independence as an approximation, and assume it applies only to momenta satisfying k < k max for a certain maximum k max as well as to the range of time η i < η for a certain initial time η i , so that there would arise no backreaction problem. In addition, to simply the computations we assume k|η i | ≫ 1 for the range of momenta of interest so that we may ignore the initial time dependence. On the other hand, in the case of false vacuum inflation where the spacetime is exactly de Sitter, since all α-vacua are de Sitter invariant, we accept the possibility of an α-vacuum as it is. In either case, we proceed to computing the leading order graviton non-Gaussianity in the α-vaccum, hoping that it may shed some light on the initial state of the universe.

Interaction picture in the in-in formalism
In terms of the Fourier mode operators h A k (η), the cubic action Eq. (2.4) is rewritten as As usual, to take this self-interactions into account, we resort to the interaction picture, where the states and operators are expressed in terms of the free field operators with the interaction Hamiltonian H I = H − H 0 being used to evolve the states and operators. Here H and H 0 are the full and free Hamiltonians, respectively. In the in-in formalism with which one computes the expectation value of an operator Q at time t, we have where |ψ H (Q H ) is the state (operator) in the Heisenberg picture and |ψ (Q) is the corresponding free state (operator) , and T (T ) denotes the time-ordering (anti-time-ordering).
In our case, we set H 0 to be the one given by L 2 and H I = −L 3 , replace t with the conformal time η, and consider the 2-point and 3-point functions,

Two-point and Three-point Functions
Following the in-in formalism presented in the previous section, we now compute the twopoint and three-point functions at their leading orders.

Two-point function
The two-point function at leading order is simply given by its free field version. It is straightforward to obtain it by using Eqs. (2.10) and (2.12). Focusing on its behavior at late times (η → 0), we obtain where k = |k 1 | = |k 2 |, and P T (k) is the power spectrum given by The spectrum for the BD vacuum is realized when α = 0. Thus the α-vacuum spectrum is generally enhanced by a factor e 2α for α ≫ 1. An intriguing fact is that, in the special case of φ = π, the spectrum is exponentially suppressed like e −2α for α ≫ 1, instead of being enhanced. This implies that the two-point function can be made arbitrarily small in principle. One might wonder how this could be consistently realized under the uncertainty principle, or under the Klein-Gordon normalization of the mode functions. The point is that although the α-vacuum mode function v k is suppressed by e −α at leading order in |kη|, its higher order terms are actually enhanced by e α . This leads to the enhancement by e α of the component of v ′ k that contributes to the Klein-Gordon normalization. In other words, the canonical conjugate of h ij , i.e., a 2 h ′ ij is the one that may be regarded a frozen on superhorizon scales in the large α limit, e 2α |kη| ≫ 1. Possible effects of this on the arguments about classicalization and decoherence on superhorizon scales is an interesting issue, but it is out of the scope of the present paper.

Three-point function
The three-point function to its leading order is given by expanding the in-in formula to the first order in the interaction Hamiltonian, Using Eqs. (2.10) and (2.12) again, we obtain the three-point function in late times where B is the bispectrum given by where P is the factor involving the polarization, and we have defined k t ≡ k 1 +k 2 +k 3 , k ′ t ≡ k 1 +k 2 −k 3 , k ′′ t ≡ k 1 −k 2 −k 3 , and it is understood that k 1 + k 2 + k 3 = 0 in the above. The vertical bar separating (k 1 , A 1 ) and (k 2 , k 3 ; A 2 , A 3 ) in the arguments of P is inserted to remark the asymmetry between them.
As discussed in the previous subsection, the two-point function can be exponentially small for α ≫ 1 if φ = π. Therefore it is of interest to consider the three-point function in this case. Putting φ = π in the above, we find The above implies that the bispectrum is enhanced by a factor e 4α for α ≫ 1. Thus if the graviton was, or the graviton modes in a certain range of wavenumbers were in an α-vacuum like state with α ≫ 1 and φ = π, the bispectrum may be exponentially enhanced relative to the amplitude of the two-point function. This suggests that the higher point functions are even more enhanced, and the perturbative expansion ceases to be valid in this particular case. Whether this is the case or not is an interesting mathematical question, and it would be intriguing if such a state is actually realized in some scenario of inflation.
To detect the bispectrum in observation, it is customary to consider theoretical templates for the three limits in the momentum space configuration; the squeezed shape (k 1 ≪ k 2 ≃ k 3 ), the folded shape (k 1 = 2k 2 = 2k 3 ), and the equilateral shape (k 1 = k 2 = k 3 ). They serve as theoretical templates. In the current case, we immediately see that the bispectrum is enhanced for the squeezed and folded shapes. Therefore we focus on these two limits below. Furthermore, it is customary to introduce the non-Gaussian parameter, say f N L for each of the three limits. However, as we are not sure if the same form of f N L used for the scalar type (i.e., curvature perturbation) non-Gaussianity could also be useful for the tensor type perturbation, because of the different k-dependence, we present it below just to get a sense of it.
Let us first consider the squeezed limit. Setting k 1 ≪ k 2 = k 3 ≡ k, the leading order term is found as × sinh 2 2α cosh 2 α + cos 2φ sinh 2 2α sinh 2 α + cos φ sinh 4α cosh 2 α . (3.8) Following the standard practice used for the curvature perturbation, we introduce the non-Gaussian parameter f sq N L for the squeezed limit defined by [19] f sq .
We note that if f sq N L is to be normalized with respect to the amplitude of the scalar curvature perturbation spectrum, it should be multiplied by r 2 where r = P T (k)/P S (k) is the tensor-to-scalar ratio. As the difference in the k-dependence between P T (k) and P S (k) is small, the shape of f sq N L remains the same by the multiplication of r 2 at leading order. Next let us turn to the folded limit, k 1 = k 2 + k 3 with k 2 = k 3 ≡ k. We immediately see that this limit is quadratic divergent because of the terms proportional to (k ′ t ) −2 and (k ′′ t ) −2 and their cyclic permutations. To clarify how this divergence would affect actual observables like the CMB anisotropy is beyond the scope of this paper. Nevertheless, since the tensor perturbation affects CMB only through its time derivative, we would expect that the observable effects are non-singular. In any case, let us evaluate the folded limit by introducing ∆k ≡ 2k − k 1 . At leading order, we obtain × P (k 1 , A 1 | k 2 , k 3 ; A 2 , A 3 ) sinh 2 2α(sinh 2 α + cos 2φ cosh 2 α) + cos φ sinh 4α sinh 2 α −2P (k 2 , A 2 | k 3 , k 1 ; A 3 , A 1 ) sinh 2 2α(cosh 2 α + cos 2φ sinh 2 α) + cos φ sinh 4α cosh 2 α .
Again, similar to the squeezed limit, by applying the standard convention of f fo N L used for the scalar type perturbation, defined by [20] f fo Thus, f fo N L is enhanced not only by a scale-dependent factor k 2 /(∆k) 2 but also exponentially enhanced for α ≫ 1; by a factor ∼ e 2α if φ = π and ∼ e 8α if φ = π. Also if we are to normalize f fo N L by the scalar curvature perturbation spectrum, it should be multiplied by r 2 . In summary, both the squeezed and folded limits of non-Gaussianity are strongly scaledependent, and are exponentially enhanced for α ≫ 1. The enhancement factor is ∼ e 2α if φ = π, and ∼ e 8α if φ = π for both limits. This result suggests that the parameter α may already be strongly constrained by the existing observational data such as the Planck CMB data [21].

Conclusion
In this paper, we computed the graviton bispectrum for the general α-vacuum in de Sitter space, assuming the conventional cubic self-interactions in general relativity. The α-vacuum is characterized by a real non-negative parameter α and a phase φ.
It was argued that the α-vacuum is strongly constrained by the condition that its backreaction to the energy density of the universe would not jeopardize inflation [17,18]. This usually leads to the condition that the α-vacuum parameter must be very small, α ≪ 1, with its wavenumber range very narrow; H < k/a ≪ M pl . In this paper, we took a flexible point of view that the constraint could be considerably weakened if the energy scale of inflation was very low, or the universe was in a pure de Sitter phase where an α-vacuum might be realized without backreaction. In the former case, recalling that the small backreaction requirement gives e 2α H 2 /M 2 pl ≪ 1, α must satisfy the condition e α ≪ M pl /H. If we consider inflation with the energy scale ∼ 10 3 GeV, for example, which is perfectly consistent with observational constraints as well as with the standard model of particle physics, we have H 2 /M 2 pl ∼ 10 −64 . This not only allows a large applicable range of wavenumbers but also implies a weak constraint, e α ≪ 10 32 or α ≪ 74. The allowable range of α seems large enough to be interesting.
We found an intriguing result that the bispectrum may become very large in comparison with the spectrum for α ≫ 1, and it becomes particularly enhanced if φ = π. Namely, even if the tensor spectrum is too small to be detected by observation, there is a possibility that the bispectrum is large enough to be detected.
We have also computed the bispectrum in the squeezed limit (k 1 ≪ k 2 ≃ k 3 ) and the folded limit (k 1 2k 2 = 2k 3 ) for the general case of α and φ. Introducing the non-Gaussian parameters f sq N L and f fo N L for these two limits, respectively, by copying their definitions in the case of the scalar bispectrum, we found that both non-Gaussian parameters are scale- It is of interest to derive observational constraints on the α-vacuum parameters from, say, the Planck CMB data [21]. But it is out of the scope of the present paper.
Another, more fundamental issue is the validity of the perturbative expansion. In the case of α ≫ 1, where the bispectrum is much larger than the spectrum, one may suspect that the higher order spectra could be even larger. This might be a sign of an unphysical property of the α-vacuum. Nevertheless, at least from a phenomenological point of view, we believe that it is scientifically interesting enough to test the graviton α-vacuum non-Gaussianities computed in this paper against observational data. We hope to come back to this in future work.