Polarizations of Gravitational Waves in Horndeski Theory

We analyze the polarization content of gravitational waves in Horndeski theory. Besides the familiar plus and cross polarizations in Einstein's General Relativity, there is one more polarization state which is the mixture of the transverse breathing and longitudinal polarizations.The additional mode is excited by the massive scalar field. In the massless limit, the longitudinal polarization disappears, while the breathing one persists. The upper bound on the graviton mass severely constrains the amplitude of the longitudinal polarization, which makes its detection highly unlikely by the ground-based or space-borne interferometers in the near future. However, pulsar timing arrays might be able to detect the polarization excited by the massive scalar field. Since additional polarization states appear in alternative theories of gravity, the measurement of the polarizations of gravitational waves can be used to probe the nature of gravity. In addition to the plus and cross states, the detection of the breathing polarization means that gravitation is mediated by massless spin 2 and spin 0 fields, and the detection of both the breathing and longitudinal states means that gravitation is propagated by the massless spin 2 and massive spin 0 fields.


I. INTRODUCTION
The first detection of gravitational waves by the Laser Interferometer Gravitational-Wave Observatory (LIGO) Scientific Collaboration and Virgo Collaboration further supports Einstein's General Relativity (GR) and provides a new tool to study gravitational physics [1][2][3][4][5].
In order to confirm gravitational waves predicted by GR, it is necessary to determine the polarizations of gravitational waves.This can be done by the network of ground-based Advanced LIGO (aLIGO) and Virgo, the future space-borne Laser Interferometer Space Antenna (LISA) and pulsar timing arrays (e.g., the International Pulsar Timing Array and the European Pulsar Timing Array [6,7]).In the recent detection of GW170814 [4], the Advanced Virgo detector joined the two Advanced LIGO detectors, and they were able to test the polarization content of GWs for the first time.The result showed that the pure tensor polarizations were favored against pure vector and pure scalar polarizations.
In GR, the gravitational wave propagates at the speed of light and it has two polarization states, the plus and cross modes.In alternative metric theories of gravity, there may exist up to six polarization states.For null plane gravitational waves, the six polarization states are classified by the little group E(2) of the Lorentz group with the help of the six independent Newman-Penrose (NP) variables Ψ 2 , Ψ 3 , Ψ 4 and Φ 22 [8][9][10][11].In particular, the complex variable Ψ 4 denotes the familiar plus and cross modes in GR.In Brans-Dicke theory [12], in addition to the plus and cross modes Ψ 4 of the massless gravitons, there exists another breathing mode Φ 22 due to the massless Brans-Dicke scalar field [10].
Brans-Dicke theory is a simple extension to GR.In Brans-Dicke theory, gravitational interaction is mediated by both the metric tensor and the Brans-Dicke scalar field, and the Brans-Dicke field plays the role of Newton's gravitational constant.In more general scalartensor theories of gravity, the scalar field has self-interaction and it is usually massive.In 1974, Horndeski found the most general scalar-tensor theory of gravity whose action has higher derivatives of g µν and φ, but the equations of motion are at most the second order [13].Even though there are higher derivative terms, there is no Ostrogradsky instability [14], so there are exactly three physical degrees of freedom in Horndeski theory, and we expect that there is an extra polarization state in addition to the plus and cross modes.If the scalar field is massless, then the additional polarization state should be the breathing mode Φ 22 .
When the interaction between the quantized matter fields and the classical gravitational field is considered, the quadratic terms R µναβ R µναβ and R 2 are needed as counterterms to remove the singularities in the energy-momentum tensor [15].Although the quadratic gravitational theory is renormalizable [16], the theory has ghost due to the presence of higher derivatives [16,17].However, the general nonlinear f (R) gravity [18] is free of ghost and it is equivalent to a scalar-tensor theory of gravity [19,20].The effective mass squared of the equivalent scalar field is f (0)/3f (0), and the massive scalar field excites both the longitudinal and transverse breathing modes [21][22][23].The polarizations of gravitational waves in f (R) gravity were previously discussed in [24][25][26][27][28][29][30].The authors in [26,27] calculated the NP variables and found that Ψ 2 , Ψ 4 and Φ 22 are nonvanishing.They then claimed that there are four polarization states in f (R) gravity.Recently, it was pointed out that the direct application of the framework of Eardley et.al. (ELLWW framework) [10,11] derived for the null plane gravitational waves to the massive field is not correct, and the polarization state of the massive field is the mixture of the longitudinal and the transverse breathing modes [30].Furthermore, the longitudinal polarization is independent of the effective mass of the equivalent scalar field, so it cannot be used to understand how the polarization reduces to the transverse breathing mode in the massless limit.
In this paper, the focus is on the polarizations of gravitational waves in the most general scalar-tensor theory, Horndeski theory.It is assumed that matter minimally couples to the metric, so test particles follow geodesics.The gravitational wave solutions are obtained from the linearized equations of motion around the flat spacetime background, and the geodesic deviation equations are used to reveal the polarizations of the massive scalar field.The analysis shows that in Horndeski theory, the massive scalar field excites both breathing and longitudinal polarizations.The effect of the longitudinal polarization on the geodesic deviation depends on the mass and it is smaller than that of the transverse polarization in the aLIGO and LISA frequency bands.In the massless limit, the longitudinal mode disappears, while the breathing mode persists.
The paper is organized as follows.Section II briefly reviews ELLWW framework for classifying the polarizations of null gravitational waves in Refs [10,11].In Section III, the linearized equations of motion and the plane gravitational wave solution are obtained.In Section III A, the polarization states of gravitational waves in Horndeski theory are discussed by examining the geodesic deviation equations, and Section III B discusses the failure of ELLWW framework for the massive Horndeski theory.Section IV discusses the interferometer response functions and the cross-correlation functions for the longitudinal and transverse breathing polarizations.Finally, this work is briefly summarized in Section V.In this work, we use the natural units and the speed of light in vacuum c = 1.

II. REVIEW OF ELLWW FRAMEWORK
Eardley et.al. devised a model-independent framework [10,11] to classify the null gravitational waves in a generic metric theory of gravity based on the Newman-Penrose formalism [8,9].The quasiorthonormal, null tetrad basis E µ a = (k µ , l µ , m µ , mµ ) is chosen to be where bar means the complex conjugation, they satisfy the relation −k µ n µ = m µ mµ = 1 and all other inner products vanish.Since the null gravitational wave propagates in the +z direction, the Riemann tensor is a function of the retarded time u = t − z, which implies that R abcd,p = 0, where (a, b, c, d) range over (k, l, m, m) and (p, q, r, The linearized Bianchi identity and the symmetry properties of R abcd imply that So R abpq is a trivial, nonwavelike constant, and one can set R abpq = R pqab = 0.One could also The nonvanishing components of the Riemann tensor are R plql .Since R plql is symmetric in exchanging p and q, there are six independent nonvanishing components and they can be expressed in terms of the following four NP variables, and the remaining nonzero NP variables are Φ 11 = 3Ψ 2 /2, Ψ 12 = Ψ21 = Ψ3 and Λ = Ψ 2 /2 where ϑ ∈ [0, 2π) represents a rotation around the z direction and the complex number ρ denotes a translation in the Euclidean 2-plane.From these transformations, one finds out that only Ψ 2 is invariant and the amplitudes of the four NP variables are not observerindependent.However, the absence (zero amplitude) of some of the four NP variables is observer-independent.Based on this, six classes are defined as follows [10].
Class II 6 : Ψ 2 = 0.All observers measure the same nonzero amplitude of the Ψ 2 mode, but the presence or absence of all other modes is observer-dependent.
Class III 5 : Ψ 2 = 0, Ψ 3 = 0.All observers measure the absence of the Ψ 2 mode and the presence of the Ψ 3 mode, but the presence or absence of Ψ 4 and Φ 22 is observerdependent.
Class N 3 : The presence or absence of all modes is observerindependent.
Apparently, Class II 6 is the most general one.By setting successive variables {Ψ 2 , Ψ 3 , Ψ 4 , Φ 22 } to zero, one obtains less and less general classes, and eventually, Class O 0 which is trivial and represents no wave.These NP variables can also be grouped according to their helicities.
In order to establish the relation between {Ψ 2 , Ψ 3 , Ψ 4 , Φ 22 } and the polarizations of the gravitational wave, one needs to examine the geodesic deviation equation in the Cartesian coordinates [10], where x j represents the deviation vector between two nearby geodesics and j, k = 1, 2, 3.
The so-called electric part R tjtk of the Riemann tensor is given by the following matrix, where and stand for the real and imaginary parts.So these NP variables can be used to label the polarizations of null gravitational waves.More specifically, Ψ 4 and Ψ 4 represent the plus and the cross polarizations, respectively; Φ 22 represents the transverse breathing polarization, and Ψ 2 represents the longitudinal polarization; finally, Ψ 3 and Ψ 3 represent vector-x and vector-y polarizations, respectively.FIG. 1 in Ref. [10] displays how these polarizations deform a sphere of test particles, which will not be reproduced here.In terms of R titj , the plus mode is specified by P+ = −R txtx + R tyty , the cross mode is specified by P× = R txty , the transverse breathing mode is represented by Pb = R txtx + R tyty , the vector-x mode is represented by Pxz = R txtz , the vector-y mode is represented by Pyz = R tytz , and the longitudinal mode is represented by Pz = R tztz .
It is now ready to apply this framework to some specific metric theories of gravity.For example, GR predicts the existence of the plus and the cross polarizations [31], and it can be easily checked that only Ψ 4 = 0.For Brans-Dicke theory [12], there is one more polarization, the transverse breathing polarization, excited by the massless scalar field, so in addition to Ψ 4 = 0, Φ 22 is also nonzero [10].In particular, Ψ 2 = 0, so the longitudinal polarization is absent in Brans-Dicke theory.So for Brans-Dicke theory, In the next section, the plane gravitational wave solution to the linearized equation of motion will be obtained for Horndeski theory, and the polarization content will be determined.It will show that because of the massive scalar field, the electric part of the Riemann tensor takes a different form from Eq. ( 10).Its components are expressed in terms of a different set of NP variables.

III. GRAVITATIONAL WAVE POLARIZATIONS IN HORNDESKI THEORY
In this section, the polarization content of the plane gravitational wave in Horndeski theory will be determined.The action of Horndeski theory is [13], where Here, X = −∇ µ φ∇ µ φ/2, 2φ = ∇ µ ∇ µ φ, the functions K, G 3 , G 4 and G 5 are arbitrary functions of φ and X, and G j,X (φ, X) = ∂G j (φ, X)/∂X with j = 4, 5. Horndeski theory reduces to several interesting theories as its subclasses by suitable choices of these functions.
For example, one obtains GR with K = G 3 = G 5 = 0 and G 4 = 1/(16πG).For Brans-Dicke theory, G 3 = G 5 = 0, K = 2ωX/φ and G 4 = φ.And finally, to reproduce f (R) gravity, one The variational principle gives rise to the equations of motion.The full set of equations can be found in Ref. [32], for instance.For the purpose of this work, one expands the metric tensor field g µν and φ in the following way, where φ 0 is a constant.The equations of motion are expanded up to the linear order in h µν and ϕ, where µν and R (1) are the linearized Einstein tensor and Ricci scalar, respectively.In addition, the symbol K ,X (0) means the value of K ,X = ∂K/∂X at φ = φ 0 and X = X 0 = 0, and the remaining symbols can be understood similarly.
In order to obtain the gravitational wave solutions around the flat background, one requires that g µν = η µν and φ = φ 0 be the solution to Eqs. ( 14) and (15), which implies that Substituting this result into Eqs.( 14) and ( 15), and after some algebraic manipulations, one gets, where G 4 (0) = 0 and the mass squared of the scalar field is For Brans-Dicke theory, K ,φφ (0) = 0, so m = 0.For the f (R) theory, one gets m 2 = f (0)/3f (0), which agrees with the previous work [30].
One can decouple Eq. ( 18) from Eq. ( 17) by reexpressing them in terms of the auxiliary tensor field hµν defined by, with σ = G 4,φ (0)/G 4 (0).The original metric tensor perturbation can be obtained by inverting the above relation, i.e., Therefore, Eq. ( 18) becomes The equations of motion remain invariant under the gauge transformation, with x µ = x µ + ξ µ .Therefore, one can choose the transverse traceless gauge ∂ ν hµν = 0, η µν hµν = 0 by using the freedom of coordinate transformation, and the linearized Eqs. ( 17) and ( 18) become the wave equations By the analogue of hµν = h µν − η µν η ρσ h ρσ in GR, it is easy to see that the field hµν represents the familiar massless graviton and it has two polarization states: the plus and cross polarizations.The scalar field ϕ is massive in general, and it decouples from the massless tensor field hµν .Suppose the massless and the massive modes both propagate in the +z direction with the wave vectors, respectively, where the dispersion relation for the scalar field is q 2 t − q 2 z = m 2 .The propagation speed of the massive scalar field is v = q 2 t − m 2 /q t .The plane wave solutions to Eqs. ( 24) and (25) take the following form hµν = e µν e −ik•x , where ϕ 0 and e µν are the amplitudes of the waves with k ν e νµ = 0 and η µν e µν = 0.

A. Polarizations
The polarizations of the gravitational wave can be extracted by studying the relative acceleration of two nearby test particles moving in the field of the gravitational wave.One assumes that the matter fields minimally couple with the metric tensor g µν , while there are no direct interactions between the matter fields and the scalar field φ.Therefore, freely falling test particles follow geodesics, and the relative acceleration is given by the linearized geodesic deviation equations Eq. ( 8).One can thus calculate the electric part R tjtk for the plane wave solution ( 27) and (28).Written as a 3 × 3 matrix, it is given by From this expression, one can easily recognize the familiar plus and cross polarizations by setting ϕ = 0, which leads to a symmetric, traceless matrix with nonvanishing components hxy /2.Indeed, hµν generates the plus and the cross polarizations.
To study the polarizations caused by the scalar field, one sets hµν = 0 and the geodesic deviation equations are, Suppose the initial deviation vector between two geodesics is (x 0 , y 0 , z 0 ), and one integrates the above equations twice to obtain the changes in the deviation vector, From these equations, one finds that if σ is independent of the mass m, so are δx and δy, but δz is proportional to m 2 , so the longitudinal state is suppressed by the mass.However, for f (R) gravity, m 2 σ = 1/3, so δx and δy are proportional to 1/m 2 , while δz is independent of it [30].
The physical meaning of the geodesic deviation equations can be understood by placing a sphere of test particles in the spacetime.When the gravitational wave passes by, this sphere deforms as shown in Figures 1 and 2 for the massive and massless scalars, respectively.In is massive.However, if it is massless, the scalar field excites merely the transverse breathing polarization.In terms of the basis introduced for the massless fields in [10], there are three polarizations: the plus state P+ , the cross state P× and the mix state of Pb and Pz .In the massless limit, the mix state reduces to the pure state Pb .

B. Newman-Penrose variables
The ELLWW framework reviewed in Section II cannot be applied to a theory which predicts the existence of massive gravitational waves.One can calculate the NP variables with the plane wave solution ( 27) and ( 28), and the nonvanishing ones are The naive application of the ELLWW framework to the massive case gives the conclusion that Horndeski gravity has the plus, cross and transverse breathing polarizations, as Ψ 4 and Φ 22 are nonzero, but Ψ 2 = Ψ 3 = 0.In particular, the absence of Ψ 2 means that there would be no longitudinal polarization if the ELLWW framework were correct.However, the previous discussion in Section III A clearly shows the existence of the longitudinal polarization, and the next section will exhibit the observable effects of the longitudinal polarization in the interferometer and the pulsar timing array.Moreover, Φ 00 is absent and both Φ 11 and Λ are proportional to Ψ 2 in the ELLWW framework, which is in contradiction with Eqs.(38) and Eqs.(39).These indicate the failure of this framework for the massive Horndeski theory.
In fact, R tjtk can be rewritten in terms of NP variables as a matrix displayed below, It is rather different from Eq. ( 10).From the above expression, it is clear that some linear combinations of Φ 00 , Φ 11 , Φ 22 and Λ correspond to the transverse breathing and the longitudinal polarizations.More specifically, −2Λ − (Φ 00 + Φ 22 )/2 represents the transverse breathing polarization, while −2(Λ + Φ 11 ) represents the longitudinal polarization.The plus and cross polarizations are still represented by Ψ 4 and Ψ 4 , respectively.In the massless limit, Φ 00 = Φ 11 = Λ = 0 according to Eqs. ( 38) and ( 39), so Eq. ( 40) takes the same form as Eq. ( 10) since the ELLWW framework applies in this case.

A. Interferometers
In this subsection, the response functions of the interferometers will be computed for the transverse breathing and longitudinal polarizations following the ideas in Refs [21,33].The detection of GW170104 placed an upper bound on the mass m g of the graviton [3], This potentially places severe constraint on the effect of the longitudinal polarization on the geodesic deviation in the massive case in the high frequency band, while the transverse breathing mode can be much stronger as long as the amplitude of scalar field ϕ is large enough.Therefore, although there exits the longitudinal mode in the massive scalar-tensor theory, the detection of its effect is likely very difficult in the high frequency band.However, in f (R) gravity, the displacement in the longitudinal direction is independent of the mass and the displacements in the transverse directions become larger for smaller mass, so this mix mode can place strong constraints on f (R) gravity.
In the interferometer, photons emanate from the beam splitter, are bounced back by the mirror and received by the beam splitter again.The round-trip propagation time when the gravitational wave is present is different from that when the gravitational wave is absent.To simplify the calculation of the response functions, the beam splitter is placed at the origin of the coordinate system.Then the change in the round-trip propagation time comes from two effects: the change in the relative distance between the beam splitter and the mirror due to the geodesic deviation, and the distributed gravitational redshift suffered by the photon in the field of the gravitational wave [33].
First, consider the response function for the longitudinal polarization.To this end, project the light in the z direction and place the mirror at z = L.The response function is thus given by, where f = q t /(2π) is the frequency of gravitational waves and the propagation speed v varies with the frequency.To obtain the response function for the transverse mode, project the light in the x direction and place the mirror at x = L.The response function is, which is independent of the mass m of the scalar field.Table I shows that the effect of the longitudinal mode on the geodesic deviation is smaller than that of the transverse mode by 19 to 9 orders of magnitude at higher frequencies.But at the lower frequencies, i.e., at 10 −7 Hz, the two modes have similar amplitudes.Therefore, aLIGO/Virgo and LISA might find it difficult to detect the longitudinal mode, but pulsar time arrays should be able to detect the mix polarization state with both the longitudinal and transverse modes.The results are consistent with those for the massive graviton in a specific bimetric theory [35].
A pulsar is a rotating neutron star or a white dwarf with a very strong magnetic field.
It emits a beam of the electromagnetic radiation.When the beam points towards the Earth, the radiation can be observed, which explains the pulsed appearance of the radiation.
Millisecond pulsars can be used as stable clocks [36].When there is no gravitational wave, one can observe the pulses at a steady rate.The presence of the gravitational wave will alter this rate, because it will affect the propagation time of the radiation.This will lead to a change in the time-of-arrival (TOA), called time residual R(t).Time residuals caused by the gravitational wave will be correlated between pulsars, and the cross-correlation function is , where θ is the angular separation of pulsars a and b, and the brackets imply the ensemble average over the stochastic background.This enables the detection of gravitational waves and the probe of the polarizations.
The effects of the gravitational wave in GR on the time residuals were first considered in Refs [37][38][39].Hellings and Downs [40] proposed a method to detect the effects by using the cross-correlation of the time derivative of the time residuals between pulsars, while Jenet et.al. [41] directly worked with the time residuals instead of the time derivative.The later work was generalized to massless gravitational waves in alternative metric theories of gravity in Ref. [42], and further to massive gravitational waves in Refs [43,44].More works have been done, for example, Refs [45][46][47][48] and references therein.
In their treatments, it is assumed that all the polarization modes have the same mass, either zero or not.If all polarizations propagate in the +z direction at the speed of light, there will be three linearly independent Killing vector fields, Using the conservation of p µ χ µ j (j = 1, 2, 3) for photon's 4-velocity p µ satisfying p ν ∇ ν p µ = 0, one obtains the change in the locally observed frequency of the radiation and integrates to obtain the time residual R(t) [37,49,50].One could also directly integrate the time component of the photon geodesic equation to obtain the change in the frequency [43,51].The later method can be applied to massive case.In the present work, a different method will be used by simply calculating the 4-velocities of the photon and observers on the Earth and the pulsar.Since different polarizations propagate at different speeds, there are not enough linearly independent Killing vector fields.The first method cannot be used any way.
In order to calculate the time residual R(t) caused by the gravitational wave solutions ( 27) and (28), one sets up a coordinate system shown in Figure 4, so that when there is no gravitational wave, the Earth is at the origin, and the distant pulsar is assumed to be stationary in the coordinate system and one can always orient the coordinate system such that the pulsar is located at x p = (L cos β, 0, L sin β).qz is the unit vector pointing to the direction of the gravitational wave, n is the unit vector connecting the Earth to the pulsar, and l = qz ∧ (n ∧ qz )/ cos β = [n − qz (n • qz )]/ cos β is the unit vector parallel to the y axis.In the leading order, i.e., in the absence of gravitational waves, the 4-velocity of the photon is u µ = γ 0 (1, − cos β, 0, − sin β) with γ 0 = dt/dλ a constant and λ an arbitrary affine parameter.The perturbed photon 4-velocity is u µ = u µ + v µ , and since g µν u µ u ν = 0, one obtains Note that one chooses the gauge such that e 11 = −e 22 , e 12 = e 21 are the only nonvanishing amplitudes for hµν , which can always be made.The photon geodesic equation is The calculation shows that v 0 = γ 0 σϕ 0 cos[(q t + q z sin β)t − q z (L + t e ) sin β] v 1 = γ 0 {−σϕ 0 cos β cos[(q t + q z sin β)t − q z (L + t e ) sin β] where t e is the time when the photon is emitted from the pulsar.Eq. ( 46) is consistent with Eq. ( 44).
The 4-velocities of the Earth and the pulsar also change due to the gravitational wave.
Take the 4-velocity of the Earth for instance.Suppose when the gravitational wave is present, its 4-velocity is given by u µ e = u 0 e (1, v e ).The normalization of u µ e implies that The geodesic equation for the Earth is, where τ is the proper time and x µ are the coordinates of the Earth.One sets x = y = 0, which is consistent with Eq. ( 51).Then, one obtains the following solution In addition, with Eq. ( 50), So the 4-velocity of the Earth is approximately σϕ 0 cos q t t, 0, 0, − q z 2q t σϕ 0 cos q t t .
Similarly, one can obtain the 4-velocity of the pulsar, which is σϕ 0 cos(q t t − q z L sin β), 0, 0, − q z 2q t σϕ 0 cos(q t t − q z L sin β) , up to linear order.The form of u µ p can be understood, realizing that χ µ 1 and χ µ 2 are still the Killing vector fields.
Therefore, the frequency of the photon measured by an observer comoving with the pulsar is and the frequency measured by another observer on the Earth is The frequency shift is thus given by where t = t e + L is the time when the photon arrives at the Earth at the leading order.
This expression (58) can be easily generalized to a coordinate system with an arbitrary orientation and at rest relative to the original frame.It can be checked that in the massless limit, the change in the ratio of frequencies agrees with Eq. ( 2) in Ref. [42].The contribution of the massive scalar field also agrees with Eq. ( 2) in Ref. [44] which was derived provided all polarizations have the same mass.
Eq. (58) gives the frequency shift caused by a monochromatic gravitational wave.Now, consider the contribution of a stochastic gravitational wave background which consists of monochromatic gravitational waves, hjk (t, x) Usually, one assumes that ϕ c (q t ) takes a form of ϕ c (q t ) ∝ (q t /q c t ) α with q c t some characteristic angular frequency.α is called the power-law index, and usually, α = 0, −2/3 or −1 [42,52].One can numerically integrate Eqs. ( 73 In Ref. [44], Lee also analyzed the time residual of TOA caused by massive gravitational waves and calculated the cross-correlation functions.His results (the right two panels in his Figure 1) differ from those on the right panel in Figure 5, because in his treatment, the longitudinal and the transverse polarizations were assumed to be independent.In Horndeski theory, however, it is not allowed to calculate the cross-correlation function separately for the longitudinal and the transverse polarizations, as they are both excited by the same field ϕ and the polarization state is a single mode.

V. CONCLUSION
This work analyzes the gravitational wave polarizations in the most general scalar-tensor theory of gravity, Horndeski theory.It reveals that there are three independent polarization modes: the mixture state of the transverse breathing Pb = R txtx + R tyty and longitudinal Pz = R tztz modes for the massive scalar field, and the usual plus P+ = −R txtx + R tyty and cross P× = R txty modes for the massless gravitons.These results are consistent with the three propagating degrees of freedom in Horndeski theory.Since the propagation speed of the massive gravitational wave depends on the frequency and is smaller than the speed of light, the massive mode will arrive at the detector later than the massless gravitons.In addition to the difference of the propagation speed, the presence of both the longitudinal and breathing states without the vector-x and vector-y states are also the distinct signature of massive scalar degree of freedom for graviton.
Using the NP variables, we find that Ψ 2 = 0.For null gravitational waves, this means that the longitudinal mode does not exist.However, the longitudinal mode exists in the massive case even though Ψ 2 = 0. We also find that the NP variable Φ 00 = 0, and Φ 00 , Φ 11 and Λ are all proportional to Φ 22 .These results are in conflict with the results for massless gravitational waves in [10], so the classification of the six polarization states for null gravitational waves derived from the little group E(2) of the Lorentz group is not applicable to the massive case.Although the longitudinal mode exists for the massive scalar field, it is difficult to be detected in the high frequency band because of the suppression by the extremely small graviton mass upper bound.In the massless case, the longitudinal mode disappears and the mix state reduces to the pure transverse breathing mode.We also apply the general analysis to particular cases: Brans-Dicke theory and f (R) gravity.Despite the fact that in f (R) gravity, the longitudinal mode does not depend on the mass of graviton, its magnitude is much smaller than the transverse one in the high frequency band, which makes its detection unlikely by the network of aLIGO/VIGO and LISA, too.It is interesting that the transverse mode becomes stronger for smaller graviton mass in the f (R) gravity, so the detection of the mixture state can place strong constraint on f (R) gravity.
Compared with aLIGO/VIRGO and LISA, pulsar timing arrays might be the primary tool to detect the massive scalar polarizations.In addition to the plus and cross states, the detection of the breathing state solely signifies the existence of massless scalar degree of freedom, and the detection of both the breathing and longitudinal states signifies the existence of massive scalar degree of freedom for gravitational interaction.

FIG. 1 .
FIG. 1.The transverse breathing and the longitudinal polarizations excited by the massive scalar field.

FIG. 2 .
FIG.2.The transverse breathing excited by the massless scalar field.

Figure 3 FIG. 3 .
Figure 3 shows the absolute values of the longitudinal and transverse response functions for aLIGO (L = 4 km) to a scalar gravitational wave with the mass m = m b .Comparing the response functions shows that the transverse response is much larger than that of the longitudinal mode in high frequencies, so the detection of the longitudinal mode becomes very difficult in the high frequency band.One can also understand the difficulty to use the interferometers to detect the longitudinal polarizations in the following way.Table I lists the magnitudes of |ẍ j /x j 0 | normalized with

FIG. 4 .
FIG.4.The gravitational wave is propagating in the direction of qz , and the photon is traveling in −n direction at the leading order.l is perpendicular to qz and in the same plane determined by qz and n.The angle between n and l is β.

FIG. 5 .
FIG. 5.The normalized cross-correlation functions ζ(θ) = C(θ)/C(0).The left panel shows the cross-correlations when the scalar field is massless, i.e., when there is no longitudinal polarization.The solid curve is for familiar GR polarizations (i.e., the plus or cross ones), and the dashed curve for the transverse breathing polarization.The right panel shows the cross-correlations induced together by the transverse breathing and longitudinal polarizations when the mass of the scalar field is taken to be m b = 7.7 × 10 −23 eV/c 2 .The calculation was done assuming T = 5 yrs.
1. Ψ 2 and Φ 22 are real while Ψ 3 and Ψ 4 are complex.So there are exactly six real independent variables, and they can be used to replace the six components of the electric part of the

TABLE I .
The dependence of the magnitudes of |ẍ j /x j 0 | for the longitudinal and transverse modes on the frequencies of gravitational waves in units of Hz 2 σϕ assuming the scalar mass is m b .