1 Introduction

In special relativity, the Lorentz symmetry is a fundamental invariance of physical laws in Minkowski spacetime. The CPT symmetry is also a fundamental invariance under the simultaneous transformations of charge conjugation (C), parity transformation (P), and time reversal (T). The CPT symmetry is exact in any Lorentz invariant local quantum field theory with a Hermitian Hamiltonian. Both Lorentz and CPT symmetries have been well tested by experiments of high energy physics (see review in Ref. [1]). For gravitational sector, the local Lorentz symmetry is one of the fundamental pillars of general relativity (GR). At extremely high energy scales, however, gravitational Lorentz symmetry is expected to be broken in theories of quantum gravity, such as deformed special relativity [2,3,4,5], Horava-Lifshitz gravity [6], loop quantum gravity [7, 8], non-commutative geometry [9, 10], superstring theory [11], etc. The CPT symmetry may not hold any more in absence of local Lorentz symmetry, while the CPT violation necessarily violates Lorentz symmetry in an interacting theory [12]. The tests of CPT symmetry are therefore essential parts of testing Lorentz symmetry.

An effective field theory of gravitational local Lorentz violation has been constructed in a framework of Standard Model Extension (SME) [13]. The local Lorentz-violating operators of arbitrary dimensions were introduced into gravitational action. In the limit of flat spacetime, a Lagrangian was expanded linearly around the Minkowski metric, and a covariant dispersion relation of gravitational waves (GWs) was deduced correspondingly [14]. The birefringence of GWs can be generated by the CPT-odd operators on which we focus. A dispersion relation is split into two branches which are related to the two circular polarization states, respectively. This implies that the two modes have a relative group velocity, which leads to an arrival-time difference between them in temporal domain. Or equivalently, there is a relative dephasing between the two modes in frequency domain. This phenomena seems like a correspondence to the birefringence of electromagnetic waves [15]. Therefore, one can test the CPT symmetry in gravitational sector by precisely measuring GWs which are emitted by compact binaries, for example, binary black holes (BBHs).

The recent discovery of GWs from compact binary coalescences, reported by Advanced Laser Interferometer Gravitational-Wave Observatory (aLIGO) and Virgo Collaborations [16,17,18,19,20,21,22], has opened an observational window to explore Lorentz and CPT violation in gravitational sector. For example, GW170817 has placed the most stringent constraints on the difference between the Lorentz-violating coefficients of mass dimension four in gravitational and photon sectors [22]. This existing study is based on multi-messenger measurements of a relative group velocity of GWs and electromagnetic waves. Due to an arrival-time difference between two circular polarizations in presence of the birefringence, there is a slight splitting of the peak at the maximal amplitude of the observed GW signal. However, there are not significant evidences for such a splitting reported by aLIGO [16], and hence an upper limit on the birefringence of GWs can be obtained by measuring the width of the peak. Based on the GW150914 signal in temporal domain, the upper limit of \(\sim {\mathscr {O}}(10^{-14})\mathrm {m}\) has been placed on the dimension-five CPT-odd Lorentz-violating operators in gravitational sector [14].

In this work, we will study the constraints on the CPT symmetry breaking and birefringence in gravitational sector from three GW experiments, including the second-generation ground-based aLIGO [23], the third-generation ground-based Einstein Telescope ET [24], and the space-based Laser Interferometer Space Antenna (LISA) [25]. By introducing the CPT-violating dispersion in SME, we will explore impacts of the birefringence on propagations of the two circular polarizations of GWs, which are emitted by distant astrophysical sources, and obtain corresponding modifications to GR gravitational waveform by following Refs. [26, 27]. We will further use Fisher information matrix to estimate experimental sensitivities of these detectors to the CPT-violating parameter. In particular, we expect to obtain some bounds on an effective characteristic length scale, below which the CPT violation in gravitational sector might emerge.

The rest of this paper is arranged as follows. In Sect. 2, we introduce the CPT-violating dispersion relation in gravitational sector, and study its modifications to GR gravitational waveform. In Sect. 3, we introduce Fisher information matrix. In Sect. 4, we show the expected constraints on the CPT-violating parameter and birefringence of GWs. The conclusion and discussion are summarized in Sect. 5.

2 Deformations of gravitational waveform due to the CPT violation

We study the CPT violation and birefringence of GWs in the model-independent SME [14]. The birefringence can be induced by the CPT-odd operators of dimension higher than four. The CPT-violating dispersion influences the propagation of GWs from astrophysical sources to detectors [26,27,28,29,30]. In principle, it also contributes to the GW emission process of the sources [31]. As argued in Ref. [32], for a distant GW source, the propagation effect can accumulate along the trajectory and could dominate over the emission effect. This is an ansatz in this study. As mentioned above, due to the birefringence of GWs, the dispersion relation is split into two branches for the two circular polarizations, which thus take different propagating group velocities [14, 30]. Therefore, there is an arrival-time difference between the two modes, or equivalently the birefringence induces a relative dephasing between the two modes in frequency domain.

We explore the birefringence of GWs in a phenomenological framework, which was followed by e.g. Refs. [14, 27, 30, 32]. The CPT-violating dispersion relation of GWs takes the following form

$$\begin{aligned} E^2=p^2\pm \zeta p^{\alpha }, \end{aligned}$$
(1)

where a ± symbol reflects the presence of birefringence, and \(\alpha \) denotes a dimensionless parameter. Here the left-handed circular polarization takes “\(+\)” while the right-handed one takes “−” in Eq. (1). Throughout this paper, we appoint \(G=c=1\) in which G and c denote Newton’s constant and the speed of light, respectively. In Eq. (1), \(\zeta \) is an undetermined parameter which is dimension in \([\mathrm {Energy}]^{2-\alpha }\). For any given \(\zeta \), we can thus define an effective length scale as \(\ell _\alpha =h_{\mathrm {P}}|\zeta |^{\frac{1}{\alpha -2}}\). In principle, the CPT violation could emerge below such a scale. Here \(h_{\mathrm {P}}\) is Planck constant. In this study, we focus on the dispersion relation (1) with odd index \(\alpha =3,5,7,\ldots \), which corresponds to a rotation-invariant limit of the leading-order dispersion in SME (see Eq. (11) in Ref. [14]).

For a given dispersion relation of form \(E^2=p^2+Ap^a\), in which A and a denote two Lorentz-violating parameters, the dephasing of GWs has been extensively studied recently [26, 27]. In such a case, the speed of GWs would be different from the speed of light, which is exactly the speed of GWs in GR. Compared to the GR gravitational waveform, in frequency domain, the modified waveform thus obtains a dephasing which accumulates due to a distant propagation of GWs. In this work, we follow the same way and adopt the similar formulae, except that we introduce different dispersions to the two circular polarizations. To be specific, we take the dispersion relation as \(E^2=p^2+\zeta _{\mathrm {L,R}}p^{\alpha }\), where the subscripts \(_\mathrm {L}\) and \(_\mathrm {R}\) denote the left- and right-handed modes, respectively. Due to \(\zeta _{\mathrm {L}}\ne \zeta _{\mathrm {R}}\), the dephasing of left-handed mode is different from that of right-handed one. Therefore, between the two modes, there is a relative dephasing which can be tested by GW detectors, as suggested by Ref. [14]. From Eq. (1), we find \(\zeta _{\mathrm {L}}=-\zeta _{\mathrm {R}}\), which implies opposite-sign dephasings for the two modes.

For each circular polarization, we can deduce modifications to its phase in the GR waveform from the dispersion relation in Eq. (1). The deduction process is as same as those followed by Refs. [26, 27], except that we consider the circular polarizations here. Denoting the GR waveform with \(h^{\mathrm {gr}}_{\mathrm {L,R}}\), we find that a GW rotates along its propagating trajectory and arrives as \(h_{\mathrm {L,R}}\) given by [30, 32]

$$\begin{aligned} h_{\mathrm {L,R}}=h^{\mathrm {gr}}_{\mathrm {L,R}}~{\mathrm {exp}}({\pm i\delta \varPsi })\ , \end{aligned}$$
(2)

where a “±” symbol takes “\(+\)” for left-handed mode and “−” for right-handed one, and \(\delta {\varPsi }\) denotes a frequency-dependent phase deformation due to the CPT violation, i.e.,

$$\begin{aligned} \delta {\varPsi }= \frac{\lambda }{\alpha -1} (\pi {\mathscr {M}}_{z} f)^{\alpha -1}. \end{aligned}$$
(3)

The dephasing (3) is as same as that in Ref. [26], since we fix the index as \(\alpha =3,5,7,\ldots \). Throughout this work, \(m_i\) (\(i=1, 2\)) is an i-th component mass of a compact binary in the source frame, \(M_z=(m_1+m_2)(1+z)\) a total mass of the binary in the observer frame, \(\eta =m_1m_2(m_1+m_2)^{-2}\) a symmetric mass ratio, \({\mathscr {M}}_z=M_z\eta ^{3/5}\) a chirp mass, and f an observed GW frequency. A parameter \(\lambda \) is defined in terms of \(\alpha \) and \(\zeta \), i.e.,

$$\begin{aligned} \lambda =\frac{h_{\mathrm {P}}^{\alpha -2} \zeta }{\pi ^{\alpha -2}{{\mathscr {M}}_z}^{\alpha -1}} \int _{0}^{z}\frac{(1+z^\prime )^{\alpha -2} dz^\prime }{H_0\sqrt{\varOmega _m(1+z^\prime )^3+\varOmega _\varLambda }}\ , \end{aligned}$$
(4)

where z is a cosmological redshift to the source. Here we assume a spatially-flat cosmological constant plus cold dark matter (\(\varLambda \) LCDM) model, the independent parameters of which are fixed to their best-fit values from Planck 2015 results [33], i.e., Hubble constant \(H_0=67.74~\mathrm {km}~\mathrm {s}^{-1}~\mathrm {Mpc}^{-1}\), the fraction of matter density today \(\varOmega _m=0.3089\), and the fraction of dark energy density today \(\varOmega _\varLambda =1-\varOmega _m\).

The circular polarization states are conventionally decomposed as the “\(+\)” and “\(\times \)” states, namely, \(h_{\mathrm {L,R}}=h_{+}\pm i h_{\times }\) and \(h^{\mathrm {gr}}_{\mathrm {L,R}}=h^{\mathrm {gr}}_{+}\pm i h^{\mathrm {gr}}_{\times }\). Therefore, we obtain \(h_{+,\times }\) as

$$\begin{aligned} h_{+}= & {} \cos (\delta \varPsi )h^{\mathrm {gr}}_{+}- \sin (\delta \varPsi )h^{\mathrm {gr}}_{\times }\ , \end{aligned}$$
(5)
$$\begin{aligned} h_{\times }= & {} \sin (\delta \varPsi )h^{\mathrm {gr}}_{+}+ \cos (\delta \varPsi )h^{\mathrm {gr}}_{\times }\ , \end{aligned}$$
(6)

where \(h^{\mathrm {gr}}_{+,\times }\) denote gravitational waveform in GR, i.e.,

$$\begin{aligned} h^{\mathrm {gr}}_{+}= & {} \frac{1}{2}(1+\cos ^2\iota )\tilde{h}^{\mathrm {gr}}\ , \end{aligned}$$
(7)
$$\begin{aligned} h^{\mathrm {gr}}_{\times }= & {} (i \cos \iota )\tilde{h}^{\mathrm {gr}}\ , \end{aligned}$$
(8)

where \(\iota \) is an inclination angle of the binary, and \(\tilde{h}^{\mathrm {gr}}\) is given by a non-spinning limit of IMRPhenomB [34], including the inspiral–merger–ringdown evolution of a binary coalescence. IMRPhenomB is available up to the asymmetric mass ratio of 4. Since we consider binaries with approximately equal component masses, it is enough for this purpose. Its explicit expression is listed in Appendix A. In addition, in the limit of \(|\delta \varPsi |\ll 1\), we expand \(h_{+,\times }\) to linear order in \(\delta \varPsi \) and obtain \(h_{+}=h_{+}^{\mathrm {gr}}-i\delta \varPsi h_{\times }^{\mathrm {gr}}\) and \(h_{\times }=h_{\times }^{\mathrm {gr}}+i\delta \varPsi h_{+}^{\mathrm {gr}}\). This is so-called amplitude birefringence [35].

Given the waveform \(h_{+,\times }\), one can define an “observed” waveform in frequency domain. For an i-th interferometer of a given experiment, the “observed” waveform is written as [36]

$$\begin{aligned} {h}^{(i)}(f)=\varepsilon F_{+}^{(i)}(\bar{\theta },\bar{\phi },\psi ){h}_{+}(f)+\varepsilon F_{\times }^{(i)}(\bar{\theta },\bar{\phi },\psi ){h}_{\times }(f)\ ,\nonumber \\ \end{aligned}$$
(9)

where \(F_{+,\times }^{(i)}\) denote a set of two pattern functions, and one has \(\varepsilon =1\) for aLIGO with arms at a \(\pi /2\) opening angle [37], \(\varepsilon =\sqrt{3}/2\) for ET with arms at a \(\pi /3\) opening angle [37, 38], and \(\varepsilon =\sqrt{3}/2\) for LISA [39, 40]. For aLIGO, we consider a single interferometer. One explicitly writes \(F_{+,\times }^{(1)}\) as

$$\begin{aligned} F_+^{(1)}= & {} \frac{1}{2}(1+\cos ^2\bar{\theta })\cos 2\bar{\phi } \cos 2\psi -\cos \bar{\theta }\sin 2\bar{\phi }\sin 2\psi , \nonumber \\\end{aligned}$$
(10)
$$\begin{aligned} F_{\times }^{(1)}= & {} \frac{1}{2}(1+\cos ^2\bar{\theta }) \cos 2\bar{\phi }\sin 2\psi -\cos \bar{\theta }\sin 2\bar{\phi }\cos 2\psi \ .\nonumber \\ \end{aligned}$$
(11)

Here we have a source location (i.e., polar angle \(\bar{\theta }\) and azimuthal angle \(\bar{\phi }\)) and a polarization angle (i.e., \(\psi \)) defined in the detector frame. For ET [38], we consider three interferometers in total. The two additional sets of antenna pattern functions are given by

$$\begin{aligned} F^{(2)}_{+,\times }(\bar{\theta },\bar{\phi },\psi )= & {} F^{(1)}_{+,\times }(\bar{\theta },\bar{\phi }+2\pi /3,\psi )\ , \end{aligned}$$
(12)
$$\begin{aligned} F^{(3)}_{+,\times }(\bar{\theta },\bar{\phi },\psi )= & {} F^{(1)}_{+,\times }(\bar{\theta },\bar{\phi }+4\pi /3,\psi )\ . \end{aligned}$$
(13)

For LISA [39, 40], we consider two interferometers in total. The second set of antenna pattern functions is given by

$$\begin{aligned} F^{(2)}_{+,\times }(\bar{\theta }, \bar{\phi },\psi )=F^{(1)}_{+,\times } (\bar{\theta },\bar{\phi }-\pi /4,\psi ) . \end{aligned}$$
(14)

When \(\psi =\pi /8\) is set in the following, we can get a relation of \(F_+^{(i)}=F_\times ^{(i)}\). We only study the optimally-oriented compact binary coalescences, which are face-on and located directly above the detectors, i.e. \(\iota =\bar{\theta }=0\). We can thus set \(\bar{\phi }=0\) here.

3 Fisher information matrix

Fisher information matrix [41,42,43] is performed to get the expected constraints on the CPT violation and on the birefringence of GWs, given a GW detection which is consistent with GR. Following the development in Ref. [44], Fisher matrix is defined as

$$\begin{aligned} F_{ab}=\sum _{i=1}^{n}\left( \frac{\partial h^{(i)}}{\partial \theta _a}\bigg |\frac{\partial h^{(i)}}{\partial \theta _b}\right) \ , \end{aligned}$$
(15)

where \(\theta _a\) denotes a-th parameter, n is a total number of interferometers for a given experiment, and we define an inner product between two waveforms \(\tilde{h}_1\) and \(\tilde{h}_2\) as

$$\begin{aligned} (h_1|h_2)=2\int _{f_\mathrm {low}}^{f_{\mathrm {high}}} \frac{{h}_1^*(f){h}_2(f)+{h}_1(f){h}_2^*(f)}{S_h(f)}df , \end{aligned}$$
(16)

where a \(^*\) symbol is a complex conjugate, and \(S_h(f)\) denotes a noise power spectral density (PSD) of the given detector. In Appendix B, we summarize the noise PSDs of aLIGO [45], ET [46], and LISA [39, 40]. Here \(f_{\mathrm {low}}\) is a lower-cutoff frequency of the detector, while \(f_{\mathrm {high}}\) is an upper-cutoff frequency, at which the GW detection terminates. aLIGO, ET, and LISA are sensitive to frequency ranges \(10-10^{4}\mathrm {Hz}\), \(1-10^{4}\mathrm {Hz}\), and \(10^{-4}-10^{-1}\mathrm {Hz}\), respectively. In addition, a signal-to-noise ratio (SNR) is defined as \(\mathrm {SNR}=\sum _{i=1}^{n}\sqrt{(h^{(i)}(f)|h^{(i)}(f))}\). A root-mean-square (rms) uncertainty on \(\theta _a\) is defined as a diagonal component of covariance matrix \(C_{ab}\), i.e. \(\varDelta \theta _a=\sqrt{C_{aa}}\). Cramer-Rao bound [47, 48] says an inequality of form \(C\ge F^{-1}\). Once Fisher matrix is obtained, one uses Cholesky decomposition to get an inverse of Fisher matrix. Therefore, a minimal uncertainty on \(\theta _a\) is given by

$$\begin{aligned} \varDelta \theta _a=\sqrt{(F^{-1})_{aa}}\ , \end{aligned}$$
(17)

which is determined by a-th diagonal component of the inverse of Fisher matrix. Furthermore, one defines a normalized covariance matrix, i.e., \(c_{ab}={(F^{-1})_{ab}}/({\varDelta \theta _{a}\varDelta \theta _{b}})\), to describe a cross correlation between \(\theta _a\) and \(\theta _b\).

In this study, we consider the optimally-oriented sources to estimate the constraints on the CPT-violating parameters. This implies that the sources are face-on and above the detectors. As discussed in Sect. 2, in frequency domain, the CPT violation induces the relative dephasing between the two circular polarizations of GWs. It is enough for a single detector to measure such a relative quantity. So we do not consider a network of GW detectors. However, one should note that a full multi-detector Bayesian analysis, that simultaneously disentangle polarizations and fit for the polarization-dependent dispersion parameters, may achieve better constraints than those obtained just from looking at the waveform dephasing in a single detector. We leave such a detailed analysis to future works. In the study the parameter space is spanned by six dimensionless parameters, i.e.,

$$\begin{aligned} {\theta }=\{\ln d_{\mathrm {L}},\ln {\mathscr {M}},\ln \eta ,\phi _c,f_0t_c,\lambda \}\ , \end{aligned}$$
(18)

where we use the chirp mass in the source frame, i.e. \({\mathscr {M}}={\mathscr {M}}_z(1+z)^{-1}\), and \(f_0\) denotes a characteristic frequency of the detector, typically chosen as a “knee” frequency or a frequency that makes the noise PSD minimal. In a fiducial model, we let \(\phi _c=f_{0}t_c=\lambda =0\), and let \(d_{\mathrm {L}}\simeq 1~\mathrm {Gpc}\) for aLIGO and ET while \(d_{\mathrm {L}}\simeq 3~\mathrm {Gpc}\) for LISA. We consider \({\mathscr {M}}\) varying within 20–10\(^{3}M_{\odot }\) for aLIGO, 20–3 \(\times 10^{3}M_{\odot }\) for ET, and \(10^{5}-10^{7}M_{\odot }\) for LISA. In addition, we study the dependence of our results on \(\eta \), which varies from 0.16 to 0.25. For a given \(\alpha \), once a constraint on \(\lambda \) is obtained, we can deduce a constraint on \(\zeta \) according to Eq. (4) and hence on the effective length scale \(\ell _\alpha \). Given the parameter space in Eq. (18), we can explicitly evaluate the Fisher matrix and the uncertainty on a given parameter by following Eqs. (15) and (17). Our results are listed in the following section.

4 Constraints on the CPT violation and GW birefringence from gravitational-wave interferometers

For a given \(\alpha \), by using Fisher matrix, we obtain the \(1\sigma \) uncertainty on \(\lambda \) and then obtain an upper limit on \(\ell _{\alpha }\). For the coalescing BBHs with equal component masses, Fig. 1 shows such upper limits on \(\ell _\alpha ~(\alpha =3,5,7)\) expected from aLIGO (green solid curve), ET (red dashed curve), and LISA (blue dash-dotted curve). The SNRs are also depicted for all the three detectors. Here \(M_{t}=m_{1}+m_{2}\) is a total mass of the binary in the source frame. \(\ell _{\alpha }\) is in units of m. We use thicker curves to denote lower-order \(\alpha \) in the figure.

Fig. 1
figure 1

The upper limits on the CPT-violating parameter \(\ell _{\alpha }\) (\(\alpha =3,5,7\)) from aLIGO (green solid), ET (red dashed), and LISA (blue dash-dotted). The SNRs are also obtained for the three detectors here

Table 1 In the case of \(\alpha =3\), for different value of \(\eta \), the rms uncertainties on \(\ln {\mathscr {M}}\), \(\ln \eta \) and \(\lambda \), as well as their cross correlations. The uncertainties on \(\ell _3\) are also listed here

We notice several generic characters about our results. For all the three detectors, the upper limits on \(\ell _\alpha \) become higher with the increase of \(\alpha \). This means that we obtain less stringent constraints on \(\ell _\alpha \) for higher-order \(\alpha \). For a given \(\alpha \), ET can always has a higher sensitivity than aLIGO in the same mass range. However, for a higher-order \(\alpha \), the difference in the sensitivities of ET and aLIGO becomes less significant. For equally spaced \(\alpha \), e.g. \(\alpha _{i}\) (i=1,2,3) with \(\alpha _1<\alpha _2<\alpha _3\) and \(2\alpha _2=\alpha _1+\alpha _3\), the ratio \(\ell _{\alpha _2}/\ell _{\alpha _1}\) is always larger than the ratio \(\ell _{\alpha _3}/\ell _{\alpha _2}\). The above general predictions could be explained as follows. Based on Eq. (3), a higher-order \(\alpha \) can generate a larger phase correction at higher frequencies. Naively, a higher-order \(\alpha \) could be corresponded to a higher post-Newtonian (PN) order. By contrast, the GW detectors can measure the higher-order phase deformations at a worse level [49,50,51]. Obviously, our results arise from a balance between the above two competitive ingredients. In addition, LISA has a worse sensitivity, partly because the birefringence is proportional to \(f^{\alpha -1}\) in Eq. (3). To get a better knowledge of the above discussions, we estimate the orders of magnitude for the following typical cases. For the chirp mass \(\simeq 30M_\odot \), ET will reach the sensitivities \(\ell _3\sim {\mathscr {O}}(10^{-16})\mathrm {m}\), \(\ell _5\sim {\mathscr {O}}(10^{-2})\mathrm {m}\) and \(\ell _7\sim {\mathscr {O}}(10)\mathrm {m}\), while aLIGO will reach \(\ell _3\sim {\mathscr {O}}(10^{-15})\mathrm {m}\), \(\ell _5\sim {\mathscr {O}}(10^{-1})\mathrm {m}\) and \(\ell _7\sim {\mathscr {O}}(10)\mathrm {m}\). For the chirp mass \(\simeq 10^5M_\odot \), LISA will reach \(\ell _3\sim {\mathscr {O}}(10^{-9})\mathrm {m}\), \(\ell _5\sim {\mathscr {O}}(10^{3})\mathrm {m}\) and \(\ell _7\sim {\mathscr {O}}(10^{5})\mathrm {m}\).

For the coalescing BBHs with different component masses, e.g. \(\eta =0.16,0.25\), Table 1 shows the rms uncertainties on the source parameters (i.e. \({\mathscr {M}}\), \(\eta \)) and the CPT-violating parameter (i.e. \(\lambda \)), as well as their normalized cross-correlation coefficients, in the case of \(\alpha =3\). The uncertainty on \(\lambda \) is converted to the upper limit on \(\ell _3\) in this table. Given the GW signal, which is consistent with GR, one would expect \(\lambda \) to be smaller than the magnitudes in the sixth column of Table 1. Therefore, the lower bound on \(\ell _{3}\) is listed in the seventh column. To show the degeneracy of parameters, we show the cross-correlations among \(\ln {\mathscr {M}}\), \(\ln \eta \) and \(\lambda \) in the last three columns.

According to Table 1, we find that the symmetric mass ratio influences the constraints on the CPT violation and birefringence of GWs. Given \(M_{t}\), the constraint on the CPT violation becomes more stringent with the increase of \(\eta \), due to an increase of SNR. Given \(\eta _1\) and \(\eta _2\) with \(\eta _1<\eta _2\), the ratio \(\ell _{\alpha }(\eta _2)/\ell _{\alpha }(\eta _1)\) becomes less significant with the increase of \(M_{t}\), due to a smaller number of frequency modes in the sensitive range of the detectors. Therefore, it is helpful to use a compact binary system with equal component masses in the study of CPT violation and birefringence. We also extend the parameter space to include \(\iota \), \(\psi \), \(\bar{\theta }\) and \(\bar{\phi }\), and vary their values around the fiducial values. Obviously, these additional parameters make the \(\lambda \) constraints less stringent. However, the modifications are less than or around one order of magnitude in the most sensitive mass range, since we only utilize the relative dephasing between the two circular polarizations of GWs. In future works, we will study the network of GW interferometers, which is expected to reduce the uncertainties [52].

5 Conclusions and discussion

In this paper, we explored the influences on the propagation of GWs from the CPT violation and birefringence in the gravitational sector [14, 27, 32]. We found a relative dephasing between the two circular polarization modes of GWs, and obtained the corresponding gravitational waveform, which is corrected by the CPT-violating parameter. Considering distant compact binary coalescences, we estimated the projected constraints on the CPT-violating parameter from aLIGO, ET, and LISA. Among these experiments, we expect ET to be most sensitive to the CPT violation and birefringence. This study involved only the relative dephasing between the two circular polarization states and was therefore model-independent. Future GW detections are expected to shed light on the CPT violation and birefringence in the gravitational sector. If such deviations from GR were detected, we can use them to infer the degree of CPT violation and birefringence. If not, we can constrain the magnitude of the CPT-violating parameter to some interesting levels.

We made several approximations which are worth being revisited in the future. For example, we only considered the kinematical propagation of GWs while disregarded the dynamical generation process, which may generate some corrections to our results. For gravitational waveform, we did not take into account additional parameters such as eccentricity, spins, precession, etc. By contrast, it has been found that the spins in binaries can worsen the projected constraints on Lorentz violation [53]. In addition, we focus on the isotropic limit for the leading-order dispersion relation of GWs in SME. In fact, there are a lot of anisotropic coefficients which can also lead to the birefringence. For the above concerns, we leave detailed analysis to future works. Furthermore, the CPT violation contributes to the phase of waveforms a corrected term proportional to \(f^{\alpha -1}\), which can be naively corresponded to the \((3\alpha /2+1)\)PN order. Due to \(\alpha \ge 3\), we have studied at least 5.5PN-order effects in this work. By contrast, the GR waveform in Eq. (A.1) is available up to the 3.5PN order. This means that we neglected the degeneracies between these higher-PN-order terms and the CPT-violating terms. This is just feasible if we were only interested in how well the birefringence can be constrained by observations.