1 Introduction

Multicarrier modulations can be separated into two main classes: cyclic prefix-based orthogonal frequency division multiplexing (CP-OFDM) and filter bank-based multicarrier (FBMC). The first class is the most widely used since it has been adopted by many major communication standards, e.g., DAB, DVB-T, IEEE802.11a/g (Wi-Fi), IEEE 802.16 (WiMAX), and LTE. However, OFDM systems still have some drawbacks such as the high side lobes of the rectangular pulse shape (transmission/reception filter) and a loss of the spectral efficiency due to the cyclic prefix extension. Because of the aforementioned weaknesses, researchers are more and more interested by FBMC modulations which employ a frequency well-localized FIR filter with small side lobes instead of the rectangular pulse shape used in OFDM. This makes FBMC systems more spectral efficient and less sensitive to frequency errors compared to OFDM [1].

When a multicarrier modulation (OFDM or FBMC) is employed, the performance of multi-cellular networks depends extensively on how well the orthogonality among subcarriers is maintained at the receiver. In an asynchronous system, this orthogonality is destroyed due to synchronization errors which include timing offsets and carrier frequency/phase offsets. The problem of asynchronism in multicarrier systems has been intensively investigated in the literature for OFDM as well as for FBMC systems [28]. The sensitivity of OFDM/FBMC systems to synchronization errors has been illustrated in terms of signal-to-interference ratio (SIR) in [1, 3, 7] and in terms of bit error rate (BER) in [8, 9], by exploiting the Gaussian approximation of the intercarrier interference. An interesting work in relationship with the scope of this paper is [2] (resp. [5]) where the authors have presented a theoretical analysis of the average error rate of an asynchronous OFDM (resp. FBMC)-based multi-cellular network in the special case of interleaved (resp. block) subcarrier assignment scheme. More recently, Medjahdi et al. [6] have presented an interference modeling based on the so called interference tables [4] for asynchronous OFDM and FBMC systems. This model and the results of [5] will be very useful to accomplish the developments of this paper.

Another serious issue with OFDM and FBMC systems is the fact that the transmitted signal is a sum of a large number of independently modulated subcarriers. Thus, they suffer from high peak-to-average power ratio (PAPR) which makes the system very sensitive to nonlinear distortion (NLD) caused by nonlinear devices such as high-power amplifiers (HPA). This problem has been largely studied for OFDM systems. The impact of NLD on OFDM signals was presented in [10, 11] and a performance analysis of nonlinear OFDM and OFDM-MIMO systems was derived in [12, 13]. As for FBMC systems, the authors in [14] carried out a theoretical analysis of BER performance for nonlinearly amplified FBMC/OQAM signals under additive white Gaussian noise (AWGN) and Rayleigh fading channels. Similarly, [15] has recently derived a closed-form expression for the BER of gray-coded M-ary (QAM or OQAM)-based OFDM in the presence of HPA NLD under a frequency flat fading Rayleigh channel.

It is worth noting that there are limited studies that investigate the problem of the joint effect of non-synchronization and HPA nonlinearities on multicarrier systems. An interesting one is [16], where the authors evaluated the BER of multicarrier DS-CDMA downlink systems subject to these impairments in frequency-selective Rayleigh fading channels, assuming QAM modulation. In this paper, we study the joint effect of synchronization errors and HPA nonlinear distortions on FBMC-based multi-cellular networks. We will derive exact expressions of average error rates for the considered system by carrying out an analytical interference analysis in frequency-selective fading channels. It is worth noting that there are limited studies that investigate the problem of joint effects of non-synchronization and HPA nonlinearities on multicarrier systems. An interesting one is [17], where the authors analyzed the interference caused by nonlinear power amplifiers together with timing errors on multicarrier OFDM/FBMC transmissions. In this work, we will consider both frequency and timing errors in addition to HPA NLD and we will plot BER curves as functions of SNR for different frequency offset (CFO) ranges and different levels of the nonlinear noise power. Regarding FBMC modulation, a cosine modulated multitone (CMT) system will be considered and the results can be easily extended to staggered modulated multitone (SMT) systems since a CMT signal can be obtained from a SMT one through a simple modulation step [18].

The remainder of this paper is organized as follows. In Section 2, the considered CMT-based multi-cellular network is introduced and the joint effect of HPA NLD and synchronization errors is described. In Section 3, we present the analytical interference analysis. The BER analysis is carried out in Section 4. Section 5 includes the evaluation of the obtained BER expressions through simulation results. Finally, Section 6 concludes this paper.

2 System model

We consider the downlink transmission scenario of the CMT-based multi-cellular network depicted in Fig. 1. MU 0 is the reference mobile user who communicates with the reference base station BS 0. The other base stations BS k ,k=1…6 represent an interfering source for MU 0. In a regular hexagonal cellular system (hexagonal coordinates system), where the reference BS is located at the origin (0,0), the reference mobile at the coordinates point (u 0,v 0) and the kth interfering BS at (u k ,v k ), then the distance between the reference mobile and the kth base station is given by ([19], pp. 140)

$$ d_{k}=R \sqrt{3} \sqrt{(u_{k}-u_{0})^{2}+(v_{k}-v_{0})^{2}+(u_{k}-u_{0})(v_{k}-v_{0})}, $$
(1)
Fig. 1
figure 1

CMT-based multi-cellular network

where R is the cell radius.

The transmitted signal from the kth base station to the reference mobile is a CMT signal given by [18]

$$ \begin{aligned} x_{k}(t)&=\sum_{n=-\infty}^{\infty} \sum_{m \in F_{k}} s_{n}^{m,k} g(t-nT)e^{j\frac{\pi}{2T} t}e^{jm(\frac{\pi}{T} t+ \frac{\pi}{2})}\\ &=\sum_{n=-\infty}^{\infty} \sum_{m \in F_{k}} s_{n}^{m,k} \gamma_{m,n}(t), \end{aligned} $$
(2)

where

  • T is the CMT symbol duration and g(t) is the prototype filter impulse response,

  • \(s_{n}^{m,k}\) is the transmitted pulse-amplitude modulated (PAM) symbol by the kth base station, at the mth subcarrier on the nth time index,

  • \(\gamma _{m,n}(t)=g(t-nT)e^{j\frac {\pi }{2T} t}e^{j\Phi ^{m}(t)}\) and \(\Phi ^{m}(t)=m(\frac {\pi }{T} t+ \frac {\pi }{2})\),

  • F k is the set of subcarriers that are assigned to the kth base station. In this paper, we will consider the block subcarrier assignment, described in Fig. 2, for frequency reuse scheme.

    Fig. 2
    figure 2

    Block subcarrier assignment

Concerning the channel model, we will consider a common model for interference analysis in microcellular mobile systems, in which the useful signal received from the reference BS is assumed to experience Nakagamim fading, whereas all the interfering signals from the neighboring BSs experience Rayleigh fading [2]. The impulse response of the multipath channel between the kth base station and the reference mobile is

$$ h_{k}(t)=\sum_{l=0}^{L-1} h_{k,l} \delta(t-\tau_{l}), $$
(3)

where τ 0<τ 1<…τ L−1<τ max and τ max is the maximum delay spread of the channel, and {h k,l ,∀k,l} are the complex path gains which are assumed mutually independent, where \(E\left [ h_{k,i}, h_{k,i}^{*} \right ]=\gamma _{k,i}\) and \(E\left [ h_{k,i}, h_{k,j}^{*} \right ]=0\), if ij. We further assume that the power is normalized such that \(\sum _{l=0}^{L-1} \gamma _{k,l}=1, \forall k\).

We assume a perfect synchronization between MU 0 and BS 0. However, MU 0 is not necessarily synchronized with the other base stations. In addition, we consider the presence of HPA nonlinear distortions. These distortions can be modeled using the Bussgang theorem which states that for a Gaussian signal x(t) which undergoes a nonlinear distortion and produce an output signal y(t), the cross-correlation function R xy (τ) is related to the auto-correlation function R xx (τ) by the following expression

$$ \mathit{R_{{xy}}}(\tau)=\alpha {R_{{xx}}}(\tau), $$
(4)

where \(\phantom {\dot {i}\!}\alpha =|\alpha | e^{\phi _{\alpha }}\) is a complex factor.

An interesting result from the Bussgang theorem is that the output y(t) can be written as [10]

$$ y(t)=\alpha x(t)+\eta(t), $$
(5)

where η(t) is a zero mean additive noise, uncorrelated to x(t) and with variance \(\sigma _{\eta }^{2}=E\left [ \left |\eta \right |^{2} \right ]\).

On the other hand, the received signal from the kth interfering base station BS k is affected by a timing offset τ k , a carrier frequency offset ε k , and a phase offset ϕ k . Therefore, we can express the composite signal at the reference receiver by the sum of the desired signal coming from BS 0 and the interference signal coming from the surrounding base stations BS k ,k=1…K

$$ {\begin{aligned} r(t)&=d_{0}^{-\frac{\beta}{2}} x_{0}(t)*h_{0}(t)+ \sum_{k=1}^{K} d_{k}^{\frac{-\beta}{2}} y_{k}(t-\tau_{k})\\ &\quad \times e^{j(2\pi \varepsilon_{k}+\phi_{k}) t} * h_{k}(t)+ n(t) \\ &=d_{0}^{-\frac{\beta}{2}} x_{0}(t)*h_{0}(t)+ \sum_{k=1}^{K} \alpha_{k} d_{k}^{\frac{-\beta}{2}} x_{k}(t-\tau_{k})\\ &\quad \times e^{j(2\pi \varepsilon_{k}t+\phi_{k})} * h_{k}(t) \\ &\quad+ \sum_{k=1}^{K} d_{k}^{\frac{-\beta}{2}} \eta_{k}(t) e^{j(2\pi \varepsilon_{k}t+\phi_{k})} * h_{k}(t) + n(t), \end{aligned}} $$
(6)

where

  • * is the convolution operator,

  • K is the total number of neighboring cells,

  • x k is the interference signal transmitted by the base station BS k ,

  • n(t) is the additive white Gaussian noise (AWGN) with two-sided power spectral density N 0/2,

  • β is the path loss exponent.

3 Interference analysis

3.1 Interference analysis in an AWGN channel

In this subsection, we will derive the interference power expression of the considered CMT-based multi-cellular network in the case of AWGN channels. To this end, let us consider the transmission of an individual PAM symbol \({s_{n}^{m}}\) from an interfering cell on its mth subcarrier (in the nth time index) to the reference mobile in the presence of all the sources of distortion presented in the last section (i.e., synchronization errors and NLD). The received signal will be then expressed as follows:

$$\begin{array}{*{20}l} {i^{m}_{n}}(t)&=\alpha {s_{n}^{m}} g(t-nT-\tau) e^{j\frac{\pi}{2T} (t-\tau)}e^{j\Phi^{m}(t- \tau)} e^{j(2\pi \varepsilon t+\phi)}\\ &\quad+ \eta(t) e^{j(2\pi \varepsilon t+\phi)} + n(t). \end{array} $$
(7)

We recall that \(\Phi ^{m}(t)=m(\frac {\pi }{T} t+ \frac {\pi }{2})\).

After demodulation, the interference signal received on the 0th subcarrier and the 0th time index (for simplicity’s sake) of the reference mobile output can be given by the following expression

$$ {\begin{aligned} \hat{i}^{0}_{0}&=Re\left\{ \left< {i^{m}_{n}}(t),\gamma_{0,0}^{*}(t) \right> \right\}= Re\left\{ \int_{-\infty}^{\infty} {i^{m}_{n}}(t) g(t)e^{-j\frac{\pi}{2T} t} dt \right\} \\ &= |\alpha|{s_{n}^{m}} Re\left\{ \int_{-\infty}^{\infty} g(t) g(t-nT-\tau) e^{j\left(\Phi^{m}(t-\tau)+2\pi\varepsilon t +\phi +\phi_{\alpha} \right)} dt \right\}\\ &\quad+ \hat{\eta}_{0}^{0} + \hat{n}_{0}^{0}, \end{aligned}} $$
(8)

where \(\hat {\eta }_{0}^{0}\) and \(\hat {n}_{0}^{0}\) are given by

$$ \begin{aligned} \hat{\eta}_{0}^{0}&=Re\left\{ \int_{-\infty}^{\infty} \eta(t) e^{j\left(2\pi\varepsilon t +\phi \right)} \gamma_{0,0}^{*}(t) dt \right\}, \hat{n}_{0}^{0}\\ &=Re\left\{ \int_{-\infty}^{\infty} n(t) \gamma_{0,0}^{*}(t) dt \right\}. \end{aligned} $$
(9)

Therefore, we can express the power of the interference as follows

$$ \begin{aligned} \mathcal{I}(\psi,m)=|\alpha|^{2} E_{s} &\left| \int_{-\infty}^{\infty} g(t) g(t-nT-\tau)\right.\\ &\left.{\vphantom{\int_{-\infty}^{\infty}}}\cos{\left[\Phi^{m}(t-\tau)+2\pi\varepsilon t +\theta \right]} dt \right|^{2}, \end{aligned} $$
(10)

where ψ=(τ,ε,θ), θ=ϕ+ϕ α and E s is the PAM symbol energy.

The interference power \( \mathcal {I}(\psi,m)\) can be explicitly expressed if we take into account some observations. First, we refer to the PHYDYAS prototype filter designed by Bellanger using the frequency sampling technique [20]. Furthermore, we assume that the timing error τ is small enough such that the prototype filter impulse response g(t) can be considered constant between t and tτ, i.e., g(tτ)≈g(t). Using the above observations and the results of [5], the expression of the interference power becomes

$$ {\begin{aligned} \mathcal{I}(\psi,m)&=|\alpha|^{2} E_{s} \left| \int_{-\infty}^{\infty} g^{2}(t) \cos{\left(\Phi^{m}(t-\tau)+2\pi\varepsilon t +\theta \right)} dt \right|^{2}\\ &=|\alpha|^{2} E_{s} \left|J(\psi,m)\right|^{2}, \end{aligned}} $$
(11)

where we have considered (for simplicity’s sake) that the interference signal is coming from the 0th time index symbol.

In Eq. (11), J(ψ,m) is the integral given by the following expression

$$ J(\psi,m)= \int_{-\infty}^{\infty} g^{2}(t) \cos{\left(\Phi^{m}(t-\tau)+2\pi\varepsilon t +\theta \right)} dt. $$
(12)

For a PHYDYAS prototype filter with an overlapping factor κ=4 and frequency coefficients G 0=1/2, G 1=0.971960, \(G_{2}=\sqrt {2}/2\), and G 3=0.235147 [5, 20], the expression of the integral J(ψ,m) will become

$$ {\begin{aligned} J(\psi,m)&= \int\limits_{0}^{4T} \cos{\left(\left(\frac{\pi}{T}m+2\pi\varepsilon \right) t +\Theta_{m} \right)} dt \\ &\quad+4 \sum\limits_{q=1}^{3} (-1)^{q} G_{q} \int\limits_{0}^{4T} \cos\left(\frac{q\pi}{2T}t\right) \cos{\left(\left(\frac{\pi}{T}m+2\pi\varepsilon \right) t +\Theta_{m} \right)} dt\\ &\quad+4 \sum\limits_{q=1}^{3} {G_{q}^{2}} \int\limits_{0}^{4T} \cos^{2}\left(\frac{q\pi}{2T}t\right) \cos{\left(\left(\frac{\pi}{T}m+2\pi\varepsilon \right) t +\Theta_{m} \right)} dt\\ &\quad+4 \sum\limits_{q=1}^{3} \sum\limits_{\substack{q'=1 \\ q'\neq q}}^{3} (-1)^{q+q'} G_{q} G_{q'} \int\limits_{0}^{4T} \cos\left(\frac{q\pi}{2T}t\right) \cos\left(\frac{q'\pi}{2T}t\right) \\ &\quad\times \cos{\left(\left(\frac{\pi}{T}m+2\pi\varepsilon \right) t +\Theta_{m} \right)} dt,\\ \end{aligned}} $$
(13)

where \(\Theta _{m}= \theta +(1-\frac {2\tau }{T})\frac {\pi }{2}m\).

After a few steps of straightforward manipulations, one finds that

$$ {\begin{aligned} J(\psi,m)&= \left.\Gamma_{0}(\psi,m,t) \right|_{0}^{4T} + 4 \sum\limits_{q=1}^{3} (-1)^{q} G_{q} \left.{\Gamma_{1}^{q}}(\psi,m,t) \right|_{0}^{4T} \\ &\quad+4 \sum\limits_{q=1}^{3} {G_{q}^{2}} \left.{\Gamma_{2}^{q}}(\psi,m,t) \right|_{0}^{4T}\\ &\quad+ 4 \sum\limits_{q=1}^{3} \sum\limits_{\substack{q'=1 \\ q'\neq q}}^{3} (-1)^{q+q'} G_{q} G_{q'} \left.\Gamma_{3}^{q,q'}(\psi,m,t) \right|_{0}^{4T}, \\ \end{aligned}} $$
(14)

where \(\Gamma _{0}(\psi,m,t)=\frac {\sin {\left (\left (\frac {\pi }{T}m+2\pi \varepsilon \right) t +\Theta _{m} \right)} }{\frac {\pi }{T}m+2\pi \varepsilon }\) and

$$ {\begin{aligned} {\Gamma_{1}^{q}}(\psi,m,t)&=\frac{1}{2}\left\{ \frac{\sin{\left(\left(\frac{\pi}{T}m+2\pi\varepsilon +\frac{q\pi}{2T}\right) t +\Theta_{m} \right)} }{\frac{\pi}{T}m+2\pi\varepsilon +\frac{q\pi}{2T}} \right. \\ & \quad\qquad\left. - \frac{\sin{\left(\left(\frac{\pi}{T}m+2\pi\varepsilon -\frac{q\pi}{2T}\right) t +\Theta_{m} \right)} }{ \frac{q\pi}{2T}-\frac{\pi}{T}m-2\pi\varepsilon} \right\}. \end{aligned}} $$
(15)

Similarly, \({\Gamma _{2}^{q}}(\psi,m,t)\) and \(\Gamma _{3}^{q,q'}(\psi,m,t)\) are, respectively, the primitives of integrals shown in the third and fourth terms of Eq. (13).

By putting

$$ \begin{aligned} \Gamma(t,\psi,m)&=\Gamma_{0}(\psi,m,t) + 4 \sum\limits_{q=1}^{3} (-1)^{q} G_{q} {\Gamma_{1}^{q}}(\psi,m,t)\\ &\quad+4 \sum\limits_{q=1}^{3} {G_{q}^{2}}{\Gamma_{2}^{q}}(\psi,m,t) \\ &\quad + 4 \sum\limits_{q=1}^{3} \sum\limits_{\substack{q'=1 \\ q'\neq q}}^{3} (-1)^{q+q'} G_{q} G_{q'} \Gamma_{3}^{q,q'}(\psi,m,t), \\ \end{aligned} $$
(16)

we can rewrite the interference power \(\mathcal {I}(\psi,m)\) as follows:

$$ \begin{aligned} \mathcal{I}(\psi,m)&=|\alpha|^{2} E_{s} \left| \left.\Gamma(\psi,m,t) \right|_{0}^{4T} \right|^{2} = |\alpha|^{2} E_{s} \chi(\psi,m), \end{aligned} $$
(17)

with \(\chi (\psi,m)=\left | \left.\Gamma (\psi,m,t) \right |_{0}^{4T} \right |^{2}\).

3.2 Interference power in a frequency-selective channel and SINR expression

According to [5], the interference power arriving through a frequency-selective channel, in the multi-cell model described in Section 2, can be calculated using the following expression:

$$ \begin{aligned} P_{i}&= \sum_{k=1}^{K} \sum_{m \in F_{k}} d_{k}^{-\beta} \mathcal{I}(\psi_{k},m,\alpha_{k}) |H_{k}(m)|^{2}\\ & = E_{s} \sum_{k=1}^{K} \sum_{m \in F_{k}} |\alpha_{k}|^{2} d_{k}^{-\beta} \chi(\psi_{k},m)|H_{k}(m)|^{2}, \end{aligned} $$
(18)

where

  • \(\mathcal {I}(\psi _{k},m,\alpha _{k})\) (Eq. (17)) is the power of the interference caused by the kth base station on the mth subcarrier and corresponding to an error vector ψ k =(τ k ,ε k ,θ k ),

  • |H k (m)|2 is the power channel gain between the kth interfering base station and the reference receiver.

Similarly, we can express the power of the total nonlinear noise signal coming from all the K interfering base stations as follows:

$$ P_{nl}=\sum_{k=1}^{K} \sum_{m \in F_{k}} d_{k}^{-\beta} \sigma_{\hat{\eta}_{0,k}^{0}}^{2}(\varepsilon_{k}) |H_{k}(m)|^{2}, $$
(19)

where \(\sigma _{\hat {\eta }_{0,k}^{0}}^{2}(\varepsilon _{k})=E\left [ \left | \hat {\eta }_{0,k}^{0}(\varepsilon _{k}) \right |^{2} \right ]\) is the variance of the received nonlinear noise \(\hat {\eta }_{0,k}^{0}(\varepsilon _{k}) \).

Concerning the power of the desired signal, which is assumed to be perfectly synchronized with the reference base station, it can be simply expressed as

$$ P_{d}=d_{0}^{-\beta} E_{s} |H_{0}|^{2}. $$
(20)

Using the following expression of the signal to interference plus noise ratio

$$ \text{SINR}=\frac{P_{d}}{P_{i}+ P_{nl} + {\sigma_{n}^{2}}}. $$
(21)

And by substituting Eqs. (18), (19), and (20) in Eq. (21), the SINR expression can be rewritten as follows:

$$ {\begin{aligned} \text{SINR}&= \frac{d_{0}^{-\beta} E_{s} |H_{0}|^{2}}{ \sum\limits_{k=1}^{K} \sum\limits_{m \in F_{k}}d_{k}^{-\beta}|H_{k}(m)|^{2} \left[ |\alpha_{k}|^{2} E_{s} \chi(\psi_{k},m)+ \sigma_{\hat{\eta}_{0,k}^{0}}^{2}(\varepsilon_{k}) \right] + N_{0} B_{{sc}} }, \end{aligned}} $$
(22)

where B sc is the bandwidth of 0th subchannel.

In order to get a more explicit expression of the SINR, we suggest in the following subsection to analyze the expression of the received nonlinear noise variance \(\sigma _{\hat {\eta }_{0,k}^{0}}^{2}(\varepsilon _{k})\).

3.3 Variance of the received nonlinear noise

In this subsection, we will simplify the expression of the received nonlinear noise \(\hat {\eta }\) in order to find a closed-form expression of its variance \(\sigma _{\hat {\eta }}^{2}\). We note that the BS index will be omitted in this subsection for simplicity’s sake.

Recall that the received nonlinear noise is expressed by

$$ \hat{\eta}_{0}^{0}(\varepsilon)=Re\left\{ \int_{-\infty}^{\infty} \eta(t)h(t)e^{j(2\pi\varepsilon+\frac{\pi}{2T}) t} dt \right\}. $$
(23)

We consider PHYDYAS prototype filter whose impulse response is given by [20]

$$ g(t) = \left\{ \begin{array}{ll} 2\sum\limits_{q=0}^{\kappa-1} (-1)^{q} G_{q} \cos\left(\frac{2\pi}{\kappa T}qt\right) & \text{if} t \in [0,\kappa T],\\ 0 & \text{elsewhere}, \end{array} \right. $$
(24)

where κ is the overlapping factor and G q ,q=0,…,κ−1 are the frequency coefficients of the filter.

Furthermore, the nonlinear noise η(t) can be expressed as the sum of his real and imaginary components : η(t)=η r (t)+j η i (t).

Therefore, the received nonlinear noise \(\hat {\eta }_{0}^{0}\) can be rewritten as

$$\begin{array}{*{20}l} \hat{\eta}_{0}^{0} & =2\sum_{q=0}^{\kappa-1} (-1)^{q} G_{q} \left\{ \int_{0}^{\kappa T} \eta_{r}(t)\right. \\ &\quad\cos\left[\frac{2\pi}{\kappa T}qt \right] \cos\left[(2\pi\epsilon+\frac{\pi}{2T})t\right] dt \\ & \quad\left. \!- \int_{0}^{\kappa T} \eta_{i}(t) \cos\left[\frac{2\pi}{\kappa T}qt \right] \sin\left[(2\pi\epsilon+\frac{\pi}{2T})t\right] dt \right\}. \end{array} $$
(25)

After a few steps of straightforward manipulations and by considering the following observations : (i) η r (t)=η i (t)=η r is constant in the interval [0,κ T], (ii) the overlapping factor κ=4, one can find that

$$ \begin{aligned} \hat{\eta}_{0}^{0} & =-\frac{4T}{\pi}\eta_{r} \sin(4\pi T\epsilon) \left[ \cos(4\pi T\epsilon) - \sin(4\pi T\epsilon) \right]\\ &\quad\times\sum_{q=0}^{3} (-1)^{q} \frac{2G_{q} (4T\epsilon+1)}{q^{2}-(4T\epsilon+1)^{2}}. \end{aligned} $$
(26)

Therefore, the variance \(\sigma _{\hat {\eta _{k}}}^{2}\) of the received nonlinear noise from the kth interfering BS can be expressed as

$$ \begin{aligned} \sigma_{\hat{\eta_{k}}}^{2}(\varepsilon_{k})&=8 \frac{T^{2}}{\pi^{2}} c^{2} \sigma_{\eta_{k}}^{2}(4T\varepsilon_{k}+1)^{2} \sin^{2}(4\pi T\varepsilon_{k})\\ &\quad\times\left[1-\sin(8\pi T\varepsilon_{k})\right], = \sigma_{\eta_{k}}^{2} a(\varepsilon_{k}), \end{aligned} $$
(27)

where \(a(\varepsilon _{k})=8 \frac {T^{2}}{\pi ^{2}} c^{2}(\varepsilon _{k}) (4T\varepsilon _{k}+1)^{2} \sin ^{2}(4\pi T\varepsilon _{k}) \left [1-\sin (8\pi T\varepsilon _{k})\right ]\) and \(c(\varepsilon _{k})=2\sum _{q=0}^{3} (-1)^{q} \frac {G_{q}}{q^{2}-(4T\varepsilon _{k} +1)^{2}}\).

Then, the SINR expression will become as follows:

$$ {\begin{aligned} \text{SINR}&= \frac{|H_{0}|^{2}}{ \sum\limits_{k=1}^{K} \sum\limits_{m \in F_{k}} \frac{d_{k}}{d_{0}}^{-\beta}|H_{k}(m)|^{2} \left[ |\alpha_{k}|^{2} \chi(\psi_{k},m)+ \frac{1}{SNR_{\eta_{k}}} a(\varepsilon_{k}) \right] + b}, \end{aligned}} $$
(28)

where \(\text {SNR}_{\eta _{k}}=E_{s}/\sigma _{\eta _{k}}^{2}\) and \(b=\frac {1}{d_{0}^{-\beta } SNR} =\frac {N_{0} B_{sc}}{d_{0}^{-\beta } E_{s}}\).

To get a more simplified SINR expression, we introduce the following new variables

$$\chi_{d}(\psi_{k},m)=\left(\frac{d_{k}}{d_{0}}\right)^{-\beta} |\alpha_{k}|^{2} \chi(\psi_{k},m) $$

and

$$a_{d}(\varepsilon_{k})=\left(\frac{d_{k}}{d_{0}}\right)^{-\beta}a(\varepsilon_{k}). $$

Finally, we obtain the following final SINR expression:

$$ {\begin{aligned} \text{SINR}= \frac{|H_{0}|^{2}}{ \sum\limits_{k=1}^{K} \sum\limits_{m \in F_{k}} \left[ \chi_{d}(\psi_{k},m)+ \frac{1}{SNR_{\eta_{k}}} a_{d}(\varepsilon_{k}) \right] \left|H_{k}(m)\right|^{2} + b}. \end{aligned}} $$
(29)

4 Bit error rate expression

We know that the BER of 4-PAM modulation in AWGN channels, when the decision variables are “pure” Gaussian RVs, is given by the following expression [21]:

$$ \begin{aligned} \mathrm{BER(SNR)} &= \frac{3}{4}erfc \left(\sqrt{\frac{1}{5}\text{SNR}} \right), \end{aligned} $$
(30)

where erfc denotes the complementary error function given by

$$ erfc(x)=\frac{2}{\sqrt{\pi}} \int_{x}^{\infty} e^{-u^{2}} du, $$
(31)

and is SNR the signal to noise ratio.

In our analysis, we can obtain the conditional BER in the presence of interference by conditioning on the set of random variables \(\mathcal {H}=\left \{ H_{0}, H_{k}(m), \forall k,m \right \}\), as shown by the following equation:

$$ {\begin{aligned} &\mathrm{BER(SNR)}|_{\mathcal{H}} \\ &\quad=\frac{3}{4} erfc\left(\frac{1}{5} \frac{|H_{0}|^{2}}{ \sum\limits_{k=1}^{K} \sum\limits_{m \in F_{k}} \left[ \chi_{d}(\psi_{k},m)+ \frac{a_{d}(\varepsilon_{k})}{SNR_{\eta_{k}}} \right] \left|H_{k}(m)\right|^{2} + b} \right)^{1/2}. \end{aligned}} $$
(32)

Our objective is to derive the average BER which can be obtained by averaging out the vector of random variables \(\mathcal {H}\) from Eq. (32). Unfortunately, this average can not be calculated directly because we do not have a closed-form expression for the probability density function (pdf) of the SINR. In order to get the BER average with a reduced computational complexity, we refer to the following lemma [2] and we propose to use the results of [5].

Lemma 1

Let x be a unit-mean gamma random variable with parameter λ=1 and a pdf f(x)=e x,x≥0, and let y be an arbitrary non-negative random variable that is independent of x. Then

$$ {\begin{aligned} E_{x,y} \left[ \!erfc\left(\frac{x}{y+b} \right)^{1/2} \right] &= 1-\frac{1}{\sqrt{\pi}} \int_{0}^{\infty} \frac{e^{-z(1+b)}}{\sqrt{z}} \mathcal{M}_{y}(z) dz, \end{aligned}} $$
(33)

where \(\mathcal {M}_{y}(z)=E_{y}\left [ e^{-zy} \right ]\) is the moment generating function (MGF) of y.

By using this lemma and the results of [5], we can express the final expression of the average BER for the considered asynchronous and nonlinear FBMC-based multi-cellular network as follows:

$$ {\begin{aligned} \text{BER}_{Av}&= \frac{3}{4}-\frac{3}{4\sqrt{\pi}} \int\limits_{0}^{\infty} \frac{e^{-z(1+5b)}}{\sqrt{z}} \prod_{k=1}^{K} \left| I_{L_{k}} \left(1+ \frac{5 a_{d}(\varepsilon_{k})}{SNR_{\eta_{k}}} \right)\right.\\ &\quad\left.+\,5D_{k}^{\chi_{d}} \Omega_{k} z \right|^{-1} dz, \end{aligned}} $$
(34)

where

  • |A| stands for the determinant of the matrix A,

  • \(\Omega _{k}=\left [\rho _{i,j}\right ]_{(i,j)\in F_{k}\times F_{k}}\) is the correlation matrix of the random variables {|H k (m)|2,mF k },

  • L k denotes the number of subcarriers allocated to the kth BS and \(I_{L_{k}}\) is the L k ×L k identity matrix,

  • \(D_{k}^{\chi _{d}}\) is a diagonal matrix with diagonal elements \(D_{k}^{\chi _{d}}(i,i)=\chi _{d}(\psi _{k},i)\), iF k .

Equation (34) shows clearly that, in addition to the interference caused by synchronization errors, the BER performance of the studied system is affected by HPA nonlinearities and especially by the nonlinear noise η. A detailed analysis of this expression will be carried out in the next section via computer simulations.

5 Simulations

In this section, we present numerical results for the analytical BER expression that we have derived in the previous section for the considered CMT based multi-cellular network. The obtained BER expression will be evaluated using different evaluation parameters such as the signal to nonlinear noise ratio, the frequency error range, and the input back-off of the power amplifier. We consider the six neighboring cells surrounding the reference mobile user as interfering base stations. The cell radius is R=1 km and the path loss exponent β is equal to 3.76. Furthermore, we consider a CMT system with N=64 subcarriers transmitting 4-PAM modulated symbols. PHYDYAS prototype filter is used with overlapping factor κ=4. The PHYDYAS prototype filter is obtained using the frequency sampling technique described in [20]. The simulation curves are obtained using a solid state power amplifier (SSPA) model with AM/AM distortion given by the following expression:

$$ f_{{am}}[|e(t)|] =\upsilon \frac{|e(t)|}{\left(1+ \left(\frac{\upsilon |e(t)|}{A_{0}}\right)^{2p}\right)^{\frac{1}{2p}}}. $$
(35)

A 0 is the HPA input saturation level, υ is the small signal gain, and p is a parameter that controls the smoothness of the transition from the linear region to the saturation region.

Regarding the frequency specifications, they match those of the IEEE 802.11 a/g standards, i.e., a bandwidth of 2 MHz, a subcarrier spacing of 0.3125 MHz and a symbol duration of T=1.6 μs. We recall that the fractional normalized CFO ε N =2T ε is defined in the range −0.5≤ε N ≤0.5, which means that the CFO ε takes values between −0.16 and 0.16. In the curves presented in this section, ε will be taken in the range [0,0.16] (unless otherwise stated). A numerical computing environment (MATLAB) is used to simulate the interference CMT signals coming from the six interfering base stations. A 4-PAM CMT receiver is also simulated and the BER is directly computed by comparing the decoded symbols with the real transmitted symbols from the reference base station. Finally, for the Rayleigh fading channel model, we have considered the pedestrian-A model with relative delays [0 110 190 410] ns and corresponding average powers [0 –9.7 –19.2 –22.8] dB [23].

In Fig. 3, we plot the BER of the system as a function of the SNR using expression (34) (proposed model) and the expression obtained using the derivations of Medjahdi et al. [5] (which ignores HPA effects). The perfectly synchronized scenario without HPA nonlinear distortions is also considered for comparison purposes. Figure 3 evaluates also the accuracy of the theoretical results and compares it with the simulation results. In order to observe the evolution of the BER performance for different nonlinear distortion levels, the BER curves of Fig. 3 are plotted for various values of the signal to nonlinear noise ratio (SNR η =6,12, and 18 dB). As shown in Fig. 3, synchronization errors cause a severe degradation in the BER. This is found in agreement with the results of [5] where no HPA NLD effect is considered. Moreover, when the signals coming from the different interfering base stations are affected by a HPA nonlinearity as illustrated by the proposed model (Eq. (34)), the BER degradation is more significant. Example: to achieve a BER of 1.3×10−2, the asynchronous nonlinear system requires a value of SNR equal to 24 dB (When SNR η =12 dB) while the asynchronous system without HPA nonlinearities requires only 19 dB of SNR, which means a loss of 5 dB in the transmitted signal power for the former. Another important observation from Fig. 3 is that the BER performance depends strongly on the signal to nonlinear noise ratio SNR η . As this parameter decreases, the gap between the BER curves corresponding to the linear system and nonlinear ones becomes significant.

Fig. 3
figure 3

CMT average BER against the SNR for different values of signal to nonlinear noise ratio SNR η =6,12,18 dB

An important parameter that defines the amount of nonlinearity introduced by a HPA is the input back-off (IBO). A high IBO value means that the power amplifier operates close to its linear region, and therefore, it introduces less nonlinear distortions. Figure 4 shows the BER plots for two values of the IBO, IBO=4 dB and IBO=8 dB. From this figure, we can see that BER performance depends heavily on the HPA IBO, which is an expected result since there is a proportionality between the nonlinear noise variance and IBO as it was stated in [14].

Fig. 4
figure 4

The average BER for two input back-off values IBO=4,8 dB

In Fig. 5, we plot the average BER curves for three different frequency offset ranges. In the first range ε∈[0.02,0.06] (scenario a), in the second range ε∈[0.06,0.1] (scenario b), and in the third range ε∈[0.1,0.14] (scenario c). The perfectly synchronized (PS) scenario is also considered for comparison purposes. The results illustrated by Fig. 5 show that the BER degradation is also related to frequency errors and becomes more significant for high values of the carrier frequency offset. For example, to achieve a BER value of 2×10−2, the required SNR in the PS scenario is SNR=16 dB and in the asynchronous scenarios (b), (c), and (d) the required SNR is respectively equal to 18, 21, and 27 dB. These results show the high sensitivity of nonlinear CMT systems to carrier frequency offset and allow us to conclude by considering also the results of Figs. 3 and 4 that synchronization errors combined with nonlinear distortions cause a severe degradation in the performance of CMT based multi-cellular networks and a joint compensation of these two sources of distortion should be considered. Finally, we can see from the presented figures that the theoretical results show an excellent match to the corresponding simulation results despite an error floor at high SNR values (expected in interference limited scenarios).

Fig. 5
figure 5

The average BER for different CFO ranges. a PS scenario. b ε∈[0.02,0.06]. c ε∈[0.06,0.1]. d ε∈[0.1,0.14], the signal to nonlinear noise ratio is SNR η =8 dB

6 Conclusions

In this paper, we have studied jointly the effect of synchronization errors and HPA nonlinear distortions on asynchronous downlink CMT based multi-cellular networks. A scenario consisting on one reference mobile perfectly synchronized with its reference BS and K interfering BSs was considered and an exact BER expression was derived by carrying out an analytical interference analysis in the presence of a frequency-selective channel. In order to obtain explicit and less complex BER expressions, an interesting lemma based on the moment generating function of the interference power was applied. The obtained theoretical BER expression has been evaluated through simulation results using different evaluation parameters such as the signal to nonlinear noise ratio, the frequency error range, and the input back-off of the power amplifier. We found a perfect match between the simulation and the developed theoretical results. Furthermore, we have highlighted the significant performance degradation caused by the joint effect of HPA nonlinear distortions and synchronization errors.