Though the conventional pre-processing filters, such as null space projection (NSP) or regularized channel inversion, can eliminate IUI, these filters require the larger number of transmit antennas than that of receive antennas. This means that these filters are unsuitable to user devices because the pair of transmit and receive antennas at each user device is connected with one RF chain, which implies that each user has the same number of transmit and receive antennas. Thus, we propose pre-processing filters for IUI cancellation as well as transmit diversity gain when each user has the same number of transmit and receive antennas. We assume that the channel remains constant over two symbol periods. Under the condition that the DL user sums two successive received signals for two symbol periods, BS and UE pre-processing filters are designed to achieve transmit diversity and to cancel IUI, respectively. To design pre-processing filters individually, we divide the received signal at the DL user into desired signal, \(\textbf {y}^{d}_{DL}\), and IUI signal, \(\textbf {y}^{IUI}_{DL}\), where \(\textbf {y}^{d}_{DL}\) means the received signal from the BS and \(\textbf {y}^{IUI}_{DL}\) denotes the received signal from the UL user.
Transmission scheme and filter design at BS for diversity gain
In this section, we design a transmission scheme and a pre-processing filter at BS to obtain transmit diversity gain. The symbol vector, s, is transmitted at the first symbol period and the symbol vector, and s∗, is transmitted at the second symbol period from the BS to the the DL user, where s∗ means the complex conjugate of s. A rotation receive filter at the DL user, i.e. GUE= G, is multiplied to the complex conjugate of the second received signal vector and this vector is added to the first received signal vector. Then, the received desired signal at the DL user can be expressed as
$$ \begin{array}{l} {\textbf{y}^{d}_{DL,1}} + \textbf{G}{\left({\textbf{y}_{DL,2}^{d}} \right)^{*}}\\ = \sqrt {\frac{{{P_{DL}}}}{2}} {\textbf{H}_{DL}}{\textbf{F}_{BS,1}}\textbf{s} + \left[ {\begin{array}{*{20}{l}} 0&{ - 1}\\ 1&{\,0} \end{array}} \right]\sqrt {\frac{{{P_{DL}}}}{2}} \textbf{H}_{DL}^{*}\textbf{F}_{BS,2}^{*}\textbf{s} + \textbf{n}\\ = \sqrt {\frac{{{P_{DL}}}}{2}} {\left[ {\begin{array}{*{20}{l}} {{h_{1,1}}}&{{h_{1,2}}}&{ - h_{2,1}^{*}}&{ - h_{2,2}^{*}}\\ {{h_{2,1}}}&{{h_{2,2}}}&{h_{1,1}^{*}}&{h_{1,2}^{*}} \end{array}} \right]}\left[ {\begin{array}{*{20}{l}} {{\textbf{F}_{BS,1}}}\\ {\textbf{F}_{BS,2}^{*}} \end{array}} \right]\textbf{s} + \textbf{n}, \end{array} $$
(4)
where hi,j denotes an element of HDL and n is the sum of noises during two symbol periods. Since the total transmit power is PDL, the transmit power at each symbol period is PDL/2. If we set \({\textbf {H}_{eff}} \buildrel \Delta \over = \left [ {\begin {array}{*{20}{c}}{{\textbf {H}_{DL}}}&{{\bf {GH}}_{DL}^{*}}\end {array}} \right ]\), the effective DL channel matrix, Heff, is the orthogonal, i.e. \(\textbf {H}_{eff}{\textbf {H}_{eff}}^{H} = \left \| {{\textbf {H}_{DL}}} \right \|_{F}^{2}\textbf {I}\), where (∙)H denotes the conjugate transpose, ∥∙∥F means the Frobenius norm, and I is the identity matrix. Then, we can set \({\textbf {F}_{eff}} \buildrel \Delta \over = {\left [ {\begin {array}{*{20}{c}}{\textbf {F}_{BS,1}^{T}}&{{{\left ({\textbf {F}_{BS,2}^{*}} \right)}^{T}}}\end {array}} \right ]^{T}} = {\sqrt {2}\bf {H}}_{eff}^{H} /\sqrt {\left \| \textbf {H}_{eff} \right \|_{F}^{2}}\), where \(\sqrt {2}/\sqrt {\left \| \textbf {H}_{eff} \right \|_{F}^{2}}\) is the power normalization factor, and (4) can be represented as
$$ \textbf{y} = \sqrt {\frac{P_{DL}}{2}} {\frac{\sqrt{2}\left\| {{\textbf{H}_{DL}}} \right\|^{2}_{F}}{\sqrt{\left\| {{\textbf{H}_{eff}}} \right\|^{2}_{F}}}}{\bf{Is}} + \textbf{n}. $$
(5)
Then, the received signal-to-noise ratio (SNR), γ, per symbol is
$$ \gamma = \frac{{\left\| {{\textbf{H}_{DL}}} \right\|_{F}^{4}}}{{\left\| {{\textbf{H}_{eff}}} \right\|_{F}^{2}}}\rho = \frac{{\left\| {{\textbf{H}_{DL}}} \right\|_{F}^{4}}}{{2\left\| {{\textbf{H}_{DL}}} \right\|_{F}^{2}}}\rho = \frac{{\left\| {{\textbf{H}_{DL}}} \right\|_{F}^{2}}}{2}\rho, $$
(6)
where ρ means transmit SNR. This received SNR is the same with the Alamouti scheme, and we can obtain the diversity order of 4 [14]. Thus, the pre-processing filters at BS to obtain diversity gain are given as
$$ {\textbf{F}_{BS,1}} = \frac{\sqrt{2}{\textbf{H}_{DL}^{H}}}{{\sqrt {\left\| {{\textbf{H}_{eff}}} \right\|_{F}^{2}} }},\,\,\,\,{\textbf{F}_{BS,2}} = \frac{\sqrt{2}{{{\left({\textbf{G}{\textbf{H}_{DL}}} \right)}^{H}}}}{{\sqrt {\left\| {{\textbf{H}_{eff}}} \right\|_{F}^{2}} }}. $$
(7)
Pre-processing filter at the UL user for IUI cancellation
To cancel IUI, we design a pre-processing filter at the UL user based on NSP. If the UL user transmits the complex conjugate of the previous transmitted symbol at the second symbol period, the UL user can make the NSP pre-processing filter by using two symbol periods. Then, the summation of the received IUI signal at the DL user can be expressed as
$$ \begin{array}{l} \textbf{y}_{DL,1}^{IUI} + \textbf{G}{\left({\textbf{y}_{DL,2}^{IUI}} \right)^{*}}\\ = \sqrt {\frac{P_{UL}}{2}}\alpha {\textbf{H}_{IUI}}{\textbf{F}_{UE,1}}\textbf{x} + \sqrt {\frac{P_{UL}}{2}}\alpha {\bf{GH}}_{IUI}^{*}\textbf{F}_{UE,2}^{*}\textbf{x} + \textbf{n},\\ \end{array} $$
(8)
where FUE,i is the UL pre-processing filter at the i-th symbol period for canceling IUI, and \(\alpha = \sqrt {\frac {2}{{\left \| {{\textbf {F}_{UE,1}}} \right \|_{F}^{2} + \left \| {{\textbf {F}_{UE,2}}} \right \|_{F}^{2}}}}\) represents the UL power normalization factor. If we define an effective IUI channel as \(\textbf {H}_{IUI,eff} \buildrel \Delta \over = \textbf {H}_{IUI}\textbf {F}_{UE,1}\), the IUI cancellation filter of the second symbol period, FUE,2, can be designed to satisfy the following:
$$ \left({{\textbf{H}_{IUI,eff}} + {\bf{GH}}_{IUI}^{*}\textbf{F}_{UE,2}^{*}} \right)\textbf{x} = 0.\ $$
(9)
Then, FUE,2 can be expressed as
$$ {\textbf{F}_{UE,2}} = - {\left({{\textbf{G}^{*}}{\textbf{H}_{IUI}}} \right)^{\dag} }\textbf{H}_{IUI,eff}^{*}, $$
(10)
where (∙)† means the pseudo inverse of matrix. If we set the UL pre-processing filter of the first symbol period to identity matrix, FUE,1=I, (10) can be represented as
$$ {\textbf{F}_{UE,2}} = - {\left({{\textbf{G}^{*}}{\textbf{H}_{IUI}}} \right)^{\dag}}\textbf{H}_{IUI}^{*}. $$
(11)
Then, the summation of the received signal at the DL user for two symbol periods can be expressed as
$$ {\begin{aligned} \begin{array}{*{20}{l}} &{{\textbf{y}_{DL,1}} + {\bf{Gy}}_{DL,2}^{*}}\\ &\quad= \sqrt {\frac{{{P_{DL}}}}{2}} {\textbf{H}_{DL}}{\textbf{F}_{BS,1}}\textbf{s} + \sqrt {\frac{{{P_{UL}}}}{2}} \alpha {\textbf{H}_{IUI,eff}}\textbf{x}\\ &\quad+ \sqrt {\frac{{{P_{DL}}}}{2}} {\bf{GH}}_{DL}^{*}\textbf{F}_{BS,2}^{*}\textbf{s} + \sqrt {\frac{{{P_{UL}}}}{2}} \alpha {\bf{GH}}_{IUI}^{*}\textbf{F}_{UE,2}^{*}\textbf{x} + \textbf{n}\\ &\quad= \sqrt {\frac{{{P_{UL}}}}{2}} \alpha \left({{\textbf{H}_{IUI,eff}} - {\bf{GH}}_{IUI}^{*}{{\left({{\bf{GH}}_{IUI}^{*}} \right)}^{\dag} }{\textbf{H}_{IUI,eff}}} \right)\textbf{x}\\ &\quad+ \sqrt {\frac{{{P_{DL}}}}{2}} \left[ {\begin{array}{*{20}{l}} {{\textbf{H}_{DL}}}&{{\bf{GH}}_{DL}^{*}} \end{array}} \right]\frac{{\sqrt 2 \textbf{H}_{eff}^{*}}}{{\sqrt {\left\| {{\textbf{H}_{eff}}} \right\|_{F}^{2}} }}\textbf{s} + \textbf{n}\\ &\quad= \frac{{\sqrt {{P_{DL}}} }}{2}\left\| {{\textbf{H}_{DL}}} \right\|_{F}^{2}{\bf{Is}} + \textbf{n}. \end{array} \end{aligned}} $$
(12)
Thus, we can cancel IUI and obtain diversity gain by using the proposed scheme. Importantly, note that the proposed scheme can obtain full diversity without SNR loss unlike the Alamouti scheme. In the Alamouti scheme, the DL user cannot obtain full diversity gain since IUI exists in two-user FD scenario. Although the Alamouti scheme can obtain full diversity without IUI cancellation by treating IUI as noise, the signal-to-interference and noise ratio (SINR) is degraded due to IUI, and this induces BER performance loss. Also, the Alamouti scheme with interference cancellation derives only two diversity gain in our system model [16, 17]. To obtain full diversity without SNR loss in the Alamouti scheme, IUI cancellation is required at the UL user by a pre-processing filter. However, since the number of spatial domain of a pre-processing filter at the UL user is not larger than the rank of effective IUI channel, the UL user cannot make a pre-processing filter to cancel IUI. However, the proposed scheme achieves both IUI cancellation and full diversity gain by using the pre-processing filters at the UL user and BS, since the number of spatial and time domains of the UL user is two times the rank of the IUI channel.
Extension to the arbitrary number of antennas system
The pre-processing filter of UL user can be designed to cancel IUI regardless of the number of user antennas if the number of transmit antennas of the UL user is not smaller than that of receive antennas of the DL user. Also, the proposed BS transmission scheme can be extended to the arbitrary number of BS transmit antennas, so that it achieves \(2M_{T}^{BS}\) diversity gain, where \(M_{T}^{BS}\) is the number of BS transmit antennas.
$$\begin{array}{*{20}l} {}\textbf{y}_{DL,1}^{d} + \textbf{G}{\left({\textbf{y}_{DL,2}^{d}} \right)^{*}} &= \left[\! {\begin{array}{*{20}{c}} {{\textbf{h}_{DL}^{\left(1 \right)}}}\\ {{\textbf{h}_{DL}^{\left(2 \right)}}} \end{array}} \right]\!{\textbf{F}_{BS,1}}\textbf{s}\! +\!\! \left[ \!{\begin{array}{*{20}{c}} 0&{ - 1}\\ 1&0 \end{array}}\! \right]\!\!{\left[\! {\begin{array}{*{20}{c}} {{\textbf{h}_{DL}^{\left(1 \right)}}}\\ {{\textbf{h}_{DL}^{\left(2 \right)}}} \end{array}}\! \right]^{*}}\!\textbf{F}_{BS,2}^{*}\textbf{s}\\ &= \left[ {\begin{array}{*{20}{c}} {{\textbf{h}_{DL}^{\left(1 \right)}}}&{ - {\left({\textbf{h}_{DL}^{\left(2 \right)}} \right)^{*}}}\\ {{\textbf{h}_{DL}^{\left(2 \right)}}}&{{\left({\textbf{h}_{DL}^{\left(1 \right)}} \right)^{*}}} \end{array}} \right]\left[ {\begin{array}{*{20}{c}} {{\textbf{F}_{BS,1}}}\\ {\textbf{F}_{BS,2}^{*}} \end{array}} \right]\textbf{s}, \end{array} $$
(13)
where \({\textbf {h}_{DL}^{\left (i \right)}}\) represents the \( 1 \times M_{T}^{BS}\) channel vector from BS to the ith receive antenna of DL user. Since the effective channel is a form of orthogonal matrix as in Section 3.1, \(2M_{T}^{BS}\) diversity gain can be obtained if the pre-processing filter of BS consists of the Hermitian matrix of Heff, i.e. \({\textbf {F}_{BS,1}} = \left [ {\begin {array}{*{20}{c}}{{\left ({\textbf {h}_{DL}^{\left (1 \right)}} \right)^{H}}}&{{\left ({\textbf {h}_{DL}^{\left (2 \right)}} \right)^{H}}}\end {array}} \right ]\)and \(\textbf {F}_{BS,2}^{*} = \left [ {\begin {array}{*{20}{c}}{ - {{\left ({\textbf {h}_{DL}^{\left (2 \right)}} \right)^{T}}}}&{{{\left ({\textbf {h}_{DL}^{\left (1 \right)}} \right)^{T}}}}\end {array}} \right ]\), where BS transmits two symbols like in the case of \(M_{T}^{BS}=2\). However, the pre-processing filter of BS is hard to be extended to the case of arbitrary number of antennas of DL user. Thus, we consider on antenna selection scheme to extend our proposed scheme to the arbitrary number of user antennas. For the arbitrary number of receive antennas of the DL user, i.e. \(M_{R}^{UE}\), we can select two receive antennas by using the rotation matrix. Since the proposed BS transmission scheme can make effective channel matrix become orthogonal when two receive antennas are selected, the channel gain-based antenna selection method can be used. However, independent antenna selection makes the rotation matrix vary where two receive antennas with the largest channel gains are selected. This means that DL user should inform the information of rotation matrix to UL user, and it induces the feedback overhead. Thus, we propose a subset based antenna selection scheme to remove the feedback of rotation matrix. Each subset can be constructed by sequentially combining two consecutive antennas; the rotation matrix is expressed as a block diagonal matrix of G:
$$ {\textbf{G}^{e}} = \left[ {\begin{array}{*{20}{c}} \textbf{G}&{{{\bf{0}}_{2 \times 2}}}&{{{\bf{0}}_{2 \times 2}}}& \cdots &{{{\bf{0}}_{2 \times 2}}}\\ {{{\bf{0}}_{2 \times 2}}}&G&{{{\bf{0}}_{2 \times 2}}}& \cdots &{{{\bf{0}}_{2 \times 2}}}\\ {{{\bf{0}}_{2 \times 2}}}&{{{\bf{0}}_{2 \times 2}}}& \ddots & \cdots & \vdots \\ \vdots & \vdots & \vdots &G&{{{\bf{0}}_{2 \times 2}}}\\ {{{\bf{0}}_{2 \times 2}}}&{{{\bf{0}}_{2 \times 2}}}& \ldots &{{{\bf{0}}_{2 \times 2}}}&\textbf{G} \end{array}} \right]. $$
(14)
where Ge is the \(M_{R}^{UE} \times M_{R}^{UE}\) rotation matrix of the DL user with the arbitrary number of receive antennas. It makes the orthogonality between two effective subchannels within each subset:
$$ {}\textbf{H}_{eff}^{e} \,=\, \left[\! {\begin{array}{*{20}{c}} {{\textbf{h}_{1}}}\\ {{\textbf{h}_{2}}}\\ \vdots \\ {{\textbf{h}_{M_{R}^{UE} - 1}}}\\ {{\textbf{h}_{M_{R}^{UE}}}} \end{array}}\! \right] + {\textbf{G}^{e}}\left[ {\begin{array}{*{20}{c}} {{\textbf{h}_{1}}}\\ {{\textbf{h}_{2}}}\\ \vdots \\ {{\textbf{h}_{M_{R}^{UE} - 1}}}\\ {{\textbf{h}_{M_{R}^{UE}}}} \end{array}} \right] \! =\! \left[\! {\begin{array}{*{20}{c}} {{\textbf{h}_{1}}}&{ - \textbf{h}_{2}^{*}}\\ {{\textbf{h}_{2}}}&{\textbf{h}_{1}^{*}}\\ \vdots & \vdots \\ {{\textbf{h}_{M_{R}^{UE} - 1}}}&{ - \textbf{h}_{M_{R}^{UE}}^{*}}\\ {{\textbf{h}_{M_{R}^{UE}}}}&{\textbf{h}_{M_{R}^{UE} - 1}^{*}} \end{array}}\! \right]\!. $$
(15)
Then, each channel gain of the ith subset after applying the proposed scheme can be derived as
$$ {S_{i}} = \left| {\left| {\textbf{h}_{DL}^{\left(i,1 \right)}} \right|} \right|_{F}^{2} + \left| {\left| {\textbf{h}_{DL}^{\left(i,2 \right)}} \right|} \right|_{F}^{2}, $$
(16)
where \({\textbf {h}_{DL}^{\left (i,1 \right)}}\) and \({\textbf {h}_{DL}^{\left (i,2 \right)}}\) represent two subchannels of the ith subset. Then, the subset selection scheme can be applied based on the gain of each subset:
$$ {i^{*}} = \mathop {\max }\limits_{i} {S_{i}}, $$
(17)
and the pre-processing filter of BS can be written as
$$\begin{array}{*{20}l} \textbf{F}_{BS,1}^{e} = \frac{{{{\left({\textbf{H}_{DL}^{\left({{i^{*}}} \right)}} \right)}^{H}}}}{{\sqrt {\left\| {\textbf{H}_{DL}^{\left({{i^{*}}} \right)}} \right\|_{F}^{2}} }},\,\,\,\textbf{F}_{BS,2}^{e} = \frac{{{{\left({{\bf{GH}}_{DL}^{\left({{i^{*}}} \right)}} \right)}^{H}}}}{{\sqrt {\left\| {\textbf{H}_{DL}^{\left({{i^{*}}} \right)}} \right\|_{F}^{2}} }}, \end{array} $$
(18)
where \({\textbf {H}_{DL}^{\left ({{i^{*}}} \right)}}\) means \(2 \times M_{T}^{BS}\) channel matrix between the BS and the i∗th subset receive antennas of DL user. Therefore, DL user can detect the signal by using the antennas contained in the i∗th subset, and each received SNR can be expressed as
$$ \gamma^{e} = \frac{\left| {\left| {\textbf{h}_{DL}^{\left(i,1 \right)}} \right|} \right|_{F}^{2} + \left| {\left| {\textbf{h}_{DL}^{\left(i,2 \right)}} \right|} \right|_{F}^{2}}{2}\rho. $$
(19)
Thus, the proposed scheme antenna selection scheme can achieve selection diversity gain in addition to the \(2M_{T}^{BS}\) diversity gain in general system with the arbitrary number of antennas.
Although the proposed scheme has limitation in extending to general multi-user environment, it is possible to increase the number of UL users if only one DL user is supported. Since the pre-processing filters to cancel IUI from each UL user are designed at each user, residual IUI of the proposed scheme does not exist regardless of the number of UL users. This means that DL user does not need to consider IUI from multiple UL users. In the case of the increase of the number of DL users, IUI can be eliminated if additional time durations are used as much as the number of DL users. However, it is inefficient since this technique causes the performance loss of UL spectral efficiency due to the repetition transmission of the same information. Thus, the proposed scheme can be applied to the system supporting multiple UL users and one DL user.