1 Introduction

The popularity of smartphone and tablet computers that can connect to wireless local area networks (LANs) has led to a rapid expansion of such systems by deploying huge numbers of access points (APs). A standard for very high throughput (VHT) wireless LAN systems has been issued as IEEE 802.11ac (11ac) [1]. In 11ac, a data rate of more than 1 Gbit/s can be achieved by expanding to the maximum transmission bandwidth to 80 MHz (or 160 MHz as an optional feature). However, since the frequency resource available for wireless LANs is limited, the number of available non-overlapping channels will be reduced in such a dense AP environment. As a result, the overlapping basic service sets (OBSSs) give rise to inter-cell interference (ICI) [2, 3]. Various techniques have been studied for mitigating this ICI through the use of time and/or frequency resource sharing. Time resource sharing (TRS) is achieved through the use of carrier sense multiple access with collision avoidance (CSMA/CA) or request to send/clear to send (RTS/CTS) protocols [4, 5], which allocate different time durations to each access. In contrast, frequency resource sharing avoids ICI by allocating a different channel (or frequency band) to each cell. In addition, a dynamic channel assignment (DCA) scheme has been proposed as a way to improve spectrum efficiency [6]. However, all of these schemes merely divide up the limited time and frequency resources in each cell and result in severe system throughput degradation when the number of OBSSs increases.

In May 2013, the high-efficiency wireless (HEW) LAN study group (SG) started its work for a new standard amendment in the post-11ac era [7] and the HEW SG became the task group (TG) ax in May 2014. The TGax is studying ways to address the OBSS problem through efficient use of spatial and/or frequency resources and has identified simultaneous transmission as an attractive candidate for improving spectrum efficiency in dense wireless LAN environments [8, 9]. In this paper, we focus on a spatial resource sharing scheme for ICI mitigation. There are many related studies on spatial resource sharing for ICI mitigation [1017]. These techniques can be categorized into two control schemes: centralized [1013] and distributed [1417]. It has been reported that centralized control schemes can achieve high throughput in [1013]; however, all the APs must be synchronized and share all of the transmit data and channel state information (CSI). In contrast, distributed schemes improve the throughput by using the CSI between the AP and stations (STAs) associated with the AP and between the AP and STAs associated with the other APs in the OBSS. In distributed schemes, all APs do not need to be coordinated to share data streams and can be connected to different wired networks. Although distributed schemes provide less of a throughput improvement than centralized ones do, they significantly lighten the requirements on the APs. Since the APs of wireless LANs generally have different owners, the distributed scheme is more suitable for them. For these reasons, in this paper, we focus on an ICI mitigation scheme using transmit beamforming based on a distributed scheme.

As cited above, a number of studies have investigated distributed schemes. Ref. [14] considered multi-cell beamforming based on zero forcing (ZF) under limited backhaul information exchange in cellular systems, and Ref. [15] considered joint beamforming to mitigate the ICI of a primary user in cognitive radio networks. We have proposed an ICI mitigation scheme using beamforming for wireless LAN systems [13, 14]. In this scheme, ICI mitigation is carried out among multiple APs by using ZF transmit beamforming in downlink (DL) multiuser multiple-input multiple-output (MU-MIMO). This scheme suppresses ICI by using CSI between the AP and STAs associated with the other APs in the OBSS, even if the APs are connected to different wired networks. However, the conventional ICI mitigation techniques perform null beamforming for STAs subjected to both large and small ICI effects, even though in the latter case almost no performance degradation occurs in spite of the limitation on the number of antennas at the AP. Consequently, the spatial resource cannot be used sufficiently to improve system throughput.

This paper proposes a cooperative ICI mitigation scheme using transmit beamforming in DL MU-MIMO transmission for wireless LAN systems. The proposed scheme makes selective use of transmit beamforming on the basis of the ICI power for each STA in the neighboring cell. Null beamforming is performed if the ICI power exceeds a threshold for mitigation to kick in; otherwise, it is not carried out and the transmit antenna is used to obtain a diversity gain for STAs in the basic service set (BSS). Since the proposed scheme refers to the ICI power level as a basis for selecting either null beamforming or using the transmit antenna to obtain a diversity gain, it enables the spatial resource to be efficiently used and the system throughput to be improved. Computer simulation results confirm that it can improve the achievable rate in an OBSS environment.

The remainder of this paper is organized as follows. We describe the system model in an OBSS environment in Sect. 2 and the proposed scheme in Sect. 3. In Sect. 4, we describe simulation results that verify the scheme’s effectiveness. Our conclusions are presented in Sect. 5.

The following notations are used throughout this paper: Column vectors (matrices) are denoted by boldface lower (upper) case letters, superscripts \(^{T}\) and \(^{H}\) respectively stand for transpose and conjugate transpose, and \(\mathbf{I}_{n}\) denotes an \(n\times n\) identity matrix.

2 System Model in an OBSS Environment

Figure 1 shows the system model in an OBSS environment. We consider that there are two APs and each AP transmits data streams to \(M\) STAs in DL MU-MIMO transmission. Each AP and STA has \(K\) and \(L\) antennas, respectively. In the OBSS environment, the received signal vectors \(\mathbf{r}_{0,i}\in {\mathbb {C}}^{L\times 1}\) and \(\mathbf{r}_{1,i}\in {\mathbb {C}}^{L\times 1}\) at \(\hbox {STA}_{0,i}\, (i=0,{\ldots },M-1)\) and \(\hbox {STA}_{1,j}\,(j=0,{\ldots },M-1)\) associated with \(\hbox {AP}_{0}\) and \(\hbox {AP}_{1}\) are respectively given by

$$\begin{aligned} \left\{ { \begin{array}{l} \mathbf{y}_{0,i} =\mathbf{H}_{D,0,i} \mathbf{W}_{0}\mathbf{x}_{0}+\mathbf{H}_{I,1,i} \mathbf{W}_{1}\mathbf{x}_{1}+\mathbf{n}_{i}\\ \mathbf{y}_{1,j} =\mathbf{H}_{D,1,j} \mathbf{W}_{1}\mathbf{x}_{1}+\mathbf{H}_{I,0,j} \mathbf{W}_{0}\mathbf{x}_{0}+\mathbf{n}_{j}\\ \end{array}} \right. , \end{aligned}$$
(1)

where \(\mathbf{H}_{D,0,i} \in {\mathbb {C}}^{L\times K}\) and \(\mathbf{H}_{D,1,j} \in {\mathbb {C}}^{L\times K}\) are CSI matrices between \(\hbox {AP}_{0}\) and \(\hbox {STA}_{0,i}\), and \(\hbox {AP}_{1}\) and \(\hbox {STA}_{1,j}\), respectively. \(\mathbf{H}_{I,1,i} \in {\mathbb {C}}^{L\times K}\) and \(\mathbf{H}_{I,0,j} \in {\mathbb {C}}^{L\times K}\) are respectively CSI matrices of the ICI component between \(\hbox {AP}_{1}\) and \(\hbox {STA}_{0,i}\), and between \(\hbox {AP}_{0}\) and \(\hbox {STA}_{1,j}\). \(\mathbf{n}_{i}\in {\mathbb {C}}^{L\times 1}\) and \(\mathbf{n}_{j}\in {\mathbb {C}}^{L\times 1}\) are the additive white Gaussian noise vectors and all of the elements in \(\mathbf{n}_{i}\) and \(\mathbf{n}_{j}\) are assumed to have a common variance \(\sigma ^{2}\). \(\mathbf{W}_{0}\in {\mathbb {C}}^{K\times LM}\) and \(\mathbf{W}_{1}\in {\mathbb {C}}^{K\times LM}\) are transmit beamforming weights at \(\hbox {AP}_{0}\) and \(\hbox {AP}_{1}\). \(\mathbf{x}_{0}\in {{\mathbb {C}}}^{LM\times 1}\) and \(\mathbf{x}_{1}\in {\mathbb {C}}^{LM\times 1}\) are transmit data symbol vectors from \(\hbox {AP}_{0}\) and \(\hbox {AP}_{1}\). They are given by

$$\begin{aligned}&\left\{ { \begin{array}{l} \mathbf{W}_{0}=\left[ {\mathbf{W}_{0,0} \ldots \mathbf{W}_{0,i} \ldots \mathbf{W}_{0,M-1}}\right] \\ \mathbf{W}_{1}=\left[ {\mathbf{W}_{1,0} \ldots \mathbf{W}_{1,j} \ldots \mathbf{W}_{1,M-1}}\right] \\ \end{array}} \right. , \end{aligned}$$
(2)
$$\begin{aligned}&\left\{ {\begin{array}{l} \mathbf{x}_{0}=\left[ {\mathbf{x}_{0,0}^{T}\ldots \mathbf{x}_{0,i}^{T}\ldots \mathbf{x}_{0,M-1}^{T}}\right] ^{T} \\ \mathbf{x}_{1}=\left[ {\mathbf{x}_{1,0}^{T}\ldots \mathbf{x}_{1,j}^{T}\ldots \mathbf{x}_{1,M-1}^{T}}\right] ^{T} \\ \end{array}} \right. , \end{aligned}$$
(3)

where \(\mathbf{W}_{0,i}\in {\mathbb {C}}^{K\times L}\) and \(\mathbf{W}_{1,j}\in {\mathbb {C}}^{K\times L}\) are transmit beamforming weights for \(\hbox {STA}_{0,i}\) and \(\hbox {STA}_{1,j}\) at \(\hbox {AP}_{0}\) and \(\hbox {AP}_{1}\). \(\mathbf{x}_{0,i}\in {\mathbb {C}}^{L\times 1}\) and \(\mathbf{x}_{1,j}\in {\mathbb {C}}^{L\times 1}\) are respectively transmit data symbol vectors from \(\hbox {AP}_{0}\) to \(\hbox {STA}_{0,i}\), and from \(\hbox {AP}_{1}\) to \(\hbox {STA}_{1,j}\). The proposed scheme suppresses the ICI by using transmit beamforming at each AP in the OBSS environment.

Fig. 1
figure 1

System model of an OBSS environment in a two AP scenario (“D” is the desired signal and “I” is the interference signal).

3 Proposed Scheme for ICI Mitigation

3.1 ICI Mitigation Using Transmit Beamforming

Table 1 shows the transmission procedure. Two APs use the same frequency channel and time slot and simultaneously carry out DL MU-MIMO transmission. Thus, each AP uses the ZF transmit beamforming to prevent ICI from affecting the STAs in the neighboring cell. The CSI matrices between each AP and the STAs in the BSS, \(\mathbf{H}_{D,0,i}\) and \(\mathbf{H}_{D,1,j}\), are estimated. The CSI matrices between each AP and the STAs associated with the AP in the OBSS and whose ICI power is larger than threshold \(P_{th}\), \(\mathbf{H}_{I,1,i}\) and \(\mathbf{H}_{I,0,j}\), are also estimated. 802.11 standard specifies both implicit and explicit feedback for CSI acquisition [18]. In high density environments, implicit feedback achieves high efficiency by reducing the amount of feedback for CSI acquisition. The simple extension of implicit feedback in 801.11n for cooperative ICI mitigation system is described in [19, 20]. Here, it is assumed with loss of generality that the power levels of ICI affecting \(\hbox {STA}_{0,i^{\prime }}\,(i^{\prime } =0,{\ldots },~U_{0}-1,~U_{0}\le M) \,P_{ICI,i^{\prime }}\) and \(\hbox {STA}_{1,j^{\prime }}\,(j^{\prime }=0,{\ldots },~U_{1}-1\), \(U_{1}\le M) \,P_{ICI,j^{\prime }}\) are larger than \(P_{th}\). The ZF transmit beamforming weights are calculated using \(\tilde{\mathbf{H}}_{D,0,i}\), \(\tilde{\mathbf{H}}_{D,1,j}\), \(\tilde{\mathbf{H}}_{I,1,j}\), and \(\tilde{\mathbf{H}}_{I,0,i}\) which are the estimates of \(\mathbf{H}_{D,0,i}\), \(\mathbf{H}_{D,1,j}\), \(\mathbf{H}_{I,1,i}\) and \(\mathbf{H}_{I,0,j}\).

Table 1 Transmission procedure of the proposed scheme

In \(\hbox {AP}_{0}\), the transmit beamforming weight for \(\hbox {STA}_{0,k}\) is determined so as to prevent the intra-cell interference from affecting \(\hbox {STA}_{0,i}\,(i=0,{\ldots },M-1,~i\ne k)\) associated with \(\hbox {AP}_{0}\) and the ICI affecting \(\hbox {STA}_{1,j^{\prime }}\,(j^{\prime }=0,{\ldots },~U_{1}-1)\) associated with \(\hbox {AP}_{1}\). By using singular value decomposition (SVD), the interference CSI matrix for \(\hbox {STA}_{0,k}\) is expressed as

$$\begin{aligned} \tilde{\mathbf{H}}^{{\varvec{\prime }}}_{I,0,k}&= \left[ {\tilde{\mathbf{H}}_{D,0,0}^T \;\ldots \;\tilde{\mathbf{H}}_{D,0,k-1}^{T}\;\tilde{\mathbf{H}}_{D,0,k+1}^T \;\ldots \;\tilde{\mathbf{H}}_{D,0,M-1}^{T}\;\tilde{\mathbf{H}}_{I,1,0}^T \;\ldots \;\tilde{\mathbf{H}}_{I,1,U_{1}-1}^{T}}\right] ^{T} \nonumber \\&= \mathbf{U}_{I,0,k} \left[ {{ \begin{array}{cc} {{\varvec{\Sigma }}_{I,0,k}}&{} \mathbf{0} \\ \end{array}}}\right] \left[ {{\begin{array}{cc} {\mathbf{V}_{I,0,k}^{(s)}}&{\mathbf{V}_{I,0,k}^{(n)}} \end{array}}}\right] ^{H}, \end{aligned}$$
(4)

where \(\mathbf{U}_{I,0,k}\in {\mathbb {C}}^{L(M+U_{1}-1)\times L(M+U_{1} -1)}\) denotes the left singular vectors and the diagonal elements of \({\varvec{\Sigma }}_{I,0,k}\) are the square roots of \(\lambda _{0,0}\), \(\lambda _{0,1},{\ldots }\), \(\lambda _{0,K-1}\), which are null space eigenvalues. \(\mathbf{V}_{I,0,k}^{(s)} \in {\mathbb {C}}^{K\times L(M+U_{1}-1)}\) and \(\mathbf{V}_{I,0,k}^{(n)} \in {\mathbb {C}}^{K\times (K-L(M+U_{1}-1))}\) respectively represent the right singular vectors for the signal space and the null space. Here, \(\mathbf{V}_{I,0,k}^{(n)}\) is a set of vectors that do not interfere with the other STAs in the BSS and OBSS. If the channel estimates of \(\tilde{\mathbf{H}}^{{\varvec{\prime }}}_{I,0,k}\) are perfect, the utilization of \(\mathbf{V}_{I,0,k}^{(n)}\) makes it possible to prevent the interference from affecting \(\hbox {STA}_{0,k}\,(i=0,{\ldots },M-1,~i\ne k)\) and \(\hbox {STA}_{1,j^{\prime }}\,(j^{\prime }=0,{\ldots },~U_{1}-1)\). To obtain a transmit beamforming weight that does not interfere with the other STAs in the BSS and OBSS, the null space channel matrix, \(\tilde{\mathbf{H}}^{{\varvec{\prime }}}_{D,0,k} \mathbf{V}_{I,0,k}^{(n)}\), is decomposed by SVD to yield

$$\begin{aligned} \tilde{\mathbf{H}}^{{\varvec{\prime }}}_{D,0,k} \mathbf{V}_{I,0,k}^{(n)} =\mathbf{U}_{D,0,k} \left[ {{ \begin{array}{cc} {{\varvec{\Sigma }}_{D,0,k}}&{} \mathbf{0} \\ \end{array}}}\right] \left[ {{ \begin{array}{cc} {\mathbf{V}_{D,0,k}^{(s)}}&{} {\mathbf{V}_{D,0,k}^{(n)}} \\ \end{array}}}\right] ^{H}, \end{aligned}$$
(5)

where \(\mathbf{U}_{D,0,k}\in {\mathbb {C}}^{L\times L}\) denotes the left singular vectors of \(\tilde{\mathbf{H}}^{{\varvec{\prime }}}_{D,0,k} \mathbf{V}_{I,0,k}^{(n)}\) and the diagonal elements of \({\varvec{\Sigma }}_{D,0,k}\) are the square roots of \(\lambda ^{\prime }_{0,0}\), \(\lambda ^{\prime }_{0,1},{\ldots }, \lambda ^{\prime }_{0,L-1}\), which are the eigenvalues. \(\mathbf{V}_{D,0,k}^{(s)} \in {\mathbb {C}}^{(K-L(M+U_{1}-1))\times L}\) and \(\mathbf{V}_{D,0,k}^{(n)} \in {\mathbb {C}}^{(K-L(M+U_{1}-1))\times (K-L(M+U_{1}))}\) represent the right singular vectors of \(\tilde{\mathbf{H}}^{{\varvec{\prime }}}_{D,0,k} \mathbf{V}_{I,0,k}^{(n)}\) for the signal space and the null space, respectively. Here, \(\mathbf{V}_{D,0,k}^{(s)}\) is the transmit weight that does not interfere with and maximizes the channel capacity for \(\hbox {STA}_{0,k}\). Therefore, the transmit beamforming weight \(\mathbf{W}_{0,k}\) is obtained as

$$\begin{aligned} \mathbf{W}_{0,k} =\mathbf{V}_{I,0,k}^{(n)}~\mathbf{V}_{D,0,k}^{(s)}. \end{aligned}$$
(6)

The transmit beamforming weight which does not interfere with the other STAs in the BSS and OBSS and maximizes to the channel capacity for \(\hbox {STA}_{1,k}\) associated with \(\hbox {AP}_{1}\), \(\mathbf{W}_{1,k}\), is obtained as

$$\begin{aligned} \mathbf{W}_{1,k} =\mathbf{V}_{I,1,k}^{(n)}~\mathbf{V}_{D,1,k}^{(s)}, \end{aligned}$$
(7)

where \(\mathbf{V}_{I,0,k}^{(n)} \in {\mathbb {C}}^{K\times (K-L(M+U_{1}-1))}\) represents the right singular vectors of \(\tilde{\mathbf{H}}_{I,1,i}\) for the null space, and \(\mathbf{V}_{D,0,k}^{(s)} \in {\mathbb {C}}^{(K-L(M+U_{1}-1))\times L}\) represents the right singular vectors of \(\tilde{\mathbf{H}}^{{\varvec{\prime }}}_{D,0,k} \mathbf{V}_{I,0,k}^{(n)}\) for the signal space.

The proposed scheme prevents not only interference between multiple STAs in the BSS but also ICI whose power is larger than the threshold \(P_{th}\) for the STAs in the neighboring cells by using ZF beamforming. Moreover, it does not perform null beamforming (or ICI mitigation) for the STAs when the ICI power is less than \(P_{th}\) since its effect is small at those STAs and does not need to be suppressed. Therefore, the spatial resource of the transmit antennas can be used not to suppress ICI but to obtain a diversity gain for the STAs in the BSS; this leads to a higher channel capacity. In this scheme, each AP uses ICI power as a basis for adaptively detecting whether null beamforming is to be carried out for the STAs in the OBSS. Here, ICI can be estimated by using the receive signal strength indicator (RSSI) in a wireless LAN system. This simply enables the spatial resource to be used effectively for improving system throughput. Furthermore, the proposed scheme does not estimate the CSI between the AP and the STAs associated with the neighboring AP when the ICI power is less than \(P_{th}\), whereas the conventional schemes need to estimate all of the CSI of the neighboring cell’s STAs. Therefore the proposed scheme can reduce the overhead for the CSI estimation.

3.2 Achievable Rate

The achievable rate at \(\hbox {AP}_{0}\) and \(\hbox {AP}_{1}\) obtained with selective beamforming, \(C_{prop}\), is approximately given by

$$\begin{aligned} C_{prop}&= \sum \limits _{i=0}^{M-1} {\log _{2}\left| {\mathbf{I}_{L}+\frac{\mathbf{H}_{D,0,i} \mathbf{W}_{0,i} \mathbf{W}_{0,i}^{H}\mathbf{H}_{D,0,i}^{H}}{\sigma ^{2}\mathbf{I}_{L}+\mathbf{A}_{0,i} +\mathbf{B}_{1,i}}} \right| } \nonumber \\&+\sum \limits _{j=0}^{M-1} {\log _{2}\left| {\mathbf{I}_{L}+\frac{\mathbf{H}_{D,1,j} \mathbf{W}_{1,j} \mathbf{W}_{1,j}^{H}\mathbf{H}_{D,1,j}^{H}}{\sigma ^{2}\mathbf{I}_{L}+\mathbf{A}_{1,j} +\mathbf{B}_{0,j}}} \right| }, \end{aligned}$$
(8)

where \(\mathbf{A}_{0,i}\) and \(\mathbf{A}_{1,j}\) represent the matrix of the multiuser interference components from the other STAs’ streams at \(\hbox {AP}_{0}\) and \(\hbox {AP}_{1}\), and \(\mathbf{B}_{0,i}\) and \(\mathbf{B}_{1,j}\) respectively represent the matrix of the ICI components from \(\hbox {AP}_{0}\) and \(\hbox {AP}_{1}\). They are expressed as

$$\begin{aligned}&\left\{ { \begin{aligned} \mathbf{A}_{0,i}&=\sum \limits _{ \begin{array}{c} {i}^{\prime }=0 \\ \ne i \\ \end{array}}^{M-1} {\mathbf{H}_{D,0,i} \mathbf{W}_{0,{i}^{\prime }} \mathbf{W}_{0,{i}^{\prime }}^{H}\mathbf{H}_{D,0,i}^{H}} \\ \mathbf{A}_{1,j}&=\sum \limits _{ \begin{array}{c} {j}^{\prime }=0 \\ \ne j \\ \end{array}}^{M-1} {\mathbf{H}_{D,0,j} \mathbf{W}_{0,{j}^{\prime }} \mathbf{W}_{0,{j}^{\prime }}^{H}\mathbf{H}_{D,0,j}^{H}} \end{aligned}} \right. , \end{aligned}$$
(9)
$$\begin{aligned}&\left\{ { \begin{aligned} \mathbf{B}_{1,i}&=\sum \limits _{{i}^{\prime }=0}^{M-1} {\mathbf{H}_{I,1,i} \mathbf{W}_{1,{i}^{\prime }} \mathbf{W}_{1,{i}^{\prime }}^{H}\mathbf{H}_{I,1,i}^{H}}\\ \mathbf{B}_{0,j}&=\sum \limits _{{j}^{\prime }=0}^{M-1} {\mathbf{H}_{I,0,j} \mathbf{W}_{0,{j}^{\prime }} \mathbf{W}_{0,{j}^{\prime }}^{H}\mathbf{H}_{I,0,j}^{H}} \end{aligned}} \right. . \end{aligned}$$
(10)

If there is no CSI error at the AP, both \(\mathbf{A}_{0,i}\) and \(\mathbf{A}_{1,j}\) become zero matrices and \(\mathbf{B}_{1,i}\) and \(\mathbf{B}_{0,j}\) become only the ICI components consisting of the interference at \(\hbox {STA}_{0,i^{\prime }}\) \((i^{\prime }=U_{0},{\ldots },M-1)\) and \(\hbox {STA}_{1,j^{\prime }}\,(j^{\prime }=U_{1},{\ldots },M-1)\) where the ICI power is smaller than \(P_{th}\).

$$\begin{aligned} \left\{ {\begin{aligned}{c} \mathbf{B}^{{\varvec{\prime }}}_{1,i}&=\sum \limits _{{i}^{\prime }=U_{0}}^{M-1} {\mathbf{H}_{I,1,i} \mathbf{W}_{1,{i}^{\prime }} \mathbf{W}_{1,{i}^{\prime }}^{H}\mathbf{H}_{I,1,i}^{H}}\\ \mathbf{B}^{{\varvec{\prime }}}_{0,j}&=\sum \limits _{{j}^{\prime }=U_{1}}^{M-1} {\mathbf{H}_{I,0,j} \mathbf{W}_{0,{j}^{\prime }} \mathbf{W}_{0,{j}^{\prime }}^{H}\mathbf{H}_{I,0,j}^{H}} \end{aligned}} \right. \end{aligned}$$
(11)

In this paper, we assume that the STA has perfect CSI, while the AP has some estimation error in the CSI. The estimated channel matrices obtained at the AP, \(\tilde{\mathbf{H}}_{D,0,i}\), \(\tilde{\mathbf{H}}_{D,1,i}\), \(\tilde{\mathbf{H}}_{I,1,i}\), and \(\tilde{\mathbf{H}}_{I,0,i}\), are defined as

$$\begin{aligned} \left\{ {\begin{array}{l} \tilde{\mathbf{H}}_{D,0,i} =\mathbf{H}_{D,0,i} +\mathbf{E}_{D,0,i} \\ \tilde{\mathbf{H}}_{D,1,j} =\mathbf{H}_{D,1,j} +\mathbf{E}_{D,1,j} \\ \tilde{\mathbf{H}}_{I,1,i} =\mathbf{H}_{I,1,i} +\mathbf{E}_{I,1,i} \\ \tilde{\mathbf{H}}_{I,0,j} =\mathbf{H}_{I,0,j} +\mathbf{E}_{I,0,j} \\ \end{array}} \right. , \end{aligned}$$
(12)

where \(\mathbf{E}_{D,0,i}\), \(\mathbf{E}_{D,1,j}\), \(\mathbf{E}_{I,1,i}\), and \(\mathbf{E}_{I,0,j}\) denote all i.i.d. zero-mean complex Gaussian variables with variance \(\sigma _{e}^{2}\). Thus, the transmission weight is calculated using the imperfect channel matrices, \(\tilde{\mathbf{H}}_{D,0,i}\), \(\tilde{\mathbf{H}}_{D,1,i}\), \(\tilde{\mathbf{H}}_{I,1,i}\), and \(\tilde{\mathbf{H}}_{I,0,i}\).

4 Simulation Results

4.1 Simulation Setup

We performed a numerical simulation to verify the proposed scheme’s effectiveness. The simulation conditions are summarized in Table 2. We used a Monte Carlo method to compute the distribution of achievable rates. The STA locations were uniformly distributed over a square that was 20 m on a side, and each AP was located at the center of the square, as shown in Fig. 2. The path loss exponential factor and shadowing loss standard variation were changed at the 10-m break point distance in accordance with Channel model D in the IEEE 802.11 TGn. The parameters of the transmit signal format were based on the IEEE 802.11ac standard [1].

Table 2 Simulation parameters
Fig. 2
figure 2

Simulation scenario of OBSS environment

4.2 Optimum Threshold

Figure 3 plots the achievable rate at a cumulative distribution function (CDF) of 50 and 10 % of the CDF as a function of the threshold \(P_{th}\) when the distance between APs is 60 and 40 m. In this simulation, \(P_{th}\) is defined as the interference-to-noise power ratio (INR), where the noise power is the sum of the thermal noise and the noise figure. The APs and STAs have respectively \(K=8\) and \(L=1\) antennas, and the number of associated STAs at each AP is \(M=4\). We assume that there is no channel estimation error \((\sigma _{e}^{2} = 0)\). As the figure shows, there is an optimum value of \(P_{th}\) at each \(D_{AP}\) and \(P_{th} = 10\) and 12 dB are the optimum values for the \(D_{AP}\) of 60 and 40 m, respectively. The optimum value exists because of the trade-off between the effect of the ICI suppression and antenna diversity gain. When \(P_{th}\) is set to a small value, the ICI effect can be suppressed. However, the scheme operates so as to suppress not only large but also small ICI effects, thereby reducing the diversity gain. On the other hand, when \(P_{th}\) is set to a large value, the diversity gain for STAs in the BSS increases, although the ICI also becomes large at STAs in the OBSS. Thus, the achievable rate improvement is a trade-off between the need for ICI suppression and a diversity gain. In addition, when the distance between APs becomes small, ICI becomes large. Consequently, ICI suppression using null beamforming improves performance by raising the threshold \(P_{th}\). In addition, we can see that the optimum \(P_{th}\) difference between 50 and 10 % of CDF is negligible. The proposed scheme used the optimum \(P_{th}\) in the simulation described in the next section.

Fig. 3
figure 3

Achievable rate at 50 and 10 % of CDF as a function of the threshold \(P_{th}\) for \(D_{AP}\) = 60 and 40 m

4.3 Performance Analysis of the Proposed Scheme

Figure 4 shows the CDF of the achievable rate for the proposed scheme. The distance between APs is \(D_{AP}\) = 60 and 40 m. The APs and STAs have respectively \(K=8\) and \(L=1\) antennas, and the number of associated STAs at each AP is \(M=4\). For comparison, the figure also plots the CDFs of the achievable rate for a scheme with TRS between APs, i.e., time division multiple access, a conventional ICI mitigation scheme (C-ICIM) for STAs in the OBSS using null beamforming \((P_{th} = -\infty )\), and a scheme without ICI mitigation (No-ICIM) for them \((P_{th} = +\infty )\). Note that all three schemes use DL MU-MIMO transmission for STAs in the BSS. We assume there is no channel estimation error. As the figure shows, the achievable rates of the proposed scheme are respectively 63 (55), 14 (8) and 17 (43) % higher than those of the TRS, C-ICIM, No-ICIM schemes at 50 % of CDF when the distance between \(D_{AP}\) is 60 (40) m. This is because the proposed scheme uses the transmit antennas to perform null beamforming for the neighboring cell’s STAs subjected to a large ICI effect and to perform transmit diversity for the associated STAs on the basis of the threshold. As a result, the spatial resource is effectively used. In addition, the improvement had by the proposed scheme over that of the No-ICIM one increases as the distance between APs decreases. This is because the proposed scheme mitigates ICI by using null beamforming. In contrast, the improvement over that of the C-ICIM increases as the distance between APs gets larger. This is because the proposed scheme obtains a diversity gain from the transmit antennas whereas the C-ICIM performs null beamforming on the small ICI. These results clearly show that the proposed scheme not only suppresses ICI to support STAs subjected to a large ICI effect but also uses the transmit diversity effect to improve the achievable rate. Figure 5 shows the CDF of the achievable rate at an STA. The achievable rate obtained with the proposed scheme is larger than that with the other three schemes throughout the CDF range. The proposed scheme is respectively 64 (56), 15 (10) and 17 (47) % higher in its achievable rate than the TRS, C-ICIM, No-ICIM at 50 % of CDF and a \(D_{AP}\) of 60 (40) m. In contrast, there is almost no performance difference between the proposed scheme and the C-ICIM at 10 % of CDF and a \(D_{AP}\) of 40 m. This is because the STA cannot obtain a diversity effect so as to suppress the ICI by using the transmit antennas of the APs.

Fig. 4
figure 4

CDF of the total achievable rate for a \(D_{AP} = 60\,\hbox { m}\) and b \(D_{AP}= 40\,\hbox { m}\)

Fig. 5
figure 5

CDF of the achievable rate of an STA for a \(D_{AP} = 60\,\hbox { m}\) and b \(D_{AP} = 40\,\hbox { m}\)

Figure 6 plots the CDF of the achievable rate in the case of a channel estimation error \(\sigma _{e}^{2}\) defined in Eq. (12) for (a) \(D_{AP} = 60\hbox { m}\) and (b) \(D_{AP} = 40\,\hbox { m}\). We assume that the variance of the channel estimation error is equivalent to that of the noise component, i.e., \(\sigma ^{2} = \sigma _{e}^{2}\). The APs and STAs have respectively \(K=8\) and \(L=1\) antennas, and the number of associated STAs at each AP is \(M=4\). When there is a channel estimation error, the achievable bit rate deteriorates since the transmit beamforming cannot be perfectly performed and the interference of the other streams in the BSS becomes large. Moreover, neither the proposed scheme nor the C-ICIM scheme can correctly perform null beamforming for the STAs in the OBSS. The achievable rate obtained with the C-ICIM scheme is especially degraded and it is worse than that with the No-ICIM when \(D_{AP}\) is large (40 m). The reason for this behavior is as follows. The channel estimation error of the STAs subjected to small ICI effect becomes large since the distance between the AP and the STAs in the OBSS is long and the signal-to-noise power ratio (SNR) for the CSI estimation is low. However, the C-ICIM scheme performs null beamforming for all STAs. As a result, the achievable bit rate obtained with the C-ICIM scheme falls because of the channel error of the STAs with a small ICI power even though they may not need their ICI to be suppressed. In contrast, the proposed scheme does not perform null beamforming for the STAs subjected to a small amount of ICI. Therefore, it can suppress the interference increase caused by the channel estimation error of the STAs.

Fig. 6
figure 6

Effect of channel estimation error for a \(D_{AP} = 60\,\hbox { m}\) and b \(D_{AP} = 40\,\hbox { m}\)

Figure 7 plots the CDF of the achievable rate with the proposed scheme when the number of transmit antennas at an AP and number of associated STAs at an AP \((K,~M)\) are (4, 2) and (8, 4). We assume there is no channel estimation error. As the figure shows, the achievable rates at 50 % of CDF with the proposed scheme increases when the number of transmit antenna at an AP and associated STAs increases. At both distance (\(D_{AP} = 60\,\hbox { m}\) and 40 m), the achievable bit rate with (K, M) = (8, 4) is 1.9 times larger than that with (4, 2).

Fig. 7
figure 7

CDF of the achievable rate when the set of the number of transmit antenna at an AP and associated STA \((K,~M)\) is (4, 2) and (8, 4)

Figure 8 plots the achievable bit rate at 50 % of CDF as a function of \(D_{AP}\) when there is no channel estimation error. The APs and STAs have \(K=8\) and \(L=1\) antennas, and the number of associated STAs at each AP is \(M=4\). It can readily be seen that the proposed scheme provides the best achievable bit rate regardless of the distance between APs. This is because when \(D_{AP}\) is small \((D_{AP} \le 10\,\hbox { m})\), the proposed scheme suppresses the effect of a large amount of ICI at STAs in the OBSS by using null beamforming, and eventually the proposed and C-ICIM schemes provide the same achievable bit rate. In contrast, when \(D_{AP}\) is large \((D_{AP} \ge 10\,\hbox { m})\), the effect of ICI is small and there are more STAs where the ICI power is lower than the threshold. Thus, the proposed scheme does not need to perform null beamforming and the probability that transmit antenna diversity is effective increases. Eventually, both the proposed and No-ICIM schemes provide the same achievable rate.

Fig. 8
figure 8

Achievable rate at 50 % of CDF as a function of the distance between APs

5 Conclusion

We proposed a simple beamforming scheme for mitigating the effects of ICI in wireless LAN systems and conducted a computer simulation to confirm that it improves the achievable rate in an OBSS environment. The proposed scheme selectively determines whether or not to perform transmit beamforming on basis of the ICI power at each STA in neighboring cells. Null beamforming is performed to reduce the ICI effect when the ICI power is larger than a threshold; otherwise, it is not carried out and the spatial resource of the transmit antennas is used to obtain a transmit diversity gain. Computer simulation results confirmed that in an OBSS environment where two APs are spaced 60 m apart, the achievable rate obtained with the proposed scheme is 64, 15 and 17 % higher than those of a TRS scheme, a conventional ICI mitigation scheme, and a scheme without ICI mitigation. In addition, we also confirmed that the proposed scheme provides a higher achievable rate than the other three schemes at any AP distance. We thus conclude that the proposed scheme is suitable for future high-density wireless LAN systems.