1 Introduction

Massive multiple-input and multiple-output (MIMO) has been envisioned as a crucial technology for the next-generation cellular systems, due to its high energy efficiency and spectral efficiency [13]. To exploit the benefits of massive antennas, a large number of radio frequency (RF) chains are needed so that each antenna can own one dedicated RF chain. Combating the expensive cost, hybrid architecture has been a promising scheme which combines cost-effective phase shifters and limited RF chains. For hybrid architecture, there are two most common precoding techniques: hybrid precoding and analog precoding [4, 5].

1.1 Related work

Hybrid analog/digital precoding has been investigated extensively including precoding design and channel estimation strategy [47], discrete Fourier transform (DFT)-based hybrid architecture [810], angle domain hybrid precoding and channel tracking [11, 12], and performance analysis on hybrid architecture [1315]. Most of the existing hybrid precoding designs heavily rely on the perfect knowledge of the channel, which brings a great challenge because only limited RF chains are available at base station (BS) and user equipment (UE). An adaptive channel estimation algorithm was proposed in [6] for hybrid architecture millimeter wave (mmWave) systems, which applies the compressive sensing (CS) technology based on the poor scattering nature of mmWave. With the CS method, the channel estimation algorithm in [6] yields a limited accuracy and a high computational complexity. For analog precoding, an RF beam search method was proposed in [16, 17], where the channel estimation was obtained by sweeping all beam directions over the spatial region. However, the beam search algorithm in [16] is time-consuming due to the exhaustive beam searching. Besides, orthogonal training sequences have also been widely used to obtain the angle information for channel estimation in the DFT-based hybrid architecture [8, 9], which caused high computational complexity and feedback cost. Therefore, our objective is to design a simple and effective signal processing scheme for hybrid architecture massive MIMO systems.

1.2 Summary of contributions

In this journal, we investigate low-complexity beam-domain channel estimation, precoding, and power allocation in hybrid architecture massive MIMO systems. The major contributions of this journal are summarized as follows:

(1) A low-complexity channel estimation algorithm is proposed, which avoids high pilot overhead by utilizing estimated beams and beam gains.

(2) The RF precoder and combiner are designed based on estimated beam information.

(3) The power allocation among multiple beams is optimized to maximize spectral efficiency.

(4) Simulation results demonstrate high channel estimation accuracy and comparable spectral efficiency achieved by our proposed channel estimation and power allocation schemes.

2 System model

We consider a multi-path massive MIMO systemFootnote 1 with its forward-link scenario as shown in Fig. 1. The transmitter is equipped with an Mt×Nt uniform rectangular array (URA) and Kt≤min(Mt,Nt) RF chains, while the receiver is equipped with an Mr×Nr URA and Kr≤min(Mr,Nr) RF chains. Let L be the number of paths between the transmitter and receiver. We assume L≤min {Kt,Kr} due to the sparsity of channel [8], and L RF chains can be selected randomly from all RF chains for usage.

Fig. 1
figure 1

A multi-beam massive MIMO system with hybrid architecture

In a massive MIMO system with URAs, the forward-link channel matrix with L paths can be written as

$$\begin{array}{@{}rcl@{}} \mathbf H=\frac{1}{\sqrt{L}}\sum_{l=1}^{L}\gamma_{l}\mathbf a_{r}(\theta_{l}^{r}, \phi_{l}^{r})\mathbf a_{t}^{H}(\theta_{l}^{t}, \phi_{l}^{t})\textrm{,} \end{array} $$
(1)

where \(\gamma _{l}\sim \mathcal {CN}(0,\sigma _{l}^{2})\) is the complex gain of the lth path, \(\mathbf a_{t}(\theta _{l}^{t}, \phi _{l}^{t})\in \mathbb C^{M_{t}N_{t}\times 1}\) and \(\mathbf a_{r}(\theta _{l}^{r}, \phi _{l}^{r})\in \mathbb C^{M_{r}N_{r}\times 1}\) denote the array steering vectors of the lth path at the transmitter and receiver, respectively. \(\theta _{l}^{t}, \theta _{l}^{r}\in [0,\pi)\) are the azimuth angles and \(\phi _{l}^{t}, \phi _{l}^{r}\in [0,\frac {\pi }{2})\) are the elevation angles of the lth path. In general, the steering vector a(θl,ϕl) can be written as

$$\begin{array}{@{}rcl@{}} \mathbf{a}(\theta_{l}, \phi_{l})=\mathbf{a}_{y}(\theta_{l}, \phi_{l})\otimes\mathbf{a}_{x}(\theta_{l}, \phi_{l})\textrm{,} \end{array} $$
(2)

where ⊗ denotes the Kronecker product, ax(θl,ϕl) is an M×1 vector with the mth element being \(e^{j\frac {2\pi }{\lambda }d(m-1)\cos \theta _{l}\sin \phi _{l}}\), and ay(θl,ϕl) is a N×1 vector with the nth element being \(e^{j\frac {2\pi }{\lambda }d(n-1)\sin \theta _{l}\sin \phi _{l}}, d=\frac {\lambda }{2}\) is the distance between two adjacent antennas, and λ is the wavelength. M and N are the numbers of antennas in the horizontal direction and vertical direction of the URA, respectively.

Lemma 1: The steering vectors of different paths are asymptotically orthogonal for sufficient large MN. That is, \(\mathop {\lim }\limits _{M, N\rightarrow \infty }\mathbf a^{H}(\theta _{l_{1}}, \phi _{l_{1}})\mathbf a(\theta _{l_{2}}, \phi _{l_{2}})=0\) (l1l2).

Proof 1.

: See Appendix 6.

In Fig. 1, the transmit processing includes power allocation and RF precoding. The post processing at the receiver consists of RF combining and baseband processing. The digital precoding is not considered due to the asymptotical orthogonality of different paths as shown in Lamme 1. Assuming a flat-fading channel, the discrete-time received signal at the receiver is given by [6]

$$\begin{array}{@{}rcl@{}} \mathbf{y}=\mathbf{F}_{r}^{H}\mathbf{H}\mathbf{F}_{t}\mathbf P\mathbf{s}+\mathbf{F}_{r}^{H}\mathbf{n}\textrm{,} \end{array} $$
(3)

where \(\mathbf {s}\in \mathbb {C}^{L\times 1}\) is the transmitted signal with \(\mathbb E\{\mathbf {s}\mathbf {s}^{H}\}=\sigma _{s}^{2}\mathbf I_{L}, \mathbb E\{\cdot \}\) is the expectation operator, \(\mathbf P\in \mathbb {R}^{L\times L}\) is the power allocation matrix, \(\mathbf F_{t}\in \mathbb C^{M_{t}N_{t}\times L}\) denotes the RF precoder, \(\mathbf F_{r}\in \mathbb C^{M_{r}N_{r}\times L}\) denotes the RF combiner, and \(\mathbf {n}\sim \mathcal {CN}(\mathbf {0},\sigma _{n}^{2}\mathbf {I}_{M_{r}N_{r}})\) denotes additive white Gaussian noise (AWGN) vector. Let [F]i,j denote the (i,j)th element of a matrix F. The RF precoder and RF combiner control phase changes by employing phase shifters with the magnitude \(|[\mathbf F_{t}]_{i,j}|=\frac {1}{\sqrt {M_{t}N_{t}}}\) and \(|[\mathbf F_{r}]_{i,j}|=\frac {1}{\sqrt {M_{r}N_{r}}}\), respectively.

3 Methods of beam-Domain channel estimation and power allocation

In this section, we present the beam-domain signal processing in hybrid architecture massive MIMO systems. First, we discuss the DOA estimation based on the hybrid architecture with limited RF chains. Then, low-complexity channel estimation is proposed based on the estimated beam information. Finally, we study the power allocation to improve the system performance.

3.1 Beam information estimation

Due to the angle reciprocity, the direction-of-departure (DOD) of the forward-link channel is the same as the DOA of the reverse-link channel, even for the frequency division duplex (FDD) systems [8].

The DOA information \(\theta _{l}^{t}\) and \(\phi _{l}^{t}\) can be estimated using the limited RF chains in two time slots, respectively. For the reverse link, Kt contiguous antennas in a row at the receiver (which is the transmitter in the forward link) are first chosen in the horizontal direction as in Fig. 1, and then linked with Kt RF chains to estimate the azimuth angle \(\theta _{l}^{t}\) in the first time slot. Similarly, the elevation angle \(\phi _{l}^{t}\) can be estimated by employing Kt antennas in a column in the second time slot. Here, the estimated angles are defined as (\(\theta _{l}^{t}, \phi _{l}^{t}\)).

By using only a small number of antennas, this estimation strategy has a low complexity and a high accuracy, which have been analyzed in our previous work [18]. For example, while Kt=16, the root mean square error (RMSE) of the estimated DOA has been less than 1 by costing only 32 snapshots [18]. Many DOA estimation algorithms can be adopted in this proposed estimation strategy, such as the well-known MUltiple SIgnal Classification (MUSIC) and Estimation of Signal Parameters via Rotational Invariance Techniques (ESPRIT) [1921]. In each time slot, there are a number of snapshots to obtain approximate covariance information for DOA estimation with MUSIC or ESPRIT algorithm.

Practically, considering the low-cost, finite-resolution phase shifters are implemented at the transmitter and receiver for RF processing. For the Mt×Nt URA, there are Mt and Nt phase shifters with finite-resolution in the horizontal and vertical directions, respectively. When all the phase shifters have the same resolution \(\frac {1}{P}\pi \), for the azimuth angle θ∈[0,π) in the horizontal direction, the beam codebook consists of P basic beams by sampling the angle space [0,π) as \(G_{M}=\left \{0,\frac {1}{P}\pi,\cdots,\frac {P-1}{P}\pi \right \}\); for the elevation angle \(\phi \in [0,\frac {\pi }{2})\) in the vertical direction, the beam codebook consists of \(\left (\left \lfloor \frac {P}{2}\right \rfloor +1\right)\) basic beams by sampling the angle space \([0,\frac {\pi }{2})\) as \(G_{N}=\left \{0,\frac {1}{P}\pi,\cdots,\frac {\left \lfloor \frac {P}{2}\right \rfloor }{P}\pi \right \}\). Thus, for an Mt×Nt URA, the beam codebook consists of \(P\left (\left \lfloor \frac {P}{2}\right \rfloor +1\right)\) basic 2-D beams (θ,ϕ).

Finally, the estimated angles (\(\theta _{l}^{t}, \phi _{l}^{t}\)) should be mapped to the beam codebooks for practical transmission. The DOA information can be easily estimated as \((\hat {\theta }_{l}^{t},\hat {\phi }_{l}^{t})=\text {arg} \mathop {\text {min}}\limits _{\theta \in G_{M},\\ \phi \in G_{N}}\left \{|\theta \,-\,\theta _{l}^{t}|\,+\,|\phi \,-\,\phi _{l}^{t}|\right \}\) for practical RF processing. Notably, the DOA estimation strategy can also be directly applied at the receiver in forward link to obtain \((\hat {\theta }_{l}^{r}, \hat {\phi }_{l}^{r})\).

With the estimated beams, RF precoder at the transmitter and RF combiner at the receiver are given as

$$\begin{array}{@{}rcl@{}} \hat{\mathbf{F}}_{t}=\frac{1}{\sqrt{M_{t}N_{t}}}\left[\mathbf a_{t}\left(\hat{\theta}_{1}^{t}, \hat{\phi}_{1}^{t}\right) \cdots \mathbf a_{t}(\hat{\theta}_{L}^{t}, \hat{\phi}_{L}^{t})\right]\textrm{,} \end{array} $$
(4)
$$\begin{array}{@{}rcl@{}} \hat{\mathbf{F}}_{r}=\frac{1}{\sqrt{M_{r}N_{r}}}\left[\mathbf a_{r}\left(\hat{\theta}_{1}^{r}, \hat{\phi}_{1}^{r}\right) \cdots \mathbf a_{r}(\hat{\theta}_{L}^{r}, \hat{\phi}_{L}^{r})\right]\textrm{.} \end{array} $$
(5)

3.2 Channel estimation

Channel estimation is crucial but intractable in massive MIMO systems [22, 23]. In this subsection, we propose a low-complexity channel estimation method in the beam domain. The proposed method includes the following three steps.

First, the magnitude of the channel gain \(|\hat {\gamma }_{l}|\) and DOAs \((\hat {\theta }_{l}^{t},\hat {\phi }_{l}^{t})\) mentioned above are simultaneously derived. In the reverse link, the received signal as the output of the URA at the receiver, which is the transmitter in the forward link, is expressed as [20, 21]

$$\begin{array}{@{}rcl@{}} \mathbf{Y}_{1}=\mathbf{H}_{1}\mathbf{X}_{1}+\mathbf{W}_{1}=\mathbf A_{t}\mathbf{G}\mathbf{X}_{1}+\mathbf{W}_{1}\textrm{,} \end{array} $$
(6)

where X1 is the transmitted signal matrix from one antenna for DOA estimation with \(\mathbb E\left \{\mathbf {X}_{1}\mathbf {X}_{1}^{H}\right \}=\sigma _{X}^{2}\mathbf {I}_{L}, \mathbf {H}_{1}=\mathbf A_{t}\mathbf {G}\) is the reverse-link channel matrix, \(\mathbf A_{t}=\left [\mathbf a_{t}\left (\hat {\theta }_{1}^{t}, \hat {\phi }_{1}^{t}\right) \cdots \mathbf a_{t}\left (\hat {\theta }_{L}^{t}, \hat {\phi }_{L}^{t}\right)\right ]\) is obtained from the estimated beam information, G=diag(γ1,⋯,γL) is a L×L diagonal matrix with beam gains {γ1,⋯,γL} on the main diagonal, and \(\mathbf {W}_{1}\sim \mathcal {CN}\left (\mathbf {0},\sigma _{W}^{2}\mathbf {I}_{M_{t}N_{t}}\right)\) denotes the AWGN matrix. From (6), the covariance matrix of the received signal can be written as

$$\begin{array}{@{}rcl@{}} {\mathbf{R}_{YY}}&{=}&{\mathbb{E}\{\mathbf{Y}_{1}\mathbf{Y}_{1}^{H}\}}\\ &{=}&{\sigma_{X}^{2}\mathbf A_{t}\mathbf{G}\mathbf{G}^{H}\mathbf A_{t}^{H}+\sigma_{W}^{2}\mathbf{I}_{M_{t}N_{t}}} \\ &{=}&{\sigma_{X}^{2}\mathbf A_{t}} \left[\begin{aligned} |&\gamma_{1}|^{2}&\cdots & &0&\\ &\vdots & \ddots & &\vdots&\\ &0&\cdots & &|\gamma_{L}&|^{2} \end{aligned}\right] {\mathbf A_{t}^{H}+\sigma_{W}^{2}\mathbf{I}_{M_{t}N_{t}}\textrm{.}} \end{array} $$
(7)

Based on (7), the magnitude of the beam gain is obtained as

$$\begin{array}{@{}rcl@{}} |\hat{\gamma}_{l}|=\sqrt{\left[\frac{1}{\sigma_{X}^{2}}\mathbf A_{t}^{\dag}(\mathbf R_{YY}-\sigma_{W}^{2}\mathbf{I}_{M_{t}N_{t}})(\mathbf A_{t}^{H})^{\dag}\right]_{l,l}}\textrm{.} \end{array} $$
(8)

where \(\mathbf A_{t}^{\dag }=\left (\mathbf A_{t}^{H}\mathbf A_{t}\right)^{-1}\mathbf A_{t}^{H}\) is the pseudo-inverse of At, and \(\left (\mathbf A_{t}^{H}\right)^{\dag }=\mathbf A_{t}\left (\mathbf A_{t}^{H}\mathbf A_{t}\right)^{-1}\) is the pseudo-inverse of \(\mathbf A_{t}^{H}\). By using the statistic characteristic, the estimated magnitude of the beam gain will have higher accuracy than the pilot-based estimation method.

Second, the phase of beam gain is estimated through a channel estimation algorithm with low pilot overhead and complexity. While the training signal is transmitted in reverse link, the received signal is given by

$$\begin{array}{@{}rcl@{}} \mathbf{Y}_{2}=\mathbf A_{t} \left[\begin{aligned} |\hat{\gamma}_{1}|&e^{j\varphi_{1}} & \cdots& & &0\\ &\vdots & \ddots & &\vdots\\ &0 & \cdots & &|\hat{\gamma}_{L}&|e^{j\varphi_{L}} \end{aligned}\right] \mathbf{X}_{2}+\mathbf{W}_{2}\textrm{,} \end{array} $$
(9)

where φl denotes the phase of \(\hat {\gamma }_{l}, \mathbf {X}_{2}\in \mathbb {C}^{L \times n_{s}}\) is the training signal consisting of ns bits pilots, and W2 is the AWGN matrix. Using (9), the phase of the beam gain is estimated as

$$\begin{array}{@{}rcl@{}} e^{j\hat{\varphi}_{l}}=\frac{\left\{\mathbf{A}_{t}^{\dag}\mathbf{Y}_{2}\mathbf{X}_{2}^{\dag}\right\}_{l,l}}{|\hat{\gamma}_{l}|}\textrm{.} \end{array} $$
(10)

where \(\mathbf X_{2}^{\dag }=\mathbf X_{2}^{H}\left (\mathbf X_{2}\mathbf X_{2}^{H}\right)^{-1}\) is the pseudo-inverse of X2. With L beams, there only needs ns= log2L bits pilots to estimate the phase of the beam gain. Due to the pseudo-inverse in (10), there is a significant complexity reduction by using ns bits pilots.

Finally, with the estimated beam gain \(\hat {\gamma }_{l}=|\hat {\gamma }_{l}|e^{j\hat {\varphi }_{l}}\), the forward-link channel matrix in (1) can be estimated as

$$\begin{array}{@{}rcl@{}} \hat{\mathbf H}=\frac{1}{\sqrt{L}}\sum_{l=1}^{L}\hat{\gamma}_{l}\mathbf a_{r}\left(\hat{\theta}_{l}^{r}, \hat{\phi}_{l}^{r}\right)\mathbf a_{t}^{H}\left(\hat{\theta}_{l}^{t}, \hat{\phi}_{l}^{t}\right)\textrm{.} \end{array} $$
(11)

In summary, the proposed channel estimation strategy has a low complexity for three reasons: (1) It directly uses the results of DOA estimation to obtain the magnitude of the channel gain as an additional product; (2) the DOA estimation strategy has a low complexity; (3) the L beam gains are estimated with short training pilots. Furthermore, the proposed channel estimation strategy has a high accuracy. The reason is that the magnitude of channel gain is estimated from the statistic characteristic as shown in (8), which has a higher accuracy than that of the conventional pilot-based channel estimation.

The proposed channel estimation method can also be adopted for multi-user systems, resulting in a great degradation of pilot overhead. When multiple users have similar DOAs and bring in multi-user interference, the proposed channel estimation method is still available. The reason is that the channel estimation of all users is obtained from the DOA estimation which needs to be implemented in a time-division method. As a result, the interference caused by the similar DOAs does not affect the time-division channel estimation of multiple users.

3.3 Power allocation

In this subsection, the power allocation problem is studied to improve the spectral efficiency of the hybrid architecture massive MIMO system. Let us denote the power allocation matrix \(\mathbf P=\text {diag}(\sqrt {p_{1}},\cdots,\sqrt {p_{L}})\). The power allocation problem can be formulated as

$$\begin{array}{*{20}l} &{}\mathop{\text{max}}\limits_{\{p_{l}\}} R=\text{log}\:\text{det}\left(\mathbf{I}_{L}\,+\,\frac{(\hat{\mathbf{F}}_{r}^{H}\hat{\mathbf{F}}_{r})^{-1}(\hat{\mathbf{F}}_{r}^{H}\hat{\mathbf H}\hat{\mathbf{F}}_{t}\mathbf{P})(\hat{\mathbf{F}}_{r}^{H}\hat{\mathbf H}\hat{\mathbf{F}}_{t}\mathbf{P})^{H}}{\sigma_{n}^{2}}\right) \end{array} $$
(12a)
$$\begin{array}{*{20}l} &\textrm{s.t.}\qquad \!\!\!\!\!\!\!\! \sum_{l=1}^{L} p_{l}= P_{t} \end{array} $$
(12b)
$$\begin{array}{*{20}l} &\,p_{l}\geq0 \end{array} $$
(12c)

where Pt is the total transmit power.

Theorem

The optimal power allocation for the lth beam is derived as

$$\begin{array}{@{}rcl@{}} p_{l}\approx\left(\frac{1}{\lambda}-\frac{L\sigma_{n}^{2}}{M_{t}N_{t}M_{r}N_{r}|\hat{\gamma}_{l}|^{2}}\right)^{+}\textrm{,} \end{array} $$
(13)

where u+=max{u,0}, and λ is the Lagrange multiplier satisfying

$$\begin{array}{@{}rcl@{}} \sum_{l=1}^{L}\left(\frac{1}{\lambda}-\frac{L\sigma_{n}^{2}}{M_{t}N_{t}M_{r}N_{r}|\hat{\gamma}_{l}|^{2}}\right)^{+}=P_{t}\textrm{.} \end{array} $$
(14)

Proof 2.

: Using (4)-(5) and (11), we obtain

$$ {\begin{aligned} (\hat{\mathbf{F}}_{r}^{H}\hat{\mathbf{F}}_{r})^{\,-\,1}\!\!\!\!\!\!&\overset{(a)}{\approx}\!\!\left(\frac{1}{M_{r}N_{r}}\!\! \left[ \begin{array}{llc} \mathbf a_{r}^{H}\left(\hat{\theta}_{1}^{r}, \hat{\phi}_{1}^{r}\right)\mathbf a_{r}\left(\hat{\theta}_{1}^{r}, \hat{\phi}_{1}^{r}\right) & \cdots & 0\\ \!\!\!\!\!\!\!\!\!\!\vdots &\ddots & \vdots\\ 0 & \cdots &\!\!\! \mathbf a_{r}^{H} \left(\hat{\theta}_{L}^{r}, \hat{\phi}_{L}^{r}\right)\mathbf a_{r}\left(\hat{\theta}_{L}^{r}, \hat{\phi}_{L}^{r}\right) \end{array}\right] \right)^{-1} \\ &\overset{(b)}{=}\!\!\mathbf{I}_{L}\textrm{,} \end{aligned}} $$
(15)
$$\begin{array}{@{}rcl@{}} \hat{\mathbf{F}}_{t}\mathbf{P}\,=\, \frac{1}{\sqrt{M_{t}N_{t}}}\left[\sqrt{p_{1}}\mathbf a_{t}\left(\hat{\theta}_{1}^{t}, \hat{\phi}_{1}^{t}\right) \!\cdots\! \sqrt{p_{L}}\mathbf a_{t}\left(\hat{\theta}_{L}^{t}, \hat{\phi}_{L}^{t}\right)\right]\textrm{,} \end{array} $$
(16)

and

$$\begin{array}{@{}rcl@{}} {\begin{aligned} \hat{\mathbf{F}}_{r}^{H}\hat{\mathbf H}&&=\!\!\frac{1}{\sqrt{M_{r}N_{r}}}\!\!\left[\!\!\begin{array}{c}\mathbf a_{r}^{H}\left(\hat{\theta}_{1}^{r}, \hat{\phi}_{1}^{r}\right) \\ \vdots \\ \mathbf a_{r}^{H} \left(\hat{\theta}_{L}^{r}, \hat{\phi}_{L}^{r}\right) \\ \end{array}\!\!\right] \!\!\frac{1}{\sqrt{L}}\!\!\sum_{l=1}^{L}\!\!\hat{\gamma}_{l}\mathbf a_{r}\left(\hat{\theta}_{l}^{r}, \hat{\phi}_{l}^{r}\right)\mathbf a_{t}^{H}\left(\hat{\theta}_{l}^{t}, \hat{\phi}_{l}^{t}\right)\\ &&\overset{(c)}{\approx}\!\!\frac{1}{\sqrt{M_{r}N_{r} L}}\!\!\left[\!\!\begin{array}{c}\hat{\gamma}_{1}\mathbf a_{r}^{H}\left(\hat{\theta}_{1}^{r}, \hat{\phi}_{1}^{r}\right)\mathbf a_{r}\left(\hat{\theta}_{1}^{r}, \hat{\phi}_{1}^{r}\right)\mathbf a_{t}^{H}\left(\hat{\theta}_{1}^{t}, \hat{\phi}_{1}^{t}\right) \\ \vdots \\ \hat{\gamma}_{L}\mathbf a_{r}^{H} \left(\hat{\theta}_{L}^{r}, \hat{\phi}_{L}^{r}\right)\mathbf a_{r}\left(\hat{\theta}_{L}^{r}, \hat{\phi}_{L}^{r}\right)\mathbf a_{t}^{H} \left(\hat{\theta}_{L}^{t}, \hat{\phi}_{L}^{t}\right) \\ \end{array}\!\!\right] \\ &&\overset{(d)}{=}\!\!\sqrt{\frac{M_{r}N_{r}}{L}}\!\!\left[\begin{array}{c}\hat{\gamma}_{1}\mathbf a_{t}^{H}\left(\hat{\theta}_{1}^{t}, \hat{\phi}_{1}^{t}\right) \\ \vdots \\ \hat{\gamma}_{L}\mathbf a_{t}^{H} \left(\hat{\theta}_{L}^{t}, \hat{\phi}_{L}^{t}\right) \end{array}\right]\textrm{,} \end{aligned}} \end{array} $$
(17)

where (a) and (c) follow from the Lemma 1 that the basis beams are asymptotically orthogonal for sufficiently large MrNr,(b) and (d) are due to \(\mathbf a_{r}^{H}(\hat {\theta }_{l}^{r}, \hat {\phi }_{l}^{r})\mathbf a_{r}(\hat {\theta }_{l}^{r}, \hat {\phi }_{l}^{r})=M_{r}N_{r}\), and all results are proved in Appendix 6.

From (16) and (17), we have

$$ {\begin{aligned} \hat{\mathbf{F}}_{r}^{H}\hat{\mathbf H}\hat{\mathbf{F}}_{t}\mathbf{P}\!\!\!\!\! &\approx\sqrt{\frac{M_{r}N_{r}}{L}}\!\!\left[\begin{array}{c}\hat{\gamma}_{1}\mathbf a_{t}^{H}\left(\hat{\theta}_{1}^{t}, \hat{\phi}_{1}^{t}\right) \\ \vdots \\ \hat{\gamma}_{L}\mathbf a_{t}^{H} \left(\hat{\theta}_{L}^{t}, \hat{\phi}_{L}^{t}\right) \end{array}\right] \cdot\frac{1}{\sqrt{M_{t}N_{t}}}\left[\sqrt{p_{1}}\mathbf a_{t}\left(\hat{\theta}_{1}^{t}, \hat{\phi}_{1}^{t}\right) \cdots \sqrt{p_{L}}\mathbf a_{t}\left(\hat{\theta}_{L}^{t}, \hat{\phi}_{L}^{t}\right)\right]\\ \!\!\!\!\!\!\!&\!\!\!\!\!\!\!\!\overset{(e)}{\approx}\!\sqrt{\frac{M_{r}N_{r}}{LM_{t}N_{t}}} \left[\!\begin{array}{llc} \sqrt{p_{1}}\hat{\gamma}_{1}\mathbf a_{t}^{H}\left(\hat{\theta}_{1}^{t}, \hat{\phi}_{1}^{t}\right)\mathbf a_{t}\left(\hat{\theta}_{1}^{t}, \hat{\phi}_{1}^{t}\right) & \cdots &\! 0\\\ \vdots &\ddots & \vdots\\ 0 &\cdots & \sqrt{p_{L}}\hat{\gamma}_{L}\mathbf a_{t}^{H} \left(\hat{\theta}_{L}^{t}, \hat{\phi}_{L}^{t}\right)\mathbf a_{t}\left(\hat{\theta}_{L}^{t}, \hat{\phi}_{L}^{t}\right) \end{array}\right] \\ &\overset{(f)}{=}\!\sqrt{\frac{M_{t}N_{t}M_{r}N_{r}}{L}} \left[\begin{array}{llc} \sqrt{p_{1}}\hat{\gamma}_{1} & \cdots & 0\\ \vdots &\ddots & \vdots\\ 0 & \cdots & \sqrt{p_{L}}\hat{\gamma}_{L} \end{array}\right] \textrm{,} \end{aligned}} $$
(18)

where (e) and (f) follow the Lemma 1 and \(\mathbf a_{t}^{H}\left (\hat {\theta }_{l}^{t}, \hat {\phi }_{l}^{t}\right)\mathbf a_{t}\left (\hat {\theta }_{l}^{t}, \hat {\phi }_{l}^{t}\right)=M_{t}N_{t}\), respectively, and all proofs are shown in Appendix 6.

Combining (15) and (18), the product \(\left (\hat {\mathbf {F}}_{r}^{H}\hat {\mathbf {F}}_{r}\right)^{-1}\left (\hat {\mathbf {F}}_{r}^{H}\hat {\mathbf H}\hat {\mathbf {F}}_{t}\mathbf {P}\right)\left (\hat {\mathbf {F}}_{r}^{H}\hat {\mathbf H}\hat {\mathbf {F}}_{t}\mathbf {P}\right)^{H}\) in (12a) is derived as

$$ {{}\begin{aligned} &\left(\hat{\mathbf{F}}_{r}^{H}\hat{\mathbf{F}}_{r}\right)^{-1}\left(\hat{\mathbf{F}}_{r}^{H}\hat{\mathbf H}\hat{\mathbf{F}}_{t}\mathbf{P}\right)\left(\hat{\mathbf{F}}_{r}^{H}\hat{\mathbf H}\hat{\mathbf{F}}_{t}\mathbf{P}\right)^{H}\\&\quad \approx\frac{M_{t}N_{t}M_{r}N_{r}}{L}\!\! \left[\begin{array}{lllc} p_{1}|\hat{\gamma}_{1}|^{2} & \cdots & 0\\ \vdots &\ddots & \vdots\\ 0 & \cdots & p_{L}|\hat{\gamma}_{L}|^{2} \end{array}\right] \textrm{.} \end{aligned}} $$
(19)

The objective function in (12a) can be formulated as

$$\begin{array}{@{}rcl@{}} R\approx\sum_{l=1}^{L}\text{log}\left(1\,+\,\frac{M_{t}N_{t}M_{r}N_{r}}{L\sigma_{n}^{2}}p_{l}|\hat{\gamma}_{l}|^{2}\right)\textrm{.} \end{array} $$
(20)

Problem (12) with the simplified objective function (20) is a convex problem, and admits a water-filling solution as Theorem 1.

To sum up, the main steps of the low-complexity beam-domain signal processing are summarized as follows:

The low-complexity beam-domain signal processing

  • In forward link, the receiver estimates beams \((\hat {\theta }_{l}^{r},\hat {\phi }_{l}^{r})\).

  • Estimate the beams \((\hat {\theta }_{l}^{t},\hat {\phi }_{l}^{t})\) and the magnitude of beam gain \(|\hat {\gamma }_{l}|\) by (8) in reverse link.

  • With the estimated beams, RF precoder at the transmitter and RF combiner at the receiver are derived from (4) and (5), respectively.

  • Perform the power allocation only with \(|\hat {\gamma }_{l}|\) according to Theorem 1.

4 Numerical results and discussion

In this section, we present the numerical results to evaluate the performance of the proposed signal processing methods in hybrid architecture massive MIMO systems. In the simulations, both the transmitter and receiver are equipped with 16 ×16 URA and 16 RF chains. The channel fading coefficients are generated from the urban micro model in the 3GPP standard [24]. The path loss is given by PL(dB)=36.7log10(d)+22.7+26log10(fc), where d=50 m is the distance between the transmitter and receiver, and fc=1 GHz. All simulation results are obtained by averaging over 1000 channel realizations.

Figure 2 illustrates the normalized mean square error (MSE) of the proposed channel estimation scheme and the scheme in [8], as a function of SNR with L=2,4,6,8. We see that the proposed channel estimation algorithm provides a much higher estimation accuracy than the method in [8]. The higher accuracy is for the reason that the magnitude of the beam gain is estimated from the statistic characteristic of all signal as shown (8). By using the statistic characteristic, the accuracy of the estimation is much higher than that of using the one-time training pilots. Moreover, in Fig. 2, the MSE increases with the number of beams L. The reason is that the channel estimation depends on the estimated DOAs, as shown in (8)-(11). As L increases, the error of DOA estimation increases, resulting in larger MSE.

Fig. 2
figure 2

MSE of the proposed channel estimation scheme and the method in [8] with L=2,4,6,8

The channel capacity (CC) of the MIMO system can be obtained from the singular value decomposition (SVD) processing. To evaluate the optimality of the proposed power allocation, Fig. 3 presents the SVD-based channel capacity with the water-filling power allocation and the spectral efficiency (SE) of the proposed optimal power allocation, while the URA with the different number of antennas is employed and 6 beams are considered. It can be seen that the spectral efficiency is getting close to the channel capacity with the antenna number increasing from 4×4 to 16×16. The reason is that the proposed power allocation is approximated from the asymptotical orthogonality of basis beams for sufficient large antenna number M×N in Lemma 1. Furthermore, Fig. 3 shows that the URA with 8×8 antennas can ensure the asymptotical results, which indicates the proposed power allocation is available for the massive MIMO system.

Fig. 3
figure 3

Channel capacity and spectral efficiency of the proposed massive MIMO system employing URA with different antenna numbers, where L=6

Figure 4 compares the spectral efficiency of the proposed optimal power allocation (OPA) and equal power allocation (EPA) among multiple beams. It is observed that the spectral efficiency of the OPA is higher than that of EPA, especially in the SNR region of 0–15 dB. Besides, it is interesting to find that when SNR is relatively low, the spectral efficiency decreases with L, as shown in the enlarged view. In contrast, the spectral efficiency increases with L at high SNR. The reason is that, at low SNR, the transmit power is low, and the water-filling algorithm only allocates the power to one beam, resulting in spectral efficiency degradation after the channel is normalized by L. At high SNR, the transmit power is filling to all beams, which brings in higher spectral efficiency when L increases.

Fig. 4
figure 4

Spectral efficiency of the proposed optimal power allocation and equal power allocation with L=2,4,6,8

Figure 5 shows the spectral efficiency comparison of the proposed OPA and random power allocation (RPA) among multiple beams. It is observed that the spectral efficiency of the OPA is higher than that of RPA in the entire SNR region. The performance results of the OPA and RPA in Fig. 5 are similar to that of the OPA and EPA in Fig. 4, except that the spectral efficiency gap between the OPA and RPA is larger than that between the OPA and EPA.

Fig. 5
figure 5

Spectral efficiency of the proposed optimal power allocation and random power allocation with L=2,4,6,8

5 Conclusion

In this journal, we have investigated the beam-domain channel estimation and precoding for hybrid architecture massive MIMO systems. We have proposed a low-complexity channel estimation algorithm based on the estimated beam information. The RF processing has been derived from the beam steering as well. Furthermore, we have optimized the power allocation to maximize the spectral efficiency. Simulation results have shown that the proposed signal processing methods can achieve high channel estimation accuracy and spectral efficiency.

6 Appendix

6.1 Proof of Lemma 1

According to (2), it can be derived that

$$ {\begin{aligned} &\mathbf a^{H}(\theta_{l_{1}}, \phi_{l_{1}})\mathbf a(\theta_{l_{2}}, \phi_{l_{2}})\\&=\sum_{n=1}^{N}\sum_{m=1}^{M} e^{\!-jD(n-1)\sin\theta_{l_{1}}\sin\phi_{l_{1}}}e^{-jD(m-1)\cos\theta_{l_{1}}\sin\phi_{l_{1}}}\\& e^{jD(n-1)\sin\theta_{l_{2}}\sin\phi_{l_{2}}} e^{jD(m-1)\cos\theta_{l_{2}}\sin\phi_{l_{2}}}\\ &=\sum_{n=1}^{N}e^{jD(n-1)\left(\sin\theta_{l_{2}}\sin\phi_{l_{2}}-\sin\theta_{l_{1}}\sin\phi_{l_{1}}\right)} \\&\sum_{m=1}^{M}e^{jD(m-1)\left(\cos\theta_{l_{2}}\sin\phi_{l_{2}}-\cos\theta_{l_{1}}\sin\phi_{l_{1}}\right)}\\ &=\frac{1-e^{jD(N_{r}-1)\alpha_{1,2}}}{1-e^{jD\alpha_{1,2}}}\cdot \frac{1-e^{jD(M_{r}-1)\beta_{1,2}}}{1-e^{jD\beta_{1,2}}} \end{aligned}} $$
(21)

where \(D=\frac {2\pi }{\lambda }d, \alpha _{1,2}=\sin \theta _{l_{2}}\sin \phi _{l_{2}}-\sin \theta _{l_{1}}\sin \phi _{l_{1}}\), and \(\beta _{1,2}=\cos \theta _{l_{2}}\sin \phi _{l_{2}}-\cos \theta _{l_{1}}\sin \phi _{l_{1}}\).

Since \(\theta _{l_{1}}\neq \theta _{l_{2}}\) and \(\phi _{l_{1}}\neq \phi _{l_{2}}\) for l1l2,α1,2 and β1,2 are not equal to 0 for l1l2. Therefore, \(\mathbf a^{H}(\theta _{l_{1}}, \phi _{l_{1}})\mathbf a(\theta _{l_{2}}, \phi _{l_{2}})\) is bounded as M,N, and there exists

$$\begin{array}{@{}rcl@{}} \mathop{\lim}\limits_{M, N\rightarrow\infty}\frac{\mathbf a^{H}\left(\theta_{l_{1}}, \phi_{l_{1}}\right)\mathbf a\left(\theta_{l_{2}}, \phi_{l_{2}}\right)}{\sqrt{MN}}=0,\qquad l_{1}\neq l_{2}\mathrm. \end{array} $$
(22)

Thus, it is proved that the basis beams are asymptotically orthogonal for sufficiently large MN.

From direct calculation, it is derived that

$$\begin{array}{@{}rcl@{}} \mathbf a^{H}(\theta_{l_{1}}, \phi_{l_{1}})\mathbf a(\theta_{l_{2}}, \phi_{l_{2}})=MN,\qquad l_{1}= l_{2}\mathrm. \end{array} $$
(23)