1 Introduction

To provide remote areas with the same telecommunication services as large cities, it is impractical to use only terrestrial cellular networks for coverage. Multi-beam satellite communication systems that operate in high frequency bands, have high throughput, and provide broadband Internet access services are indispensable [1,2,3,4,5,6,7,8]. Multi-beam satellite system has been studied because of the frequency reuse accomplished in the satellite network, channel capacity of which has been increased [4]. High-throughput satellite systems are aimed at providing fixed and mobile personal communication services covering the world. Mobile communication in the future is also developing towards the seamless integration of satellite and ground communication systems, and the differences between the two systems will no longer exist [5]. The future mobile communication is developing towards the seamless integration of satellite and terrestrial communication systems, and multi-beam satellites have become an important research direction [6].

The optimal precoding algorithm can realize the capacity limit of any channel state. The optimal precoding algorithm is an optimal method, which perfectly preprocesses the channel states of users of all beams to form a logical channel, and the implementation complexity is very high. However, the complexity of the minimum mean square error (MMSE) precoding algorithm is relatively low and can be applied in practice [7]. The Wiener model of cellular mobile communication systems can benefit from inter-beam interference. However, the multi-beam model suitable for multi-beam satellite communication systems is always limited by interference, that is, the system performance deteriorates sharply when the interference level exceeds a certain level. Due to the rapid development of terrestrial mobile communication technology, there are many studies on the Wiener model and the channel capacity of terrestrial cellular mobile communication systems [8]. For example, reference [9] uses Toeplitz and the asymptotic equivalence of circulant matrices to give the channel capacity of Wiener 1D and 2D models applying MMSE precoding. The reference [10] is the first to study the emerging multi-beam satellite, and analyzes the performance of the low-earth orbit multi-beam satellite system using convolutional coding and soft-decision Viterbi decoding. It can be seen that user interference outside the satellite beam coverage area, but still visible within the line-of-sight (LOS), has a significant impact on satellite system capacity. On this basis, the reference [11] studies the traversal capacity of the reverse link of the multi-beam satellite system. Considering the shadow fading and rain fading problems of satellite channels, a Rice/log-normal composite model is proposed to analyze the lower bound of ergodic capability. Reference [12] proposes a hierarchical modulation scheme to increase the channel capacity of satellite communication systems by adding more data streams to the transmission of existing digital broadcasting systems. Reference [13] mitigates the impact of co-channel interference through a cooperative transmission scheme, thereby improving the forward link capacity of a multi-beam satellite system, bringing substantial improvements in spectral efficiency. Reference [14] studies the architecture of next-generation multi-beam satellite systems, including various satellite payload options, ground terminal advancements, and scalable system-level software control and management techniques. Although there have been in-depth studies on the capacity enhancement technology and frequency band limitation of the multi-beam satellite system, the research on the channel capacity limit of the multi-beam satellite system is still lacking, and further theoretical research is needed to improve it.

This paper firstly analyzes the Wiener-Gaussian access model of terrestrial cellular systems. After considering the difference between the spatial characteristics and gain characteristics of the multi-beam satellite system compared with the terrestrial cellular system, a Wiener model suitable for the multi-beam satellite communication system is proposed. On this basis, when using optimal linear precoding and minimum mean square error precoding, the channel capacity limits of multi-beam satellites under different fading conditions are deduced. It provides theoretical support for the in-depth study of the channel capacity mechanism and system performance of multi-beam satellites.

The rest of this paper is organized as follows: Sect. 2 constructs a multi-beam satellite beam coverage model; Sect. 3 analyzes the channel capacity of optimal linear precoding and minimum mean square error precoding in a fading-free channel model; Sect. 4 further introduces the fading channel model to analyze its channel capacity; Sect. 5 presents the simulation results and analysis; Sect. 6 gives the conclusion.

2 Multibeam satellite beam coverage model

A multi-beam satellite forms N beams to cover N different areas, and each beam contains K users, assuming that the system can achieve perfect symbol and frame synchronization, and uses a fading-free channel model. Then at any time interval l, the signal received by the multi-beam antenna can be represented by a \(N \times 1\)-dimensional vector as:

$$y\left( l \right) = {\mathbf{A}}x\left( l \right) + n\left( l \right)\;,$$
(1)

where \(x\left( l \right) = \left( {x_{1,1} \left( l \right), \ldots ,x_{1,K} \left( l \right), \ldots ,x_{N,1} \left( l \right), \ldots ,x_{N,K} \left( l \right)} \right)^{t}\) is the \(NK \times 1\) dimensional user signal, \(x_{i,k} \left( l \right)\) is the \(k_{th}\) user signal within the i satellite beam and satisfied the average power limit \(E\left[ {x_{i,k} \left( l \right)x_{i,k}^{H} \left( l \right)} \right] \le P\), \(n\left( l \right)\) is Gaussian noise with zero mean and variance \(\sigma^{2}\), and the \(N \times NK\) dimensional matrix \(A\) is used to describe the impact of inter-beam interference, which can be expressed as:

$${\mathbf{A}} = \left( {\begin{array}{*{20}c} {a_{1,1} } & {a_{1,2} } & \cdots & {a_{1,N} } \\ {a_{2,1} } & {a_{2,2} } & \cdots & {a_{1,N} } \\ \vdots & \vdots & \ddots & \vdots \\ {a_{N,1} } & {a_{N,2} } & \cdots & {a_{N,N} } \\ \end{array} } \right)\;,$$
(2)

where \(a_{n,i} = \left( {a_{n,i,1} , \ldots ,a_{n,i,K} } \right)\) represents the gain of \(K\) users in the \(1 \times K\) dimensional beam, each element \(a_{n,i,k}\) represents the path gain from \(n_{th}\) beam to user k in the \(i_{th}\) beam, and the structure of A depends on the topology of the system and the radiation characteristics of the beam.

In this communication mode, it is assumed that K users in the beam are located near the center of the beam, and the received beam signals have the same gain coefficient. At the same time, assume \(a_{i,n} = b_{i,n} 1_{K}\), where \(b_{i,n}\) represents the \(n_{th}\) beam to \(K\) users in the \(i_{th}\) beam. Therefore, the \(N \times NK\) dimensional matrix A can be expressed as the Kronecker product of the \(N \times NK\) dimensional matrix B and the all-one \(1 \times K\) dimensional matrix, that is, \({\mathbf{A}} = {\mathbf{B}} \otimes {\mathbf{1}}_{K}\), where the matrix B depends on the beam coverage model. This paper will study the matrix B corresponding to the Wiener model and the multi-beam model respectively.

2.1 Wiener model

In the terrestrial cellular mobile communication system, there is only the case of adjacent beam interference. Therefore, the Gaussian cellular multiple access model of Wyner, namely the Wiener model, is adopted, and its one-dimensional linear model is shown in Fig. 1. Based on the one-dimensional model, the two-dimensional Wiener model is a hexagonal continuous beam arrangement, and the number of beams is \(N = LM\), as shown in Fig. 2.

Fig. 1
figure 1

Wiener 1D Model for Terrestrial Cellular Mobile Communication System

Full size image
Fig. 2
figure 2

Wiener 2D Model for Terrestrial Cellular Mobile Communication System

Full size image

In the Wiener model of the terrestrial cellular mobile communication system, it is assumed that the interference signal received by the user in the specified beam from other beams is \(\alpha\) times the transmitted signal and \(0 \le \alpha \le 1\). Then for the Wiener one-dimensional model, the corresponding matrix B can be expressed as is a tridiagonal matrix \({\mathbf{T}}_{1D}\):

$${\mathbf{T}}_{1D} = \left( {\begin{array}{*{20}c} 1 & \alpha & {} & {} & {} & {} \\ \alpha & 1 & \alpha & {} & {} & {} \\ {} & \alpha & 1 & \ddots & {} & {} \\ {} & {} & \ddots & \ddots & \alpha & {} \\ {} & {} & {} & \alpha & 1 & \alpha \\ {} & {} & {} & {} & \alpha & 1 \\ \end{array} } \right)\;.$$
(3)

Similarly, for the two-dimensional model of Wiener hexagonal arrangement as shown in Fig. 2, its matrix B can be expressed as a TBT (Toeplitz-block-Toeplitz) matrix \({\mathbf{T}}_{2D}\) composed of two \(L \times L\) and \(M \times M\) dimension submatrices:

$${\mathbf{T}}_{2D} = \left( {\begin{array}{*{20}c} {{\mathbf{T}}_{M} } & {{\mathbf{S}}_{M} } & {} & {} & {} & {} \\ {{\mathbf{S}}_{M}^{t} } & {{\mathbf{T}}_{M} } & {{\mathbf{S}}_{M} } & {} & {} & {} \\ {} & {{\mathbf{S}}_{M}^{t} } & {{\mathbf{T}}_{M} } & {{\mathbf{S}}_{M} } & {} & {} \\ {} & {} & {{\mathbf{S}}_{M}^{t} } & {{\mathbf{T}}_{M} } & \ddots & {} \\ {} & {} & {} & \ddots & \ddots & {{\mathbf{S}}_{M} } \\ {} & {} & {} & {} & {{\mathbf{S}}_{M}^{t} } & {{\mathbf{T}}_{M} } \\ \end{array} } \right)\;,$$
(4)

where

$${\mathbf{T}}_{M} { = }{\mathbf{T}}_{1D} { = }\left( {\begin{array}{*{20}c} 1 & \alpha & {} & {} & {} & {} \\ \alpha & 1 & \alpha & {} & {} & {} \\ {} & \alpha & 1 & \ddots & {} & {} \\ {} & {} & \ddots & \ddots & \alpha & {} \\ {} & {} & {} & \alpha & 1 & \alpha \\ {} & {} & {} & {} & \alpha & 1 \\ \end{array} } \right),\;\;{\mathbf{S}}_{M} = \alpha \left( {\begin{array}{*{20}c} 1 & {} & {} & {} & {} \\ 1 & 1 & {} & {} & {} \\ {} & 1 & 1 & {} & {} \\ {} & {} & \ddots & \ddots & {} \\ {} & {} & {} & 1 & 1 \\ \end{array} } \right).$$

2.2 Wiener model of multi-beam satellite communication system

The traditional Wiener model only considers the inter-beam interference of adjacent beams. In a multi-beam satellite communication system, the level of inter-beam interference depends on the beam distribution and beam spacing, and the radiation gain of a spot beam antenna decreases with the boresight angle [15]. Therefore, this paper proposes a Wiener model suitable for multi-beam satellite communication systems, hereinafter referred to as the multi-beam model, and its 61-beam arrangement scheme is shown in Fig. 3.

Fig. 3
figure 3

61-beam arrangement scheme for multi-beam satellites

Full size image

Assuming that \(K\) users in each beam are located near the center of the beam, the distance from the user in beam \(i\) to the center of the beam \(n\) can be expressed as \(d = 2\left( {i - n} \right)R_{cell}\), where \(R_{cell}\) is the beam radius (-3 dB area). Thus, the element in \({\mathbf{B}}\) can be denoted as \(B_{i,n} = \alpha^{{\left| {i - n} \right|}}\), where \(\alpha\) is the isolation degree between beams, \(\alpha = 0\) represents complete isolation between beams without interference, and \(\alpha = 1\) represents the maximum interference between beams. Therefore, the matrix \({\mathbf{T}}_{1D}\) corresponding to the one-dimensional Wiener model in the multi-beam model can be expressed as matrix \({\hat{\mathbf{T}}}_{1D}\):

$${\hat{\mathbf{T}}}_{1D} = \left( {\begin{array}{*{20}c} 1 & \alpha & {\alpha^{2} } & \cdots & {\alpha^{N - 1} } \\ \alpha & 1 & \alpha & \cdots & {\alpha^{N - 2} } \\ {\alpha^{2} } & \alpha & 1 & \cdots & {\alpha^{N - 3} } \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ {\alpha^{N - 1} } & {\alpha^{N - 2} } & {\alpha^{N - 3} } & \cdots & 1 \\ \end{array} } \right)\;.$$
(5)

Similarly, the elements in the matrix \({\mathbf{B}}\) corresponding to the two-dimensional multi-beam model can be expressed as \(b_{i,n} = \alpha^{{d_{i,n} /R_{adj} }}\), where \(R_{adj}\) is the distance from the center point of adjacent beams, \(d_{i,n}\) is the distance from the center point of the \(i\) beam to the center point of the \(n\) beam, Thus, according to the beam coverage scheme, the inter-beam interference matrix \({\hat{\mathbf{T}}}_{2D}\) of the multi-beam model can be obtained.

3 Channel capacity analysis of fading-free channel model

3.1 Channel capacity of optimal linear precoding algorithms

For the optimal linear precoding algorithm, the capacity limit of a fading-free channel can be expressed as:

$$C_{opt} \left( {{\mathbf{A}};\gamma } \right) = \max \frac{1}{N}{\rm I}\left( {x;y|{\mathbf{A}}} \right)\;.$$
(6)

Among them, the capacity limit is defined as the maximum capacity that can be achieved by traversing the beam arrangement form and user distribution form, and is constrained by the average power, \(\gamma = P/\sigma^{2}\) represents the signal-to-noise ratio, and \({\rm I}\left( {x;y|{\mathbf{A}}} \right)\) represents the amount of mutual information of the system. Since each user signal is transmitted independently, when the user sends an independent and identically distributed Gaussian signal under the constraint of maximum average power, the amount of normalized input and output mutual information will reach the maximum [16], that is,\(E\left[ {xx^{H} } \right] = PI_{{_{NK} }}\). At this time, the theoretical capacity limit under the broadband transmission mode can be expressed as:

$$C_{opt} \left( {{\mathbf{A}};\gamma } \right) = \frac{1}{N}\log_{2} \det \left( {{\mathbf{I}}_{N} + \gamma {\mathbf{AA}}^{H} } \right) = \frac{1}{N}\sum\nolimits_{i = 1}^{N} {\log_{2} \left( {1 + \gamma \lambda_{i} \left( {{\mathbf{AA}}^{H} } \right)} \right)} \;,$$
(7)

where \(\lambda_{i} \left( {{\mathbf{AA}}^{H} } \right)\) represents the ith eigenvalue of \({\mathbf{AA}}^{H}\).

From Wyner's earlier research, when using intra-beam time division multiple access, the capacity of the system is the same as that of the broadband transmission scheme. Therefore, when the number of beams tends to be infinite, the achievable capacity of a single beam will no longer increase, and the capacity limit value can be obtained. For the Wiener one-dimensional model, there is \({\mathbf{AA}}^{H} = K{\mathbf{T}}_{1D}^{2}\), and for the Wiener two-dimensional hexagonal model, there is \({\mathbf{AA}}^{H} = K{\mathbf{T}}_{2D}^{2}\). Next, the Wiener one-dimensional model is used as an example to deduce the channel capacity. The eigenvalues \({\mathbf{T}}_{1D}\) shown in Eq. (4) can be expressed as:

$$\lambda_{i} \left( {{\mathbf{T}}_{1D} } \right) = 1 + 2\alpha \cos \left( {\frac{i\pi }{{N + 1}}} \right),\quad i = 1, \ldots ,N.$$
(8)

On this basis, the capacity limit of the Wiener one-dimensional linear beam arrangement when \(N \to \infty\) can be expressed as:

$$\mathop {\lim }\limits_{N \to \infty } C_{opt} \left( {{\mathbf{T}}_{1D} \otimes {\mathbf{1}}_{K} ;\alpha ,\gamma } \right) = \int_{0}^{1} {\log_{2} \left( {1 + \hat{\gamma }\left( {1 + 2\alpha \cos 2\pi \theta } \right)^{2} } \right)d\theta } \;,$$
(9)

where \(\hat{\gamma } = K\gamma\) represents the total power of \(K\) users in a single beam.

For the multi-beam model, the eigenvalues of \({\hat{\mathbf{T}}}_{1D}\) shown in Eq. (5) can be expressed as:

$$\lambda_{i} \left( {{\hat{\mathbf{T}}}_{1D} } \right) = \frac{{1 - \alpha^{2} }}{{1 + 2\alpha \cos \left( {\frac{i\pi + \varepsilon }{{N + 1}}} \right) + \alpha^{2} }}\;.$$
(10)

Similarly, the system capacity of the multi-beam model when \(\varepsilon \to \infty ,N \to \infty\) can be expressed as:

$$C_{opt} \left( {{\hat{\mathbf{T}}}_{1D} \otimes {\mathbf{1}}_{K} ;\alpha ,\gamma } \right) = \int_{0}^{1} {\log_{2} \left( {1 + \hat{\gamma }\left[ {\frac{{1 - \alpha^{2} }}{{1 + 2\alpha \cos \left( {\pi \theta } \right) + \alpha^{2} }}} \right]^{2} } \right)d\theta } \;.$$
(11)

According to Eq. (11), when \(\alpha \to 1\), the system capacity of the multi-beam model with any power tends to be zero. This is because \(\alpha \to 1\) means that no matter how far apart the beams will have very serious mutual interference. However, this situation does not occur in the Wiener one-dimensional linear model represented by Eq. (9), because each beam only interferes with adjacent beams at this time, that is, \({\mathbf{T}}_{1D}\) is of full rank. It can be seen that the multi-beam model is always interference-limited, and its ability to deal with inter-beam interference is limited.

In addition, there is \(\frac{{1 - \alpha^{2} }}{{1 + 2\alpha \cos \left( {\pi \theta } \right) + \alpha^{2} }} \approx 1 - 2\alpha \cos \left( {\pi \theta } \right) + O\left( {\alpha^{2} } \right)\) when \(\alpha\) is small, and the performance of the multi-beam model and the Wiener one-dimensional linear model is basically the same.

For the two cases of in-beam power \(\hat{\gamma } < 1\) and \(\hat{\gamma } \ge 1\), the capacity performance of the system under low SNR and high SNR is studied respectively. Equation (11) is equivalent using the extended formula \(\log_{2} \left( {1 + x} \right) = \sum\nolimits_{n = 1}^{\infty } {\left( { - 1} \right)^{n - 1} \frac{{x^{n} }}{n}} ,0 < x < 1\). When the signal-to-noise ratio \(\hat{\gamma } < 1\) is low, we can get:

$$\begin{aligned} C_{opt} \left( {{\hat{\mathbf{T}}}_{1D} \otimes {\mathbf{1}}_{K} ;\alpha ,\gamma } \right) & = \sum\nolimits_{n = 1}^{\infty } {\left( { - 1} \right)^{n - 1} \frac{{\hat{\gamma }^{n} \left( {1 - \alpha^{2} } \right)^{2n} }}{n}} \int_{0}^{1} {\frac{d\theta }{{\left( {1 + 2\alpha \cos \left( {\pi \theta } \right) + \alpha^{2} } \right)^{2n} }}} \\ & = \sum\nolimits_{n = 1}^{\infty } {\left( { - 1} \right)^{n - 1} \frac{{\hat{\gamma }^{n} }}{n}} \sum\nolimits_{k = 0}^{2n - 1} {\frac{{\left( {2n + k - 1} \right)!}}{{\left( {k!} \right)^{2} \left( {2n - k - 1} \right)!}}} \left( {\frac{{\alpha^{2} }}{{1 - \alpha^{2} }}} \right)^{k} . \\ \end{aligned}$$
(12)

For the case where \(\alpha\) is relatively small, \(\left( {\frac{{\alpha^{2} }}{{1 - \alpha^{2} }}} \right)^{k}\) can be ignored. When \(k \ge 2\), Eq. (12) can be expressed as:

$$C_{opt} \left( {{\hat{\mathbf{T}}}_{1D} \otimes {\mathbf{1}}_{K} ;\alpha ,\hat{\gamma }} \right) \approx \left( {\frac{{1 + \alpha^{2} }}{{1 - \alpha^{2} }}} \right)\log_{2} \left( {1 + \hat{\gamma }} \right) - \left( {\frac{{4\alpha^{2} }}{{1 - \alpha^{2} }}} \right)\left( {\frac{{\hat{\gamma }}}{{1 + \hat{\gamma }}}} \right)^{2} .$$
(13)

According to Eq. (13), when \(\alpha\) is small and the SNR is small, the channel capacity of the multi-beam model increases with the increase of \(\alpha\). In addition, when the SNR is high, Eq. (12) can be rewritten as:

$$\begin{aligned} C_{opt} \left( {{\hat{\mathbf{T}}}_{1D} \otimes {\mathbf{1}}_{K} ;\alpha ,\gamma } \right) & \ge \log_{2} \hat{\gamma } + 2\log_{2} \left( {1 - \alpha^{2} } \right) - 2\int_{0}^{1} {\log_{2} \left( {1 + 2\alpha \cos \pi \theta + \alpha^{2} } \right)d\theta } \\ & = \log_{2} \hat{\gamma } + 2\log_{2} \left( {1 - \alpha^{2} } \right)\;. \\ \end{aligned}$$
(14)

According to (14), the multi-beam model has no threshold effect at high SNR, and the channel capacity simply decreases as \(\alpha\) increases. It is different from the traditional Wiener 1D model where there is a threshold effect at \(\alpha = 1/2\) at high SNR. Therefore, in the multi-beam satellite system, the isolation between beams can reduce the interference, which is very important to the system performance.

3.2 Channel capacity of minimum mean square error precoding algorithm

The capacity limit of optimal linear precoding is a theoretical state. In practical applications, MMSE precoding with lower complexity and better performance is more commonly used. This paper will derive its channel capacity limit and compare the difference with the optimal linear precoding algorithm. The \(NK \times N\) dimensional MMSE precoding matrix can be expressed as:

$${\mathbf{G}} = \gamma {\mathbf{A}}^{H} \left( {I_{N} + \gamma {\mathbf{AA}}^{H} } \right)^{ - 1} \;.$$
(15)

Its average MMSE can be expressed as:

$$mmse\left( {{\mathbf{A}};\gamma } \right) = P\left[ {\frac{1}{NK}\sum\nolimits_{i = 1}^{N} {\frac{1}{{1 + \gamma \lambda_{i} \left( {{\mathbf{AA}}^{H} } \right)}}} + 1 - \frac{1}{K}} \right]\;.$$
(16)

The system capacity with linear MMSE precoding can be expressed as:

$$C_{mmse} \left( {{\mathbf{A}};\gamma } \right) = K\log_{2} \frac{P}{{mmse\left( {{\mathbf{A}};\gamma } \right)}} = - K\log_{2} \left( {\frac{1}{NK}\sum\nolimits_{i = 1}^{N} {\frac{1}{{1 + \gamma \lambda_{i} \left( {{\mathbf{AA}}^{H} } \right)}}} + 1 - \frac{1}{K}} \right)\;.$$
(17)

Using the asymptotic equivalence of Toeplitz and circulant matrices, Wyner gives the system capacity of the Wiener 1D and 2D models applying MMSE precoding in reference [17], which are expressed by Eqs. (18) and (19) respectively as:

$$C_{mmse} \left( {{\mathbf{T}}_{1D} \otimes {\mathbf{1}}_{K} ;\alpha ,\gamma } \right) = - K\log_{2} \left( {\frac{P}{K}\int_{0}^{1} {\left( {1 + K\gamma \left( {1 + 2\alpha \cos 2\pi \theta } \right)^{2} } \right)^{ - 1} } d\theta + 1 - \frac{1}{K}} \right)\;.$$
(18)
$$C_{mmse} \left( {{\mathbf{T}}_{2D} \otimes {\mathbf{1}}_{K} ;\alpha ,\gamma } \right) = - K\log_{2} \left( {\frac{P}{K}\int_{0}^{1} {\int_{0}^{1} {\left( {1 + K\gamma \left[ {1 + 2\alpha u\left( {\theta_{1} ,\theta_{2} } \right)} \right]^{2} } \right)^{ - 1} } } d\theta_{1} d\theta_{2} + 1 - \frac{1}{K}} \right)\;.$$
(19)

Similarly, substituting into Eq. (10), the channel capacity when the MMSE precoding algorithm is applied in the multi-beam model can be expressed as:

$$C_{mmse} \left( {{\hat{\mathbf{T}}}_{1D} \otimes {\mathbf{1}}_{K} ;\alpha ,\gamma } \right) = - K\log_{2} \left( {\frac{P}{K}\int_{0}^{1} {\left( {1 + K\gamma \left[ {\frac{{1 - \alpha^{2} }}{{1 + 2\alpha \cos \left( {\pi \theta } \right) + \alpha^{2} }}} \right]^{2} } \right)^{ - 1} d\theta } + 1 - \frac{1}{K}} \right)\;.$$
(20)

4 Fading channel model channel capacity analysis

Since the distance between the antenna feeds is negligible compared to the distance between the user and the satellite, it is assumed that the same user has the same fading coefficient on the antenna feed. And let \(d_{n,k}\) represent the fading coefficient from the \(n\) beam to the \(k\) user, then all user signals jointly received by the receiver under the fading channel can be expressed as:

$$y = {\mathbf{AD}}x + z = \left( {{\mathbf{B}} \otimes {\mathbf{1}}_{K} } \right){\mathbf{D}}x + z\;,$$
(21)

where D is an \(NK \times NK\) dimensional diagonal matrix with elements \(d_{n,k}\) that are complex, independent, strictly stationary, ergodic, and assuming normalized power for the fading process. Due to large free space loss and limited on-board transponder power, satellite communication systems usually work under LOS propagation conditions to transmit high-speed data. Therefore, the Rice fading model is used to describe the influence of the strength of the LOS component on the system performance [18]. The fading coefficient consists of real and imaginary parts and obeys an independent Gaussian distribution \(N\left( {\frac{{\mu_{R} }}{\sqrt 2 },\sigma_{R}^{2} } \right)\), where \(\mu_{R}^{2} = \frac{\kappa }{\kappa + 1}\),\(\sigma_{R}^{2} = \frac{1}{\kappa + 1}\), and \(\kappa\) is the Rice factor.

According to Eq. (6), the channel capacity limit when optimal linear precoding is applied in a fading channel can be expressed as:

$$C_{opt} \left( {{\mathbf{AD}};\gamma } \right) \triangleq \frac{1}{N}{\mathbb{E}}\left[ {{\rm I}\left( {x;y|{\mathbf{AD}}} \right)} \right] = \frac{1}{N}{\mathbb{E}}\left[ {\log_{2} \det \left( {I_{N} + \gamma {\mathbf{ADD}}^{H} {\mathbf{A}}^{H} } \right)} \right],$$
(22)

Through simplification, \({\mathbf{ADD}}^{H} {\mathbf{A}}^{H}\) in Eq. (22) is expressed as:

$${\mathbf{ADD}}^{H} {\mathbf{A}}^{H} = \sum\nolimits_{n = 1}^{N} {\left( {\sum\nolimits_{k = 1}^{K} {d_{n,k} d_{n,k}^{*} } } \right)} b_{n} b_{n}^{H} = {\mathbf{B}}\Theta {\mathbf{B}}^{H} \;,$$
(23)

where \(b_{i}\) represents the \(i\) column vector of matrix \({\mathbf{B}}\), and \(\Theta = {\text{diag}}\left( {\xi_{1} , \ldots ,\xi_{N} } \right),\xi_{n} \triangleq \sum\nolimits_{k = 1}^{K} {d_{n,k} d_{n,k}^{*} }\) is defined, then Eq. (22) can be rewritten as:

$$C_{opt} \left( {\left( {{\mathbf{B}} \otimes {\mathbf{1}}_{K} } \right){\mathbf{D}};\gamma } \right) \triangleq \frac{1}{N}{\mathbb{E}}\left[ {\log_{2} \det \left( {{\mathbf{I}}_{N} + \gamma {\mathbf{B}}\Theta {\mathbf{B}}^{H} } \right)} \right]\;.$$
(24)

Similarly, the channel capacity limit when MMSE precoding is applied in a fading channel can be expressed as:

$$C_{mmse} \left( {\left( {{\mathbf{\rm B}} \otimes {\mathbf{1}}_{K} } \right){\mathbf{D}};\gamma } \right) = {\mathbb{E}}\left[ { - K\log_{2} \left( {\frac{1}{NK}{\text{trace}}\left( {\left( {I_{N} + \gamma {\mathbf{B}}\Theta {\mathbf{B}}^{H} } \right)^{ - 1} } \right) + 1 - \frac{1}{K}} \right)} \right]\;.$$
(25)

Since the beam signals in the multi-beam model use broadband transmission, it is assumed that each beam contains a large number of users, that is, \(K \to \infty\). Since \(d_{n,k}\) is independent and \(E\left[ {d_{n,k} d_{n,k}^{H} } \right] = 1\), \(\mathop {\lim }\limits_{K \to \infty } \frac{1}{K}\sum\nolimits_{k = 1}^{K} {d_{n,k} d_{n,k}^{*} } = 1\) can be obtained according to the law of large numbers, which is \(\mathop {\lim }\limits_{K \to \infty } \frac{1}{K}{\mathbf{B}}\Theta {\mathbf{B}}^{H} = {\mathbf{BB}}^{H}\). Let \(\hat{\gamma } = K\gamma\) represent the total power of \(K\) users in a single beam and \(\hat{\gamma }\) remains constant as the number of users in the beam increases, then Eq. (24) can be simplified as:

$$\begin{aligned} \mathop {\lim }\limits_{K \to \infty } C_{opt} \left( {\left( {{\mathbf{B}} \otimes {\mathbf{1}}_{K} } \right){\mathbf{D}};\gamma } \right) & = \mathop {\lim }\limits_{K \to \infty } \frac{1}{N}\log_{2} \det \left( {{\mathbf{I}}_{N} + \hat{\gamma }\frac{1}{K}{\mathbf{B}}\Theta {\mathbf{B}}^{H} } \right) \\ & = \mathop {\lim }\limits_{K \to \infty } \frac{1}{N}\log_{2} \det \left( {{\mathbf{I}}_{N} + \hat{\gamma }{\mathbf{BB}}^{H} } \right) \\ & = \mathop {\lim }\limits_{K \to \infty } C_{opt} \left( {{\mathbf{B}};\hat{\gamma }} \right). \\ \end{aligned}$$
(26)

According to Eq. (26), no matter how the fading is distributed, the fading process will be averaged as the number of users per beam increases. At this time, the capacity of the optimal linear precoding scheme for the fading channel will be close to the case of the non-fading channel, that is, for the case of low SNR \(\hat{\gamma } < 1\), the channel capacity increases as \(\alpha\) increases. When \(\hat{\gamma } \ge 1\), the channel capacity decreases as \(\alpha\) increases.

5 Simulation verification and analysis

Geostationary orbit satellites are used in this paper, and the carrier frequency is 20 kHz. Simulation analysis is carried out to verify the conclusions derived in this paper. The channel capacity of the Wiener model and the multi-beam model with and without fading channels is studied. The simulation adopts the optimal linear precoding algorithm and MMSE precoding respectively. The low SNR environment is set to SNR = 0.08, the high SNR environment is set to SNR = 8, the number of beams formed is \(N = 50\), and the channel bandwidth is 500 MHz.

Figure 4 shows the channel capacity of a fading-free channel in a low SNR environment. When the inter-beam interference \(\alpha\) is small, the channel capacity of both models increases with the increase of \(\alpha\). However, the channel capacity of the multi-beam model decreases gradually with the increase of \(\alpha\) when \(\alpha > 0.8\). Therefore, the multi-beam model in the low SNR environment has an optimal level of inter-beam interference, beyond which the channel capacity will be reduced, and the system will turn into a state of limited inter-beam interference.

Fig. 4
figure 4

Channel capacity of fading-free channels in low SNR environments

Full size image

Figure 5 shows the channel capacity of a fading-free channel in a high SNR environment. When the inter-beam interference \(\alpha\) is small, the channel capacity difference between the two models is small, and both decrease with the increase of \(\alpha\). At \(\alpha > 0.5\), the channel capacity of the Wiener model starts to increase as \(\alpha\) increases, but appears the threshold effect. However, the channel capacity of the multi-beam model continues to decrease until it approaches zero. It can be seen that the Wiener model suitable for cellular mobile communication systems can benefit from inter-beam interference in the case of high SNR. While the multi-beam model is always interference-limited, and the best case is complete isolation between beams. In addition, according to Figs. 4 and 5, the curve of optimal linear precoding and MMSE precoding have the same changing trend, but the capacity limit of MMSE precoding algorithm is lower than that of optimal linear precoding.

Fig. 5
figure 5

Channel capacity of fading-free channels in high SNR environments

Full size image

Figure 6 shows the performance of two precoding algorithms in a multi-beam model fading channel. In a low SNR environment, the variation trend of channel capacity is similar to that of a fading-free channel. The channel capacity can benefit from the inter-beam interference when the inter-beam interference \(\alpha\) is small, but the capacity drops sharply when \(\alpha\) exceeds a certain level. This is because the inter-beam interference can increase the identification of useful signals to a certain extent in a low SNR environment. In addition, the MMSE precoding algorithm is not much different from the optimal linear precoding algorithm when \(\alpha\) is small, but its performance is obviously inferior to the optimal linear precoding algorithm when \(\alpha\) is large.

Fig. 6
figure 6

Channel capacity of fading channels in low SNR environments

Full size image

Figure 7 is a comparison of the performance of two precoding algorithms in a multi-beam model. In the case of high SNR, the multi-beam model is always interference-limited, and with the increase of the inter-beam interference \(\alpha\), the capacity performance decreases until it approaches 0, which is consistent with the conclusion of the fading-free channel.

Fig. 7
figure 7

Channel capacity of fading channels in high SNR environments

Full size image

Figure 8 shows the effect of the number of users and the Rice factor on the channel capacity under fading channel conditions. The vertical axis \(C_{{{\text{diff}}}}\) represents the capacity difference between the non-fading channel and the fading channel, and the number of users is 1, 5 and 25 users per beam, respectively. It can be seen that when the number of users is small, the LOS direct path strength has a great influence on the channel capacity. When the number of users is large, the LOS direct path strength basically does not affect the channel capacity, and the channel capacity benefits from the interference of users in the beam under the same Rice factor. When each beam contains a large number of users, the capacity of the optimal linear precoding scheme for fading channels will be close to that of fading-free channels.

Fig. 8
figure 8

Channel capacity versus the number of users and Rice factor

Full size image

6 Conclusion

The optimal precoding algorithm is an optimal method, which perfectly preprocesses the channel states of users of all beams to form a logical channel, and the implementation complexity is very high. The complexity of the MMSE precoding algorithm is relatively low but the performance is relatively perfect. This paper analyzes the Wiener-Gaussian access model for terrestrial cellular systems. After considering the difference between the spatial characteristics and gain characteristics of the multi-beam satellite system compared with the terrestrial cellular system, a Wiener model suitable for the multi-beam satellite communication system is proposed. On this basis, the channel capacity limits of multi-beam satellites under different fading conditions are derived when using optimal linear precoding and MMSE precoding. Whether in a fading-free channel or a fading channel, the channel capacity of the multi-beam model in a low SNR environment increases with the increase of the inter-beam interference. In the high SNR environment, the multi-beam model has no threshold effect, and the channel capacity decreases with the increase of the inter-beam interference until it approaches zero. Optimal linear precoding and MMSE precoding have the same trend with inter-beam interference, but the capacity limit of MMSE precoding algorithm is lower than that of optimal linear precoding. Under fading channel conditions and each beam contains a large number of users, its channel capacity will be close to that of a non-fading channel. These conclusions provide theoretical support for the in-depth study of the channel capacity mechanism and system performance of multi-beam satellites.