1 Introduction

In wireless networks, spectrum efficiency and energy efficiency are two most important metrics to consider, as their improvement is one of the promising means to realize broadband-green communication concept. Compared to the conventional orthogonal multiple access (OMA) scheme, non-orthogonal multiple access (NOMA) technique can further improve spectrum efficiency and system achievable rate, as the interference cancelation between difference users can be successfully implemented as long as one user’s received signal-to-interference-plus-noise ratio (SINR) for the other user’s signal is larger than or equal to the received SINR of the other user for its own signal [1]. Therefore, it has been deemed as a promising technique to mitigate spectrum congestion in the fifth generation (5G) networks [24]. On the other hand, with the assistance of relay(s), greater system capacity can be obtained with the same power consumption or lower power consumption is required while keeping the system capacity intact [5, 6]. However, the spectral efficiency of cooperation system will be reduced greatly if the relay operates in half-duplex mode, as such relay(s) cannot transmit and receive signals simultaneously. Hence, two-path relay protocol is commonly used in cooperation systems. Consequently, when NOMA and two-path relay cooperation techniques are combined effectively, the performance of spectral efficiency and power consumption can be improved greatly. Currently, NOMA-based relaying systems have been an emerging research topic in the recent days.

For NOMA-based relaying systems, when relays operate in half-duplex decode-and-forward (DF) mode, many system models and resource allocation schemes already have been proposed. In [7], for a system in which NOMA technique is applied in both direct and relay transmissions, analytical expressions for outage probability and ergodic sum capacity are derived. In [8], a NOMA-based relaying system is proposed to improve spectral efficiency as well as its achievable capacity is investigated. A buffer-aided NOMA relaying system is proposed in [9], its performance is investigated, and an adaptive transmission scheme for such system is proposed in [10]. For a system with slowly faded NOMA-equipped multiple-relay channels, the benefit of joint network channel coding and decoding is studied in [11]. In [12], an analytical framework for a NOMA-based relaying system is developed, and then, its performance over Rician fading channels is studied. In [13], the impact of relay selection on the performance of cooperative NOMA is studied, and then, a two-stage relay selection strategy is proposed. In [14], a novel signal detection scheme for a simple NOMA-based relaying system is proposed, and then, the ergodic sum rate and outage performance of the system are investigated. In [15], based on Alamouti space-time block-coded NOMA technique, a two-phase cooperative DF relaying scheme is proposed. In [16], a dual-hop cooperative relaying scheme using NOMA is proposed, where two sources communicate with their corresponding destinations in parallel over the same channel via a common relay. To maximize the throughput of a NOMA-equipped wireless network with multiple relays, in [17], a novel approach to dynamically select an optimal relay mode and optimal transmit power is proposed.

On the other hand, when relays operate in half-duplex amplify-and-forward (AF) mode, many schemes have been proposed to improve the performance of NOMA-based relaying systems. In [18], a NOMA-based multi-antenna-equipped relaying network is designed, and then, its outage performance is analyzed. When a base station communicates with multiple mobile users simultaneously through the help of a relay over Nakagami-m fading channels, the overall performance is analyzed in [19]. For a NOMA-equipped single-cell relay network, where an OFDM-based AF relay allocates its spectrum and power resources to source-destination (SD) pairs, a many-to-many two-sided SD pair-subchannel matching algorithm is proposed in [20]. In [21], a joint power allocation and relay beamforming design problem is investigated, and then, an alternating optimization-based algorithm is proposed to maximize the achievable rate. In [22], the outage performance of a cooperative NOMA-equipped relay system is studied, and then, an accurate closed-form approximation of the outage probability is derived. In [23], when multiple users transmit messages to two destinations under the help of multiple AF relays, an optimal relay selection criterion is proposed to improve outage performance, and closed-form analytical expressions for the outage probability are derived. In [24], a relay-aided NOMA technique is proposed for uplink cellular networks, where the cooperative relay transmission is used to accommodate more than one user per orthogonal resource block in the context of interference-limited scenarios.

To enhance system flexibility, in [25], for NOMA-equipped cooperative networks with both the DF and AF relaying protocols, where one base station communicates with two mobile users with the aid of multiple relays, a two-stage relay selection strategy is proposed while considering different quality-of-service (QoS) requirements of the users. In [26], the analytical performance of a NOMA-equipped cellular system, with multiple AF and DF relays, is analyzed and then compared with that of four traditional schemes. To enable continuous signal transmission, a full-duplex cooperative NOMA system between dual users is proposed in [27] and then the outage probability and achievable ergodic sum capacity of the corresponding system are investigated. In [28], a NOMA-based multi-pair two-way relay network is developed, and then, a rate splitting scheme and a successive group decoding strategy are proposed to optimize its performance.

Under saturated traffic scenarios, a source node usually transmits messages continuously at every time slot. If the corresponding system has a relay that forwards messages to the destination, this relay should have full-duplex functionality or more than one relay should be deployed in order to avoid buffer overflow at the source node. Two-path relaying is one of the complementary techniques to mitigate the hardware difficulty in realizing full-duplex functionality of a single relay node. To the best of our knowledge, only the authors in [28] considered the joint two-path relay and NOMA-based scenario. Nevertheless, they just focused on rate splitting and group decoding for NOMA-equipped multi-pair scenario and ignored optimal power allocation among the source and relay nodes. In this paper, we develop a NOMA-based two-path relay system with the full interference cancellation (FIC) concept, where a source node transmits signals to a single user under the assistance of two AF relays with half-duplex mode. In such system, while targeting on the maximization of achievable rate and the minimization of required frequency band, the power allocation among the source and relay nodes, is formulated as an optimization problem. Following the guidelines of Karush-Kuhn-Tucker (KKT) conditions, the dual decomposition method is adopted to solve both problems. Though the commonly used multicarrier scenario is not considered, the proposed scheme can be extended the multicarrier case with imperfect CSI, such as the system model in literature [29]. Overall, the main contributions of the paper are listed as follows.

  • We have shown the idea of a NOMA-based novel two-path relaying system, where the target information of the source node is divided into two equal streams and they are transmitted to the destination via two half-duplex AF relays in superposed mode. Because of adopting the idea of superposition model in the NOMA technique, intuitively, lower level of system frequency is required compared to a system with OMA scheme. In such system, while targeting on the maximization of the system capacity, the power allocation among the source and relay nodes is an optimization problem given that there are power constraints at the individual nodes as well as the entire system. The dual decomposition method is adopted to obtain the closed form expressions of the optimal power allocation.

  • Once the optimal power allocation at each node is known, in order to separate the superposed received messages at the destination, the optimal power allocation between the two data streams is formulated as an optimization problem while targeting on the minimization of the required frequency band. Similar to the solution scheme of the first problem, the dual decomposition method is adopted to obtain the closed form expression of the optimal power allocation.

  • We conduct extensive numerical simulation in order to verify the effectiveness and accuracy of the proposed power allocation scheme from different angles. Under different scenarios, we verify the analytical insights of the proposed power allocation scheme with the simulation results.

The rest of the paper is organized as follows. In Section 2, a novel NOMA-based two-path AF relay system model is presented, and then, the corresponding optimization problem is formulated. The optimal power allocation strategy is proposed in Section 3. We evaluate the proposed scheme in Section 4. Finally, we conclude the paper with some future research direction in Section 5.

2 System description

In Fig. 1, we demonstrate a novel NOMA-based two-path successive relay system. The system consists of a source node, a destination node, and two relays that operate in AF mode. For notational simplicity, the channel gain between two corresponding terminals is denoted as hi, i∈{1,2,⋯,6}. It is supposed that the relays work in half-duplex mode. Therefore, when the source node transmits signals continuously at every time slot, relay 1 and relay 2 receive the transmitted signals and forward them to the destination successively in an alternative mannerFootnote 1.

For such two-relay cooperative systems, the location of the relays affects system performance greatly. Hence, to maximize the system achievable rate, the relays should be put on the optimal positions. Generally speaking, when the interference between two relays can be canceled completely, the relays are usually put in the same place due to the sake of management. Hence, the distance from the relays to the source and the destination are equal.

The superposed signal, transmitted from the source node, can be written as \(\sqrt {a_{1} p_{s}}s_{1} + \sqrt {a_{2} p_{s}}s_{2}\), where ps denotes transmit power, s1 and s2 are transmitted data symbols with E[|s1|2]=E[|s2|2]=1, a1 and a2 are power allocation coefficients between the transmitted signals s1 and s2 while satisfying a1+a2=1 and a1>a2 constraints. In each slot, the destination node tries to recover the superposed signals and then decode s1 and s2 individually through the successive interference cancelation (SIC) technology. Similar to the system model in [6, 30], it is assumed that the channel between the source node and the destination node is in deep fade, and hence, the direct link between them can be ignored. The channel gain between two nodes is constant in a symbol duration, and it is inversely proportional to the fourth power of the distance between the transmitter and the receiver. Noise signal of each link follows Additive White Gaussian Noise (AWGN) distribution with zero mean and σ2 variance.

To facilitate the formulation, it is assumed that relay 1 and relay 2 receive signal from the source node and forward it to the destination in odd and even symbol duration, respectively. When the source node transmits its superposed signals, the forwarded signal from relay 1 in the odd nth slot can be shown as:

$$ x_{1}[\!n]= {\alpha_{1}}r_{1}[\!n-1] $$
(1)

where pr is the transmit power at relay 1 and α1 is amplifying coefficient, which can be expressed as \(\alpha _{1} = \sqrt {p_{r1}}/{|r_{1}[n-1]|}\) to normalize transmit power. Note that r1[ n−1] is the received signal in the (n−1)th symbol period and it can be formulated as (2), where x2[n−1] is the transmitted signal from relay 2 in the (n−1)th symbol duration, η1[ n−1] is the noise added to received signal, and s1[ n−1] and s2[n−1] are the transmitted superposed signals in the (n−1)th symbol duration. From now on, for the sake of simplicity, \(\sqrt {a_{1} p_{s}}s_{1}[n-1]+\sqrt {a_{2} p_{s}}s_{2}[\!n-1]\) is denoted as xs[ n−1] until the final system capacity is computed.

$$ \begin{aligned} r_{1}[\!n-1]= & h_{1} \left\{\sqrt{a_{1} p_{s}}s_{1}[n-1]+\sqrt{a_{2} p_{s}}s_{2}[\!n-1]\right\}\\ & +h_{3} x_{2}[\!n-1] + {\eta}_{1}[\!n-1]. \end{aligned} $$
(2)

Therefore, the received signal at the destination in the nth symbol duration can be shown as:

$$ \begin{aligned} y_{d}[n] & =h_{6} x_{1}[\!n]+n_{d}[n]\\ & = \alpha_{1} h_{1} h_{6} x_{s}[\!n-1] + \alpha_{1} h_{3} h_{6} x_{2}[\!n-1] \\ &\quad + \alpha_{1} h_{6} {\eta}_{1}[\!n-1] + {\eta}_{d}[\!n] \end{aligned} $$
(3)

where ηd[n] is the additive Gaussian noise with zero mean and σ2 variance.

In (3), α1h1h6xs[ n−1] is the desired signal to be recovered, α1h6η1[ n−1]+ηd[n] is the additive noise at the destination, and the term α1h3h6x2[ n−1] is the interference from relay 2 which should be canceled or suppressed.

In the (n+1)th even symbol duration, the received signal at the destination can be obtained in the similar manner, which can be shown as (4), where xs[n] is the transmitted superposed signal from the source in the nth symbol duration, ηd[n] is the noise signal, and α2 is an amplifying factor.

$$ \begin{aligned} y_{d}[n+1] & = \alpha_{2} h_{2} h_{5} x_{s}[\!n] + \alpha_{2} h_{4} h_{5} x_{1}[\!n] \\ &\quad + \alpha_{2} h_{5} n_{2}[\!n] +n_{d}[\!n+1]. \end{aligned} $$
(4)

It is assumed that the transmit power of the relays are equal in the system. Hence, the corresponding amplifying factor at the relays can be represented as \(\alpha _{1} = \sqrt {p_{r}}/\sqrt {\left |h_{1}\right |^{2} p_{s} + \left |h_{3}\right |^{2} p_{r} + \sigma _{1}^{2}}\) and \(\alpha _{2} = \sqrt {p_{r}}/\sqrt {\left |h_{2}\right |^{2} p_{s} + \left |h_{4}\right |^{2} p_{r} + \sigma _{2}^{2}}\), where the transmit power at both relay 1 and relay 2 are considered as the same and it is pr. Without loss of generality, we assume that all the additive Gaussian noise be independent and identically distributed (i.i.d) with zero-mean and σ2 variation.

3 Achievable rate analysis and optimal power allocation

3.1 Achievable rate analysis and problem formulation

The received superposed signal \(\sqrt {a_{1} p_{s}}s_{1} + \sqrt {a_{2} p_{s}}s_{2}\) at the destination can be shown as \(S = \left |h_{tot}\right |^{2}\left (\sqrt {a_{1} p_{s}}s_{1} + \sqrt {a_{2} p_{s}}s_{2}\right) + {\eta }_{tot}\), where htot represents the equivalent fading coefficient during the signal transmission process from the source to the destination and ηtot is the total equivalent additive noise added to the original signal. For the transmitted data stream s1 and s2, after SIC is employed to cancel interference between them, their achievable rate can be shown as \(C_{1} = \text {log}_{2} (1 + \left |h_{tot}\right |^{2} a_{1} p_{s}/\left (\left |h_{tot}\right |^{2} a_{2} p_{s} + \sigma _{tot}^{2}\right)\) and \(C_{2} = \text {log}_{2} \left (1 + \left |h_{tot}\right |^{2} a_{2} p_{s}/\sigma _{tot}^{2}\right)\), respectively. Hence, the sum achievable rate can be shown as (5), which implies that only when the combined signal achieves maximal rate can C1+C2 reach its maximal point. Consequently, in order to maximize the system achievable rate, the achievable rate of the superposed signals should be maximized. In this paper, under predetermined constrains, we first explore the optimal power allocation between the source and relay nodes in order to obtain the maximal achievable rate. Then, we propose a power allocation strategy to balance the transmission rate between different data streams while targeting on the minimization of the required frequency band.

$$ {{} \begin{aligned} C &= C_{1} + C_{2} \\ &= \text{log}_{2} \left(1 \,+\, \frac{\left|h_{tot}\right|^{2} a_{1} p_{s}}{\left|h_{tot}\right|^{2} a_{2} p_{s} \,+\, \sigma_{tot}^{2}}\right) \,+\, \text{log}_{2}\left(1 \,+\, \frac{\left|h_{tot}\right|^{2} a_{2} p_{s}}{\sigma_{tot}^{2}}\right)\\ &= \text{log}_{2}\left(1+ \frac{\left|h_{tot}\right|^{2} p_{s}}{\sigma_{tot}^{2}}\right). \end{aligned}} $$
(5)

In the nth odd symbol duration, the received signal at the destination is shown as (3), from which we can see that x2[ n−1] is the forwarded signal from relay 2. On the other hand, the received signal at the destination in the (n−1)th symbol duration can be shown as yd[ n−1]=h5x2[ n−1]+ηd[n−1], based on which the interference from Relay 2 can be canceled. Hence, (3) can be rewritten as (6), where η[ n]=α1h6η1[ n−1]+α1h3h6ηd[ n−1]/h5+ηd[ n].

$$ y_{d}[\!n]=\alpha_{1} h_{1} h_{6} x_{s}[\!n-1] + {\eta}'[\!n]. $$
(6)

The resultant SNR can be shown as (7).

$$ \Gamma = \frac{\alpha_{1}^{2} \left|h_{1}\right|^{2} \left|h_{6}\right|^{2} p_{s}}{\left(\alpha_{1}^{2} \left|h_{6}\right|^{2} + 1 + \alpha_{1}^{2} \left|h_{3}\right|^{2} \left|h_{6}\right|^{2}/\left|h_{5}\right|^{2}\right) \sigma^{2}}. $$
(7)

As all the additive noise is assumed to have zero mean and unit variance, after substituting the expression of α1, the achievable rate in the odd symbol duration can be shown as Codd=log2[1+|h1|2|h5|2|h6|2pspr/{|h1|2|h5|2ps+(|h5|2|h6|2+|h3|2|h6|2+|h3|2|h5|2)pr+|h5|2}].

In the same way, the achievable rate in the even symbol duration can be obtained, which is shown as Ceven=log2[1+|h2|2|h5|2|h6|2pspr/{|h2|2|h6|2ps+(|h5|2|h6|2+|h4|2|h5|2+|h4|2|h6|2)pr+|h6|2}]. Maximizing the system achievable rate is equivalent to maximizing the achievable rate at both the odd and even symbol duration. However, under given constraints on transmit power, it is impossible to obtain maximal achievable rate in both the even and odd symbol duration simultaneously due to the variation channel fading. Therefore, we investigate the system achievable rate at the odd and even symbol duration separately while realizing the optimal power allocation. As both the processes and schemes are similar, we only present the scheme to maximize the achievable rate at the odd symbol duration for brevity, and the maximal achievable rate in the even symbol duration can be obtained in the similar manner. For notational simplicity, we use fi to denote |hi|2, i∈{1,2,⋯,6} hereafter.

It is straightforward to see that maximizing the achievable rate is equivalent to maximizing the SNR of the received signal. Hence, the optimization problem at the odd symbol duration can be formulated as:

$$ \begin{aligned} \max_{p_{s}, p_{r}}~~& \frac{f_{1} f_{5} f_{6} p_{s} p_{r}}{f_{1} f_{5} p_{s} + \left(f_{5} f_{6} + f_{3} f_{6} + f_{3} f_{5}\right) p_{s} + f_{5}}\\ \text{s.t.},~~ & p_{s}+p_{r} \leq p_{max}\\ & p_{s}\leq{p_{max}^{s}} \\ & p_{r}\leq{p_{max}^{r}},\\ \end{aligned} $$
(8)

where \(p_{max}^{s}\) and \(p_{max}^{r}\) are the maximal transmit power that the source node and the corresponding relay node can provide, respectively, and pmax is the total transmit power due to the consideration of interference threshold to other users when a cognitive scenario is investigated.

It is easy to see the formulation in (8) is a convex problem, so there is a optimal solution. However, though pr only appears in the numerator of expression, ps appears both in the numerator and denominator. Furthermore, three constrains on them should be considered jointly. Hence, it is difficult to obtain the solution straightforwardly.

3.2 The optimization of the achievable rate

To solve the problem in (8), the dual decomposition method is adopted. As a result, the resultant Lagrange function can be obtained, which can be shown as:

$$ {{} \begin{aligned} L\left(p_{s}, p_{r},\lambda_{1},\lambda_{2},\lambda_{3}\right)&=\frac{f_{1} f_{5} f_{6} p_{s} p_{r}}{f_{1} f_{5} p_{s} + \left(f_{5} f_{6} + f_{3} f_{6} + f_{3} f_{5}\right) p_{r} + f_{5}}\\ &\quad + \lambda_{1}\left(p_-p_{s} - P_{r}\right)\,+\,\lambda_{2}\left(p_{max}^{s}-p_{s}\right) \\ &\quad +\lambda_{3}\left(p_{max}^{r}-p_{r}\right), \end{aligned}} $$
(9)

where λi, i∈{1,2,3} are Lagrange multipliers with constraint λi≥0.

Based on the dual method, the relations in (10) can be derived while taking KKT conditions into account.

$$ {{} \begin{aligned} \frac{\partial L}{\partial {p_{s}}}\,=\,\lambda_{1} + \lambda_{2} - \frac{f_{1} f_{5} f_{6} p_{r} \left[\left(f_{5} f_{6} \,+\, f_{3} f_{6}+ f_{3} f_{5}\right) p_{r} + f_{5}\right]}{\left[f_{1} f_{5} p_{s} \,+\, \left(f_{5} f_{6} + f_{3} f_{6} + f_{3} f_{5}\right) p_{r} + f_{5}\right]^{2}}\,=\,0, \end{aligned}} $$
(10a)
$$ {{}\begin{aligned} \frac{\partial L}{\partial {p_{r}}}\,=\,\lambda_{1} + \lambda_{3} - \frac{f_{1} f_{5} f_{6} p_{s} \left(f_{1} f_{5} p_{s} + f_{5}\right)}{\left[f_{1} f_{5} p_{s} \,+\, \left(f_{5} f_{6} \,+\, f_{3} f_{6} + f_{3} f_{5}\right) p_{r} + f_{5}\right]^{2}}=0, \end{aligned}} $$
(10b)
$$ \qquad \qquad \qquad \lambda_{1}\frac{\partial L}{\partial {\lambda_{1}}}=\lambda_{1}\left(p_{max}-p_{s} - p_{r}\right)=0, $$
(10c)
$$ \qquad \qquad \qquad \; \lambda_{2}\frac{\partial L}{\partial {\lambda_{2}}}=\lambda_{2} \left(P_{max}^{s}-p_{s}\right)=0, $$
(10d)
$$ \qquad \qquad \qquad \; \lambda_{3} \frac{\partial L}{\partial {\lambda_{3}}} = \lambda_{3}\left(p_{max}^{r} - p_{r}\right)=0. $$
(10e)

According to (10), the scheme to achieve the optimally allocated power is presented as follows.

On the condition that \(p_{max}^{s} + p_{max}^{r} \leq p_{max}\) holds, it is straightforward to see that the maximal achievable rate can be obtained by setting \(p_{s} = p_{max}^{s}\) and \(p_{r} = p_{max}^{r}\). Therefore, we focus on the \(p_{max}^{s} + p_{max}^{r} > p_{max}\) case. To maximize the system achievable rate, ps+pr=pmax should be set.

On the condition that \(p_{max}^{s} + p_{max}^{r} > p_{max}\) holds, it is unlikely that both λ2>0 and λ3>0 occur, which implies that \(p_{r} = p_{max}^{r}\) and \(p_{s} = p_{max}^{s}\) should be set. Hence, there is no available solutions for this case. Consequently, to maximize the system achievable rate in the odd symbol duration, three cases need to be considered: (1) λ2=0 and λ3>0, (2) λ3=0 and λ2>0, and (3) λ2=0 and λ3=0.

Case 1: λ2=0 and λ3>0

For this case, based on (10e), \(p_{r} = p_{max}^{r}\) can be concluded. Combining with the ps+pr=pmax condition, \(p_{s}= p_{max} - p_{max}^{r}\) can be derived.

To identify its application scenario, substituting the allocated power into (10a) and (10b), and combining with the λ3>0 constraint, the following can be derived:

$$ \begin{aligned} &\left(f_{1} f_{5} - f_{5} f_{6} - f_{3} f_{6} - f_{3} f_{5}\right) {p_{max}^{r}}^{2} - 2\left(f_{1} f_{5} p_{max} + f_{5}\right)\\ & p_{max}^{r} + \left(f_ 1 f_{5} p_{max} + f_{5}\right)p_{max} > 0. \end{aligned} $$
(11)

It implies that when the relation in (11) is satisfied, \(p_{r} = p_{max}^{r}\) and \(p_{s}= p_{max} - p_{max}^{r}\) are the optimal level of power to maximize the system achievable rate. To simplify notation, let a=(f1f5f5f6f3f6f3f5), b=f1f5pmax+f5. It is easy to see that b>0 always holds. Accordingly, the relation in (11) can be rewritten as (12).

$$ a {p_{max}^{r}}^{2} - 2b p_{max}^{r} + b p_{max} > 0. $$
(12)

From (11), when a > 0 holds, it can be concluded that \(p_{max}^{r} > \left (b + \sqrt {b\left (b -a p_{max}\right)}\right)/a\) or \(p_{max}^{r} < \left (b -\!\!{\vphantom {\sqrt {b(b -a p_{max})}}}\right. \left.\sqrt {b(b -a p_{max})}\right)/a\) should hold. Moreover, it can be testified that \(\left (b + \sqrt {b(b -a p_{max})}\right)/a > p_{max}\) and \(0 < \left (b -\!\!{\vphantom {\sqrt {b(b -a p_{max})}}}\right. \left.\sqrt {b(b -a p_{max})}\right)/a < p_{max}\) hold. Hence, the available scenario is \(p_{max}^{r} < \left (b - \sqrt {b(b -a p_{max})}\right)/a\) under the a>0 conditionFootnote 2. On the other hand, if a<0 holds, the range of \(p_{max}^{r}\) can be shown as \(\left (b + \sqrt {b(b -a p_{max})}\right)/a < p_{max}^{r} < \left (b - \sqrt {b(b -a p_{max})}\right)/a\). As \(b - \sqrt {b(b -a p_{max})} <0\) holds, \(p_{max}^{r}\) locates in a positive range which is practical. Furthermore, as \(p_{s} = p_{max} - p_{max}^{r}\) and \(p_{max}^{r} < \left (b - \sqrt {b(b -a p_{max})}\right)/a\) hold, \(p_{s} > \left [-b + a p_{max} +\!\!{\vphantom {\sqrt {b(b -a p_{max})}}}\right. \left.\sqrt {b(b - {ap}_{max}}\right ]/a\) can be concluded, which implies that \(p_{max}^{s} > \left [-b + a p_{max} + \sqrt {b(b - {ap}_{max}}\right ]\) should hold.

To summarize, to maximize the system capacity, \(p_{r} = p_{max}^{r}\) and \(p_{s}= p_{max} - p_{max}^{r}\) should be set for the scenario that \(p_{max}^{r} < \left (b - \sqrt {b(b -{ap}_{max})}\right)/a\) and \(p_{max}^{s} > \left [-b + a p_{max} + \sqrt {b(b - {ap}_{max}}\right ]/a\) hold.

Case 2: λ3=0 and λ2>0

For this case, the solution can be obtained in the similar manner as that of case 1. As a result, \(p_{s} = p_{max}^{s}\) and \(p_{r} = p_{max} - p_{max}^{s}\) can be concluded. As λ2>0, combined with (10a) and (10b), the corresponding applicable condition can be shown as:

$$ {{} \begin{aligned} & \left(f_{1} f_{5} - f_{5} f_{6} - f_{3} f_{6} - f_{3} f_{5}\right) {p_{max}^{s}}^{2} + 2\left[(f_{5} f_{6} + f_{3} f_{6} + f_{3} f_{5})\right. \\ & \left. p_{max} + f_{5}\right] p_{max}^{s} - \left[(f_{5} f_{6} + f_{3} f_{6} \,+\, f_{3} f_{5})p_{max} \,+\, f_{5}\right] p_{max} < 0. \end{aligned}} $$
(13)

Taking the defined notations a, b, and c in case 1, (13) can be rewritten as:

$$ a {p_{max}^{s}}^{2} + 2\left(-a p_{max} + b\right)p_{max}^{s} + (a p_{max} - b) p_{max} < 0 $$
(14)

It is easy to see that Δ=4(−apmax+b)2−4a(apmaxb)pmax=4b(bapmax)>0 always holds. Hence, when a>0 and \(p_{max}^{s} > 0\) are considered, \(0 < p_{max}^{s} < \left (-b + a p_{max} + \sqrt {b(b-{ap}_{max})}\right)/a\) can be derived, which locates in a positive range. Hence, it is consistent with practical scenarios. When a<0, \(p_{max}^{s} > \left (-b + a p_{max} - \sqrt {b(b-{ap}_{max})}\right)/(2a)\) can be derived or \(p_{max}^{s} < \left (-b + {ap}_{max} + \sqrt {b(b-{ap}_{max})}\right)/a\) should hold. Moreover, it is easy to prove that \(0<\left (-b + {ap}_{max} + \sqrt {b(b-{ap}_{max})}\right)/a < p_{max}\) and \(\left (-b + a p_{max} - \sqrt {b(b-{ap}_{max})}\right)/a> p_{max}\) hold. Hence, the feasible solution is \(p_{max}^{s} < \left (-b + {ap}_{max} +\!\!{\vphantom {\sqrt {b(b -a p_{max})}}}\right. \left. \sqrt {b(b-{ap}_{max})}\right)/a\) when a<0.

To summarize, when \(p_{max}^{s} < \left (-b + a p_{max} + {\vphantom {\sqrt {b(b -a p_{max})}}}\right. \left.\sqrt {b(b-{ap}_{max})}\right)/a\) and \(p_{max}^{r}\! > \left (b - \sqrt {b(b-{ap}_{max})}\right)/a\), the optimally allocated power can be set as \(p_{s} = p_{max}^{s}\) and \(p_{r} = p_{max} - p_{max}^{s}\).

Case 3: λ2=0 and λ3=0.

For this case, substituting λ2=0 and λ3=0 into (10a) and (10b), the following can be obtained:

$$ \label {combined_equation} \left\{ \begin{array}{l} \lambda_{1} - \frac{f_{1} f_{5} f_{6} p_{r} \left[\left(f_{5} f_{6} + f_{3} f_{6} + f_{3} f_{5}\right) p_{r} + f_{5}\right]}{\left[f_{1} f_{5} p_{s} + \left(f_{5} f_{6} + f_{3} f_{6} + f_{3} f_{5}\right) p_{r} + f_{5}\right]^{2}}=0,\\ \lambda_{1} - \frac{f_{1} f_{5} f_{6} p_{s} (f_{1} f_{5} p_{s} + f_{5})}{\left[f_{1} f_{5} p_{s} + \left(f_{5} f_{6} + f_{3} f_{6} + f_{3} f_{5}\right) p_{r} + f_{5}\right]^{2}}=0,\\ \qquad \qquad \qquad \;\; p_{max}-p_{r} - p_{s} =0. \\ \end{array} \right. $$
(15)

In (15), once λ1 and ps are eliminated, (16) can be derived. Then, it can be rewritten as (17).

$$ \begin{aligned} & \left(f_{1} f_{5} - f_{5} f_{6} - f_{3} f_{6} - f_{3} f_{5}\right) p_{r}^{2} - 2(f_{1} f_{5} p_{max} + f_{5}) p_{r}\\ & + (f_{1} f_{5} p_{max} + f_{5}) p_{max} = 0. \end{aligned} $$
(16)
$$ \qquad \qquad \;\;\; a p_{r}^{2} - 2b p_{r} + bp_{max} = 0. $$
(17)

For the relation in (17), the feasible solution can be derivedFootnote 3, which can be shown as (18) under both the a>0 and a<0 conditions. As a result, the corresponding allocated power for the source node can be obtained, which can be shown as (19).

$$ p_{r} = \frac{b - \sqrt{b(b-ap_{max})}}{a}. $$
(18)
$$ p_{s} = \frac{-b + ap_{max} + \sqrt{b(b-ap_{max})}}{a}. $$
(19)

As allocated power should satisfy the \(p_{r} \leq p_{max}^{r}\) and \(p_{s} \leq p_{max}^{s}\) constraints, the applicable scenario is \(p_{max}^{r} \geq \left (b - \sqrt {b(b-ap_{max})}\right)/{a}\) and \(p_{max}^{s} \geq \left (-b +\!\!{\vphantom {\sqrt {b(b -a p_{max})}}}\right. \left. ap_{max} + \sqrt {b(b-ap_{max})}\right)/a\).

Combining all the above cases, the optimal power allocation scheme can be summarized as follows.

  • When \(p_{max}^{r} < \left (b - \sqrt {b(b -ap_{max})}\right)/a\) and \(p_{max}^{s} > \left (-b + a p_{max} + \sqrt {b(b - ap_{max})}\right)/a\) hold, \(p_{r} = p_{max}^{r}\) and \(p_{s} = p_{max} - p_{max}^{r}\) can be set.

  • When \(p_{max}^{r} > \left (b - \sqrt {b(b-ap_{max})}\right)/a\) and \(p_{max}^{s} < \left (-b + a p_{max} + \sqrt {b(b-ap_{max})}\right)/a\) hold, \(p_{s} = p_{max}^{s}\) and \(p_{r} = p_{max} - p_{max}^{s}\) should be set.

  • When \(p_{max}^{r} \geq \left (b - \sqrt {b(b-ap_{max})}\right)/{a}\) and \(p_{max}^{s} \geq \left (-b + ap_{max} + \sqrt {b(b-ap_{max})}\right)/a\), \(p_{r} = \left (b - \sqrt {b(b-ap_{max})}\right)/a\) and \(p_{s} = \left (-b + ap_{max} + \sqrt {b(b-ap_{max})}\right)/a\) should be set.

  • When \(p_{max}^{r} < \left (b - \sqrt {b(b-ap_{max})}\right)/{a}\) and \(p_{max}^{s} < \left (-b + ap_{max} + \sqrt {b(b-ap_{max})}\right)/a\) hold, \(p_{r} =p_{max}^{r}\) and \(p_{s} = p_{max}^{s}\) should be set.

In the similar manner, to maximize the achievable rate in the even symbol duration, the allocated power can be obtained, which is shown as follows, where c=f2f6f5f6f4f5f4f6 and d=f2f6pmax+f6.

  • On the condition that \(p_{max}^{r} \!< \!\left (d \,-\, \sqrt {d(d -cp_{max})}\right)\!/c\) and \(p_{max}^{s} > \left (-d + c p_{max} + \sqrt {d(d - cp_{max})}\right)\!/c\) hold, \(p_{r} = p_{max}^{r}\) and \(p_{s} = p_{max} - p_{max}^{r}\) can be set.

  • On the condition that \(p_{max}^{r} \!\!>\! \!\left (d - \sqrt {d(d-cp_{max})}\right)\!/c\) and \(p_{max}^{s} \!<\! \left (-d\! + c p_{max} + \sqrt {d(d-cp_{max})}\right)/c\) hold, \(p_{s} = p_{max}^{s}\) and \(p_{r} = p_{max} - p_{max}^{s}\) should be set.

  • On the condition that \(p_{max}^{r} \!\geq \! \left (d \,-\, \sqrt {d(d-cp_{max})}\right)\!/{c}\) and \(p_{max}^{s} \geq \left (-d + cp_{max} + \sqrt {d(d-cp_{max})}\right)/c\), \(p_{r} = \left (d - \sqrt {d(d-cp_{max})}\right)/c\) and \(p_{s} = \left (-d + cp_{max} + \sqrt {d(d-cp_{max})}\right)/c\) should be set.

  • On the condition that \(p_{max}^{r} \!\!<\! \left (d\! -\! \sqrt {d(d-cp_{max})}\right)/{c}\) and \(p_{max}^{s} < \left (-d + cp_{max} + \sqrt {d(d-cp_{max})}\right)/c\), \(p_{r} =p_{max}^{r}\) and \(p_{s} = p_{max}^{s}\) should be set.

3.3 The optimization of the required frequency band

As shown in (5), when total transmit power ps is fixed and channel state information is given, the optimal system achievable rate can be determined. Nevertheless, for the given power ps obtained optimally from the previous subsection, the proportional factors between the transmit data streams s1 and s2 affect the system spectral efficiency. As we know, for the given data stream C, when they are divided equally into two halves and transmitted in parallel mode with NOMA technique, the required frequency bandwidth is minimal, that is C/2. If C1C2, the required frequency is the bigger one, which means it is greater than C/2. Consequently, to minimize the required system bandwidth, the allocated power on two data symbols and relay should guarantee C1=C2. Hence, (20) can be concluded, and its solution can be shown as (21).

$$ \left|h_{tot}\right|^{2} p_{s} a_{2}^{2} + 2 \sigma_{tot}^{2} a_{2} - \sigma_{tot}^{2} = 0. $$
(20)
$$ a_{2} = \frac{-\sigma_{tot}^{2} + \sqrt{\sigma_{tot}^{2}\left(\sigma_{tot}^{2} + |h_{tot}|^{2} p_{s}\right)}}{\left|h_{tot}\right|^{2} p_{s}}. $$
(21)

From (7), \(\left |h_{tot}\right |^{2} = \alpha _{1}^{2} \left |h_{1}\right |^{2} \left |h_{6}\right |^{2}\) and \(\sigma _{tot}^{2} = \left (\alpha _{1}^{2} \left |h_{6}\right |^{2} + 1 + \alpha _{1}^{2} \left |h_{3}\right |^{2} \left |h_{6}\right |^{2}/\left |h_{5}\right |^{2}\right) \sigma ^{2}\) can be obtained. It can be proved that \(a_{2} = \sigma _{tot}^{2}/\left (\sigma _{tot}^{2} + \sqrt {\sigma _{tot}^{2} \left (\sigma _{tot}^{2} + \left |h_{tot}\right |^{2} P_{s}\right)}\right) < 1/2\), which is consistent with the a1+a2=1 and a2<a1 conditions.

4 Performance evaluation

In this section, a linear network is simulated to verify the effectiveness of our proposed schemes. Generally speaking, wireless channel is affected by Rayleigh and shadow fadings. For the resultant channels (in between the source, the relays and the destination) in the considered system, Rayleigh fading is statistically the same, and hence, we only consider the shadow fading in the simulation. To simplify the simulation process, the parameters of the shadow fading, represented by fi=|hi|2, i∈{1,2,⋯,6}, are generated randomly in the (0,1) range. Furthermore, as two relays are put together, their distance towards the source node and the destination node are equal, and so the allocated power in the odd and even symbol duration are the same. Consequently, for the sake of brevity, we only evaluate the performance of the odd symbol duration. Without loss of generality, it is furthermore assumed that all the Gaussian white additive noises are independent and identically distributed (i.i.d.) with zero mean and unit variance. For the sake of notational simplicity, all the transmit power and power consumption are normalized by the power of the additive noise in the following simulations.

Figure 2 shows the achievable rate of the system in the odd symbol duration with respect to the maximal total transmit power pmax. In this figure, two scenarios are considered, which are \(p_{max}^{s} = p_{max}^{r}=10~\mathrm {w}\) and \(p_{max}^{s} = P_{max}^{r}=20~\mathrm {w}\). The equal power allocation scheme is considered as the benchmark scheme, the idea of which is ps=pr with the pr+pspmax constraint. From the results, it can be seen that the achievable rate of our proposed scheme is always larger than that of the equal allocation scheme for the same total transmit power pmax. It is understandable that when \(p_{max} < p_{max}^{s} + p_{max}^{r}\), the proposed scheme can allocate the total transmit power optimally in accordance with the specific channel condition to maximize the system capacity. However, the allocated power obtained by the equal allocation scheme always remains fixed irrespective of the instant channel state information, which is not necessary the optimal allocated power. At the same time, when pmax≥16 w, the system capacity of scenario \(p_{max}^{s} = P_{max}^{r}=20~\mathrm {w}\) is greater than that of scenario \(p_{max}^{s} = p_{max}^{r}=10~\mathrm {w}\) though their total power consumption is the same. As the constrains \( p_{s}\leq {p_{max}^{s}}\) and \( p_{r}\leq {p_{max}^{r}}\) are considered during power allocation, the allocated power allocated with larger maximal transmit power threshold (\(p_{max}^{s}\) or \(p_{max}^{r}\)) is more optimal than smaller one in maximizing system capacity.

Fig. 2
figure 2

The system achievable rate with the increasing total transmit power threshold

The corresponding power consumption in Fig. 2 is also shown in Fig. 3 for the \(p_{max}^{r} = p_{max}^{s} = 10~\mathrm {w}\) scenario. Combined with Fig. 2, it can be concluded that the proposed scheme can obtain greater achievable rate while consuming the same level of power. It can also be seen that, at the pmax=20 w point, resultant both the relay transmit power and the source transmit power are the same no matter these are obtained by our scheme or the benchmark scheme. We know that under the pmax=20 w and \(p_{max}^{r} = p_{max}^{s} =10~\mathrm {w}\) conditions, \(p_{max} \geq p_{max}^{s} + p_{max}^{r}\) holds. As a result, according to the proposed scheme, \(p^{r}= p_{max}^{r}\) and \(p^{s} = p_{max}^{s}\) should be set. It also can be seen that when pmax<20 w, it holds that ps>pr. It is easy to know that the received signal at a relay is the combination of two transmitted signals that are from source node and the other relay simultaneously. The transmitted signals from relay is the intended signal for destination, but it is interference for the other relay. Hence, in order to obtain maximal system capacity, the transmit power of source node should always be greater than or equal to the allocated power of relay. Therefore, we can say that the simulation results are consistent with the theoretical analysis discussed above.

Fig. 3
figure 3

Power consumption with the increasing total transmit power threshold

Under the same settings, once the allocated power to maximize the system achievable rate is obtained, the power consumption of superposed symbols s1 and s2 are also simulated, which are shown in Fig. 4. From the results, it can be seen that the transmit power of s1 and s2 always increase with the total transmit power threshold pmax, and the allocated power on s1 is always greater than that on s2. All these results are consistent with our aforementioned theoretical analysis.

Fig. 4
figure 4

Transmit power of the separated superposed signal with the increasing total transmit power threshold

To show the performance of our proposed scheme under different cases, all possible scenarios are simulated separately. The system achievable rate and the corresponding power consumption are shown in Figs. 5 and 6, respectively. In order to be closer to the practical scenarios, the maximal transmit power at the relay node and the source node are generated randomly in the (0,10) range, and the maximal total transmit power is set to pmax=10 w.

Fig. 5
figure 5

Achievable rate under four different cases: case 1, case 2, case 3, and case 4\(\left (\mathrm {i.e.}, p_{max}\geq p_{max}^{s} + p_{max}^{r}\right)\)

Fig. 6
figure 6

Power consumption under four different cases: case 1, case 2, case 3, case 4 \(\left (\mathrm {i.e.}, p_{max}\geq p_{max}^{s} + p_{max}^{r}\right)\)

From Figs. 5 and 6, it can be seen that the achievable rate in case 3 is the largest. Moreover, the power consumption of case 2, case 1, and case 3 are equal. It implies that when \(p_{max}^{r} \geq (b - \sqrt {b(b-ap_{max})})/{a}\) and \(p_{max}^{s} \geq (-b + ap_{max} + \sqrt {b(b-ap_{max})})/a\) hold, the largest achievable rate can be obtained under the same power consumption. The \(p_{max}^{r} \geq (b - \sqrt {b(b-ap_{max})})/{a}\) and \(p_{max}^{s} \geq (-b + ap_{max} + \sqrt {b(b-ap_{max})})/a\) conditions imply that the optimally allocated power between the relay node and the source node always satisfy their individual constrains. Hence, the resultant power allocation is globally optimal in which the achievable rate is the largest.

5 Conclusion

For a NOMA-equipped cooperative relay system with the SIC mode, where the cognitive relays operate in the AF mode, a power allocation scheme was proposed to maximize the achievable rate under some pre-determined transmit power constraints. Based on the KKT conditions, network scenarios were divided into two different types. The power allocation to maximize the achievable rate of each type was presented in a closed-form expression. Then, based on the optimally allocated source power, a power allocation scheme was proposed for the transmit data streams while targeting on the minimization of the required frequency band. Extensive simulations were conducted to evaluate the performance of the proposed power allocation schemes. As of future research, we would like to extend the single-user scenario to the multi-user one while keeping the same NOMA-based cooperative relay idea. Incorporating massive millimeter multiple-input multiple-output (MIMO) technology in such a system could be another possible research direction.