A serious gaming approach for optimization of energy allocation in CubeSats

Energy consumption remains an open challenge in aerial systems such as CubeSats and therefore optimization of its allocation is a top priority for maximizing operational capacity. Our research review reveals a plethora of approaches for optimization of energy allocation and all achieving varying degrees of success and not without any compromises. In this paper, we exploit the use of serious gaming in a novel energy allocation algorithm that aims at minimizing energy consumption to maximize the utilities of both CubeSats and terrestrial sensors. To demonstrate this, we use Stackelberg for serious gaming and standalone topology for CubeSat configuration. The experimental results show that the use of a Stack-elberg game approach for optimization has led to reduction in the required transmission energy in sensors, an improved link performance between the CubeSat and ground sensors, and an increase in network lifetime and performance without resorting into sensor power enhancements or other external power sources. The overall average operational capacity improvement predictions range between 22 to 27% across all performance indicators of energy efficiency across RF chains of link budgets.


Introduction
Serious gaming is an alternative testbed for testing complex decision-making in numerous subject areas through simulation and analysis of decision-making processes.The use of rewards, which are incentives based on performance, and the user interactions that follow inform on the choice of actions.Serious gaming techniques are increasingly applied widely for data analytics by retrieving valuable information from complex large data sets that support generating smart decision.The learning and training concepts rely on discovering data patterns and those data elements that when adapted may change the data patterns.This is driven by the competition as it unfolds during interaction and helps support decision making and performance optimization, despite the continuous challenge of measuring the learning impact of competition [8,9,28].
Serious gaming is regularly used to examine the behavior of rational and strategic decisionmakers by considering possible equilibriums in conflict-of-interest situations and then providing appropriate mechanisms for enabling cooperation between two or more players.Stackelberg, and Nash Equilibrium are prime examples of the use of hierarchical serious gaming in observing the sequential interactions among players foe the purpose of making strategic decisions across many applications.Observing different attributes and behaviors among players during interactions enables these serious gaming methods to seek optimum decision-making solution(s) [7,12,17].Serious gaming has been applied successfully for smart decision making to enhance performance across a wide range of applications ranging from education to healthcare, to manufacturing and energy allocation is among these.Optimization of energy allocation will prolong network terminal lifetime and which in turn will help with sustaining the system performance, especially of wireless systems.This requires optimization of channel modeling and use of the limited transmission energy resources [6,32].
Satellite systems including Cubesat are crucial in global wireless communication systems, due to their soft deployment, high frequency bands, wide coverage footprint, and line-of-sight connectivity.Yet, despite the advancement of Cubesat technology, energy allocation and consumption remain an open challenge that reportedly affect performance and/or operational lifetime.CubeSats use solar cells to convert solar light to electricity that is stored in rechargeable lithium batteries that are an energy source during an eclipse as well as during peak load times.Moreover, the energy subsystem in Cubesat is responsible for storing, distributing, and controlling the spacecraft's electrical energy.The efficiency of the Cubesat solar cells may be affected, for instance, by the radiation intensity of the sun, and the incident angle at which the sunlight strikes the cells [5,23,26].
Motivated by the open challenge of energy consumption, the overall aim of this paper is to apply serious gaming in optimizing energy allocation in a Cubesat.Section 2 reviews related work in support of the design and concludes by highlighting the specific research gaps that motivate our proposed contribution.Section 3 presents the design of our proposed model that deploys serious gaming into the underlying communication infrastructure.Section 4 evaluates our proposed model and presents a first proof-of-concept.Section 5 concludes.

Related research review
This section reviews related research works whose optimization approach includes the use of serious gaming and their target is energy allocation in aerial systems.For ensuring consistency and keep within the scope, a primary and a secondary set of criteria have been used to source and review the related works; the primary criteria include the use of optimization, the network configuration, and the vehicle type whereas the secondary criteria include propagation path loss, altitude, and coverage range.The section concludes with a summary of our findings and whose purpose is to bring to the fore research gaps which serve to motivate our own work.
[16] introduces a study on green UAV communications for 6G that also includes a review of various optimization techniques for power consumption includes serious gaming.The authors suggest a set of optimization parameters, i.e., UAV trajectory, configuration, and link budget, for efficient energy allocation.[22] proposes a resource allocation scheme for satellite networks based on the bargaining model of a two-player resource allocation game theory approach.Their simulation shows reasonable results in relation to time and throughput for their application area of emergency service provision.The proposed solution requires bit energy over power density.[11] utilises Nash Equilibrium in a power allocation strategy for cognitive satellite networks.The results in relation to bit error rate (BER), and signal-to-noise ratio (SNR) suggest an acceptable level of performance.However, as the proposed technique has been deployed in only a specific scenario, scalability is an issue to be resolved.[31] introduces power allocation for inter-satellite links for which a utility function is designed and optimized with Nash Equilibrium using signal-to-interference and noise ratio (SINR), utility values, and transmission power as the performance indicators.The results suggest improvements in power resource reservation.
[18] considers a Stackelberg model for managing interference in a multibeam satellite system that helps with power allocation and consumption.The results prove that the proposed game model can adjust the price of interference to achieve a trade-off between intercell interference and operating profit.[34] presents a two-layer Stackelberg equilibrium to allocate power and computation resources among users while satisfying the quality of service (QoS) constraints and utility price.In response to the resource allocation problem the authors aim at maximizing the utility between operator and users, which leads to energy minimization.The Stackelberg equilibrium achieves optimal pricing.
[33] optimizes satellite transmission energy allocation at a 1000 km altitude applying empirically different weight factors.The results show improvement in energy allocation.However, satellite energy use, transponder state and service requirements need further consideration.[30] uses a Lyapunov optimization approach for energy efficiency over multibeam satellite downlinks.This helps with optimal control of a dynamical system for queueing networks.The results on energy efficiency show noticeable improvements, but with some delay trade-off.
[20] considers Artificial Intelligence (AI) approaches for satellite power allocation.The approaches include genetic algorithms, simulated annealing, particle swarm, and deep reinforcement learning for multibeam management to maintain robustness and fair power consumption.The results vary across different approaches with noticeable trad-offs.The approaches do not find a global optimum, only local optima.Therefore, they offer no guarantee of good performance when user behaviour changes.[10] uses deep reinforcement learning to optimize power allocation for high throughput satellites.The results focus on time and throughput management for maintaining power not for dynamic power allocation.
[24] presents a mathematical integer programming (IP) formulation for task scheduling and energy management of CubeSats.The proposed model mainly focuses on maximizing the number of tasks of a CubeSat while maintaining low energy consumption.The results of the proposed model suggest an optimal energy effective scheduling plan, allowing the best possible use of available energy resources while ensuring quality of service (QoS).[25] proposes a model that focuses on energy efficiency for Unmanned Aerial Vehicle (UAV) using Exact Potential Games (EPG) and Nash Equilibrium.The model seeks a compromise between coverage deployment and energy consumption.The results confirm the success of the proposed approach and confirm the reliability of the proposed model.However, modifying the frequency band results in interference.[29] proposes a cooperative joint cooperative beam association and power allocation scheme in LEO Satellites using many-to-many a match game-based beam association (MGBA) algorithm and a successive convex approximation (SCA) power allocation (SPA) algorithm.The results show that the proposed MGBA-SPA scheme outperforms other contrast schemes.

Research motivation and proposed contribution
Table 1 summarises the findings from our review of the literature separating between issues that are being addressed and those that are not, hence, creating gaps that motivate further research work, including ours.Our proposed contribution not only aims to address on-going challenges, but it also offers added value.Optimizing energy allocation in Cube-Sats is scarcely reported in the literature and currently lacking consideration of a serious gaming approach.Therefore, a key difference between the proposed contribution and what is reported in the literature is, primarily, the novel energy allocation approach that aims at joint minimization energy consumption to maximize utilities for both Cubesat and terrestrial sensors using a Stackelberg game approach.This noticeable shift from mainstream approaches promises smart decisions and achieving the objective of efficient energy allocation.This requires making multiple decisions to minimize energy consumption, i.e., from fair transmission to maximization of feasible sleep time, to optimization of convergence range, to adjustment of cluster size and to node selection.

Proposed model
Figure 1 shows a bird's-eye-view of the proposed model that uses Stackelberg serious gaming for optimizing energy allocation on a Cubesat.A proof-of-concept implementation verifies.
Wireless communications between a Cubesat and all its terrestrial nodes, whether mobile or stationary, is challenging as this involves managing a potentially large number of terrestrial nodes connected to the Cubesat whilst in orbit.Introducing node clustering and a head node would help with managing the number of nodes that are directly communicating at any given time with the Cubesat.A cluster head will have the responsibility of data transmission directly to the Cubesat, which reduces the probability of data collision between nodes and in turn would lead to better energy consumption.The cluster size and the number of nodes in each cluster can be pre-set to ensure energy allocation fairness.Cluster allocation considers criteria, such as energy level consumption, number of neighbor nodes, and harvested energy levels.This approach can help with maximizing network lifetime and achieving energy dissipation load balancing utilization in respective clusters.
Figure 1 depicts two segments [2,13], the space segment with its Cubesat and payloads, and the ground segment with its ground control station that is linked to the Cubesat with a RF control link that supports all Cubesat operational functions and aerial navigation.The Cubesat serves K randomly distributed sensors on the ground via the RF links, and this can be optimized using Stackelberg serious gaming.Sensing the link margin data depends on a set of parameters including path loss, transmitter power, Multiple Input Multiple Output (MIMO) antenna gain, frequency band, and bandwidth.Data links are established between cluster heads and ground sensors.Stackelberg comes into play to prioritise access and optimize the overall network utility and energy allocation for the Cubesat.Guaranteeing a fair energy allocation requires multi-optimization decisions on fair transmission, feasible sleep time maximization, convergence range optimization, cluster size adjustment and node selection.
Estimating a Cubesat link margin is heavily dependent on various parameters that effect the free space link from the Cubesat to the sensors on the ground.Thus, we have considered the Carrier to Noise ratio (C/N) in Eqs. 1 and 2, and the bit energy per noise density ratio (E b /N 0 ) in Eq. 3.These parameters are widely considered as quality indicators of a satellite communication channel, from which a fuller range of link budget parameters can be drawn, e.g., throughput, Signal to Interference Noise Ratio (SINR), Received Signal Strength (RSS), coverage range, [1,4,5].
(1)  Efficient and fair energy allocation is vital in maximizing the usage of shared resources like bandwidth, which in turn can benefit both transmitter and receivers.An optimization model with serious gaming may optimize priority-based access and overall network utility.The aim of this paper is to use Stackelberg as the serious gaming approach in achieving that.With serious gaming, the learning concept and strategies can be developed based on real conditions or data from predicted results.With the proposed model the Cubesat, acting as the leader, sets the interference prices to the terrestrial sensors, acting as the supporters, including both the cluster heads and cluster nodes.The Cubesat, although affected by the cost of interference, focuses on maximizing its utility from interference returns.The terrestrial sensors focus on maximizing their utilities per unit of power consumption.Hence, Stackelberg aims at obtaining the optimal prices for Cubesat and optimal transmission Fig. 1 A conceptual framework of the proposed model powers for terrestrial sensors.The formulation of the serious Stackelberg game approach is mathematically expressed in Eqs. ( 6) to (17) [15,32].
The Cubesat can transmit data at a particular level of energy P P , while terrestrial sen- sors transmit at a variable energy P i , thus the SINR at receivers is as in Eq. ( 6).The energy allocation price problem is formulated as a two-stage Stackelberg serious game with the Cubesat being the leader and the N terrestrial sensors being the followers.The sensors chose their own transmission energy according to the interference energy price p i set by the Cubesat.Hence, the Cubesat aims to maximize its utility by forcing a charge for interference I ik produced by the terrestrial sensors by consideration of the interference effects on the overall utility as in Eq. (7).
where g i refers to channel energy gain between a cluster head and one of its nodes, h i refers to channel energy gain between a Cubesat and a cluster head, K refers to the number of ter- restrial sensors.Channel energy gains are distributed using additive white Gaussian noise and variance σ 2 , where α refers to the conversion factor of a Cubesat's economic loss per unit of interference produced by sensors, I i P i refers to the interference between a cluster head and its nodes, P denotes the vector of energy allocation for all sensors.
In the serious game, the Cubesat seeks to achieve optimal utility through its interference price (objective O1) , this is expressed as in Eq. ( 8), whereas terrestrial sensors aim at maximizing their utility through receiving optimal allocated energy (objective O2) this is expressed as in Eq. ( 9).Both objectives are pursued without any degradation of the quality of service for transmission of large data packets with the lowest transmission delay possible.
where W i refers to time where a sensor is waiting in a queue for packets to be transmitted, E W i refers to service time, T i refers to delay constraint which is converted into a trans- mission rate constraint as in Eqs.(10) and (11).O2 is solved in a non-cooperative game G to maximize utilities as in Eq. ( 12). ( 6) where R i refers to transmission rate,υ i refers to packet arrival rate of terrestrial sensors, Z refers to packet size, and Ω to a set of terrestrial sensors {1, 2, ……k}.
Equations ( 13) to (17) present a Stackelberg equilibrium which aims at finding the fractional structure of the terrestrial sensors' objective function and convert it into an equivalent parametric programming problem.
where p * and P * refer to optimal solution of objectives O1 and O2 respectively, thus p * and P * refer to the point of the Stackelberg equilibrium.p * represents the optimal strat- egy of the Cubesat, whereas P * refers to the optimal power of the terrestrial sensors.ξ refers to a conversion factor to represent the economic gain of terrestrial sensors, I i (P i ) refers to the interference that terrestrial sensors will experience from the Cubesat under the interference price p i , P c refers to the additional circuit power consumption of sensors through transmission, and q i refers to the searching function for the optimum solution.
A feasible sleep strategy is introduced to minimize energy consumption and in turn extend the lifetime of wireless sensors.To enable that, sensor nodes switch between two modes: active and sleep.Sensor nodes observing the feasible sleep strategy help with balancing energy consumption by switching between active and sleep modes so that not all sensor nodes are active at any given moment.This is formulated as in Eqs.(18) to (20).
where u i denote the utility function of the i th player, N and K denote respectively the player and their strategic space in the wireless sensors network, s i denotes the mode of the i th sensor node, s −i denotes the mode of all other nodes, U i s i , s −i and C i s i , s −i denote respectively the revenue and cost functions of the i th sensor node, R and P denote respec- tively the reward and probability of a sensor node sending successfully a data packet to the next sensor node.The probability of s i being either in active or sleeping mode is 1 and 0 respectively.
O3 ∶ max Optimization of transmission energy P * i using Stackelberg serious gaming is formulated in Eqs. ( 21) to (27) [19,21], with the Cubesat interference price set in consideration of utility maxU MBS and bandwidth.
where Z(p) refers to the interference price competition constraints, p(t + 1) refers to the updated process price.
Achieving optimal or fair transmission energy allocation helps the Cubesat to make relevant decisions such as adjusting the cluster sizes, optimizing the convergence ranges, node selection, and maximizing feasible sleep time, which in turn helps guaranteeing a fair energy allocation.Figure 2 shows the interactions among players and with the utility, Cubesat, during game play where each player represents a cluster head, P N is the energy demand of players and determining K and M will maximize energy reduction.Algorithm 1 is our energy allocation approach using Stackelberg serious gaming.

Simulation results and discussion
We have used MATLAB's CubeSat blockset simulation library, as shown on Fig. 3, to simulate, visualize and analyse the Cubesat link margin.The data is used as input to our model.Figure 4 illustrates early predictions of the Cubesat's link margin using GUI plots.These plots present a general overview of the Cubesat communication performance based on indicators such as PL and power consumption, which in turn help with observing the likely effect of likely improvement before deploying any serious gaming.
Figure 5 shows the Cubesat and cluster head utilities after Stackelberg deployment.Maximizing utilities per unit power of consumption helps in sustaining a reliable connection with Cubesat, thus enabling Internet of Everything (IoE) communications with ground sensors.Sharing the transmission spectrum will undoubtably cause interference.
The Cubesat objective is to maximize use of its utility by imposing a monetary charge for the interference caused by cluster heads whilst considering the effect of interference on the utility.Figure 5 reveals that as the weight assigned to interference rises so does the utility benefit.In contrast, increases in throughput affect utility use which in turn causes higher power consumption.
Figure 6 presents the evaluation of the performance of the proposed power allocation scheme, both Stackelberg (Cubesat) utility and price, against the Quality of Service (QoS) threshold.This is a vital performance indictor as it reveals the optimal interference prices and power consumption.The Fig. 6 reveals a clear pattern of pricing and utilities Fig. 2 Interactions among players and with the utility, Cubesat, during game play Algorithm 1 Energy allocation using Stackelberg serious gaming equilibrium with increasing QoS threshold which in turn suggests that the proposed power allocation scheme presents a noticeable improvement in power-saving.
Guaranteeing fair energy allocation and thus fair transmission, an RF chain of MIMO antennas is introduced, and its number optimized, which not only enhances energy efficiency, but also yields the best data-rate performance, as Fig. 7 demonstrates.The RF  Figures 8 and 9 show the predicted results of the RSS and Th link budget parameters between the Cubesat and ground sensors at a distance range between 400 and 800 km; firstly, non-optimized, i.e., prior to applying serious gaming and, secondly, optimized, after applying the proposed approach.Both RSS and Th show an increase with distance and shadowing effects, with their values floating within an acceptable average.After applying the proposed serious gaming approach, the predicted results of both RSS and Th shows an increase of 27% on average.This suggests that optimization of energy consumption is possible without resorting to sensor power enhancements or external power sources.
Figure 10 shows the Eb/No performance before and after applying the proposed serious gaming approach at the lowest BER achieved of 1x10 −6 using the "Semilogy" function in MATLAB.The best link performance is the one that allows for the lowest possible BER with the lowest possible Eb/No, i.e., after applying the proposed serious gaming approach.This prescribes a robust channel, where a low error rate is achieved without requiring a high transmission power.The overall predictions of these two QoS indicators reveal reasonable improvements.This may lead to a reduction in the required transmission energy from sensors and an improved link performance between Cubesat and ground sensors, thus, increasing network lifetime and performance.The predicted Eb/No performance, after applying the proposed serious gaming approach, shows an increase of 21% on average in relation to non-optimized predictions.Once again this supports the conclusion that energy consumption optimization is possible without having to resort to sensor power enhancements or external power sources.This may lead to a reduction in the required transmission energy from sensors and an improved link performance between Cubesat and ground sensors, thus, increasing network lifetime and performance.Most importantly, this improvement yields to optimization of energy consumption without using sensor power enhancements or external power sources.After applying the Stackelberg gaming approach, the Eb/No performance has improved by an average of 21% in comparison to non-optimized predictions.This is a noticeable improvement that results in fair transmission between nodes and clusters.Figure 11 Fig. 8 RSS predictions before and after applying the proposed serious gaming approach Fig. 9 Th predictions before and after applying the proposed serious gaming approach shows the residual energy of wireless sensors at their lowest levels, with a feasible sleep strategy.

First proof-of-concept
Figure 12 shows a bird's-eye view of a first proof-of-concept IoT space-to-ground system for smart farming primarily aimed at crop irrigation and water management.The space part comprises of the CubeSat with its payloads and a transceiver module, which is responsible for collecting data from a wireless server gateway and then transmitting it to the cloud for storage and predictive analysis.The ground part comprises of wireless sensors collecting data on soil moisture, air humidity, air temperature, and water levels, and a cluster head Figure 13 shows a Blynk IoT platform dashboard depicting real-time data values.Figure 14 displays the python code used for extracting the sensed data for evaluation and validation of the link quality between the CubeSat and the ground sensors before and after Stackelberg.Figure 15 visualizes the performance of a 5G Wi-Fi Module using a set of key performance indicators that includes coverage, capacity, access point, security, signal level, interference, noise.The focus is on RSSI and signal coverage levels, as they help measure the performance of the communication link between the CubeSat and ground sensors.The RSS shows reasonable average floats between -81 and -85dBm, and maximum data rates between 130 and 135Mbps.Figure 16 visualizes the performance of the 5G Wi-Fi Module after Stackelberg and this shows improvements between 10 and 15% for RSSI, which is an acceptable level for power consumption.
Fig. 12 The proof-of-concept IoT space-to-ground system where EIRP is Effective Isotropic Radiated Power, C/N is carrier power, BW is bandwidth, R b is data rate, P t is transmission power , G t and G r is transmission and receiver antenna gains respectively, L is connector and cable loss, A R is rain and atmospheric gas attenua- tions, K is Boltzmann's constant, G/T is the ratio of the receiver antenna gain to the system noise temperature in dB, T is affective temperature, and ERFC of ( 28) is a complementary error function that defines the cumulative probability curve.Figure 17 compares predicted BER and Eb/No values for power consumption before and after Stackelberg.Pre-Stackelberg BER achieves 1 × 10 −5 whereas post-Stackelberg achieves improved performance of an average of 5 dB which suggests a channel with low error rates that utilizes minimum transmission power.Furthermore, post-Stackelberg, maximum capacity is achieved without increasing transmission power between the Cube-Sat and ground gateway, and average power consumption records an improvement of 15%.

Concluding discussion and future work
This work presents the use of Stackelberg serious gaming for energy allocation between a Cubesat and ground sensors, which on the one hand minimizes energy consumption and, on the other hand, maximizes the transfer of utilities from Cubesat to terrestrial sensors.
where P t indicates transmission power, G C indicates CubeSat antenna gain, G S indicates ground station antenna gain, d indicates distance from the Cubesat to earth station and/or ground sensors, λ indicates wavelength, a fraction of speed of light to frequency.d is cal- culated with consideration of the elevation angle θ , which is vital in satellite wireless com- munication systems for achieving better LoS connectivity, L refers to losses, E r indicates the earth radius (6,371 km), h t indicates the Cubesat altitude, F indicates the receiver noise figure, K indicates the Boltzmann's constant (1.38.10-23Ws/K), T 0 indicates the absolute temperature (290 K), B indicates the receiver noise bandwidth, N indicates the Noise fig- ure, I indicates interference, Bw indicates the required bandwidth (MHz), R b indicates data bit rate, and Th refers to predicted throughput in (Mb/s).

Fig. 5 Fig. 6
Fig. 5 Cubesat and cluster head utilities after serious gaming deployment

Fig. 7
Fig. 7 Energy efficiency of the proposed approach in relation to RF chains

Fig. 10 Fig. 11
Fig. 10 Eb/No performance at the lowest BER predictions

Fig. 13
Fig.13 Blynk IoT platform dashboard that shows edge-processed sensor data

Fig. 14
Fig. 14 Python code for extracting sensed data

Fig. 15
Fig. 15 RSSI and signal coverage levels

Fig. 16 Fig. 17
Fig. 16 RSSI and signal coverage levels after Stackelberg Fig. 17BER and Eb/No for power consumption before and after Stackelberg

Table 1
Related research against the proposed contribution