Mechanism Design for Exchanging Resources in Federated Networks

Abstract

This paper introduces a mechanism for pricing and exchanging resources in federated networks of task-processing elements. An operational model is developed to allocate processing, storage and communication resources to computational demands. This model finds an efficient and stable solution to combinatorial routing and allocating resources among networked elements with technical constraints. Using mixed-integer linear programming (MILP) formulation, we find optimal solution to processing tasks, allocating links, storing and delivering data to destination. A trusted auctioneer uses a mechanism to allocate resources to computational tasks and suggests prices for exchanging resources across a federation using minimum number of MILP solutions to a network topology. The proposed mechanism maximizes the collective value for a federation and ensures an expected value for each federate and minimizes the computational cost associated with the operational runs. The auctioneer doesn’t have access to utility functions and private information on resources a priori while assumes a federation with self-centric and rational participants. An application of federated satellite systems is developed with endogenous components such as adaptive bidding and opportunity cost of using resources. Numerical results show that the proposed mechanism improves the collective and expected values in a federation with strategic federates.

Appendices

Appendix A: Objective Functions

The operational model uses a temporal network of elements in consecutive time steps to process tasks and schedule delivery. This appendix introduces notations and MILP formulation for the operational models and objective functions.

The set ${\mathbf {ts}} = \{t_0, t_1, \ldots , t_{n-1}\}$ lists n time steps for a model state at time $t_0$. A routing solution with integer variables to model data transmissions on links requires a series of sub-steps to model individual actions. The number of required sub-steps during each time step is equal to maximum path length. With a maximum path length m in a network and n time steps, define sub-steps as:

$$\begin{aligned} {\mathbf {s}} = \{s_{(0)}, s_{(1)}, s_{(2)}, \ldots s_{(n*m-1)}\} \text { s.t. } s_{(k*m)} \widehat{=}t_{k}: k \in \{0, 1, \ldots , n-1\} \end{aligned}$$

(12)

where values of ${\mathbf {s}}_v$ with $(k-1)*m<v < k*m$ are distributed between the pair of consecutive time steps: $t_{k-1}$ and $t_k$, in ascending order. Figure 11 illustrates sub-steps in a sample multi-task routing case. In this figure, $T_i$ shows computational tasks, $D_j$ shows destination elements, and $P_i$ shows a delivery path for task $T_i$. Sub-steps $s_{km+b}$ for time step $t_k$ model data link transmission. Two tasks are delivered in time step $t_k$ while MILP models $P_1$ with two $P_2$ with three sub-steps.

Fig. 11

Substeps for defining delivery paths when two tasks are delivered simultaneously

Each time step of the MILP defines target variables as:

$$\begin{aligned} x_{process}&: (T \in {\mathbf {T}}, e \in {\mathbf {E}}) \rightarrow \{0,1\} \\ x_{transmit}&: (T \in {\mathbf {T}}, l \in {\mathbf {L}}, s \in {\mathbf {s}}) \rightarrow {\mathbb {Z}}^+ \\ x_{resolve}&: (T \in {\mathbf {T}}, e \in {\mathbf {E}}, s \in {\mathbf {s}}) \rightarrow \{0,1\} \\ x_{read}&: (T \in {\mathbf {T}}, e \in {\mathbf {E}}, t \in {\mathbf {ts}}) \rightarrow \{0,1\} \\ x_{store}&: (T \in {\mathbf {T}}, e \in {\mathbf {E}}, t \in {\mathbf {ts}}) \rightarrow \{0,1\} \end{aligned}$$

where s identifies the sub-steps in Eq. 12, excluding time t for notational simplicity.

1.1 A.1 Maximize Value

The value-maximizing objective function for a federate is defined as:

$$\begin{aligned} J_{value}(t)= & {} \sum _{T\in {\mathbf {T}}_a}\sum _{s \in {\mathbf {s}}_t} \begin{pmatrix} {\mathbb {1}}(e \in E_d) \times {\mathcal {V}}(T, t) \\ + {\mathbb {1}}(e \notin E_d)\times T.penalty\end{pmatrix} x_{resolve}(T, e, s) \nonumber \\&\quad - \sum _{l_{ikt} \in L}\sum _{T\in {\mathbf {T}}_a}\sum _{s \in {\mathbf {s}}_t} T.size \times \epsilon \times x_{transmit}(T, l_{ikt},s) \nonumber \\&\quad - \sum _{T \in {\mathbf {T}}_a} T.size \times SP_{e_T}(t)\times x_{store}(T, e_T, t) \end{aligned}$$

(13)

where $e_T = T.element$ and $SP_{T.element}(t)$ is the storage penalty for the owner of task T at time step t defined by Eq. 1. The intuition is that resolving a task affects federation value through value function of delivering it, or the penalty function of failure to deliver the task. In addition, data transmission through a link affects the federation value with the network communication cost or the opportunity cost of storage penalty ($i = k$) defined by cost function$\zeta$ in Eq. 1.

The MILP model of an operational run at time t subject to capacity and financial constraints is defined as:

$$\begin{aligned}&\sum _{T \in {\mathbf {T}}}\sum _{s \in {\mathbf {s}}_t}{T.size \times x_{transmit}(T, l, s)} \le l.capacity, \forall \; l\in L \end{aligned}$$

(14)

$$\begin{aligned}&\sum _{T \in {\mathbf {T}}}{T.size \times x_{store}(T, e, t)}\le e.capacity - {\mathcal {D}}(e),\forall \; e \in E \end{aligned}$$

(15)

$$\begin{aligned}&T.size \left( x_{process}(T, e_T) - \sum _{s_t}{x_{resolve}(T, e, s)}\right) \end{aligned}$$

(16)

$$\begin{aligned}&\quad +\,T.size \left( x_{store}(T, e, t) - x_{read}(T, e, t)\right) \nonumber \\&\quad +\,T.size \left( \sum _{l \in inlink(e, t)}{\sum _{s_t}{x_{transmit}(T, l, s)}}\right) \nonumber \\&\quad -\,T.size\left( \sum _{l \in outlink(e, t)}{\sum _{s_t}{x_{transmit}(T, l, s)}}\right) = 0, \qquad \forall \; e \in E, \forall \; T \in {\mathbf {T}}.\nonumber \\&\sum _{t'\le t}{\sum _{s\in {\mathbf {s}}_t}{x_{resolve}(T, e, s)}}=1,\forall \; T \in {\mathbf {T}}: T.exp\le t. \end{aligned}$$

(17)

where the inlink and outlink are the set of links into and out of an element:

$$\begin{aligned} inlink(e_k, t)&= \{l_{jkt}\in L: e_j \in E\}\\ outlink(e_i, t)&= \{l_{idt}\in L: e_d \in E\} \end{aligned}$$

Constraint 14 defines the limits on link capacity, Constraint 15 defines the storage capacity of an element, Constraint 16 balances the inflow and outflow of data into and out of an element, lastly, Constraint 17 resolves expired task to free up memory of expired data.

1.2 A.2 Minimize Cost

$$\begin{aligned} \text {find}&: x_{transmit}(T,l_{ikt},s), x_{resolve}(T, e, s) \end{aligned}$$

(18)

$$\begin{aligned} \text {given}&: x_{read}^@(T, e, t), x_{store}^@(T, e, t), x_{process}^@(T, e, s) \nonumber \\&T \in {\mathbf {T}}, l_{ikt} \in L, s \in {\mathbf {s}}_t, t \in {\mathbf {t}}, e \in E \end{aligned}$$

(19)

$$\begin{aligned} \text {minimize}&: \sum _{l_{ikt} \in L} \sum _{s \in {\mathbf {s}}_t} \zeta ({\mathcal {F}}_t(T), l_{ikt}) x_{transmit}(T, l_{ikt},s) \nonumber \\&+ \sum _{T\in {\mathbf {T}}_a} \sum _{s \in {\mathbf {s}}_t} \left( \begin{aligned}&{\mathbb {1}}(e \in E_d) {\mathcal {V}}(T, t) \\&+ {\mathbb {1}}(e \notin E_d) T.penalty \end{aligned} \right) x_{resolve}(T, e, s) \end{aligned}$$

(20)

$$\begin{aligned} \text {subject to}&: \textit{Constraints in Eqs.}~{14},~{15},~{16},~{17}\,\, and: \nonumber \\&\sum _{s \in {\mathbf {s}}_t}\sum _{e \in E} x_{resolve}(T, e, s) = \sum _{s \in {\mathbf {s}}_t}\sum _{e \in E} x_{resolve}^@(T, e, s) \nonumber \\&\qquad \forall t \in {\mathbf {ts}} \end{aligned}$$

(21)

where $x_{store}^@(T, e, t)$, $x_{read}^@(T, e, t)$ and $x_{process}^@(T, e)$ are the calculated decisions from Eq. 6. Constraint 21 ensures tasks resolved at time step t using Eq. 6 would also be resolved in above solution, although, tasks may be delivered to different elements.

Appendix B: Q-Learning

A generic open-source Q-learning module models adaptive bidding.² Nonetheless, three compatibility issues between bidding behavior and the basic Q-learning must be resolved: (1) temporal distance between actions and reward (task pick up and tasks delivery) (2) interdependency between actions and rewards in consecutive times steps, and (3) continuous action space in bidding ($c_f \in {\mathbb {R}}$). The first and second concerns are addressed in updating multiple Q-values given a reward value. Regarding the state-action dimensionality, as the action space increases in size, the probability of visiting the same state again decreases [42]. Then, we define Gaussian distance between states to update the Q-values. Assuming a state action pair as $x_i = (s_i, a_i)$, the learning parameter is:

$$\begin{aligned} \alpha _{ij} = \frac{\alpha }{K_i}\exp \left( \frac{\varDelta s_{ij}^2}{2\sigma _s^2}+\frac{\varDelta a_{ij}^2}{2\sigma _a^2}\right) \end{aligned}$$

where $\alpha$ is the learning factor from Q-learning (Eq. 11), $\varDelta s_{ij} = |s_j-s_i|$, $\varDelta a_{ij} = |a_j-a_i|$, and $K_i$ is the normalizing factor that ensures the sum of all Q values are updated with $\alpha$.

After receiving each reward $R^{t'}$, Q-values are updated according to their $\alpha _{ij}$:

$$\begin{aligned} Q(x^t_{ij}) \leftarrow Q(x^t_{ij}) +\alpha _{ij} \left[ \frac{R_{x^t}}{N} + \gamma Q(x^{t+1}_{ij})-Q(x^t_{ij})\right] \; \forall \; Q(x^t_{ij}): t> t' - \varDelta t \; \forall \; j\in S \end{aligned}$$

where $\varDelta t$ represents the number of time steps after which the actions are uncoupled from rewards (i.e. actions cannot affect further rewards), and N is the total number of actions during $\varDelta t$.