Auction design for the allocation of carbon emission allowances to supply chains via multi-agent-based model and Q-learning

Abstract

To increase competition, control price, and decrease inefficiency in the carbon allowance auction market, limitations on bidding price and volume can be set. With limitations, participants have the same cap bidding price and volume. While without the limitations, participants have different values per unit of carbon allowance; therefore, some participants may be strong and the other week. Due to the impact of these limitations on the auction, this paper tries to compare the uniform and discriminatory pricing in a carbon allowance auction with and without the limitations utilizing a multi-agent-based model consisting of the government and supply chains. The government determines the supply chains' initial allowances. The supply chains compete in the carbon auction market and determine their bidding strategies based on the Q-learning algorithm. Then they optimize their tactical and operational decisions. They can also trade their carbon allowances in a carbon trading market in which price is free determined according to carbon supply and demand. Results show that without the limitations, the carbon price in the uniform pricing is less than or equal to the discriminatory pricing method. At the same time, there are no differences between them in the case with limitations. Overall, the auction reduces the profit of the supply chains. This negative effect is less in uniform than discriminatory pricing in the case without the limitations. Nevertheless, the strong supply chains make huge profits from the auction when mitigation rate is high.

Appendices

Appendices

1.1 Appendix A

The following indices are applied for each supply chain’s mathematical modeling:

Supply chain index	$s=\{\mathrm{1,2},\dots ,S\}$
Product index	$i=\{\mathrm{1,2},\dots ,I\}$
Machine index	$j=\{\mathrm{1,2},\dots ,J\}$
Manufacturing center index	$m=\{\mathrm{1,2},\dots ,M\}$
Warehouse index	$w=\{\mathrm{1,2},\dots ,W\}$
Customer zone index	$c=\{\mathrm{1,2},\dots ,C\}$
Transportation mode index	$v=\{\mathrm{1,2},\dots ,V\}$
Period index	$t=\{\mathrm{1,2},\dots ,T\}$

The input parameters include the followings:

${de}_{sict}$	Forecasted demand for product $i$ in customer zone $c$ in period $t$ in supply chain $s$
${Fc}_{sm}$	Fixed costs for manufacturing center $m$ to operate in each period in supply chain $s$
${{Fc}^{^{\prime}}}_{sw}$	Fixed costs for warehouse $w$ to operate in each period in supply chain $s$
${Uc}_{sim}$	Unit holding cost per period for product $i$ in manufacturing center $m$ in supply chain $s$
${{Uc}^{^{\prime}}}_{siw}$	Unit holding cost per period for product $i$ in warehouse $w$ in supply chain $s$
${Hc}_{sm}$	Holding capacity in manufacturing center $m$ in each period in supply chain $s$
${{Hc}^{^{\prime}}}_{sw}$	Holding capacity in warehouse $w$ in each period in supply chain $s$
${Vo}_{i}$	The volume of product $i$
${P}_{sij}$	Processing time (hours) to produce a unit of product $i$ on machine $j$ in supply chain $s$
${L}_{sij}$	Labor/hour cost to produce a unit of product $i$ on machine $j$ in supply chain $s$
$C{m}_{si}$	Cost of raw material for producing a unit of product $i$ in supply chain $s$
${Oc}_{sim}$	Variable overhead cost for producing a unit of product $i$ in manufacturing center $m$ in supply chain $s$
${Cp}_{sjm}$	Capacity hours for the production in manufacturing center $m$ on machine $j$ in each period in supply chain $s$
${Cr}_{sim}$	Capacity units of raw material supply for product $i$ in manufacturing center $m$ in each period in supply chain $s$
${tc}_{simwv}$	Unit transportation cost of product $i$ from manufacturing center $m$ to warehouse $w$ through mode $v$ in supply chain $s$
${{tc}^{^{\prime}}}_{siwcv}$	Unit transportation cost of product $i$ from warehouse $w$ to customer zone $c$ through mode $v$ in supply chain $s$
${{t}_{1}}_{\begin{array}{c}max\\ smwv\end{array}}$	Maximum transportation capacity from manufacturing center $m$ to warehouse $w$ through mode $v$ in each period in supply chain $s$
${{t}_{2}}_{\begin{array}{c}max\\ swcv\end{array}}$	Maximum transportation capacity from warehouse $w$ to customer zone $c$ through mode $v$ in each period in supply chain $s$
${I}_{{1}_{sim}}$	Inventory level of product $i$ in manufacturing center $m$ at the start of the planning horizon
${{I}^{^{\prime}}}_{{1}_{sim}}$	Inventory level of product $i$ in manufacturing center $m$ at the end of the planning horizon
${I}_{{2}_{siw}}$	Inventory level of product $i$ in warehouse $w$ at the start of the planning horizon
${{I}^{^{\prime}}}_{{2}_{siw}}$	Inventory level of product $i$ in warehouse $w$ at the end of the planning horizon
${ep}_{sij}$	Estimated carbon emissions to produce a unit of product $i$ on machine $j$ in any period per unit time in supply chain $s$
${et}_{simwv}$	Estimated carbon emissions for the shipment a unit of product $i$ from manufacturing center $m$ to warehouse $w$ through mode $v$ in supply chain $s$
${{et}^{^{\prime}}}_{siwcv}$	Estimated carbon emissions for the shipment a unit of product $i$ from warehouse $w$ to customer zone $c$ through mode $v$ in supply chain $s$
${eh}_{sim}$	Estimated carbon emissions for holding a unit of product $i$ in manufacturing center $m$ in each period in supply chain $s$
${{eh}^{^{\prime}}}_{siw}$	Estimated carbon emissions for holding a unit of product $i$ in warehouse $w$ in each period in supply chain $s$
$G$	A large number

The decision variables include the followings:

${Qp}_{sijmt}$	Quantity of product $i$ produced in manufacturing center $m$ on machine $j$ at period $t$ in supply chain $s$
${X}_{simwvt}$	Quantity of product $i$ shipped from manufacturing center $m$ to warehouse $w$ through mode $v$ at period $t$ in supply chain $s$
${{X}^{^{\prime}}}_{siwcvt}$	Quantity of product $i$ shipped from warehouse $w$ to customer zone $c$ through mode $v$ at period $t$ in supply chain $s$
${Y}_{simt}$	Inventory amount of product $i$ in manufacturing center $m$ at the end of period $t$ in the supply chain $s$
${{Y}^{^{\prime}}}_{siwt}$	Inventory amount of product $i$ in warehouse $w$ at the end of period $t$ in the supply chain $s$
${F}_{smt}=\left\{\begin{array}{c}1,\\ 0,\end{array}\right.$	If the manufacturing center $m$ operates in period $t$ in the supply chain $s$
	Otherwise
${{F}^{^{\prime}}}_{swt}=\left\{\begin{array}{c}1,\\ 0,\end{array}\right.$	If warehouse $w$ operates in period $t$ in the supply chain $s$
	Otherwise
$Cost\_{EC}_{s}$	Total costs of supply chain $s$ in the cap-and-trade system
${EC}_{s}$	Total carbon emission produced in supply chain $s$
${Cap}_{s}$	Carbon allocated to supply chain $s$ in the auction
$TEP$	Total carbon emission produced in all supply chains
$\psi$	Carbon trading price

According to these parameters and variables, each supply chain's objective function is formulated using mixed-integer linear programming based on a given carbon trading price. Each supply chain aims to minimize its objective function (Eq. 13). It includes fixed costs for operating and opening manufacturing centers, and warehouses, production cost, inventory holding costs in manufacturing centers and warehouses, transportation costs for the shipment of products from manufacturing centers to warehouses and warehouses to customer zones, and the revenue (cost) of selling (buying) carbon allowances, respectively:

$$Cost\_{EC}_{s}=Min \sum_{m}\sum_{t}{Fc}_{sm}{F}_{smt}+\sum_{w}\sum_{t}{{Fc}^{^{\prime}}}_{sw}{{F}^{^{\prime}}}_{swt} + \sum_{i}\sum_{j}\sum_{m}\sum_{t}{Qp}_{sijmt}({P}_{sij}{L}_{sij}+{Cm}_{si}+{Oc}_{sim})+\sum_{i}\sum_{m}\sum_{t}{Uc}_{sim}{Y}_{simt}+ \sum_{i}\sum_{w}\sum_{t}{{Uc}^{^{\prime}}}_{siw}{{Y}^{^{\prime}}}_{siwt}+\sum_{i}\sum_{m}\sum_{w}\sum_{v}\sum_{t}{tc}_{simwv}{X}_{simwvt}+ \sum_{i}\sum_{w}\sum_{c}\sum_{v}\sum_{t}{{tc}^{^{\prime}}}_{siwcv}{{X}^{^{\prime}}}_{siwcvt}+\psi ({EC}_{s}-{Cap}_{s})$$

(13)

The carbon emission generated by each supply chain $s$ is calculated according to Eq. (14). It includes carbon emissions generated in manufacturing centers, shipment of products from manufacturing centers to warehouses and warehouses to customer zones, and inventory holding in manufacturing centers and warehouses.

$$ EC_{s} = \mathop \sum \limits_{i} \mathop \sum \limits_{j} \mathop \sum \limits_{m} \mathop \sum \limits_{t} P_{sij} ep_{sij} Qp_{sijmt} + \mathop \sum \limits_{i} \mathop \sum \limits_{m} \mathop \sum \limits_{w} \mathop \sum \limits_{v} \mathop \sum \limits_{t} et_{simwvt} X_{simwvt} + \mathop \sum \limits_{i} \mathop \sum \limits_{w} \mathop \sum \limits_{c} \mathop \sum \limits_{v} \mathop \sum \limits_{t} et^{^{\prime}}_{siwcvt} X^{\prime}_{siwcvt} + \mathop \sum \limits_{i} \mathop \sum \limits_{m} \mathop \sum \limits_{t} eh_{sim} Y_{simt} + \mathop \sum \limits_{i} \mathop \sum \limits_{w} \mathop \sum \limits_{t} eh^{^{\prime}}_{siw} Y^{\prime}_{siwt} $$

(14)

The objective function in Eq. (13) is subject to the following constraints:

Limitation on raw material supply:

$$ \mathop \sum \limits_{j} Qp_{sijmt} \le Cr_{sim}\quad \forall s,i,m,t $$

(15)

Restriction on the total available working hours for each machine:

$$ \mathop \sum \limits_{i} Qp_{sijmt} P_{sij} \le Cp_{sjm}\quad \forall s,j,m,t $$

(16)

Storage capacity restriction in manufacturing centers and warehouses:

$$ \mathop \sum \limits_{i} Vo_{i} Y_{simt} \le Hc_{sm}\quad \forall s,m,t{ } $$

(17)

$$ \mathop \sum \limits_{i} Vo_{i} Y^{\prime}_{siwt} \le Hc^{^{\prime}}_{sw}\quad \forall s,w,t $$

(18)

Limitations on transportation capacity for shipment of products from the manufacturing centers to the warehouses and the warehouses to the customer zones:

$$ \mathop \sum \limits_{i} (Vo_{i} X_{simwvt} ) \le t_{1}{{\begin{array}{*{20}c} {max} \\ {smwv} \\ \end{array} }}\quad \forall s,m,w,v,t $$

(19)

$$ \mathop \sum \limits_{i} (Vo_{i} X^{\prime}_{siwcvt} ) \le t_{2} {{\begin{array}{*{20}c} {max} \\ {swcv} \\ \end{array} }}\quad \forall s,w,c,v,t $$

(20)

The inventory balance in manufacturing centers, warehouses, and customer zones:

$$ Y_{simt} - Y_{{sim\left( {t - 1} \right)}} = \mathop \sum \limits_{j} Qp_{sijmt} - \mathop \sum \limits_{w} \mathop \sum \limits_{v} X_{simwvt} \quad \forall s,i,m,t $$

(21)

$$ Y^{\prime}_{siwt} - Y^{\prime}_{{siw\left( {t - 1} \right)}} = \mathop \sum \limits_{m} \mathop \sum \limits_{v} X_{simwvt} - \mathop \sum \limits_{c} \mathop \sum \limits_{v} X^{\prime}_{siwcvt}\quad \forall s,i,w,t $$

(22)

$$ \mathop \sum \limits_{j} \mathop \sum \limits_{m} \mathop \sum \limits_{t} Qp_{sijmt} = \mathop \sum \limits_{c} \mathop \sum \limits_{t} de_{sict} + \mathop \sum \limits_{m} I^{\prime}_{{1_{sim} }} - \mathop \sum \limits_{m} I_{{1_{sim} }} + \mathop \sum \limits_{w} I^{\prime}_{{2_{siw} }} - \mathop \sum \limits_{w} I_{{2_{siw} }}\quad \forall i,s $$

(23)

Restriction on satisfying demand:

$$ \mathop \sum \limits_{w} \mathop \sum \limits_{v} X^{\prime}_{siwcvt} = de_{sict}\quad \forall s,i,c,t $$

(24)

Inventory levels at the start and end of the planning horizon in manufacturing centers and warehouses:

$$ Y_{sim0} = I_{{1_{sim} }} \& Y_{simT} = I^{\prime}_{{1_{sim} }}\quad \forall s,i,m $$

(25)

$$ Y^{\prime}_{siw0} = I_{{2_{siw} }} \& Y_{siwT} = I^{\prime}_{{2_{siw} }}\quad \forall s,i,w $$

(26)

Limitation on decision variables:

$$ 0 \le Qp_{sijmt} \le G F_{smt}\quad \forall s,i,m,t $$

(27)

$$ 0 \le X_{simwvt} \le G F_{smt} \& 0 \le X_{simwvt} \le GF^{\prime}_{swt}\quad \forall s,i,m,w,v,t $$

(28)

$$ 0 \le X^{\prime}_{siwcvt} \le GF^{\prime}_{swt}\quad \forall s,i,w,c,v,t $$

(29)

$$ 0 \le Y_{simt}\quad \forall s,i,m,t $$

(30)

$$ 0 \le Y^{\prime}_{siwt} \quad \forall s,i,w,t $$

(31)

Appendix B

The centers of each supply chain are dispersed randomly in a square area with a size of $10\times 10$ units of distance. Euclidean distance is considered between the two centers. The distance between manufacturing center $m$ and warehouse $w$ and between warehouse $w$ and customer zone $c$ are represented by ${dis}_{mw}$ and ${dis}_{wc}$, respectively. These distances are used for the following parameters that their values are dependent on distance. They are calculated according to the following formulas (the brackets indicate the generation of a random number from a uniform distribution in the interval):

$${tc}_{simwv}={Vo}_{i}[\mathrm{2,4}]*{dis}_{mw}*{ts}_{v}$$

(32)

$${{tc}^{^{\prime}}}_{siwcv}={Vo}_{i}[\mathrm{2,4}]*{dis}_{wc}*{ts}_{v}$$

(33)

$${et}_{simwv}={Vo}_{i}[\mathrm{2,4}]*{dis}_{mw}*{cts}_{v}$$

(34)

$${{et}^{^{\prime}}}_{siwcv}={Vo}_{i}[\mathrm{2,4}]*{dis}_{wc}*{cts}_{v}$$

(35)

The parameters ${ts}_{v}$ and ${cts}_{v}$ indicate the cost and carbon emissions for shipping a unit of product in a distance unit through mode $v$, respectively. These two parameters are generated according to Table 10:

Table 10 Specifications of transportation mode-dependent parameters

The inventory level for all products at the start and the end of the planning horizon in the manufacturing centers and warehouses are zero. The parameters regarding the manufacturing machine are calculated based on Table 11. The others are generated according to the details given in Table 12.

Table 11 Specifications of the machine-dependent parameters

Table 12 Specifications of the generated examples