Model Predictive Optimization for Energy Storage-Based Smart Grids

In recent years, energy storage systems (ESS) have started to play the role of an active electricity supplier so as to minimize overall electricity costs in a smart grid. However, ESS lifetime decreases with each cycle of charge/discharge. There is a tradeoff between ESS lifetime and electricity cost saving. As a solution, this work proposes a Model Predictive Optimization (MPO) method for distribution management in smart grids. Future energy states are predicted using an Autoregressive Integrated Moving Average (ARIMA) model. Based on the predicted electricity status, a near-optimal schedule for ESS usage is found using Genetic Algorithm such that a tradeoff is made between cost saving from electricity trading and the loss of life (LoL) in ESS. Experiment results show that the error rate of the prediction model is less than 10%. The MPO method achieves an overall cost saving of 0.85% and an ESS LoL reduction of 12.18%.


Introduction
According to the Tracking Clean Energy Progress 2015 report [1] from the International Energy Agency (IEA), the global annual increase in electricity demand has grown at a rate of 3.6% from 2002 to 2012. With the rapid growth of electricity demand, people try to explore various renewable energy resources. To increase the reliability of energy supply, the scalability of energy management, and the flexibility of energy utilization, the architecture of smart grids [5] has been proposed. The smart grid is a distributed system and is composed of multiple micro-grids. Each micro-grid includes multiple renewable power generators (e.g., photovoltaic, wind turbines), the energy storage system (ESS) and power consumers.
Although using renewable energy resources can help reduce the cost of electricity, there is a problem of intermittency in power generation by renewable energy resources. The intermittent problem of renewable energy resources leads to decrease in quality of electricity supply. As a remedy, ESS can be used to improve the power quality [18]. When there is surplus electricity in a micro-grid, it can be used to charge ESS and/or it can be sold to the utility or to other micro-grids. On the contrary, when there is deficient electricity in a micro-grid, ESS can be discharged to cover the deficit and/or electricity can be bought from the utility or from other micro-grids. However, the rate of charging/discharging, over-charging and over-discharging shorten the lifetime of ESS [11].
To find an optimal ESS schedule with a trade-off between its lifetime and cost saving for consumers, we propose an energy storage distributed management system based on Model Predictive Control (MPC) technique [8]. The proposed system includes a prediction model and an optimizer. The prediction model predicts the future electricity supply and demand with the time series analysis for history data of the power generators and power loads in micro-grids. According to the predicted results, the optimizer will not only satisfy the requirement with electricity trading, but also effectively choose the appropriate time points and the amount of energy to charge/discharge ESS.
This work is organized as follows. In Sect. 2, we introduce the related work on smart grid. Section 3 gives the overall framework and the details of proposed distribution management method. The experimental results and test cases will be explained and compared in Sect. 4. In Sect. 5, we give the conclusions for this work.

Related Work
Since the 20th century, MPC method which incorporates both forecasts and newly updated information has been used in the industrial process [19]. In recent years, MPC has drawn the attention of power systems also [16,17]. The core concept in MPC [3] is to forecast the development in the future timeslots, and to implement control actions in the next timeslot only. MPC is a rolling process that runs the results from optimizer repeatedly with updated forecasts. With the state feedback from the plant, the prediction model will be adjusted.
According to the stability or the instability of history data, the prediction methods can be classified into the linear method and non-linear method. However, the linear prediction method cannot deal with complicated data such as generation of solar power and wind speed. They are both linear, as well as, non-linear data. Thus, a prediction model which can handle both linear and non-linear data is more often to be used. For example, Box et al. proposed an Autoregressive Integrated Moving Average (ARIMA) model [2] for time series. The ARIMA model can reduce the prediction error because of the precise noise model. With the differences added to the mode, a non-stationary series can be transformed into a stationary series. The ARIMA model has the capability to handle many kinds of prediction questions. As a result, the ARIMA model was introduced to predict linear and non-linear data, simultaneously.
Optimization in smart grid enables consumers to use power related data to make more informed and profitable decisions such as power distribution and cost management [6]. Therefore, over the past few decades, a large number of research has focused on solving optimization problems. For example, Simulated Annealing (SA) [22] algorithm and Tabu Search (TS) [7] algorithm. However, both of them are not good at solving large-scale problems because of the slow convergence and long execution time. In the recent years, Genetic Algorithm (GA) [10] is frequently applied to optimize parameter configurations for different tasks in several years. For example, Yildirim et al. used GA to minimize energy consumption [24], Lee et al. used GA to determine the power consumption scheduling [12], and Chung et al. used GA to decide production frequencies of economic lot scheduling [4]. Miao et al. also used GA to minimize the entire energy expense with the power consumption scheduling [15].
To solve the problem of intermittency generated by renewable energy sources, ESS has played an important role [21]. ESS improves the stability of the electricity usage. It helps perform ancillary services such as peak-shaving and outage protection. As a service in peak-shaving, ESS can determine whether to charge or discharge for cost saving in peak time of electricity. As a service in outage protection, ESS can be an electricity supplier in which case it is used as a backup. However, the higher usage frequency of the battery module in ESS, the greater will the battery lifetime decrease. Therefore, a trade-off between the cost saving of electricity and lifetime of batteries should be considered.

Distribution Management System Design
In this section, the proposed design of distribution management system is described. We introduce the core techniques used in this system including MPC, the prediction model, and the optimization algorithm. Figure 1 shows the architecture system of the proposed Model Predictive Optimization (MPO) method for smart grids. Each micro-grid is equipped with the capabilities of predicting power generation and demand loads. A micro-grid not only satisfies demand loads via the utility, but also through other micro-grids via trading so as to save costs and earn additional economic benefits. The distribution management system includes two main parts, namely prediction models and an optimizer. Each micro-grid has its own prediction model for predicting demand loads and renewable energy generation. Through collecting history demand loads and renewable energy generation data, a micro-grid can forecast the situation of electricity such as deficient or surplus. Then, all micro-grids send the prediction information including forecasted demand loads and forecasted renewable energy generation to the optimizer, which is basically an ESS scheduler in this work.

Model Predictive Optimization
When the optimizer receives the prediction information of demand loads and generation, it will use an optimization algorithm to schedule future control inputs like trading electricity capacity such as buying deficient or selling surplus electricity capacity from the utility or other MGs, and charging/discharging ESS capacity. According to the control inputs, the distribution equipment controller tries to satisfy the electricity requirements. Feedback on the actual situations of electricity demand-response are given to the prediction models. If there is a significant difference between actual data and predicted data, the prediction is rectified (re-trained) based on the feedback.

Prediction with ARIMA Model
The inputs of proposed prediction model include history electricity data of power loads and renewable energy resources. We use the ARIMA model proposed in the Box-Jenkins methodology [2] to forecast the demand loads and power generation in the future. The ARIMA (p,d,q) model is defined in Eq. 1.
The forecasted data x t is calculated by the autoregressive parameters from ϕ 1 to ϕ p , the history series data from x t−1 to x t−p , the moving average parameters from θ 1 to θ q , and a series of random errors (or residuals) from ε t to ε t−q . This model observes the regression on history data x t−i for p periods and has a series of random errors ε t−i for q periods. The two values p and q are selected by observing the autocorrelation function (ACF) and partial autocorrelation function (PACF). However, since the ARIMA model is only for stationary series, we have to make differencing of the non-stationary series. Let the order of differencing be demoted as d.
After the identification of ARIMA (p,d,q) model, we would use the maximumlikelihood estimation (MLE) to estimate the parameters of the model. We use the Akaike Information Criterion (AIC) [9] and the Bayesian Information Criterion (BIC) [20] to check for an adequate model. To get the best fitness of ARIMA model, we should find the minimum values of AIC and BIC as follows: where n is the sample size, k is the number of estimated parameters, and RSS is the residual sum of squares, which are the factors from an estimated model for model selection. Finally, we can use the adequate ARIMA model to forecast the future electricity data. After forecasting electricity data, the future electricity situation such as surplus or deficient in electricity can be estimated.

Genetic Algorithm Optimization for ESS Scheduling
In general, the deficiency in electricity in a MG can be satisfied by purchasing electricity from the utility and/or other MGs and/or by discharging the ESS. However, the charging/discharging of ESS will shorten the lifetime of ESS, as a result of which additional cost is incurred on the batteries for purchase of new ESS. Thus, in this work, the charging/discharging of ESS is associated with an additional cost penalty. Our goal is to reduce the overall electricity cost, while considering the ESS lifetime at the same time. In this subsection, we will illustrate the formulation of optimization problem. Then, the process of GA optimization will be explained.

Proposed Cost Function.
To find a feasible schedule for a set of micro-grids such that an appropriate trade-off is made between the total electricity cost and the ESS lifetime, the target optimization problem is formulated as follows, where the cost function is specified under three constraints for each micro-grid.
Gen total,n = GenSup L,n + GenSup ESS,n + GenSup MG,n + GenSup U,n Load total,n = GenSup L,n + ESSSup L,n + MGSup L,n + USup L,n For next time slot (n = 1), a schedule is generated by considering future N time slots, where N > 0. The parameter α lif e is a weight factor, and is set to ten times of Capacity Battery , where Capacity Battery is the energy capacity value of each battery in ESS. α save is a constant, and is set to 1. The second constraint is to ensure that the CSOC n of ESS is between the lower bound (CSOC min ) and the upper bound (CSOC max ) at each time slot n. The third and fourth constraints are to ensure that the electricity power is balanced between the generation and the load at each time slot n, where Gen total,n is the total generation, and Load total,n is the total demand loads.
With the above formulation, positive values of the cost function would indicate a profit on selling surplus electricity. Two parameters representing the surplus amount of electricity sold include the amount of generation (kWh) sold to other MGs, GenSup MG,n , and that sold to the utility denoted by GenSup U,n . Negative values of the cost function represent the cost incurred by buying deficient electricity in an MG, and/or when the ESS needs to be charged. The two parameters representing the amount of deficient electricity bought include MGSup L,n the capacity (kWh) supplied by other MGs to loads, and USup L,n is that supplied by the utility to demand loads. USup ESS,n is the capacity (kWh) that the utility supplies to ESS. The loss of life (LoL) of ESS [21] is affected by three factors, namely electricity usage, the amount of generated power (kWh) used to charge ESS GenSup (ESS) , the amount of capacity (kWh) that ESS discharges for demand loads ESSSup L , and USup ESS,n .
The Process of GA Optimization. After prediction models have predicted the electricity data for the consumption of power loads and the generation of renewable energy resources, the process of optimization can be performed. First, we will randomly generate the initial set of schedules. Pairs of the schedules are selected as parents, which are merged into a single schedule during the crossover and mutation operations. A new schedule set evolves by replacing a schedule in the old set. This process is repeated until the optimization time is finished or the schedule set converges. Finally, we can get the optimal trading electricity capacity and charging/discharging capacity of ESS for each micro-grid.
First, randomly generate the initial set of schedules. As shown in Fig. 2, a schedule is encoded by a three dimensional matrix, where rows, columns, and pages represent micro-grids, units need to be scheduled, and N time periods in the future, respectively. Each micro-grid has its own schedulable units. For example, in Fig. 2, the schedulable units are GenSup L,n , GenSup ESS,n , GenSup MG,n , GenSup U,n , ESSSup L,n , MGSup L,n , USup L,n , and USup ESS,n . The values in the matrix represent electricity power (kWh), and generated randomly, but satisfying the constraints given in the cost function (Eq. 4).

Fig. 2. The initial set of a schedule
Second, a selection operation will choose two schedules in the current scheduling set. The chosen schedules will be the parents of the next generation. This work adopts the most common way, which gives the best schedule a high probability to be selected than the worst one. The value of each schedule is calculated by the cost function (Eq. 4). In other words, a schedule with the highest cost will be the best schedule in the scheduling set. After evaluating a schedule, its normalized rank is calculated [12]. The normalized rank is used to adjust the probability of a schedule being selected. It is enforced that the probability of the best schedule to be selected is four times more than that of the worst.
After parent selection, a crossover is executed. We use a classical one-point crossover operation by cutting the pages of a matrix as shown in Fig. 2 in random to generate a crossover point, and then exchanging segments of the two parents to produce a child. After crossover operation, we should adjust some schedulable units to satisfy constraints of the cost function. A schedule produced via the classical one-point crossover would be an infeasible one that relates the constraints given in Eq. 4. Thus, an adjustment is made such that a child completely inherits part of schedulable units from parent 1, and the remaining schedulable units are filled by parent 2. This adjustment plays the role of mutation.
When a new child schedule is generated, it will substitute a schedule in the current scheduling set to generate a new scheduling set. The proposed method replaces the schedule that has the lowest cost in the current scheduling set by the newly generated one.

Experiments
In this section, we present evaluation of the proposed MPO method for smart grids. First, the experimental setup used for experiments is introduced. Then, the experimental results are illustrated.

Experimental Setup
In this subsection, the experimental environment and energy data used in our experiments are described.
Experimental Environment. The proposed method is implemented in the Python and Matlab programming language on a PC with Intel(R) Core(TM) i7 2.39 GHz CPU, 4 GB RAM running Windows 7 64-bit OS. The Python programming language is used for implementing the weather information parser such as irradiance and wind speed. The Matlab programming language is used for realizing the proposed distribution management.

Demand Load Data, Generation Data, Electricity Price, and Power
Proportion. Three different types power consumers including commercial consumers, industrial consumers, and residential consumers are considered. We refer the different conditions of electricity usage with different times in research [23] to simulate one day consumption for each type of demand load. The demand load percentages for each type of consumers are commercial consumers 13%, industrial consumers 73%, and residential consumers 14%. By observing the usage frequency of demand loads throughout a day, demand loads behavior can be classified into peak time and non-peak time. To reduce peak load demands, the utility sets the price higher at peak time than at non-peak time. In this work, the utility selling price is set to $3.62 per kWh at peak time and $1.69 at non-peak time. However, the utility buying price is set to $2 per kWh at peak time and $1 at non-peak time. Between all micro-grids, the trading price of selling and buying are the same. They are set to $3 per kWh at peak time, and $1.5 per kWh at non-peak time.

Experimental Results
In this subsection, we first give an evaluation of the ARIMA prediction model. Then, the cost saving of electricity and ESS LoL for smart grids due to the proposed method are described.

Evaluation of ARIMA Prediction Model.
We use the ARIMA model to predict the demand loads, wind speed, and irradiance. For predicting the values precisely, the past 100 hours history data are used. The prediction accuracy is evaluated by the Root Mean Squared Error (RMSE) as shown in Eq. 5. RMSE makes an excellent general purpose error metric for a prediction method.
where n is the number of predicted data, y i is the real data, and y i is the predicted data. The smaller the RMSE value, the higher the prediction accuracy is. To normalize the RMSE, Eq. 6 is used to explain the error rate between different scales of data, where the maximum error is the difference between the maximum and minimum value of data. The error rates of demand loads, wind speed, and irradiance are 4.77%, 1.60%, and 7.18%, respectively. We can observe that all error rates are smaller than 10%, which is quite good.
Optimization Method. In the following experiments, we assume the smart grid has 30 micro-grids. First, we consider several look-ahead time slots, such as 2, 3, 4, 5, and 6 time slots and conclude on an appropriate number of time slots to use for MPO. Finally, electricity cost, ESS LoL, and overall cost are compared to the MPC look ahead-dispatch [14] which is also MPC based method for electricity cost saving, and economic dispatch system [13] which is state-ofthe-art method for electricity cost saving. First, we consider several look-ahead time slots and conclude on an appropriate number of time slots to use for MPO. As shown in Fig. 3, we analyze different look-ahead time slots to be used for the proposed MPO method. The electricity  Finally, electricity cost, ESS LoL, and overall cost are compared to other methods. Figure 4 shows the comparison of ESS usage in one day, for MPO, MPC look ahead-dispatch method, and economic dispatch system. In the MPC look ahead-dispatch method [14], ESS not only plays a backup role, but also acts as an active electricity supplier. ESS will be discharged as demand loads are higher than a given threshold, where the threshold is set to the average of past demand loads, and will be charged as demand loads are lower or equal to the given threshold. In the economic dispatch system [13], ESS will be discharged/charged to minimize the cost of electricity. Both of them also use predicted future electricity to determine the trading electricity amount. Figure 4 shows the demand loads (kWh) and power generation (kWh) varying with the hour of the day along with the ESS capacity for two different micro-grids, one with a low demand load consumption (MG3) and one with a high demand load consumption (MG20). In MG3, the maximum ESS usage percentages of MPO, MPC look ahead-dispatch method, and economic dispatch system are 30%, 59%, and 35%. The average ESS usage percentages of MPO, MPC look ahead-dispatch method, and economic dispatch system are 6%, 10%, and 12%, respectively. In MG20, the maximum ESS usage percentages of MPO, MPC look ahead-dispatch method, and economic dispatch system are 12%, 37%, and 28%. The average ESS usage percentages of MPO, MPC look ahead-dispatch method, and economic dispatch system are 6%, 15%, and 11%, respectively. Therefore, the proposed method has lowest percentages in the maximum and average ESS usage. We can observe that not only in a micro-grid with low demand loads (MG3), but also in a micro-grid with high demand loads (MG20), the proposed method can avoid suddenly burst in the usage of ESS such as 5 time slots in MG3 and 6 time slots in MG20.
An estimation of overall cost incurred in a smart grid is designed as Eq. 7, where ElectricityCost is the cost to be paid for satisfying electricity requirements, Cost ESS is the total installation cost of ESS, and ESSLoL is LoL estimated for the ESS, that is, ESS LoL is considered as a cost factor in this work.
To evaluate the impact of overall cost and ESS LoL with different weight factors α lif e , the simulation results are shown in Fig. 5. We can observe that giving two times emphasis to α lif e results in the least overall cost and ESS LoL (i.e. 82%). However, giving three times emphasis to α lif e , leads to a higher overall cost and ESS LoL (i.e. 87%). Because of discharging/charging ESS is too expensive, ESS will not be discharged/charged at non-peak time, and be discharged for a higher capacity at peak-time with a limited discharged capacity. Therefore, the more emphasis consumers put on ESS, it will not always lead to a better overall cost and ESS LoL. Compared to the MPC look ahead-dispatch method, the overall cost saving and ESS LoL reduction achieved by MPO is 0.85% (i.e. $1,362,000) and 12.18% for 20 months, respectively.
The long term trend of ESS LoL and overall cost saving are shown in Fig. 6. To compare the overall cost with other methods, we set the overall cost of proposed MPO method as a baseline, and observe the difference with other methods which is called as the overall cost saving ($). In Fig. 6, we can observe that the overall cost savings ($) of proposed MPO method compared with other methods grow gradually with time. After 22 months, ESS LoL in MPO, the MPC Look Ahead-dispatch Method, and the Economic Dispatch System are 94%, 103%, and 100%, respectively. In other words, ESS in the proposed MPO method has a longer life than that in other methods. Even other methods have run out of ESS lifetime, the proposed MPO method still has 6% ESS lifetime to be used. Moreover, to compare the overall cost with MPC look ahead-dispatch method and economic dispatch system, the proposed method can save $1,498,200 and $1,467,840, respectively. Therefore, to save the overall cost and extend ESS lifetime, it is better to consider ESS LoL while using it as an active supply.

Conclusions
In this work, we proposed a model predictive optimization (MPO) method for distributed management system in smart grids. MPO leverages prediction models for demand loads and power generation, while scheduling the charging/discharging of energy storage systems (ESS) and the trading of electricity among micro-grids with each other and with the utility, using ESS as an active power supply. The Autoregressive Integrated Moving Average (ARIMA) model was used to predict the energy states of power loads and renewable energy resources. Our prediction model exhibited a RMSE error of less than 10% for predicting demand loads and renewable energy generation. The load and generation prediction results were used as inputs to the optimizer. The Genetic Algorithm (GA) optimizer was used to schedule the usage of ESS, and to reduce the overall cost, which includes the total electricity cost and the cost incurred due to ESS loss of life (LoL). Experimental results showed that 100 schedules in the initial scheduling set and 4 look-ahead time slots are appropriate settings for the optimizer. Compared with the model predictive control (MPC) look ahead-dispatch method, MPO demonstrated an overall cost saving of 0.85% (i.e. $1,362,000) and an ESS LoL reduction of 12.18% for 20 months, respectively, if two times of α lif e is selected. With the history data of demand loads and generation, the proposed MPO method can also provide a previous analysis for smart grids to decide an appropriate ESS capacity for installation.