1 Introduction

The global landscape of electrical energy faces formidable challenges from conventional fossil fuel pollution, exemplified by coal power plants emitting 800 g of carbon dioxide per kilowatt-hour of electrical energy produced. Additionally, aging coal power stations release 5–10% more nitrogen and carbon dioxide than their newer counterparts [1]. This pollution, as identified by the World Health Organization (WHO), has severe consequences, contributing to the annual death of 4–7 million individuals and widespread illness affecting hundreds of millions more [2]. These emissions also play a role in ozone layer depletion, acidic rainfall, and the proliferation of atmospheric microbes, collectively posing a substantial threat to Earth’s sustainability. Over 85% of countries within the International Energy Agency (IEA) are currently susceptible to moderate to severe global warming, with nations like India, China, and Mexico ranking among the most vulnerable [3].

The power industry faces significant risks from climate change, impacting fuel resources, energy generation, physical resilience of energy infrastructure, and energy demand. Escalating extreme weather events, such as cyclones, wildfires, heatwaves, floods, and cold spells, pose a growing threat, disrupting energy generation and complicating demand management [4]. Recent blackouts in Australia, California, Japan, and Korea, caused by wildfires, heatwaves, and cyclones, underscore the vulnerability of energy systems to climate-related risks.

Furthermore, a confluence of climate change, pollution, surging energy consumption, economic concerns, and energy security anxieties after the recent Russian war collectively unsettles the existing power distribution network. Previous works extensively discuss conventional power plants’ limitations and drawbacks [1, 4]. Studies also advocate for constrained carbon capture and storage, extending not solely within the electricity sector but also to other domains like transportation, cooling/heating, and the industrial sector [5, 6]. Coordination between every sector of energy, heat, transportation, and desalination is a must to improve the overall system’s flexibility. The results indicate that electrification and collaboration between sectors provide more flexibility [7].

According to [8], US emissions experienced a 35% reduction from 2000 to 2022, measured in metric tons of CO2 per capita. The Intergovernmental Panel on Climate Change (IPCC) sets the targeted reduction range for carbon dioxide CO2 emissions by 2050, relative to the year 2000, between 50 and 85% per capita. In contrast, China’s emissions increased by a significant 350%. Overall, estimated global emissions rose by 12.5% during the same period.

A microgrid (MG) is a self-sufficient system designed to generate electricity through renewable energy sources (RES) and energy storage systems (ESSs), capable of functioning independently or connected to the primary power grid. The utilization of microgrids is often described as distributed, scattered, decentralized, district, or embedded generation. Microgrids have emerged as a preferred solution to address the inherent variability and uncertainty associated with RES, significantly mitigating the risk of blackouts and enhancing the overall reliability of the power supply [9].

Some countries will reach 100% renewable energy in the coming years. In [10], 139 countries can be fully covered by renewable energy by using their potential in the wind, hydro, and solar energy. Furthermore, many countries, including Bangladesh, Barbados, Cambodia, Colombia, Ghana, Mongolia, Vietnam, and Hawaii, intend to use only renewable energy by 2045 or 2050 [11].

Other countries have taken strong steps towards increasing RE integration in their long-term plans, including developed country such as Sweden [12] and Developing country such as Ethiopia [13].

Several cities have made similar commitments, pledging to meet all of their energy needs from renewable sources by 2050. Over 920 cities in 73 countries have established a renewable energy target, and over 1100 city governments announced net zero targets [14]. The transition towards 100% RE isn’t only at the country and city level but also at the company level. With a maximum goal year of 2050, a similar tendency occurs among larger corporations such as BMW and IKEA, as well as technological giants such as Apple, Google, eBay, and Facebook, LG, KIA, and many more [15]. In real work there is some promising examples of Several countries, including Norway and Costa Rica, now get practically all of their power from renewable sources [11].

Recently, numerous review studies have focused on 100% renewable energy (RE) microgrids (MGs) from various perspectives, such as MG control systems classification [16], protection systems [17], sustainability [18], etc. In contrast, this review paper primarily addresses real-time energy management systems with short-term uncertainty considerations.

After the introductory phase, Sect. 2 explains diverse microgrid models and expounds on uncertain variables. The third section explains existing energy management systems, outlining their pre-proposed optimization techniques and describing relevant technical and economic objective functions. Section 4 addresses uncertainty models and prediction methodologies. Finally, Sect. 5 summarizes the key conclusions and contributions of the paper. Figure 1 illustrates the framework of the rest of the paper.

Fig. 1
figure 1

The paper’s graphical framework

Full size image

2 Microgrid modelling

Typical microgrids encompass renewable sources like PV and wind plants, energy storage systems, and various loads. Each component within a microgrid necessitates mathematical technical models to analyze the microgrid’s dynamic behavior comprehensively. The cost analysis is integral to microgrid planning and operational studies, combining the expenses associated with each component.

2.1 Generator models

2.1.1 Photovoltaic PV

PV arrays can convert global solar irradiance into direct current electrical power. Using proper inverters, PV energy can be fed into the power grid. PV power is variable by nature. It depends mainly on PV incident solar radiation, which varies according to the geographical location, hourly, and seasonally. Also, it is significantly affected by various weather conditions, such as clouds, fog, dust, and snow [19]. Many equivalent circuits are proposed to simulate PV-generated power dynamics. These models differ considerably according to their applications. PV Output Power (Ppv) is expressed mathematically as follows [20]:

$${P}_{pv}={G}_{T}\eta {A}_{pv}$$
(1)
$$\eta ={\eta }_{m}{\eta }_{pc}$$
(2)
$${\eta }_{m}={\eta }_{r}[1-\beta ({T}_{C}-{T}_{r})$$
(3)
$${T}_{C}={T}_{a}+\frac{{\propto }_{\tau }}{{U}_{L}} {G}_{T}$$
(4)
$$\frac{{\propto }_{\tau }}{{U}_{L}}=\frac{{G}_{NOCT}}{NOCT-{T}_{a,NOCT}}$$
(5)

where \({G}_{T}\) is the total radiation per square meter in 1 h, \(\eta ,{\eta }_{m},{\eta }_{pc}{\eta }_{r}\) are system, module, power conditioning (PV auxiliary equipment) and rated references efficiency, respectively, \({A}_{pv}\) PV panel area, \(\beta\) PV cell temperature coefficient of efficiency, \({T}_{C}\) PV cell temperature, \({T}_{r}\) temperature at rated efficiency, \({T}_{a}\) ambient temperature, NOCT normal operation cell temperature, \({T}_{a, NOCT}\) ambient temperature of NOCT and \({G}_{NOCT}\) solar radiation in NOCT. Solar incident irradiance is the main motive of all PV models, while the dynamics of other secondary parameters can be ignored to simplify the analysis process [19]. For instance, temperature uncertainties result in less than a 3% effect on the power output from the module [21].

2.1.2 Wind power

A wind turbine generates electricity by converting the force of the wind into torque exerted on the rotor blades. The power is transferred from the wind to the generator rotor. It is affected by the air’s density, the rotor’s area, and the wind speed. For optimization and scheduling purposes, the wind output power model depends on available wind power and the turbine speed characteristics. A standard wind power model is formulated mathematically as follows: [20]:

$${P}_{w}=\frac{1}{2}\rho A{V}_{w}^{3}{C}_{p}$$
(6)

where \(\rho\) the air density (kg/m3), A is the swept area by the rotor, \(Cp\) is the rotor power coefficient, and \({V}_{w}\) is the wind velocity at the hub height(m/s).

Wind power is proportional to wind speed; a wind speed forecast inaccuracy would result in significant inaccuracies. Also, wind direction estimation is an essential consideration when selecting the type of wind turbine, which can be measured using a standard anemometer.

2.2 Distributed storage

The switch to a green energy system necessitates energy storage to guarantee a consistent supply of RE. Backup storage is utilized to keep the balance between the load and generation during times of limited generation. RES are susceptible to power availability uncertainty caused by weather patterns and climate dependencies. Backup storage helps eliminate this uncertainty and offers a more stable electrical energy supply. The storage systems are designed to charge when there is an excess of energy produced by renewable generators and discharge electricity to the microgrid during periods of low energy generation.

Batteries are widely used for their ease of installation and portability. They can quickly store and release electrical energy, making them suitable for short-term energy needs [22]. The fuel cell converts hydrogen and oxygen into electricity, heat, and water through hydrogen oxidation in a catalytic layer, producing an electrical current. Due to their clean and efficient electricity production, fuel cells are widely used in vehicles and stationary power systems. The hydrogen generated can be stored in pressurized containers, in liquid form, or as metal hydrides; however, the multiple steps involved in this process lead to a reduced overall efficiency of 40%, making it relatively expensive [23]. The most suitable ESSs for dealing with uncertainty are batteries, fuel cells, and supercapacitors, as they have no special requirements and possess the proper response time to reduce the power mismatch in microgrids due to the unpredictable behavior of renewables.

2.2.1 Batteries

Batteries are energy storage devices that use electrochemical processes to store energy with an efficiency range of 75–95% [22, 24]. The choice of battery model depends on the specific application and the accuracy and complexity required. Electrochemical models are generally the most accurate but can also be complex and time-consuming to develop. In the relevant literature, the battery model in terms of its output voltage [25], energy capacity [20], or state of charge (SOC), which is the common model for sizing, optimization, and scheduling purposes and can be expressed mathematically as follows [26]:

$$SOC\left(t\right)=\left( PVp\left(t\right)+WTp\left(t\right)-\frac{PL\left(t\right)}{{\beta }_{v}} \right)x{\beta }_{ec}+SOC\left(t-1\right)X\left(1-dh\right)$$
(7)
$$SOC\left(t\right)=-\frac{\left( -\frac{PL\left(t\right)}{{\beta }_{v}}-\left[PVp\left(t\right)+WTp\left(t\right)\right]\right)}{{\beta }_{ed}}+SOC\left(t-1\right)X\left(1-dh\right)$$
(8)

where: SOC (t), SOC (t−1) is the state of charge of the batteries for the current and previous time in kilo-watthour, dh the hourly rate of self-discharge, PL (t) demand load, \({\beta }_{v}\) inverter efficiency and \(\mathrm{\beta ec},{\beta }_{ed}\) is the charging and discharging efficiency.

SOC is a critical factor that influences the energy stored in ESSs. One of the key restrictions in the ESS operating studies is the SOC limitation between the minimum and maximum stored energy permitted for the ESSs. Indeed, the SOC is a crucial uncertain parameter in ESS energy management strategies. [27]

$${SOC}_{min}\le SOC\left(t\right)\le {SOC}_{max}$$

2.2.2 Fuel cell

Hydrogen-based energy storage systems utilize two mechanisms for storing energy and generating power. Typically, hydrogen is produced using a water electrolysis unit. The fuel cell, the primary technology in hydrogen-based energy storage systems, converts the chemical energy in hydrogen into electricity. While a reversible fuel cell can perform both functions, it is more economical to have two separate subsystems [28]. Fuel cell models have been created with two distinct charging and discharging modes to develop optimization and energy management dispatching strategies [25, 29]. During the charging mode, an electrolyzer fills the hydrogen tanks. The molar flow of hydrogen in the electrolyzer can be expressed as a function of the provided electrical power, as follows [25]:

$${\upeta }_{{\text{H}}2.{\text{EL}}}=\frac{{\upeta }_{{\text{EL}}}{{\text{P}}}_{{\text{EL}}}}{{{\text{LHV}}}_{{\text{H}}2}}$$
(9)

In the discharging mode, hydrogen is employed to produce electricity using a fuel cell, and the amount of hydrogen consumed by the fuel cell is directly proportional to the power it generates via the following relationship:

$${\upeta }_{{\text{H}}2.{\text{FC}}}=\frac{{{\text{P}}}_{{\text{FC}}}}{{{\upeta }_{{\text{FC}}}\mathrm{ LHV}}_{{\text{H}}2}}$$
(10)

where \({\eta }_{H2.EL}\) the hydrogen molar flow of the electrolyzer, \({P}_{EL}\) the supplied electric power to the electrolyzer, \({LHV}_{H2}\) the lower heating value of hydrogen (240 MJ/Kmol), \({\eta }_{EL}{\eta }_{FC}\) electrolyzer and fuel cell efficiency, which take into account electrochemical, thermodynamic, and ancillary losses and \({P}_{FC}\) is the output power.

2.3 Load model

Various models and methodologies, including both static and dynamic ones, are used to represent the load. Typically, the load is represented as a continuous power load, but using a managed load model is more desirable as it enables demand-side management by the energy management system. The load can either receive full power or be disconnected entirely. Different methods and techniques for load modeling in microgrids exist, such as measurement-based [30], analytical, and machine learning-based approaches. One of the challenges of load modeling is to capture the dynamic and stochastic nature of loads and their interactions with other microgrid components. Many studies have focused on developing static [31], dynamic [32], and composite models [33] for load representation. However, a significant proportion of microgrid loads are dynamic, including electric machines and loads connected via power converters, making microgrid dynamics predominantly influenced by the performance of power converters and dynamic loads [23].

3 Energy management systems

The evolution of Energy Management Systems (EMS) can be traced through various milestones. In the 1950s, it began with load frequency control (LFC), followed by the introduction of supervisory control and data acquisition SCADA, the EMS database, automatic generation control (AGC), economic dispatch (ED), and unit commitment (UC) in the 1960s. In the 1970s, network analysis (state estimator), load forecasting, event processing, system redundancy, and backup were incorporated. The 1980s saw the addition of transmission security (optimal power flow), while the 1990s brought advancements in inter-control center communications protocols, open platforms, and distributed architecture. After the 2000s, new features such as demand side management (DSM), multidirectional power flow, decentralized control, and network management were introduced [34]. Many definitions are suggested to express EMS-related analyses [35, 36]. According to the International Electrotechnical Commission IEC in the standard IEC 61970 [37], EMS can be defined as follows: “a computer system comprising a software platform providing basic support services and a set of applications providing the functionality needed for the effective operation of electrical generation and transmission facilities to assure adequate security of energy supply at a minimum cost.”

In microgrids, the control and supervision within the EMS can be classified as either centralized or decentralized. In a centralized EMS [36], the control section collects data and information on generation, demand, cost functions, and other relevant data. Based on this information, the control section determines the optimal scheduling and objectives, which are then transmitted to the local controllers for implementation. In a decentralized EMS [38], the responsibility for determining real-time and future generations or loads lies with the local controllers. These controllers exchange data with the central controller as needed. The importance of an energy management system becomes particularly critical when utilizing 100% RES. Since RE availability is only sometimes consistent due to weather and environmental conditions, efficient energy management is vital. The aim is to meet electrical energy demand while minimizing costs and energy losses. Several factors should be considered when selecting an optimization model for an EMS. These factors include the objective function, constraints, linearity of the model, solution methods (analytical, numerical, etc.), complexity, and sensitivity to changes, which will be discussed in the following subsections.

3.1 Objective functions

The primary goal of an energy management system should be to achieve a balance between supply and demand. However, additional objectives will vary depending on factors such as user preferences, the location of MG, the type of storage system utilized, the configuration of the microgrid, market prices, the capacity of the microgrid, regulations specific to the country, and electrical energy production.

In [39], the objective function focuses on achieving power balance using a simple algorithm for its simplicity. On the other hand, [40] considers technical parameters such as maximizing H2 generation, increasing lifetime, minimizing battery use, ensuring stability, and maintaining steady demand. The primary advantage of this approach is its moderate level of complexity. Furthermore, economic objectives can be incorporated alongside the primary power balance objective by utilizing various cost functions, as explained in [41].

3.2 Microgrid optimization techniques

They involve applying advanced optimization methods to control and optimize RE microgrid systems effectively. These techniques encompass the integration of RES with energy storage systems and utilize sophisticated computer algorithms to accomplish the desired objective functions. Several constraints, such as market prices, are considered in the optimization process. Figure 2 provides a summary of the most used optimization approaches and algorithms, which are described in detail below.

Fig. 2
figure 2

Pre-proposed EMS optimization techniques

Full size image

3.2.1 Classical methods

In [42], linear programming (LP) is employed to find the optimal solution for problems with a linear objective function and linear inequality constraints. The application of LP in [42] focuses on microgrid scheduling and cost optimization. In [43], LP combined with artificial intelligence (AI) is utilized for multi-objective optimization. Non-linear programming (NLP) is a mathematical optimization technique for solving optimization problems with non-linear objective functions or constraints. These problems involve non-linear equations that cannot be expressed as a linear combination of variables. In [44], NLP is applied to minimize operational costs. Dynamic programming DP is a mathematical optimization technique that tackles complex problems by breaking them down into smaller subproblems and solving each subproblem only once over a specific period. In [45], a dynamic model is proposed for electrical energy demand scheduling, highlighting the use of dynamic programming in addressing the problem.

3.2.2 Artificial intelligent (AI)

Fuzzy Logic (FL) is a mathematical model designed to handle uncertainty in decision-making processes, unlike traditional logic that deals with true or false values. Fuzzy logic allows for degrees of truth. In [46], a fuzzy technique is utilized for optimizing the battery storage system. The Neural Network (NN) method is a machine-learning technique inspired by the human brain’s structure and operation. Neural networks comprise linked nodes, or neurons, that process and send data. Each neuron gets information from other neurons, computes it, and then delivers its output to other neurons. NN is used in [47] to obtain the most significant amount of power from wind and solar energy sources. Genetic Algorithm (GA) is an optimization algorithm inspired by natural selection and evaluation principles. It solves complex optimization problems by simulating the process of natural selection, where the fittest individuals are selected for reproduction, and their genetic material is combined to create new offspring. In [48], a genetic algorithm is applied to solve the operation scheduling problem in microgrids.

3.2.3 Meta-heuristic approaches

The particle swarm optimization (PSO) technique involves a collection of particles representing potential solutions, moving in search of the optimal solution. These particles track the movement of the best solution discovered thus far, gradually converging toward the optimal solution as they learn and adapt based on their experiences. In [49], PSO minimizes microgrids’ operation and maintenance (O&M) costs. In the Ant Colony Optimization (ACO) technique, a collective of ants collaborates to discover the most efficient route connecting their colony and a food source. They leave pheromones along their path as markers, allowing other ants to trace these pheromone trails and locate the shortest path. This approach has been extended to tackle multi-objective optimization problems [50].

3.2.4 Model predictive control

Model Predictive Control (MPC) predicts a system’s future behavior using a mathematical model within a defined time span. Based on this model, optimal control actions are computed to minimize a specified cost function over the given time horizon. These control actions are then implemented in the system, and the process is iterated at each time step. In [51], MPC addresses the optimization problem of maximizing economic benefits. Researchers have explored various alternative approaches to tackle the energy management challenge in microgrids. For instance, the Round-Robin approach [52], the rolling horizon optimization method [53], and other methodologies have been applied. Table 1 summarizes the main features of EMS optimization techniques.

Table 1 EMS optimization techniques features
Full size table

3.3 EMS software programs

Numerous software tools are available for practical use to support the applications of EMSs, as documented in the existing literature, such as HOMER, DigSILENT Power Factory [38], MATLAB/ Simulink, iPower from General Electric company, MicroSCADA Pro Network Manager SCADA from ABB, Spectrum PowerTM from siemens, GenCom, GAMS, MATPOWER, Vykon and many other as referenced in [36, 54].

3.4 EMS constraints

During real-time operations, the energy management system encounters several constraints. These constraints pertain to the power supply and encompass limitations on power generation, maintaining energy balance, and adhering to specific constraints regarding the state of charge (SOC) and depth of discharge (DOD) of the battery in the storage system. Similarly, the fuel cell has its operating limits. Furthermore, when dealing with a 100% RE microgrid MG, the primary constraint revolves around the uncertainty associated with renewable RES, as will be discussed in the following section.

4 Uncertainty sources

Intermittency, variability, and volatility in RE generation pose severe issues for the adaptability of the power system. Furthermore, the price of electricity in the power market has changed significantly because of the fast growth of renewable energy. Research is now focused on increasing power system flexibility in an unpredictably changing environment to ensure the power system’s security, dependability, and economic effectiveness. Additionally, this limited flexibility issue has not been fixed [55]. According to the International Energy Agency (IEA) and the North American Reliability Corporation (NERC), the flexibility issue is mainly related to the uncertainty of RES. Flexibility is the broadest range of uncertainty that the power system can tolerate, according to [56]. A power system must also have much flexibility to accommodate variations in solar and wind energy if it uses more RES than 80% [57]. The possibility of a difference between projected and actual measurements is how uncertainty is typically characterized [58]. The uncertainty in the power system can be classified as follows:

4.1 Load demand

The load power uncertainty is a time-dependent problem, especially for residential loads. It depends on the load types and their operation. Many influencing factors may increase the uncertainty of the residential loads, such as building indoor conditions, building parameters, climate, social economy, etc., as discussed in [58]. Consequently, it is essential to use a combination of successful forecasting techniques.

4.2 Market energy price

The variance between the transaction volume mentioned in the contract and the actual volume is caused by various unanticipated variables, such as source, load, and system failures, which have an impact on both local and worldwide markets due to changes in energy, installation, and supply prices [58]. A real-time tariff is adapted to cope with the power mismatch between the load and generated power that complicates handling the uncertainty problem.

4.3 Renewable energy RES

According to how they interact with weather and climatic fluctuations, RES can be divided into three classes [59]:

  1. 1.

    The weather has little effect on tidal and geothermal energy.

  2. 2.

    Seasonal weather can affect hydroelectricity generated from water reservoirs.

  3. 3.

    The weather directly impacts both wind and solar power.

The latter are the principal sources of RE generation, and their fluctuations cannot be scheduled like conventional power. Both wind and solar power uncertainties should be studied to be handled in high-penetration RES grids. For instance, in Europe, wind and PV generation fluctuations are simulated using data from the past 8 years, and the mismatches between supply and demand are investigated; as discussed in [59], the data from 83 regions were analyzed. The results were divided into three time periods, as follows:

  • Daily, wind and solar energy exhibit a prominent peak in their power spectrum. While solar energy’s peak displays increased power on the outskirts of the peak during the day, wind power demonstrates a distinctly sharp peak. The wind power generated during daylight hours is approximately 2.7% lower. Conversely, wind power gradually increases at night, benefiting from the thermal stratification becoming more stable due to the surface’s radiation-induced cooling.

  • Within the time frame of 2–10 days, wind energy exhibits significantly greater spectral power compared to solar energy. A notable and robust weekly pattern is observed in energy consumption during this period.

  • Regarding seasonality, both the generation and utilization of PV and wind energy showcase a pronounced pattern. During the winter, there is an 11% rise in energy demand, accompanied by a 32.2% increase in wind power and a 34.1% decrease in PV power. Conversely, throughout the summer, consumption is 9.1% lower than usual, with PV generation exceeding the average by 28.2% and wind power experiencing a 33.6% decline.

Historical data gives a clear vision of the uncertainty level of RES. For example, solar radiation shows such variation; Fig. 3 represents the variation in solar radiation between the predicted values [60] and the measured values [61] for a day in December 2020 in one of the solar belt countries, i.e., Egypt. The difference between the actual measured data and the average month’s predicted one reached 62%, which is a large error. In this paper, RES uncertainty is reviewed. Generally, uncertainty studies can be classified into long-term uncertainty and short-term uncertainty.

Fig. 3
figure 3

Solar radiation predicted model versus measured

Full size image

The generation of electric power introduces uncertainty in power systems, which can lead to potential issues with system stability and dependability. Numerous research studies have been dedicated to understanding uncertainty in solar energy since 1963 [62]. These studies have addressed various aspects, including the long-term performance of flat-plate solar energy collectors [62], diffuse radiation on horizontal surfaces [63], predictive models for solar collector energy supply [64], and the impact of clouds on solar radiation [65]. These investigations have collectively advanced our understanding of solar energy’s reliability and potential applications. Uncertainty in power systems exists across a wide range of timescales, spanning from just a few minutes to several years, depending on the fluctuation rate and the specific study objectives. Researchers have classified uncertainty studies into two main categories: long-term uncertainty and short-term uncertainty [66].

Long-term uncertainty in RE pertains to the challenges and uncertainties faced when undertaking planning, sizing, and investment decisions for RE projects over extended periods, typically from days to years. Various factors, including technological advancements, environmental considerations, and social factors, influence long-term uncertainty. The utilization of long-term uncertainty and variability proves beneficial in several applications, such as load scheduling [66]. This aids grid operators in enhancing reliability and load balancing. It also finds application in managing microgrids [67] and sizing RES [68]. In long-term studies, probabilistic approaches are commonly employed, relying on historical data and utilizing Probability Density Functions (PDF) to represent uncertain parameters like wind speed patterns, often described by a Weibull PDF [69]. Moreover, cloud cover index in solar energy studies [70]. However, it is essential to note that this research focuses on real-time applications necessitating short-term investigations, distinct from the scope of long-term probabilistic studies.

5 Short-term uncertainty methods

Short-term uncertainty significantly influences both investment decisions and the operational flexibility and reliability of power systems [71, 72]; neglecting variability and uncertainty in decision-making can lead to inefficiencies and unrealistic outcomes, as uncertainty is an inherent aspect of all planning problems [72]. To effectively model short-term uncertainty in renewable energy sources, several approaches can be considered, such as robust optimization (RO), information gap decision theory (IGDT), Fuzzy chance constraint programming (FCCP), machine learning (ML), Interval analysis (IA), and various other methods. In the context of short-term uncertainty modelling for renewable energy sources, selecting a suitable technique requires thoroughly examining the specific attributes of the energy source, the availability and reliability of historical data, and the desired level of accuracy. The primary objective of these modelling methods is to capture the effects of variable inputs on system outputs accurately. A succinct overview of the prevalent approaches employed for short-term uncertainty modelling is presented here. Furthermore, Table 2 offers an in-depth assessment of the advantages and disadvantages of each method, providing a comprehensive understanding of their capabilities and limitations.

  1. A.

    Robust optimization (RO)

Table 2 Short-term uncertainty methods comparison
Full size table

Robust optimization is a resilient approach for modelling uncertainty in renewable energy systems. It is particularly suitable in scenarios with insufficient accurate data available about the sources of uncertainty, such as weather variations, market pricing, or equipment failures [73]. This method involves formulating a mathematical model that accounts for unknown characteristics related to renewable resource availability, like solar irradiance or wind speed. These unknown parameters are represented as uncertainty sets. The critical aspect of robust optimization lies in selecting appropriate uncertainty sets within which the values of the unknown variables must fall, thereby mitigating the need for a probability density function (PDF) [58].

The primary goal of RO is to optimize decision variables, such as capacity planning or dispatch techniques, to achieve specific performance metrics, like cost and reliability, while minimizing the influence of uncertain parameters. As a result, RO exhibits high computational efficiency in decision-making problems, making it an efficient method. Researchers have utilized robust optimization to model microgrids, considering uncertainties to ensure economic viability and operational reliability [74]. Furthermore, it has been utilized to solve dependability issues caused by many uncertainties, such as the adequate capacity of renewable energy systems and a 1-min power fluctuation rate [75]. Robust optimization methods are categorized into four forms based on the type of uncertainty sets chosen: box robust optimization, ellipsoid robust optimization, polyhedron robust optimization, and budget robust optimization. Each variant offers distinct advantages and trade-offs in handling uncertainty in renewable energy systems [55]. The common set used for RO is:

$${\text{minmax}}J \left(u,w\right)s.t.\left\{\begin{array}{c}G \left(x.u.w\right)\le 0\\ u\in U\\ w\in W\end{array}\right.$$

where \(J \left(u,w\right)\) is the objective function, \(G \left(x.u.w\right)\) is a constraint condition, x is a system state variable, and U and W are decision and random variables, respectively.

  1. B.

    Interval analysis (IA)


IA shares similarities with robust optimization as both methods rely on upper and lower uncertainty bounds without necessitating a complete probability distribution. IA represents uncertainty using interval numbers, making it suitable for scenarios with numerous unknown components and limited statistical data knowledge [76]. Reports that the normal distribution effectively models active and reactive load uncertainty, while [73] recommends IA for addressing load uncertainty modeling challenges. In this context, uncertainty in electrical energy system demand response and wind power is analyzed, and a multi-cycle coordinated optimization operation plan model for an integrated energy system is developed using interval optimization theory [77].

  1. C.

    Information gap decision theory (IGDT)


IGDT is an approach to tackle uncertainty when historical data is absent and conventional methods like PDF, Membership Function (MF), or interval estimation of uncertain parameters cannot be applied. IGDT is categorized into two models: the risk-averse model and the risk-seeking model [58, 69]. In [78], IGDT is proposed to address uncertainty in market prices [73], power generation, and load uncertainty [58]. Additionally, in [79], IGDT is utilized to confront market dynamics and wind power generation uncertainties.

  1. D.

    Fuzzy chance constraint programming (FCCP)


FCCP is an optimization approach that incorporates fuzzy variables into the optimization problem. It is grounded in both probability and fuzzy set theories, making it suitable for handling multiple sources of uncertainty [58, 73]. In [80], the optimization model introduces fuzzy sets and their membership functions to express the level of satisfaction within the variable uncertainty sets. This enables modelling uncertainties related to photovoltaic (PV), wind, and thermal power using a fuzzy constraint planning model [81].

  1. E.

    Hybrid methods


When studying scenarios involving several unknown parameters or a mix of probabilistic and possibilistic uncertain parameters, it becomes imperative to adopt a combination of the abovementioned strategies. This involves leveraging various methods like the possibility-stochastic optimization technique, the interval-stochastic optimization method, fuzzy-chance-constrained programming, and other hybrid approaches. For instance, in [82], a noteworthy application of this hybrid approach was demonstrated in the study of the effects of renewable energy generators on distribution system performance. The method considered uncertainties related to load variations, renewable generation fluctuations, and the decision-making processes of owners and operators. By employing a hybrid probabilistic-possibilistic method, the study was able to comprehensively address the complexities arising from different sources of uncertainty and their potential interactions, providing valuable insights for the efficient and reliable operation of the distribution system.

  1. F.

    Machine learning (Ml)


Sophisticated methodologies like neural networks, support vector machines, or random forests can be utilized to analyze historical data and facilitate short-term renewable energy generation forecasts. These models can adapt to dynamic conditions and integrate diverse data sources, making them effective in capturing and modelling uncertainties associated with renewable energy supply’s intermittent and unpredictable nature.

Artificial Neural Networks (ANNs) have been utilized in a wide range of practical applications, including process monitoring, fault detection, adaptive human interference, natural events, and artificial intelligence, such as atmospheric processes. In [83], using ANN, Short-term uncertainty in solar power for a small PV panel is discussed at different time frames.

Notably, advancements in artificial intelligence and machine learning present promising avenues for tackling these challenges, as they enable the processing of uncertain data and offer numerous opportunities for addressing the complexities related to renewable energy forecasting [84]. Other less commonly known strategies have been suggested to address renewable energy uncertainty, including the Worst-Case Scenario method, kernel density estimation, and Hyper Heuristics [85]. Table 2 summarizes the main features of the prescribed short-term uncertainty methods. The impact of short-term and long-term uncertainties on the optimal investment option differs from an economic standpoint. Long-term uncertainties fluctuate annually, presenting the possibility for planners to benefit from waiting and observing how these uncertainties evolve. In contrast, short-term uncertainties in the model lack an annual correlation, eliminating the need for the planner to wait for their resolution [86].

6 Simulation tools

In addition to mathematical methods employed to handle unknown parameters, the practical implementation of engineering applications is closely intertwined with using software for simulation and various technologies. The selection of appropriate simulators plays a crucial role in planning and optimization. Table 3 provides a summary of commonly utilized simulation tools [58]. Apart from the EMS tools mentioned in Table 1, there are other widely used simulation tools adept at handling uncertainty, also listed in Table 3.

Table 3 Software tools for uncertainty modeling
Full size table

7 Short-term uncertainty handling

Effectively addressing short-term uncertainty in the RE and demand balance context requires integrating diverse financial, technical, and operational strategies. The following approaches have been identified as crucial for the efficient management and mitigation of uncertainty:

  1. A.

    Storage and backup systems


Introducing a specific variable becomes imperative to account for discrepancies between actual and forecasted figures. Incorporating a storage system has emerged as a widely adopted technique to stabilize demand effectively. This system consists of two components: the primary storage, responsible for supplying electrical energy during renewable power failures, and the reserve storage, promptly utilized to address uncertainties causing disparities between demand and generation. Reserve storage typically exhibits a relatively minor capacity [94], and its economic effectiveness has also been proven throughout the life of the project by comparing the scenario of 100% renewable energy with the scenario of diesel generators for the same load profile. The study showed that the hydrogen-battery hybrid energy storage system was the most cost-effective scenario, with a levelized cost of energy (LCOE) of 0.28/kWh and a net present value (NPV) of 2.3 million. During natural disasters, such as hurricanes, earthquakes, and floods, it is imperative to design and operate backup systems for microgrids. These backup systems play a crucial role in covering essential loads within the microgrid, contributing to the preservation of their resilience. Many studies and projects have investigated the resilience of microgrids to natural disasters, employing a variety of approaches and measures. For example, in [95] suggests a customized site-specific assessment of resilience strength for unique microgrids based on their ability to absorb, restore, and adapt to changing conditions.

  1. B.

    Demand side management


Demand Side Management DSM encompasses a range of techniques, including load shifting, demand response programs, and innovative grid technologies aimed at optimizing energy consumption patterns based on the availability of renewable energy. This approach synchronizes consumer demand with RE generation, reducing grid instability [86].

  1. C.

    Surplus power


In addressing short-term uncertainty, enhancing the capacity of RE generation presents a potential solution. For instance, in [96], transitioning from a diesel generator-based feeding system to a PV and storage system required the PV power to be 4–8.5 times the load, contingent on the type of storage system employed. In conventional grids, short-term uncertainty is typically viewed as contributing to power system reliability and can be effectively mitigated through control mechanisms applied to classical thermal generated power. However, in 100% RE grids, ensuring grid stability and reliability relies predominantly on the continuous rescheduling of the load, generated power, and stored energy. This is primarily due to the limited penetration of fully controlled generated power in such systems. As a result, the dynamic management of these variables becomes essential to maintain grid stability and reliability in a 100% RE grid.

8 Conclusion

The global review accentuates disparities in emission reduction efforts, emphasizing the critical need for enhanced global collaboration to achieve effective and equitable reductions worldwide. Microgrids, powered by renewable energy sources and storage, emerge as a preferred solution. Their ability to function independently or in connection with the primary power grid addresses the inherent variability and uncertainty linked to renewable energy, significantly reducing the risk of blackouts and enhancing overall power supply reliability.

Nevertheless, challenges persist, particularly in real-time energy management systems facing short-term uncertainty. Developing robust and adaptive systems capable of navigating dynamic conditions in microgrid environments presents an ongoing research challenge. The modeling approaches explored in this review have demonstrated their value in understanding the dynamic behavior of Renewable Energy Sources (RES) and microgrid components. While advanced modeling techniques have proven essential in predicting system performance and optimizing microgrid design, further research is warranted to enhance the accuracy and granularity of these models. This involves capturing complex interactions between different RES and considering energy generation’s spatial and temporal variability.

The examination of real-time energy management systems underscores their significance in achieving efficient energy dispatch, load balancing, and grid stability within RE microgrids. However, research gaps persist in developing adaptive and intelligent energy management systems capable of dynamically responding to changing grid conditions, ensuring seamless integration of RES. Additionally, integrating demand-side management strategies into real-time energy management systems requires further exploration to maximize the potential of demand response and energy efficiency.

Short-term uncertainty, a significant challenge addressed in this review, stems from the intermittent unpredictability of RE generation. Promising tools have emerged to handle uncertainties effectively. Still, further research is needed to refine these methods and develop hybrid approaches that balance robustness and computational efficiency.

The 100% RE microgrids field exhibits several research gaps, necessitating comprehensive case studies and real-world implementations to validate feasibility across diverse contexts. Standardization of evaluation metrics and performance benchmarks is crucial for comparing and replicating research findings, ensuring a more systematic assessment of microgrid projects.

Regrettably, it is observed that existing literature on real-time energy management, specifically considering short-term uncertainty, tends to primarily assess the accuracy of unmet load percentage and served load profit during operational scenarios within limited time frames. Notably, there appears to be a gap in evaluating energy efficiency and sustainability measures, despite their significant relevance and importance in enhancing overall system performance.

Despite being often accorded less importance in power system research, short-term uncertainty is a primary characteristic of RES, demanding extensive investigation for the feasibility and viability of 100% RE grids. Comprehensive research on uncertainty becomes imperative to foster the development and effective implementation of 100% RE grids.