1 Introduction

Decentralization in the electricity sector is a major step in the spread of renewable energy sources that can reduce dependence on fossil fuels [56]. Global growth of photovoltaics (PV) and wind power in recent years has been 4% and 7%, respectively. The average increase over the past 5 years reached 27% PV and 13% wind [37, 80, 109, 116]. Variable renewable energy (VRE) has differences, in various ways, from conventional generation. There are six main characteristics of VRE generator output, such as: the main resource has variable, small and modular VRE generators, which are different from conventional generators and are non-synchronous and an unpredictable type of VRE, although there may be low costs in the short-term [5, 50, 59]. These characteristics can create various challenges to the existing power system. In this case, power system performance characteristics can be affected because of some predefined challenges, e.g. the capacity for transmission line loss or inadequate generation. In addition, the inability of portfolio generation available for matching the demand for power to the needs at any time [11, 31, 39, 40, 63, 88, 113, 129].

Existing energy technologies can be used to overcome these challenges. In this case, modification technology and renewable technology can reduce some of the effects, such as the expansion of transmission networks and centralized or distributed storage devices. Integration of VREs connected to power systems requires technological solutions to achieve the decarbonization target. However, the application of a technology can cause complications caused by three main factors. First, technology choices include the implicit or explicit application of the costs, and the maturity and technological preferences of policymakers as well as companies [46, 90, 95, 115]. Second, the decision on a specific solution technology is not via a single entity but rather several actors, such as utilities, system operators and regulators [57, 66, 94, 124]. Finally, designated technologies vary by region including the VRE share of generator portfolios or individual power configurations for interconnected island systems [21, 69, 82].

From the opinions of several practitioners and researchers on energy transition, we can say that there is not enough transparency on the scope of the technologies to overcome these challenges [53, 60, 75]. The individual analysis offered by some proposes specific technologies, e.g. voltage management solutions for networks distributed through VRE penetration [70, 77, 98, 131]. However, there are several technologies presented in this paper that have the potential to overcome broader challenges such as battery storage. In addition, scenarios for investigating the deployment of specific technologies to increase storage and transmission capacity have also been discussed [33, 49, 101]. However, from several studies, the substitution effects of different technology solutions are very rarely considered. Other studies focus only on some aggregate challenges, especially the challenges of flexibility [10, 74, 81, 84, 110, 118]. However, challenges are defined at an aggregate level such that they do not necessarily lead to a particular solution technology. While some technology solutions and individual challenges might be known, some of the available literature does not provide a transparent picture. It is very important that decision-makers and researchers alike are aware of these factors when considering energy transition. When so informed, they will be better able to determine the road map and strategy on technology for the development of power system plants.

Renewable energy technology is widely covered in the literature and clearly various challenges still exist. The review carried out in this study aims to map the challenges of VRE by describing what technology solutions are appropriate to overcome these challenges. The approach taken in this paper is the analysis of data from the literature used to compile and map the list of technology solutions and challenges based on their interrelations, and to identify any lack of consistency and classify challenges to VRE. This approach aims to distinguish the observed symptoms, e.g. performance characteristics that change. Furthermore, this analysis is complemented with information from several experts to strengthen and ensure more accurate results. The findings on challenges and their linkages to technology solutions are also discussed. The relevant implications for policymakers and companies are presented in the next section. The main contribution of this review is to provide up-to-date information and useful knowledge in the deployment of RET so that energy access across the country can be improved. The systemic approach within an RE framework for information on important components of the RE ecosystem is a feature of this article.

The outline of this paper is as follows. Part one is an overview. Part two describes the materials and methods used. Part three gives the results and discusses the review and analysis regarding RET. Part three presents the findings and solutions of RET in detail. The final part is the conclusion.

2 Materials and methodology

2.1 Collecting challenges and technology solutions

Analysis of the challenges and technological solutions contained in this study were collected from literature published in journals, conferences and from some institutions in the English language. The samples analysed in this paper were mostly collected from internationally recognized journals and sources from established publishers such as Elsevier (Science Direct), Springer, Wiley, etc. [13, 38, 117] and from various online websites published by several official government and private institutions and research institutions. The journals analysed and reviewed in this paper contained 132 articles deemed relevant to technological challenges and solutions, especially for renewable energy.

The literature review conducted in this paper is divided into several categories to map various technological challenges and solutions comprehensively. The first category reviewed related to challenges and technological solutions from a systemic viewpoint, looking at the differences between systematic studies that focus specifically on technological solutions and challenges as well as other foci relating to VRE in an integrated manner in certain areas such as islands or villages. Reviews relating to market share issues or regulations are set in perspective from a technological or operational solution integrated directly with VRE. The final category analysed is the basis for extracting technological solutions and challenges. Studies relating to perspective technology and operations are used to eliminate ambiguity for the identification of challenges. This is due to dependence on fundamental technical phenomena. Various sequential effects in increasing the yield of VRE penetration have been reported in several studies [35, 71, 97, 120]. This is done because it does not have the marginal cost that is important to the challenges of integrating renewable energy. However, the ambiguity of challenge that is defined on the economic perspective has a lower spot price so that it is following the wishes of the community in perspective. To define various technical challenges including generation, it is inadequate to adjust ambiguity because it has potential effects that are not desired by stakeholders. For example, the selection of problems, in particular, is not an institutional or organizational challenge. As such, it is very easy to overlook storage from a technical point of view. Organizations or institutions that have changed are in fact steps for technical reconfiguration. In addition, it can increase more than one market share for technology solutions to power systems.

Integration of challenges and technological solutions collected and analysed from a variety of literature is a function as well as interview input for further research processes. This challenge is not tangible, in this case, the description and the words conveyed have differences. First, the challenges are collected in a long form, then iteratively collected and repeated. The technological solutions collected are determined with two requirements, first; independently this technology must be able to mitigate one another and automatically the challenges are integrated directly into VRE. Such requirements are very necessary to prevent the grouping of sub-technologies used as technological solutions. One example of sub-technology is Smart Meter, which is very possible to respond to requests as needed. However, it cannot independently reduce challenges that are integrated directly with VRE. Therefore, it is important to classify responses to requests for technological solutions, however, not for Smart Meters. As for the second category, it is done to define technology solutions based on their respective functions as explained by [16, 76, 93]. Thus, the exclusion of technological solutions can gradually be helped by the differences between one another. Given the example of the request-response, the main function of this technology is to reduce power at certain times and devices. However, response requests are operated on different devices, for example, electric heaters and heat pumps so that different technological solutions cannot serve similar functions. This study develops the challenges and technological solutions based on the various literature reviewed. The identification of all interrelated technological solutions is described with specific challenges.

The list of challenges as explained earlier will be refined with literature and reviews relating to challenges according to their level and challenges related to overall causality (Table 1). The relationship between the challenges and the technological solutions analysed shows that the two are mutually exclusive. Therefore, the analysis methodology applied in this study aims to find out the causes, management tools and the standard tools. Besides, the purpose of applying this method is to identify the main causes of certain problems and events as the root causes [14, 36, 112]. Categories with failure modes on micro-networks that can be used to find various errors and resolutions are found in the method [34, 48, 52]. The method is applied to identify the increasing symptoms of penetration of VRE collected from various literature. The symptoms analysed represent various effects that have adverse effects on performance characteristics for the power system. The identification of challenges found in the literature is then mapped based on the symptoms of each specific VRE characteristic that is the root of the problem.

Table 1 An overview of interviews by experts

3 Result and discussion

3.1 Defiance

There are eight categories of problems in increasing VRE penetration found in some of the literature as shown in Table 2. Furthermore, the problems that have been identified were divided into four main categories as requirements for basic performance for power systems. The dominant performance requirement for end consumers is one of sufficient power quality. This power quality consists of a continuous and uninterruptible power supply with a steady-state of voltage and current. In addition, if there is an instant matching, it is better to stay awake and safe. The basic category of VRE can be responsible for power quality challenges that include the modularity of the VRE generator and the fact of dissonance. Furthermore, the flow was categorized as transmission and distributed power efficiency. Multiple stream categories were the cause of the challenge compared to the other categories. Modularity, location constraints and VRE were the biggest part of the flow of challenges. The frequency of controls and challenges was categorized as stability to the power system to restore the system after a blackout. The cause of the stability of this challenge was due to the modularity of the VRE generator and the synchronization of the generator. The relationship between the challenges with the balance of supply and demand for active power in the short and long term of the system was categorized into power balance. This included a wider coordination system of speed capacity in the power system to the generator and ramp to a minimum. The main cause of the challenges was the uncertainty and variability of VRE. The main problem from the results of the analysis has given a bottom-up challenge category that was consistent by adjusting the problems contained in the power system to increase VRE penetration. A detailed review of the interrelated challenges between VRE characteristics and challenges is the basis of the review in this paper.

Table 2 The categorization of challenges according to the root cause analysis

The results of the analysis of the main problems contained in an electricity network problem that includes a mismatch of demand and electricity supply are shown in Fig. 1. Schematic description of the analysed problem was categorized into five chains, i.e. the causal effects of different VRE characteristics. Further analysis was carried out to ascertain the level of detail of each so that the problem can be resolved as quickly as possible before the selection of challenges interrelation analysis. Demand and supply that do not have in common certainly have a variety of different reasons besides increasing VRE penetration. For example, delivery limitation from nuclear power plants and coal is one of the reasons because the power system is less flexible [74]. However, the main focus of this paper discusses the challenges and integrated technological solutions and causes of the connection to the increased VRE penetration. The main problems analysed are eight causes caused by the increased VRE penetration as summarized in Fig. 1. A list of the challenges that has been summarized includes descriptions and categories of each as well as the symptoms observed and references as shown in Table 3. Twenty six challenges have been identified as a whole and most of them are challenges related to power system stability and power flow.

Fig. 1
figure 1

Analysing the root cause to balance challenges

Table 3 Result of VRE Implementation Challenges

3.2 Technologies of Solutions

Categorical and technological solutions and challenges are generally not specifically available in the literature. This is because most categories are implicit and have differences in the focus of each research. The study of power systems are flexible such as technology that can consume and produce power actively [25, 97]. Meanwhile, research on electricity networks tends to focus on technology for power distribution and transmission only ([99, 100]. Technology solutions that are comprehensively registered are not included in the technology identification as reported in the study [63]. Categorization of technology solutions is determined such as transformation in the energy sector and conclusions with a higher level. Research on top-line classification using two characteristics assigned to technological solutions has been reported by [54]. Transformations in the energy sector that lead to distributed or centralized systems are characteristics as reflected in the literature [19, 22, 26]. Therefore, the difference between distributed and centralized technology solutions can be used at a higher or lower level of system challenge. Technology with one side of generation and transmitted technology that is distributed with the other side can be categorized into the second as reported in several kinds of literature. Technology flexibility can be classified as technological solutions such as technology that contributes to system flexibility producing or consuming active power or better known as grid technology that is also classified as a technological solution. The characteristics of technological solutions can be divided into four groups through two assignments. The group which is categorized as two assignments includes a description, e.g. potential applications and solutions for each technology solution as shown in Table 4. Twenty one technology solutions have been identified; 10 of which are distributed technology solutions, while the remaining 11 technological solutions are centralized. Besides, 21 technological solutions are also distinguished from the flexibility and grid technology systems. Whereas, there are 8 flexibility technologies and 13 grid technologies.

Table 4 Solution VRE integration technologies

Grid technology is considered more attractive than flexibility technology because grid technology can serve both centralized and distributed systems. An estimation solution in a grid distribution system can estimate or measure a particular grid area. While responding to requests to serve multiple applications can be done with technology flexibility. Centrally distributed and distributed technology systems are very similar when they were first seen. However, more closely, the design between the two shows the difference. Where the ability to serve the application is distinguished from the operator and the owner himself. This difference is illustrated in the case of a stored and distributed system. On the other hand, storage with a distributed system is generally a battery unit installed at the household level with a closed state. Optimized independent consumption of these units is generally found in households, e.g. end consumers or stand-alone. While centralized storage systems such as water pump storage units or batteries are connected. The purpose of this application is for a short period during peak periods or to maintain the system’s power stability. Whereas centralized distributed storage is generally found in the operator or utility system.

3.3 Interrelationships between solutions to challenges

After completing the identification of technological solutions and challenges for integrated VRE, an analysis was carried to overcome the challenges as shown in Table 5. Challenges contained in the scope of solutions can ignore the number of technological solutions so that defined challenges can be addressed. Successful solution spaces are identified as illustrated in Table 6. Where the potential solutions contained in technological solutions that refer to several challenges can be addressed as quickly as possible. Because the space and potential of qualitative solutions are numerical comparisons and very limited to be used. Observation matrices made from the perspective of solutions such as high potential solutions and overall challenges are technological flexibility. VRE generators and distributed conventional generators that have a high level of potential solutions are included in the flexibility technology group, for example, large conventional generators with low potential solutions and conventional generation. Furthermore, distributed technological solutions tend to be higher compared to centralized systems. However, distributed grid technology has special exceptions especially for limiter or harmonic filter devices. Finally, the unique value that grid technology has on specific challenges include direct current control systems that have high voltage (HVDC) and power flow that can accurately solve problems such as long transmission distances. However, these challenges can generally be addressed by utilizing flexible technology.

Table 5 The relationship between challenges and technological solutions to distributed technology
Table 6 The relationship between challenges and technological solutions to centralized technologies

Contributions made by the solution technology to solve the challenges are described in Tables 5 and 6. Challenges that are local and site-specific have a narrower scope because the solution can only be done by the distributed solution technology, modified distributed VRE generators or additional technology solutions, e.g. harmonic filter. The whole technology group can solve various flow challenges, except technology-centred flexibility that has limitations in solving flow problems. The difference in solution space is included in the category of flow challenges starting from a narrow space to a wider space. The challenge of stability can be solved by a system technology solution by controlling at the system level centrally. Thus, the challenges of flow and distributed technology networks cannot solve challenges to stability, unless the system level can be aggregated. Stability categories such as challenges have wider solution space; however, systems in control interactions cannot be improved. To be able to balance, challenges can only be done by flexibility technology so that existing challenges can be tailored to the needs and active power consumption, excerpt for the increase in the more important VRE estimates. In general, the challenges in the balance category have a wider solution space than the availability of generations in the long run.

Three insights are very important in integrating VRE and decarbonization for the energy sector. The first process discusses two insights for overcoming integrated VRE challenges, e.g. a different power system. The last insight illustrates the results of research that can improve policymaking in the energy sector transition. Solution space for different challenges is the first point, while earlier observations are made for several types of technology that can solve specific challenges. However, the intuitive analysis of the results of expert interviews shows that business people and policymakers are not very familiar with the technological solutions that can be used to solve certain challenges. It is very clear that this technology falls into different categories. However, the development of different solution technologies can reduce the economic viability of a single technology and diminish market potential. Contributions in the decline in market price levels have a relationship with the things mentioned above. This is the same as the balancing power market in Germany. In this case, storage institutional frameworks, increasing VRE forecasts, changing demand responses simultaneously can significantly reduce market prices [43, 51, 87].

An illustration of the balance and challenges of stability can be used further as an example. The results of the interviews with experts clearly show that each different technology category can function as technology e.g. request responses available only focus on a centralized solution. Therefore, large scale and conventional generation are competitive technologies. However, the distribution of technological flexibility is not focused on analysing the more competitive technological landscape. This can be said as a prominent relationship to the potential influence of grid technology on technology flexibility, e.g. VRE estimates that increase significantly. This is because the size of the market is reduced to the demand response and storage technology. Technology like this, in general, can be used as a counterweight to a certain size of the market by looking at the quality of market participants. Lack of knowledge of technology and its groups is the main reason since competitive technology can be used for decision-making information for processes in a smoother energy transition.

The distribution of solution technology portfolios in each region for VRE integration contained in the literature seems to be very generic. Thus, the guidance given to companies and policymakers always fails to develop business policies and strategies. For future decision making, it can be assisted through an interrelation matrix such as preparing proposals and technology roadmaps both nationally and internationally. This aims to be able to decarbonize the energy sector. Interrelation material functions to match each category as well as some of the history of each country. Every quality challenge has occurred regionally for high distributed VRE penetration so that the spread of flexibility is needed especially distributed technology networks. Countries with a high penetration of VRE generators are southern England, southern and northern Italy and southern Germany [109]. Although the availability of data spread flexibility is not available for distributed technology networks in certain regions, projects such as the RD&D smart grid are technologies with very high priority for policymakers and companies in these countries [24, 28, 78]. The challenge of flow for the transmission rate reached by countries such as Germany, in general, requires a technology system with a centralized network. Such systems, such as transmission networks or amplifications, must be expanded, active power control and HVDC transmission systems. Germany is currently preparing several large projects that can be utilized by using technology. This is done after the assessment phase in determining the design and size of the complex installation has been completed.

Countries such as Ireland and Spain have done similar things, both of which have faced stability challenges. On the other hand, the transmission operator system is set as the centralized controller of the VRE generator. It aims to the needs of VRE generators to support network stability [3, 102, 108]. Besides, the investigation was carried out to ease the limitation of the stability criteria. Finally, solving the challenge of balance can only be done through technology flexibility. California, for example, is a country that have difficulty of being able to maintain power balance when the sun changes night so that the VRE generation has decreased significantly [32]. To encourage investment in storage with more flexible generators and environmentally friendly renewable energy, the State of California has introduced several new products [2, 29, 30]. Thus, interrelation matrix can be concluded that its function can be carried out by business people and those who make policies in identifying solutions technology groups. Finally, the challenges that are prevalent in certain areas can be reduced and the formulation of steps and policy strategies in supporting the dissemination of technology can be easily carried out.

Frequent debates between actors to prioritize technological solutions in VRE and irrigation management in the energy sector have often been carried out. Priority for technology solutions in integrating VRE with costs and ease of implementation is reported by several researchers ([21, 35, 99, 100]. This perspective has short-term benefits, also, the potential solutions that are perpetuated from this perspective are differences in facing challenges. Technology solutions are prioritized based on their respective solutions so that technology flexibility can be used as a solution to the challenges of VRE. This is as stated by experts in supporting the potential of technological flexibility ([99, 100, 126]). The results of the analysis can support the call for decision-makers adjusted to market rules or the placement of newly applied policies. Remuneration schemes for reactive power are introduced in the regional market. However, technology ratings are determined solely based on their respective potential and do not take into account other technological solutions that contribute to solving challenges. Besides, the solution space is different among all challenges. To consider these factors, the ranking of technologies can be adjusted to their potential in solving challenges. The preference for the deployment of this flexibility technology is specifically found in distributed and centralized VRE. Protection strategies with appropriate equipment can solve specific challenges, and higher interests can be achieved by the following perspectives. Response to requests both small and large is part of the technology solution. In addition, there are large generators with lower priority because of the limitations of the potential for more unique solutions. Relevantly to distinguish VRE integration, there are two examples large, small demand response spreads and large flexible conventional generators. Cost savings from existing solutions can be realized in the short term. However, it is not enough to only deal with the scope of the existing challenges or potential. The aspects discussed can be assumed to confirm the benefits of the results of the analysis for policymakers as a whole.

The results of the analysis carried out have important limitations to be considered when interpreting the final results. A review of specific research on existing challenges can improve VRE penetration. However, additional challenges which are not listed in this study can also face challenges such as the electric power system. At the same time, analysis of challenges was also found in power systems with lower VRE penetration. Specifically, the analysis conducted in this study is a challenge that is directly related to technology solutions. This analysis does not measure one technology solution that can solve only certain challenges. In addition, the future developments beyond the scope of this analysis can be reduced, e.g. the emergence of new solution technologies that can change frequency stability criteria or more robust end-user equipment such as variable frequency drives. Furthermore, the specific costs of the solution technology, the urgency of the challenges or the feasibility of implementing the solution technology are not considered. This is due to environmental constraints such as high land, social areas such as the public for receiving the final transmission line. This quantification is adapted to specific contexts with differences in power system characteristics. Furthermore, high levels of uncertainty are more vulnerable when considered such as revenue and costs than differences in applications and technology solutions. This need is needed for the need to think in grouping portfolios or technologies that focus on the completion of integrated VRE.

4 Conclusion

Specifically, the review in this research is to study the integration of VRE systems that are connected with modern power systems and technology to overcome challenges. Besides, the need for power system technology in increasing VRE market share with complex integration is also discussed. The collection of challenges undertaken in this study was drawn from a variety of literature relating to technology solutions in integrating VRE. The challenges developed can consistently integrate VRE which is the root problem of this analysis. The results of this analysis are supplemented by data from interviews of experts who have helped in investigations related to technology solutions and their challenges.

The results of the analysis with some insights outlined in the study can be summarized as follows

  1. 1.

    VRE integrated with challenges can affect the characteristics of the power system.

  2. 2.

    Technology solutions that vary with the number of challenges can be significantly overcome. In general, technology flexibility has a higher solution potential than the use of grid technology.

  3. 3.

    The identified technological solution facilities are intended to be able to overcome challenges in several categories.

  4. 4.

    Identification of challenges from various practice literature can be arranged and collected based on the root of the problem to produce each of the more exclusive challenge categories.

  5. 5.

    Categories and collections of technology solutions are used to test challenges that can be overcome by a single technology.

  6. 6.

    The size of potential solutions becomes very important for companies or policymakers in promoting certain technologies and their respective solutions.

Some of the descriptions presented in this review are a starting point for future research related to this topic. The relationship between technology solutions and challenges is one of the new fields of research. This is done with an estimated cost compared to the use of different solution technologies and can be introduced comparatively to the environment as a whole. Life Cycle Assessment (LCA) can be used to measure costs integrated with VRE because the installed capacity with future projections is available [41, 107, 114, 125]. This system can significantly improve recommendations on policies issued. Overall, the development of individual solutions technology that is integrated with VRE is an issue that has a high price for the transition in the energy sector in a sustainable manner. In this case, a further investigation between the characteristics of different power systems and geographies is on one side of the technology solutions and challenges with different sides.

5 Nomenclature

VRE Variable Renewable energy

HVDC High-Voltage Direct Current

RE Renewable Energy

LCA Life Cycle Assessment

RET Renewable Energy Technology

PV Photovoltaics