Introduction

Power produced from solar and wind resource-based generators is variable in nature. The inherent uncertainty and variability in solar radiation and wind speed resource cause fluctuations in power generation from these resources. When the share of such VRE becomes significant, various stakeholders’ face several new challenges in managing and operating an electrical power system efficiently, e.g. complexity in load-generation balancing within a control area [64]. Power forecasting is one key tool in this regard, which directly supports in better operation and management of electric network [100]. Generation forecasts can be at various spatial aggregation levels like control area-level aggregated forecasts, useful for net load forecasting, reserve dimensioning, ramp management, uncertainty handling, etc. [9, 10, 27, 91, 92, 112], whereas site-specific or nodal/pooling transformer station-level aggregated forecasts are useful for bidding in power exchange or ancillary service market, congestion management, reactive power support potential forecast, etc. [53, 64, 76, 85, 129, 129]. Generation forecasts can also be for various forecast horizons, minutes-ahead, hours-ahead, day-ahead, or longer-term basis [20, 73, 83]. Generation forecast has usage not only for TSO, but also for DSO in the domain of active distribution networks [46, 114. The appropriateness of the generation forecast method and input data source depends on the forecast lead time and resolution suitable for the particular application. Very short-term forecasts of the order of minutes ahead at a high temporal resolution can be obtained from machine learning [99, 106] and sky imager-based predictions [74, 128]. Machine learning techniques such as ANN, SVR, ([37, 38, 39,40,42], fuzzy logic, and wavelet approach [37,38,39,40, 42,43,44,45] have already been successfully applied in various fields. Various configurations of sky imagers have also been reported in the literature for solar irradiance applications [30, 88]. Satellite image-based forecasts are suitable for the forecast lead time of 30 min to 4 h ahead, and has been widely applied [50, 58]. However, NWP-based forecasts provide the most reliable predictions at the day-ahead stage [79]. In India, the concept of VRE forecasting has been introduced for some time and is undergoing the initial evolution phase. This paper focuses on solar power forecasting and aims at, after briefly reviewing the applications of VRE forecasting and the methods to generate this forecast, documenting the recent evolution of solar power forecasting in the scope of Indian power system and other power systems of various countries, facing similar challenges in grid integration of solar PV and wind resources.

This article is a combination of review analysis and original research efforts, and makes the following contributions, (i) It reviews the applications of generation forecasts in various decision-making processes relevant to the stakeholders in electric power system and the ongoing developments in generation forecasting regulations and infrastructure in various power systems, including the Indian power system, (ii) it reviews the various forecasting techniques and points out potential applications, feasible within the regulatory infrastructure, (iii) it presents a case study of single-site forecasting as well as aggregated forecasting for the State of Rajasthan in India, based on ECMWF NWP model output, (iv) finally, it discusses scope for further improving forecasting activities in India.

Review of forecasting applications and solar PV generation forecasting methods

Solar PV power generation introduces variability into the electricity grid. The output from such generation units have a maximum generation limit that varies throughout the day even when it is accurately predictable (under clear sky) and this characteristic of solar PV output is referred to as its temporal variability. However, even the maximum generation limit is not always known with reliability, and this aspect is referred to as its uncertainty [34]⁠⁠. This situation could arise due to the presence of cloud structures or change in aerosol content, which enhances the spatial variability of irradiance and consequently results in  the deviation of the actual solar PV power output predicted from the prediction. Solar PV power forecasting finds utility in various applications such as, UC and ED, reserve setting, residual load forecast, congestion management, active distribution network management, and VPP operation. Solar PV power output forecasts of different spatio-temporal resolutions are broadly speaking, necessary for the following stakeholders, (i) TSO (ii) Plant owner/ operator (ii) VPP operator or trader or aggregator and (iv) DSO.

Relevance of generation forecast to different stakeholders in the electricity industry

For a power system in which power generation from RE has priority feed-in, estimating the residual or net load forecast is important for the TSO. In order to generate net load forecast, control area-level aggregate generation forecast is necessary. Residual or net load forecast can be used by the TSO to schedule the dispatchable generators in order to supply the net load. It is also useful for ascertaining that sufficient ramping capability is available to compensate for the ramps in net load. The uncertainty information associated with net load can be used as an input into reserve requirement estimation [96]. Nodal (pooling station level) forecasts are useful to the TSOs for both intra- and inter-control area congestion forecast and management [57]. Power markets are also affected by network bottlenecks and hence can utilize nodal forecast information [7] for cases where power markets set nodal energy prices [63, 87]. Individual power plant-level generation forecast and its associated uncertainty can be of use to the TSO, in cases where gross pool mechanism [47] is followed or mandated by regulation, for scheduling the generators.

Steep ramps in solar PV plant power output during morning and evening times can lead to increased cycling of the dispatchable generation units, and put more stress on the power system [89]. Flexibility is necessary in order to deal with power ramps, and the requirement of flexibility also needs to be inferred from ramp forecasts or ramp alerts [86]. Many authors have described a trinity of metrics, namely, ramp rate, ramp magnitude and energy, for quantifying the ramp in power from VRE units [123]⁠.

Plant-level PV generation forecast information is useful to the plant operator for scheduling the expected generation with the grid operator. For solar PV plants participating in day-ahead power market, day-ahead generation forecast is a vital source of information for placing bids [129]. Intra-day or imbalance markets provide opportunity to the solar PV plant operators or their traders to revise their bids based on updated and more accurate intra-day forecasts [64]. Apart from single-valued deterministic forecast, forecast uncertainty information can be utilized to optimize the bids for power in the day-ahead and intra-day markets. This is especially the case when differential penalties are applied for over and under generation [23, 129].

VRE units, such as solar PV, are increasingly being expected to provide ancillary services to the grid operator, similar to conventional generating units. The ability of solar PV plants to provide ancillary services has already been documented in literature [29, 80] and demonstrated in field test [85]. Wind turbines are already being used for providing negative tertiary reserve. The role of accurate generation forecast and its uncertainty information is crucial here [53]⁠.

VPP is an aggregator of generation units, and in certain cases load entities as well. According to [107]⁠, “a VPP aggregates the capacity of many diverse DERs, creates a single operating profile from the composites of the parameters characterizing each DER and incorporates the impact of the network on aggregate DER output. A VPP is a flexible representation of a portfolio of DER that can be used to make contracts in the whole-sale market and to offer services to the system operator”. [105] defined VPP as “a portfolio of DERs, which are connected by a control system based on information and communication technology. The VPP acts as a single visible entity in the power system, is always grid-tied and can be either static or dynamic”. Individual and aggregate forecast information is necessary for VPPs to provide optimal bids in power markets, as well as for providing ancillary services to the grid operator [56, 72, 109]⁠.

Historically, distribution system has played a passive role, functioning only as the interface for the transfer of power from transmission network to end-consumers. However, with the addition of numerous distributed generators such as grid connected rooftop PV to the medium- and low-voltage distribution grid, passive operation is no longer possible. This can in turn pose barriers to the secure and economic operation of the distribution grid [46]. There is an ever-growing need for an intermediate entity between the TSO and the DERs, due to the limited visibility and control of TSO over these resources. Two primary roles that this entity (called DSO) is expected to fulfill are: (i) operator of the distribution grid and (ii) Facilitator of market access to DERs [5]⁠. One of the main challenges is to develop suitable tools and systems, which would allow DSOs to make informed decisions based on data from various sources [60], including generation forecasts, in order to fulfil their various roles. Voltage and current limitations also act as obstacles to the distribution grid’s PV hosting capacity and is the primary cause of overvoltage during high solar PV generation at noon-time [120]. As can be seen in the relevant literature [1, 15, 67, 118]⁠, several authors are of the opinion that DSOs need to start scheduling their system (including demand, generation and storage), and also manage voltage and power flow violations with the help of available resources like OLTCs, advanced inverter control techniques, capacitor banks, etc. However, for all these cases, generation forecast is the basic building block based on which the operational planning is done.

Solar PV generation forecasting techniques

Solar PV power forecasting comprises of several steps which convert NWP output, satellite images, real-time online measured data or sky images into a forecast product, by utilizing various models. The initial set of steps for deriving the forecasted surface GHI depends on the source of data and the required spatio-temporal resolution. Further steps are necessary for producing the final AC power forecast as shown in Fig. 1. The measured data of GHI and AC power output are often used for site-adaptation in order to enhance the accuracy of the prediction in real operating systems. Brief descriptions of the different solar irradiance forecasting, forecast combination, and up-scaling techniques are provided in the following subsections.

Fig. 1
figure 1

Simplistic steps involved in deriving AC power generation forecast from forecasted GHI at the surface

Full size image

Solar irradiance forecasting techniques

GHI forecasts may be generated, depending on the forecast horizon, by several methods. NWP model output, satellite image-based CMV, and statistical or machine learning methods are used worldwide commercially for generating forecasts. Application of CMV methods on images captured by ground-based cameras for sub-hourly forecasting at high spatio-temporal resolution has shown promising results [19, 103, 115, 117, 124]⁠. Figure 2 compares the spatio-temporal horizon of different solar PV forecasting techniques.

Fig. 2
figure 2

Different solar PV forecasting methodologies and their spatio-temporal horizon [13]

Full size image

⁠NWP-based solar power forecast is the only physics-based technique available for generating day-ahead to days-ahead forecast at present. NWP models predict the future state of the atmosphere by numerically solving physical equations based on initial conditions obtained through data assimilation [77]. Model runs are initiated 2–4 times a day (0, 6, 12, and 18 UTC) [103]. Further post-processing of NWP model GHI is necessary in order to obtain data at the required temporal resolution and removing systematic errors or biases [22]. Spatial averaging over a predetermined optimum number of grid points is also done to improve accuracy [84, 90, 102, 126, 127]. Figure 3 shows the general scheme for temporal interpolation.

Fig. 3
figure 3

Temporal interpolation of NWP model GHI to higher resolution

Full size image

Satellite image-based CMV methods and machine learning methods are currently used on an operational basis worldwide for producing intra-day irradiation and generation forecasts. Some of the main steps in satellite image-based solar PV generation forecasting have been shown in Fig. 4.

Fig. 4
figure 4

Steps in solar irradiance forecasting using satellite image data [11, 25, 58]

Full size image

Machine learning based methods “learn” from past datasets, and then generate irradiance forecast using the trained models. There are numerous articles already existing in the literature on different “learning techniques” [93, 101, 116, 125] of which the simplest model is persistence. Advanced models, e.g. ANN, SVM, can use additional parameters such as sky or cloud image,  relevant NWP output parameters, and measured meteorological parameters [21, 108]⁠. Times series models can also be used to forecast solar PV power output, as shown in [78] and [119]. In general, machine learning is mostly preferred when information about the underlying PV system, such as tilt orientation and technology, is not available [126, 127]⁠.

Combination techniques for irradiance and PV power forecasts

Combining the results of different forecast models can improve the overall accuracy, if the errors of the different forecast models are to a certain extent uncorrelated. Combination methodologies vary from simple linear combination to more advanced statistical methods which use optimized weights for the individual forecasts considering their historical performance [8, 24, 26, 35]. When the outputs of different forecasting techniques suitable for different forecast horizons are statistically combined, then the weights are also of a time-varying nature, depending on the lead time [73].

Up-scaling of regional PV power forecast

The errors for control area-level aggregated forecasts are significantly lower than single-point forecasts [2, 81, 104, 113]. The error reduction percentage depends on climate diversity and PV system distribution within the region [73]. In [83]⁠, the authors found that rRMSE for an ensemble of 77 PV systems was less (4.6%), when compared to single-site solar PV generation forecast (8.5%).

It is not feasible to retrieve accurate data of each and every PV installation in a power system, and so a representative set of solar PV units are selected. The quality of up-scaling depends on the proper choice of representative systems. Further set of necessary information include the spatial distribution of nominal PV power, different module technologies, and the geometric orientation of the PV systems in the region [84]. Module orientation information for distributed solar PV units can be obtained from GIS data [68].

Forecast accuracy increases with larger number of reference power plants. However, beyond a certain percentage of reference plants, the advantage of increase in accuracy is negligible. The error in up-scaling is low when the configuration of the reference plants and test plants is similar [113].

Development of VRE generation forecasting infrastructure and forecasting applications

Over time, there have been broadly two approaches towards VRE generation forecast information and its utilization in electrical power system: (a) Centralized aggregated forecast at the system operator level and (b) Site-specific forecast for individual generator or groups of generators. Often these two approaches are found to coexist in power systems, with each serving the requirements of different specific stakeholders. This section discusses the regulatory and operational reforms introduced in some power systems for handling the intermittent resource characteristic of VRE sources like solar irradiation and wind speed. It then provides a brief review of various established generation forecasting infrastructures. Finally, in the light of these discussions, possible operational/regulatory adaptations and improvisations in the Indian power system are discussed.

Regulatory interventions in VRE generation forecasting

In order to better handle the intermittency of VRE generation, while allowing its economic and secure integration, regulatory reforms have been introduced from time to time in different power systems. In Germany, the EEG act accords priority dispatch to renewable generation and makes the TSO responsible for selling power from price-taking VRE generators at the energy exchange, who are then paid under the FIT scheme. Aggregate control area-level VRE generation forecasts are utilized by TSOs for grid security assessment and aggregate VRE generation bidding at the exchange. The imbalance costs of such VRE generators are borne by the TSOs. “Price-making” VRE generators on the other hand participate in the energy exchange like any other entity and require site-specific generation forecasts. They are themselves responsible for generation forecasting and settlement of imbalances [36]. The Spanish Royal Decree RD 436/2004 introduced mandatory generation forecast for special regime RES-E generators or VRE generators, of installed capacity equal to or greater than 10 MW. This limit was further reduced by subsequent regulations. At the same time, the Spanish TSO REE also produces its own aggregate generation forecast for the Spanish electricity grid [62]. Furthermore, all VRE generators of capacity greater than 10 MW need to be connected to a centralized control centre as a necessary requirement for obtaining FIT or market premium. RD 1565/2010 has also included clusters of VRE generators connected to the same evacuation line of combined capacity greater than or equal to 10 MW within the ambit of the central control centre [32]. FERC Order 764 enabled VRE generators in USA to better manage their imbalances by introducing sub-hourly scheduling. It also allows TSOs to access sub-hourly forecast-relevant data of VRE generators for the purpose of producing power production forecasts [18, 95]. The AEMO has installed a solar power forecasting system called ASEFS under a project funded by ARENA. This was deemed essential for assisting with the operation of accurate supply and demand forecast models to increase commercial viability and ensure grid stability, in view of the large penetration of solar PV into the grid [6]⁠. The need for forecasting and scheduling of VRE generators in the Indian context was first emphasized by CERC in 2010 through the IEGC [17]⁠, and then subsequently by other SERCs and FOR, who introduced similar regulations [4, 49, 51, 66]⁠. The concept of QCA as an aggregator of VRE units connected to a pooling station as well as aggregator of multiple such pooling stations has also been introduced by FOR model regulations and other SERC regulations. A detailed analysis and inter-comparison of various aspects of the different regulations related to forecasting are included in Table 1. Recently, FOR has come up with a proposal of introducing a 5-min resolution scheduling interval in the Indian power system for higher granularity and improved ramp management [48]⁠.

Table 1 Cross-comparison of different forecasting regulations in India
Full size table

Recent developments of centralized forecasting infrastructure in Spain, Ireland, Germany, and India

CECRE

CECRE was set up by the Spanish TSO REE for the purpose of coordinating the operation of VRE generators with the operation of other generators, transmission, and distribution system. The principal function of CECRE is to relay real-time VRE generation information to the TSO. Generators larger than a threshold capacity can also receive down-regulation signal to restore system balance or relieve congestion [32]. SIPREOLICO and SIPRESOLAR are the short-term statistical solar and wind production forecast generating tools of REE, respectively, which are used by them for scheduling reserve, identifying potential congestion issues, and monitoring system security [3, 61]⁠.

WSAT

WSAT is a tool introduced by the Irish TSO EirGrid for calculating the SWL on the system by assessing power system stability with network modelling and simulation [31]⁠. Aggregate system-wide wind forecasting is a part of one group of processes related to forecasting, which are performed by the TSO, for the purpose of scheduling. Forecasts are used are obtained from two external providers via the WEF system [33].

Balancing groups in Germany

VRE generators that sell their energy for a fixed price in the energy exchange, referred to as price-taking generators, constitute a distinct balancing group, and TSOs are responsible for compensating imbalances [36]. Thus, TSOs employ the services of FSPs and obtain VRE generation forecasts from them, in their effort to accurately predict and bid the volume of energy in the EPEX spot exchange market. Some TSOs also use a combination tool, which produces an optimal forecast by a combination of products from multiple FSPs [61]. Apart from this, aggregate TSO control area-level forecasts are also used for grid security management.

REMC

REMC was conceptualized due to the need of an entity that would be able to generate control area-level aggregate VRE generation forecast in order to assist LDCs in calculating net load forecasts and managing grid security, which is quite different from the site-specific forecasts mandated by forecasting and scheduling regulations for scheduling and accounting. Apart from this, REMCs are also envisaged to act as the hub for all information regarding RE power generation in its area of responsibility, at the SLDC, RLDC or NLDC level, and are proposed to be co-located with the existing LDCs [54, 97]. The conceptual architecture of an REMC (at state or regional level) with logical interconnectivity is shown in Fig. 5. Regional and state REMC’s would be equipped with forecasting and scheduling tools and real-time monitoring of RE generation. The REMC at NLDC will have a forecasting and scheduling aggregator tool. This aggregator tool will receive forecasted data. The conceptual architecture of the REMC located in NLDC with logical interconnectivity is shown in Fig. 6. The foremost purpose of the forecasting tool is to predict the output from solar and wind generators accurately and provide the information to the system operator for ensuring smooth grid operations and enhanced system security. A schematic diagram of the forecasting tool is shown in Fig. 7, while Fig. 8 shows a schematic diagram of the forecasting and scheduling aggregator tool.

Fig. 5
figure 5

Scheme of logical interconnectivity between various software modules in an REMC

Full size image
Fig. 6
figure 6

Scheme of logical interconnectivity between various software modules in national REMC

Full size image
Fig. 7
figure 7

Schematic architecture of a forecasting tool

Full size image
Fig. 8
figure 8

Schematic architecture of a forecasting and scheduling aggregator tool

Full size image

Potential applications of generation forecast in Indian power system

Many of the SERCs have made it mandatory for all solar PV plants (with a capacity of 1/5/10 MW or above) to provide generation forecasts on a day-ahead basis to the SLDC. These regulations also allow a fixed number of intra-day revision or updates based on the latest forecasts. In particular, the requirements here are for day-ahead site-specific forecast, which can be obtained from NWP sources, and intra-day site-specific forecast, which can be generated by applying CMV to satellite images. Persistence or advanced machine learning based methods can also be used for very short-term forecast. Sky image-based methods are still at a research stage, and moreover, the use of sky imaging system for solar forecast may not be currently very useful for the Indian power system, as the minimum update interval in most of the states is 1.5 h, while the sky imaging technique can only predict for 20 to 30 minutes in the future. As per the regulations [4, 49, 66], these site-specific forecasts, different from the forecasts generated by REMCs, are provided to the SLDC for setting the schedule and consequently for determining the deviation charges.

The proposed REMCs are tasked with providing the SLDC with aggregated variable RE forecasts, which would be beneficial for the overall management of the electricity grid. The requirement here is of spatially aggregated forecast at the pooling station, state and even regional level. Such forecasts can be obtained by applying up-scaling techniques to a representative set of site-specific forecasts. State- or regional-level forecasts can find application in residual load forecasting, which would then enable state or regional LDCs to schedule the dispatchable generators for supplying the residual load. The ramps in residual load, due to the diurnal pattern of solar PV generation, can also give information on the ramping capability required from the dispatchable generators. It has been shown that VRE intermittency primarily affects the need for tertiary reserve (RRAS as per Indian power system) and not primary reserve (FGMO) [28, 70]. Thus in future, the uncertainty information associated with residual load forecast can be utilized for setting the RRAS requirement. On the other hand, nodal or pooling station-level forecasts can find application in congestion forecast and management, particularly for inter-state and inter-regional tie-lines. Load flow calculations, using nodal-level load and generation forecasts, can be employed to determine the ATC of critical transmission segments. Uncertainty information associated with nodal-level generation forecast, can also be used in future for congestion management and scheduling, as has been shown by several authors [76, 71, 59]. QCA, which has been introduced by many of the Indian states, can utilize nodal or pooling station-level aggregate forecasts for providing generation schedules to the SLDC.

According to the CEA, a distributed generator is a generating station that feeds electricity at a voltage of 33 kV or below into the electricity grid [16]. Many SERCs have already brought out draft regulations or final regulations on rooftop solar PV systems and net metering [116,117,118,119,121]. Karnataka so far allows the highest penetration of solar PV up to 80% capacity of the distribution transformer level. Increase in penetration of roof-top solar PV alters the shape of the load duck curve, during the day time in particular, and thus affects the load demand scheduled by the DISCOM. To tackle this, DISCOM control centres can utilize the aggregate forecast of roof-top and distributed PV systems connected to the distribution grid. However, module information and dynamic generation data of all the solar PV units may not be available to the DISCOM directly, due to the sheer number and the spatial distribution of the units. Different up-scaling techniques can be used in such situations to obtain the aggregate output, as shown in [68, 109,110,111] . Apart from this, large number of distributed solar PV plants connected to the HT and LT levels (11 and 0.4 kV) can cause spike in voltage along the feeder line, at the coupling points with the grid. In the distribution grid, active power generated by solar PV influences the voltage at the coupling point with the feeder. However, transformer with OLTC is typically only available till EHT voltage levels (33 or 66 kV) [55]. Even then, very frequent switching of the OLTC leads to wear and tear, which is also undesirable. Several methods are available in literature which show how to utilize nodal aggregated forecasts for minimizing the switching operation, while at the same time maintaining voltage limits on the different feeders [15, 114]. Some works have also proposed active power curtailment strategy for voltage regulation by utilizing solar PV generation forecasts.

Many of the applications discussed here are already used in operation internationally, and therefore in theory can also be implemented in the Indian system. Based on Table 2, a summary of the possible applications of generation forecast suggested in this section is presented along with the forecasting methods employed, its spatio-temporal resolution, and the intended end-user. However, the details for removing barriers to their implementation need further detailed investigation.

Table 2 Summary of the possible applications of solar PV generation forecasts in the Indian power system
Full size table

Experiment with day-ahead solar resource forecasting in Rajasthan, India

The existing practices and procedures adopted worldwide for solar power forecasting have been discussed. This section evaluates first approaches of a day-ahead solar power forecasting scheme in Rajasthan.

Selection of NWP Model, ground measured irradiance datasets for solar resource forecasting

A comparative analysis of several NWP models [90] has shown that the ECMWF NWP model provides rather accurate forecast with the least uncertainty. Again, the global NWP model from ECMWF has the finest grid resolution among all the existing global NWP models. Hence for this work, the NWP model solar and cloud parameter data sets with a resolution of 0.125° × 0.125° were obtained from ECMWF for the area of the state Rajasthan. The temporal resolution of data points obtained was 3 h with a forecast horizon of 72 h.

For the investigation and the evaluation of irradiance forecasts, solar irradiance data sets were obtained from 10 SRRA stations in Rajasthan measured during the period 1 January 2014 to 31 December 2014. The distribution of these stations in Rajasthan is shown in Fig. 9.

Fig. 9
figure 9

Locations of the SRRA stations in Rajasthan used for evaluation

Full size image

Evaluation of Irradiance forecast for Rajasthan from NWP model datasets

The forecasts are to be evaluated for point locations or single sites and for an ensemble of sites with SRRA stations in Rajasthan. Irradiance values at each time step are isolated and normalized to the time interval to obtain NWP model output GHI values. The temporal resolution of the calculated NWP model GHI is 3 h.

After obtaining ECMWF GHI forecasts of 3-h temporal resolution at each grid point, averaging of forecasted GHI values over a 0.25° × 0.25° grid, i.e. over pixels surrounding the point locations is performed. Irradiance values obtained after spatial averaging have 3-h temporal resolution, which are to be interpolated to GHI values with 1-h temporal resolution.

Hereafter for point forecasts, two different techniques are used for temporal interpolation. In technique1, direct linear interpolation technique is employed to reduce temporal resolution of 3-hourly NWP model GHI values to 1 h, while in technique2, interpolation was performed on 3-hourly CSI (Kt*) values (using the Bird clear sky model) to obtain hourly Kt* values [14]. Then, hourly Kt* values are multiplied by hourly clear sky model values to obtain hourly NWP mainly adopted to account for the typical is mainly adopted to account for the typical diurnal pattern of irradiance. For multiple-site analysis, similar procedures are repeated with averaged forecasted NWP model GHI values from all point locations under study. The forecasted GHI series, obtained by employing these two techniques, are validated against acquired GHI datasets from the SRRA stations in Rajasthan [75, 122].

A comparison of the two techniques was performed by obtaining different measures of accuracy [12]. As can be seen in Table 3, a reduction in rRMSE for technique2 is observed, which signifies an improvement of forecast accuracy for the second approach. But the forecasts from technique2 could be further improved by using the MOS approach that removes systematic bias errors which depend on the time of the day and on sky conditions, etc. So, in technique3, the intention was to model the bias as a function of hourly clear sky index (Kt*) and the cosine of solar zenith, using the measurement data obtained from SRRA stations for the first 15 days of each month in 2014 as training set and validate the model using test set which comprises of measurement datasets for remaining days. Curve fitting parameters are evaluated and a good correspondence between the modelled and measured bias values is observed. Then, the improved forecast is obtained by subtracting the modelled bias values from the forecasts obtained using technique2 [81, 84].

Figure 10 shows the forecasted GHI series obtained from technique3 with the actual GHI values measured at Aburoad for a single-site analysis, and Fig. 11 compares the forecasted GHI series with averaged GHI values for a multiple-site analysis. After calculating error measures for technique3 (Table 3), it is observed that the forecasted GHI series evaluated using technique3 has a higher accuracy compared to the previous two techniques. Again, MOS updated forecasts with multiple-site analysis has a lower uncertainty compared to MOS updated forecasts with single site/local area analysis. For single-site analysis, the forecast accuracy is very good during clear sky conditions. However, the level of uncertainty increased during cloudy conditions. For MOS updated GHI with wide area analysis, the uncertainty reduced drastically because of smoothing due to spatial averaging reducing the deviation between forecasted and mean of measured GHI values (Table 3).

Fig. 10
figure 10

Forecasted GHI series obtained from technique3 with actual GHI measured at Aburoad for single site/point location analysis. Note the phase error during a cloudy sky situation on 26 May 2014

Full size image
Fig. 11
figure 11

Forecasted GHI series obtained from technique3 with averaged value of GHIs measured at multiple site for wide area analysis

Full size image
Table 3 Error analysis of forecasted GHI series
Full size table

From Fig. 12, it can be observed that the least rRMSE value of around 9% is achieved for forecasted GHI series from technique3 with wide area analysis. The least percentage rRMSE signifies highly accurate forecasted GHI series, and this result is in agreement with the accuracy values observed in [90]. Figure 13 depicts relative MBE values obtained for different day-ahead forecasting techniques adopted. Very small rMBE values are obtained for MOS techniques with bias removal (Table 4).

Fig. 12
figure 12

Percentage rRMSE values obtained for different day-ahead forecasting techniques

Full size image
Fig. 13
figure 13

Percentage rMBE values obtained for different day-ahead forecasting approaches

Full size image
Table 4 Potential development of VRE generation forecast applications in Indian power system over time [110]⁠
Full size table

Discussion

In India, there is a strong need to overcome existing challenges in regulations and available infrastructure related to forecasting of VRE, so that forecasting accuracy improves with time. Hence, various stakeholders could ultimately get benefited from accurate forecast information.

Here, day-ahead forecasts are mandatory in many states for solar power plants with size greater than a specified threshold as mandated by different regulations. The spatial resolution of atmospheric high-resolution (HRES) NWP forecasts by ECMWF are 0.1 ̊ * 0.1 ̊ (which is approx. 11.2 km *11.2 km). A 5 MW power plant occupies typically a size of 0.06 km2. Therefore, same values of GHI could be applied to different solar power plants within same grid box. However, the incident GHI could be quite different in practice. The temporal resolution of the output of NWP model is either hourly or 3-hourly, while for scheduling, the output of solar PV power plant is required at 15 min interval. Some practical difficulties are encountered while forecasting using NWP models. Currently, there is a lack of experience in NWP applications for RE sector. However, there is a growing interest in the NWP products from meteorological organizations, namely IMD, NCMRWF, and IITM and also in the space organization ISRO.

Intra-day forecasting by solar power plant operators/grid operators has been necessitated by the different forecasting regulations in India. According to the literature, forecasts based on satellite image CMV performs well for a time frame up to six hours ahead. Presently, operational solar PV power forecasting based on CMV is not performed in India. Geo-stationary satellite visible and thermal image products are available from INSAT 3D and INSAT 3DR. However, there is a need for benchmarking the different forecasting methods in Indian conditions. Interesting insights could be revealed through the benchmarking exercise, e.g. a typical meteorological condition related to convective clouds prevalent in India is its quick formation and dissolution. This is different from common weather situations in mid-latitudes where there is a mostly advective nature of cloud motion. It also has to be considered that during the monsoon season in India the dissolution of clouds is happening at a slower rate. Sky image-based forecasts are still at a research stage. Furthermore, existing forecasting regulations mandate a minimum update interval of 1.5 h, while sky image based forecasts have a sub-hourly forecast horizon. This implies that a change in forecasting regulation is necessary to utilize sky image based forecasts for commercial use. There is also a need to further explore the various machine learning and statistical techniques for application in intra-day forecasting in India. In addition to this, there is a need to combine the different forecasting techniques in order to produce optimal forecasts for a large range of forecast horizon.

SLDCs/REMCs may face certain challenges while performing control area wide RE generation forecast. There is an issue with robust infrastructure for visibility at SLDC/REMC of the production information from VRE plants. Metering infrastructure is not sufficiently robust for the measurement of production information. VRE plants in the low voltage grid are connected to the load feeders making it difficult to segregate the production and consumption measurements. Validation of forecasts is extremely important for accuracy improvement over time. The information passed on to the REMC/ SLDC’s from the individual solar PV power plants is mostly restricted to AC power production. Information regarding other important dynamic time series data, e.g. DC power, GHI, GTI, Module Temperature, Wind speed, etc., is generally absent. Therefore, developing solar power forecast model at the level of individual plants or at the control area level is challenging considering the data currently available with SLDCs/REMCs. Regulation mandating the supply of above-mentioned information to the SLDCs/REMCs will be highly beneficial.

There is a requirement in REMC’s to have forecast produced on a week-ahead basis. The requirement is mainly for planning the maintenance of thermal generators. However, there is severe limitation in the accuracy of NWP forecast for this time horizon. This hinders the accuracy of forecasts for week-ahead time horizon.

Furthermore, the introduction of hybrid plants and VPPs will also necessitate development of site-specific or aggregate forecasts for participating in power exchange markets. Forecast uncertainty information also becomes very important when VRE units are expected to provide ancillary services.

The skill sets of the power system operators have to be upgraded with respect to usage of VRE generation forecasting information. Different aspect of system operation, namely control reserve requirement estimation, ramp management, congestion management, active distribution network management, etc., are influenced by the specified forecast information. Careful utilization of forecast information is necessary for an effective system operation.

There is a necessity for better clarity of particular aspects in certain regulations on RE generation forecasting in India. Many regulations in India stipulate that generator specific forecasts from SLDC are required. SLDC forecast should solely be for the purpose of grid security. Therefore, forecasting the individual output of every RE generator is not required for the above and would burden the SLDC unnecessarily. There is also a necessity for harmonization between inter-state and intra-state regulations. At present, most of the power plants are connected to the intra-state network. In the future, if the same power plant delivers power to both inter-state and intra-state networks, it has to deal with two separate accounting systems. Aerosol is expected to be a major factor affecting solar PV power generation. Therefore, R&D efforts have to be initiated to quantify the effect of aerosol on solar PV power output forecasts.

In future, the applications and utilization of VRE forecasting in India should increase significantly. There could be numerous other applications of generation forecast apart from the control area wide net load forecast and RE generator scheduling, which are currently envisaged. These include bidding in power and ancillary service markets, ramp forecasting, dynamic reserve dimensioning, grid congestion forecast, voltage management, etc. There has been an upward trend in the recent years of the total electricity traded at the power exchange. With the evolution of smart grids in India, VPPs may come into existence in the future. VRE generation forecasting is an essential part of VPP operations. The proposed QCA is similar in principle to VPP, although VPP can aggregate diverse types of resources and is not restricted to only VRE units. Currently, there is no concept of DSO in India. With major thrust given on roof top PV and the increasing complexity of distribution network, DSO might come into existence in the future. Forecasting is an essential functionality of DSO in order to prevent congestions in the network.

Conclusion

In this paper, existing literature on various applications of VRE generation forecasts and solar PV generation forecasting methods by which the generation forecast products suitable for these applications can be obtained have been discussed. Regulations pertaining to VRE generation forecasting and the ongoing development of centralized aggregate forecasting systems have been highlighted. Based on the ongoing research developments, potential applications of generation forecasts have been suggested for Indian power system from existing scientific literature. Further detailed study is necessary in order to overcome the challenges related to their practical implementation. Results of a research study on site level and regional aggregate solar irradiance forecasting, conducted for the state of Rajasthan, using ECMWF NWP data have been presented and the benefit of aggregation can be seen in the improvement of bias corrected accuracy from 17.29 to 9.53%.

Accurate VRE forecast is an important factor among others, e.g. control reserve, flexibility, market design etc., for facilitating integration of VRE into the electricity grid. In India, PV power forecasting is a new practice, and therefore, it would take few years to gain maturity in terms of technology, applications and uniform effective regulations. Today, there is still a lack of scientific experience in performing solar power forecasting for different spatio-temporal scales in India. Robust infrastructure for VRE forecasting will assist the Indian Government in achieving its RE targets. REMCs are an important initiative that would help integrate more RE into the electricity grid. Continuous improvement of forecasting accuracy over time can be achieved by synergetic activities of the relevant stakeholders. CERC, FOR and many SERCs have already brought out draft, model or final regulations on VRE forecasting. Intra-day forecasting techniques such as satellite and sky imager-based methods, are yet to be fully explored in the Indian context. However, it is expected that with the improvements in day-ahead and intra-day forecasting techniques, the accuracy of forecast will also improve considerably in India. This has positive implications for the secure, economic and sustainable operation of the Indian power system.