The impact of the relaxation experiments described in the previous section on East African long has been assessed. Analysis focuses on the mean state and spatial structure of precipitation anomalies, the seasonal prediction skill and representation of forecasts for a recent extreme event, as well as on experiments to diagnose errors in reanalysis humidity fields. Results are discussed both for the 3-month March–May (hereafter MAM) average as well as for individual months.
Impact on mean bias and spatial structure of precipitation anomalies
This first section considers mean precipitation biases and the spatial structure of precipitation anomalies. Figure 2 shows mean MAM precipitation biases for all experiments in Table 1. The mean rainfall from CHIRPS observations is shown in Fig. 2a, indicating rainfall maxima over Tanzania, Lake Victoria and Ethiopia. Over Kenya the maximum rainfall is located near the border with Tanzania and near the coast, whilst the semi-arid remainder of the country and most of Somalia receive around 50 mm across the season. The bias of the control run is shown in Fig. 2b. In general the model underestimates the rainfall by 10–30 mm with a larger dry bias over the maximum precipitation regions of Ethiopia and southern Tanzania and a wet bias over Lake Victoria. The experiments with the largest reduction in the bias are INDI and NWIO (Fig. 2e, k), whilst the experiments WPAC, EPAC, HTRO and SWIO (Fig. 2f, g, j, l) show no impact on the bias beyond the control. This suggests that model processes in the Pacific, upper troposphere and Southwest Indian Ocean are not responsible for this dry bias, which is arising primarily in the northwest Indian Ocean.
It can be also seen that several relaxation regions (GLOB, TROP, ATLA, LTRO) introduce a large wet bias in the normally-dry regions of northern Ethiopia and Sudan. The common region to these experiments is the lower tropical troposphere in the ATLA region, indicating that this wet signal originates from the relaxation boundary there. Given the high transpiration over the Congo Basin it may be that this erroneous signal is related to some overestimation of the atmospheric moisture content in the ERA-Interim relaxation dataset. Indeed it has been found that the ERA-Interim dataset overestimates humidity over Africa in general (ECMWF 2019).
No relaxation experiments improve the wet bias over Lake Victoria, suggesting a local origin for this bias. Previous work with a coupled atmosphere-lake model over Lake Victoria (Song et al. 2004) have shown that wind-driven horizontal heat transport in the lake affects precipitation and concludes that inclusion of these coupled atmosphere-lake hydrodynamics in models leads to a reduction of precipitation compared to using a one-dimensional representation of the lake. From CY41R1, lakes have been included in the ECMWF system through the FLAKE subgrid parameterization (Balsamo et al. 2012), representing those at least 1% of the model gridbox (for these experiments this corresponds to a lake size of around 70 km\(^2\)). This parameterization is based on a one-dimensional lake representation without a full structure and so it may therefore suffer from the same overestimation of precipitation and so is a candidate for the source of this bias. Similarly the dry bias over southern Tanzania encompasses a significant regional lake (Lake Malawi) and is unaffected by any relaxation experiments; this large dry bias here may also be related to lake processes.
The spatial structure of precipitation anomalies in model simulations is assessed in Fig. 3, shown as the first empirical orthogonal function (EOF) spatial pattern for each experiment. GPCP precipitation is used as a reference, as this includes precipitation values over the ocean and gives a complete picture of the structure of anomalies. Given that these EOF patterns are found to be quite different between the months, results for March, April and May are shown separately in Figs. 3, 4, 5. EOFs for the experiments are calculated across all years and members.
For March, the GPCP EOF over the region is a monopole aligned diagonally along with the coast of the Horn of Africa, extending west to Lake Victoria and south to Kenya. The central region of the pattern is relatively uniform, and falls off in power over the ocean. The total variance explained here is 29.6%. In comparison, ERA-Interim variance explained is slightly lower (25%), and the control variance explained is even lower (19%). The structure in ERA-Interim shows broad similarity with GPCP although anomalies generally do not penetrate far enough into the continent and extend too far over the ocean.
For the control (Fig. 3c), the structure of March anomalies over land does not extend sufficiently up into the tip of the Horn of Africa and extend too far into the ocean. This ‘bleeding’ of the EOF pattern into the ocean is also reproduced in several experiments (WPAC, EPAC, HTRO and SWIO), with others (GLOB, TROP, INDI, NWIO) generally constraining anomalies to the coast in a more realistic way. The division of these experiments here reflects their observed impacts on the mean dry bias, suggesting that inadequate constraint of precipitation anomalies to the coast is a key factor producing the overall dry bias. However the variance explained is too low in the control (two-thirds of the GPCP value) and not improved with any relaxation experiment. This suggests the model is fundamentally unable to reproduce the power of this EOF, and/or local enhancement of variability is unrepresented in model processes. A boostrap-based estimate of the uncertainty in the estimate of the model variance gives a standard deviation ranging between 1–3% across experiments and targets, suggesting the difference from observations is significant.
Results for April are shown in Fig. 4. The reference pattern is quite different to March, with the structure splitting into two poles; a smaller one over Lake Victoria and a larger one over the ocean. ERA-Interim is able to reproduce the pole over the ocean but not over Lake Victoria, whilst the control does not have sufficient power over the oceanic pole (Fig. 4b, c). Similarly to March the experiments including NWIO improve this spatial structure, implying that processes in this region are implicated in spatial structure errors. The variance explained by the first EOF in GPCP is slightly lower in April compared to March (24%) and still underestimated in the control (18%) with no relaxation experiment offering improvement. Indeed, NWIO has the largest underestimation, explaining only 13% of the variance by this first EOF.
Results for May are shown in Fig. 5. The structure of the pattern in GPCP is now located back tightly on land, over the tip of Somalia, whilst the model EOF patterns have large errors. ERA-Interim incorrectly has most power over the ocean, with one pole near to the GPCP pole and another off coast near the equator where the magnitude of the GPCP pattern is low (Fig. 5b). Interestingly the control has a large pole over the ocean and another near Lake Victoria which is quite similar to the GPCP pattern in April, suggesting the possibility of errors in simulating the correct timing of the seasonally-locked processes (Fig. 5c). Beyond the control, NWIO is able to center the pattern over Somalia and increase the variance by around 3%, but still has an erroneous pole off the coast at the equator (Fig. 5l).
The possibility of errors in the timing of the monthly processes motivates an examination of the inter-month correlation and its representation in the model and experiments. This is shown by calculating the correlation between subsequent months across all years of observations (e.g. March precipitation correlated with April precipitation across 1981–2014). N.B. For the experiments, the average correlation calculated separately for each ensemble member separately is shown (as the ensemble mean correlation boosts the signal/noise ratio and so is not a fair comparison against a ‘single-member’ correlation calculated for observations). For brevity, results are shown in supplementary material and discussed below.
For the March and April, a significant positive relationship is seen over southern Somalia, Kenya and Ethiopia Somalia and this is not present in the model (supplementary Fig. 1). Rather, individual model members show low correlation between March and April across the whole region, with only some significant positive relationship apparent over the Indian Ocean. Between April and May the correlation is actually negative over Somalia and positive over Tanzania. Again, this is not captured within individual members (supplementary Fig. 2). These results suggests that the model is not perfectly representing the persistence of precipitation influences throughout the long rains. In general these month–month precipitation correlations are improved in the relaxation experiments, particularly so for the NWIO experiment.
Impact of remote relaxation on forecast skill and predictions of the 2011 long rains drought
We turn now to the impact of the relaxation experiments on forecast skill. Figure 6 shows the ensemble mean Pearson’s product-moment correlation (hereafter refered to simply as correlation) of MAM precipitation against CHIRPS for all experiments. Results for individual months March, April and May are shown in Supplementary Figs. 3–5. The control (Fig. 6a) shows the expected low correlation over most of the region with most areas showing correlations below 0.4, consistent with previous work (Batté and Déqué 2011; Dutra et al. 2013; Mwangi et al. 2014; Kilavi et al. 2018). Parts of Kenya, northern Tanzania and Ethiopia show significant correlations, reaching up to 0.6 over Lake Victoria. This region also has a large wet bias, suggesting that factors causing this large biases are not necessarily limiting local seasonal forecast skill.
The impact of the relaxation experiments on the ensemble mean correlation can be seen in Fig. 6b–k. These show the level of accuracy of the seasonal forecast which would be achieved if the time-varying forcing from the specified regions was perfect. Note that this assumes the daily ERA-Interim sub-daily relaxation data is perfect (very likely not the case), and that assumes exact knowledge of the evolution of individual mesoscale systems, months in advance (very likely impossible). However the comparison of different experiments offers some idea of the relative contribution of the regions to potential forecast skill, as well as providing an upper bound estimate on seasonal forecast skill in this season. N.B. The correlation of ERA-Interim precipitation itself with precipitation observations could in theory provide an alternative estimate of the ‘best-possible’ precipitation simulation, since it assimilates all possible observations. This analysis is described as part of the following subsection, whilst here we focus purely on the correlation of relaxation reforecasts against observations.
The GLOB, TROP, INDI and LTRO experiments show the largest increases in correlation (Fig. 6b, c, d, h), with correlations over Kenya reaching over 0.8. This identifies the Indian Ocean lower troposphere as a strong contributor to interannual variability. The Atlantic sector (Fig. 6g) also provides some increase in correlation; this may be as the relaxation essentially provides a perfect source of MJO propagation into the region, which has been shown to be a key contributor to seasonal total rainfall (Vellinga and Milton 2018).
The Pacific regions (Fig. 6e, f) show a minor impact, particularly the East Pacific. Analysis of individual months reveals that this improvement overwhelmingly occurs in March (Supplementary Fig. 3). This is consistent with results from (Camberlin and Philippon 2002), who hypothesise that cool SST in the eastern Niño 1.2 region is linked to above average Kenya/Uganda rainfall through low surface pressure in the northern Indian Ocean and a northward shift in the inter-tropical convergence zone. Analysis of individual months also indicates that correlations for April (supplementary Fig. 4) are lower in the control and all experiments compared to March and May, consistent with previous work identifying April as a month dominated by internal chaotic variations (Camberlin and Philippon 2002). The Northwest Indian Ocean provides a much greater increase in correlation compared to the Southwest, whilst the lower tropics relaxation provides a much greater increase than the upper tropics, which have a mild impact.
A summary of the impact of these experiments across all months is provided in Fig. 7. In each panel a series of related experiments are compared, indicating which experiment (if any) gives the highest improvement in correlation over the control. In the top panel the four tropical regions (INDI, WPAC, EPAC and ATLA) are compared, showing the Indian Ocean as the key driver across all months over the Horn of Africa, with its influence strongest in May and weakening into April. The Atlantic sector shows most impact for western Kenya and northern Tanzania in March, western Kenya in April and a large part of the southern region in May. In addition southern parts of the domain are influenced strongly by the Atlantic, particularly in May.
The middle row of Fig. 7 shows the comparison between the NWIO and SWIO regions, testing the VW18 result. The Northwest Indian Ocean shows a clear influence over the Horn of Africa region in March and April and a reduced influence in May, whilst the Southwest region has some influence over northern Tanzania and southwest Kenya. The bottom row of Fig. 7 shows the comparison between the lower and upper tropical relaxation experiments, indicating that the lower troposphere has the dominant influence over interannual variability during the long rains. Sensitivity of these results to observational uncertainty is tested by recalculating correlations against GPCP and results are unchanged (Supplementary Fig. 6).
These results indicate the broad pattern of the influence of remote regions on the ensemble mean of seasonal forecasts. In order to make this impact more explicit, the forecasts for the long rains failure of 2011 are shown in Fig. 8 in each experiment. This event followed a failed short rains and led to one of the largest humanitarian emergencies in the region in recent memory (OCHA 2011). Figure 8a shows the regions which experienced a one in five year rainfall deficit and Fig. 8b indicates that few control forecast ensemble members captured this event [consistent with analysis of the operational ECMWF seasonal forecast at the same lead time which showed little skill (Dutra et al. 2013)]. Figure 8c–l show the number of members indicating a dry event in each of the relaxation experiment forecasts. Results are consistent with the analysis of ensemble mean correlation across the hindcast presented in Fig. 7. With the NWIO relaxation (Fig. 8k) almost every member is pushed toward dry conditions over the precise region experiencing a dry season over East Africa, indicating that processes in the Northwest Indian Ocean were strongly connected to this drought.
The significant improvement in model biases and forecast skill provided by relaxing the NWIO region demonstrates the importance of this region to long rains rainfall over East Africa. It also appears to corroborate one result from VM18, where a link is found between SST in this region and rainfall over East Africa. The mechanism proposed in VM18 is through direct heating of the boundary layer: warm SST anomalies heat the overlying atmosphere, leading to ascent and a reduction of the climatological subsidence cap over East Africa. To explore this further, extra relaxation experiments were run based on NWIO (full details in Table 2). Firstly, repeating the experiment but with the relaxation box limited to levels below 700 mb diagnoses the importance of the lower troposphere in the region relative to the rest of the atmospheric column (NWLO). Subsequently experiments based on NWLO are carried out with only single parameters (dynamics, temperature surface pressure and moisture), relaxed alone in each separate experiment.
Results are shown in Fig. 9 for all forecast months separately. To show improvement over the control, the correlation is shown as a percentage scaled between the control and GLOB correlations. That is, a score of 0% indicates no improvement over the control, whilst 100% indicates that the GLOB correlation can be reproduced from that experiment alone. To isolate regions where improvement is large, the regions where the Z test statistic of the difference of GLOB and control correlations is below 1 have been masked. For reference the unscaled correlations are shown in Supplementary Fig. 7.
Figure 9a, g, m indicate that in the northern part of the domain over 90% of the improvement from GLOB can be achieved by relaxing NWIO alone, for all three months. For March this result is unchanged when only levels below 700 mb are relaxed, confirming the importance of near-surface processes. Relaxing humidity fields alone in this lower troposphere reproduces a large amount of the correlation, particularly for the semi-arid lands of Kenya (Fig. 9f). By contrast, relaxation of temperature or surface pressure (Fig. 9d, e) alone is unable to improve the correlation, whilst dynamics alone reproduces between 30–50% of the correlation improvement.
During April, NWIO reproduces the majority of the GLOB improvement, but in this month less of the improvement is reproduced when relaxation is limited to levels below 700 mb (Fig. 9g, h). As in March, humidity alone is able to reproduce a large fraction of the correlation over the Horn of Africa (over 90%), and to a lesser extent across Kenya and Ethiopia (around 50%). Temperature relaxation alone has more of an impact in April compared to than March, with a fractional increase over the control of 10-30% over most of the region, and up to 70% over the southern part of Somalia and western Ethiopia (Fig. 9j).
Results for May indicate a different relationship between East Africa and the Northwest Indian Ocean compared to March and April. Whilst most of the GLOB correlation is reproduced by NWIO below 700mb alone, relaxation of the dynamical fields alone now gives the biggest improvement over the control; reproducing over 50% of the GLOB improvement over most of Ethiopia and Somalia, and over 90% in places (Fig. 9o). Humidity still has an impact, particularly over the tip of Somalia where over 90% of the GLOB correlation is reproduced, but is less influential than dynamics across the rest of the region. This is consistent with the idea that the end of the long rains (and therefore May precipitation) is controlled by the strengthening of the Somali Jet in advance of the Indian Monsoon, a dynamical phenomenon leading to enhanced divergence and wind shear that limits convection (Camberlin et al. 2010). Notably over the tip of Somalia large improvements over the control can be achieved by relaxing most variables individually, suggesting that at this time interannual variability in dynamics, humidity and temperature at this time is tightly coupled.
Diagnosis of errors in humidity relaxation fields
Previous analysis has shown that all the experiments which include relaxation on the western boundary show a wet bias (Fig. 2). It has speculated that this bias is related to errors in ERA-Interim humidity over the region to the west of East Africa. In addition, Fig. 6 indicates a significant reduction of correlation from the control in over Uganda in the same experiments, suggesting the ERA-Interim relaxation fields over central Africa as a source of the error. To explore this hypothesis, an additional experiment has been carried out for the GLOB region but with relaxation of humidity fields turned off (GLOB-Q).
The impact on bias and correlations of this GLOB-Q experiment is shown in Fig. 10. Results for relevant experiments are also shown for comparison (control, GLOB and NWIO) and in addition the bias and correlation of ERA-Interim reanalysis itself against CHIRPS is also shown. In order to make a fair comparison with the ERA-Interim correlation, the correlations here are calculated for each ensemble member separately across all years and averaged across all members (rather than showing again ensemble mean correlations, which implicitly increase the signal to noise ratio).
By turning off relaxation of humidity the wet bias over the northern part of the region is reduced (Fig. 10b, c). However, turning off the humidity also reduces the correlation over most of the region (Fig. 10g, h). ERA-Interim humidity fields west of the domain therefore have a negative impact on the precipitation bias over East Africa whilst contributing positively to the correlation. In addition, switching off the humidity fields does not fix the reduction of correlation over Uganda, suggesting further errors in ERA-Interim to the west (e.g. in dynamics and temperature).
It is interesting to evaluate ERA-Interim precipitation here in the same way as the other model outputs. A comparison to CHIRPS reveals a large wet bias over much of East Africa, over the northwest as well as the coastal region (Fig. 10e). Given that reanalysis takes advantage of available observations to constrain a model simulation, it is surprising that this bias is larger than the unrelaxed control experiment; one might expect that a model taking advantage of partial observations of system reproduces the full state compared to the same model without access to those observations. Furthermore for the northeast region the correlation of the NWIO experiment (Fig. 10i) actually provides a better correlation with the observations than ERA-Interim does (Fig. 10j).
One explanation for this surprising finding is that the assimilation of observations are somehow having a negative effect on precipitation processes over the region. For example if the model has significant errors in the placement of climatological features [e.g. the Turkana jet (Nicholson 2017)], then continually pushing the model away from its own climate through data assimilation may disrupt its ability to represent forced variability around its own mean state. Another likely factor may be that the older model version used to produce ERA-Interim. Although ERA-Interim uses the same resolution as these experiments (T255), the cycle used in the production of the reanalysis was released in 2006 (Cy31r1), compared to the cycle used for these experiments released in 2015 (Cy41r1). Significant improvements to the model have occurred during that time which may partially or fully explain this result. The implication of this is that any future work involving relaxation experiments (particularly over East Africa) should make use of the upcoming ERA-Interim replacement, ERA5, for which analysis of initial data has demonstrated improvements in humidity biases over Africa (ECMWF 2019).