Model evaluations for winter orographic clouds with observations

The Real-Time Four-Dimensional Data Assimilation (RT-FDDA) system is used for orographic snowpack enhancement. The model has three nested domains with the grid spacing of 18, 6 and 2 km. To evaluate the simulations of winter orographic clouds and precipitation, comparisons are made between model simulations and observations to determine how the model simulates the cloud distribution, cloud height, cloud vertical profiles and snow precipitation. The simulated results of the 02:00 UTC cycling with 2-km resolution are used in the comparison. The observations include SNOTEL, ceilometer, sounding and satellite data, from the ground to air. The verification of these observations indicates that the Weather Research and Forecast (WRF) RT-FDDA system provides good simulations. It is better to use data within the forecast period of 2–16 h simulations. Although the horizontal wind component near the ground has some bias, and the simulated clouds are a little higher and have a little more coverage than observed, the simulated precipitation is a little weaker than observed. The results of the comparison show that the WRF RT-FDDA model provides good simulations and can be used in orographic cloud seeding.

Since the first conceptual models of weather modification were presented well over 50 years ago [1,2], there have been many attempts to increase the snowpack on mountain ranges by seeding clouds with silver iodide or dry ice. Experiments have shown that seeding does increase precipitation under certain favorable conditions and can result in increases in the snowpack [3][4][5]. Average increases of 10%-15% have been reported in some experiments, while other experiments have not provided conclusive results [6][7][8].
Cloud models have been used in weather modification to formulate cloud-seeding hypotheses; i.e. assessing the cloud-seeding potential related to the "seedability" of a given cloud or cloud type or collection of clouds in a geographical region. Orville [9] reviewed important advances in modeling efforts for weather modification. Increasingly sophisticated models allow quantitative estimations of the effects of seeding and the conditions that optimize the treat-*Corresponding author (email: louxf@cams.cma.gov.cn) ment. Models have also been applied in operational decision making, project evaluation and understanding seeding effects. The Real-Time Four-Dimensional Data Assimilation system (RT-FDDA) [10] is being used by the Wyoming Weather Modification Five-Year Pilot Project (WWMPP) to establish an orographic cloud seeding program in three target areas (the Medicine Bow, Sierra Madre and Wind River ranges) and evaluate the feasibility and effectiveness of the cloud seeding ( Figure 1). The model outputs various fields in nested domains. For the three cloud seeding ranges, besides cloud, ice and snow mixing ratios, the model also presents possible seeding trajectories from ground-based generators.
The need for accurate forecasting techniques has become increasingly important because the products of models are being used operationally in field experiments and consequence analysis. Model validations with observations are important. Many papers have described the validations or comparisons of numerical simulations with observations [11][12][13][14][15]. Snow enhancement through seeding is interesting in terms of not only the snow fall amount and temperature but also the water substances, cloud distribution, cloud height, and cloud base. Furthermore, the deviation in wind near the ground affects the placement of seeding generators and thus the seeding targets. There are usually sparse data for high and remote mountains. The WWMPP program has recorded special data on clouds, such as ceilometer data, radiometer data, and sounding data. In this paper, comparisons are made with SNOTEL, ceilometer, sounding and satellite data, from ground to air, to determine how the model simulates the cloud distribution, cloud height, cloud vertical profiles, and snow precipitation.

Description of the WRF RT-FDDA system
The RT-FDDA system [10] was developed to provide high-resolution short-term analyses/forecasts (0-12 h) similar to the Rapid Update Cycle (RUC). However, recent advances in computing power have allowed for a much longer forecast cycle, up to 36 h at current operational sites given the present grid and configuration of the model physics. RT-FDDA employs time-continuous assimilation of a variety of synoptic and asynoptic observation data including the following: (1) METAR observations (includes "specials"), (2) Ship/buoy observations, (3) Local surface observations included in the MADIS dataset, (4) WMO rawinsonde observations, (5) NESDIS satellite-derived winds, (6) ACARS aircraft observations, (7) WYDOT observations. Incorporated datasets have time frequencies varying from 5 min to 3 h and are assimilated into the RT-FDDA system at corresponding times. In comparison, traditional twicedaily forecasts are limited to incorporating only observation data available at synoptic times. These data are only used to improve the first guess at the start of the forecast cycle. Therefore, twice-daily forecasts have a strong dependence on errors in the first guess. RT-FDDA analyses/forecasts do not generally suffer from model "spin up" issues. Thus, at any time, the RT-FDDA forecasts contain realistic and detailed mesoscale atmospheric structures, including cloud and precipitation systems, and local thermally forced circulations. It should be noted that RT-FDDA does not assimilate cloud/precipitation data at this time. The diagnosed cloud and precipitation systems in the analysis cycles result from the vertical motion and humidity assimilated from the available data. The model is "cold-started" once a week to eliminate system biases that may develop as the system continues to cycle. A cold start is when the RT-FDDA system uses model grids other than its own to start a forecast cycle. These grids are commonly provided by models such as Eta, GFS and RUC.
A forecast cycle is denoted by the UTC time at which the model starts running again. The forecast period can be as long as 36 h. For this particular implementation of the system, the forecast cycles run a minimum of once every 3 h at 02, 05, 08, 11, 14, 17, 20 and 23 UTC. The model has three nested domains with grid spacing of 18, 6 and 2 km and mesh sizes of 150×120, 136×118 and 238×193 respectively. All grids use 36 unevenly spaced vertical computational levels, extending from 15 m to about 17 km above ground level. The explicit cloud microphysical scheme of Thompson [16] was used. Although the RT-FDDA system is run for eight cycles with a 24-h forecast length every day, only the simulation results of the 02:00 UTC cycling with 2-km resolution are used in this research.

Model verification with four kinds of observations
A concentrated effort to validate the WRF RT-FDDA model was initiated in March 2008 and 2007 winter season seeding cases and follow several paths. These include comparisons with satellite cloud top temperatures, cloud base observations made with a ceilometer, sounding parameters (temperature and wind) and total precipitation accumulation recorded by SNOTEL gauges.
For comparisons in this section, the model simulation results are for the 02:00 UTC cycle; therefore, data from the first two hours are "final analyses" (e.g. fields that are fitted to the data) and other data are from forecasted fields. Table  1  Two conventional verification scores were calculated for these fields: bias and mean absolute error (MAE). These statistics were computed as a function of forecast lead time, and further stratified by the time of day.

Cloud top temperature (CTT) comparison of Goes 12
The comparison of model cloud top temperature with satellite observations is made for averages over the month of March 2008. The satellite data are from hourly GOES-12 IR satellite data. In total, 744 data points are used. The hourly model output is from domain 3 (2-km resolution), and the IR data resolution is 4-km.
The average cloud top temperature over the central region of Wyoming, using hourly data in March from WRF model output and IR satellite observations, is shown in Figure 2. The CTT patterns in both analyses are similar once the 3-6°C cold bias in the model-derived data is accounted for. One notable difference is the warm region (the north-central region of the figure) in the lee of the Wind River Range, which is much more pronounced in the satellite observations. The 3-6°C cold bias (e.g. difference with satellite data) in the WRF output could be due to several factors, including more cloud cover, higher clouds, colder surface temperatures (in clear areas), or other differences.
A closer look at the difference or bias between the model output and satellite data is presented in Figure 3, which   shows both the hourly bias (a) and daily bias (b). Two measures are calculated: the average bias (pixel to pixel difference averaged over the entire region) and the absolute average bias (absolute pixel to pixel difference averaged over the entire region). Here we concentrate on the average bias-the brown line in each plot. The hourly average bias in Figure 3(a) has a diurnal trend. The change in bias after the first hour is likely because of model adjustment in the data assimilation or analysis. Likewise, the decreasing trend (to larger differences) over the last 8 hours may be because of larger forecast uncertainties in the 24-h forecast run. The most stable values are between 2 and 16 h and indicate an overall average bias of 4-5°C, which is consistent with the regional pattern shown in Figure 2. The daily average biases (Figure 3(b)) fluctuate considerably. The bias values are actually positive on some days (as opposed to the hourly analysis). Although not shown here, the largest differences (coldest bias values) were generally associated with cloudy days over the region and conversely the smallest differences were associated with mostly clear days. However, the results suggest that the model may be simulating colder cloud tops, which could have a significant effect on modeled precipitation fields.

Comparison with SNOTEL precipitation data
Daily precipitation accumulations at SNOTEL sites in the month of March 2008 are compared with WRF model simulations of precipitation. Data from 19 SNOTEL sites in southern Wyoming and northern Colorado in or near the target ranges are used. Their names and locations are given in Table 2. To match the time and day of the SNOTEL observations (in Pacific Standard Time) to the WRF simulations (in UTC), the model-simulated daily precipitation is accumulated from 1600 UTC on the day before to 1500 UTC on the day of interest.
A summary of all 19 sites for both the model (WRF) precipitation accumulations and the SNOTEL accumulations is given in Figure 4. Additionally, the difference between the WRF and SNOTEL amounts for each site is plotted in Figure 5.  One cautionary note is that the Whiskey Park SNOTEL meter malfunctioned in March and its data should not be used in the comparisons. Considering this issue, the comparison plots show fairly good agreement between the model and SNOTEL data overall. Over the 18 good sites (i.e. excluding Whiskey Park), a general difference in total accumulation of about 10 mm (about 10%) is observed. The higher values of the WRF simulations are around 90 mm total and the higher values from the SNOTEL sites are around 100 mm. Individual sites have varying fluctuations, in terms of both precipitation amount and timing. Some general trends are also evident, particularly the lack of precipitation during the period 21-25 March in the WRF simulations versus changes in precipitation amounts at several SNOTEL sites during the same period. However, the project gauges indicated that no precipitation fell from about 00Z 22 March through to 00Z 27 March. Therefore, the SNOTEL data are not entirely consistent either.
Overall, the SNOTEL sites with the highest and lowest precipitation totals are basically the same as those simulated by the WRF model with a slight bias toward lower values of total accumulation in the WRF output.

Comparison of cloud base height with ceilometer data
Ceilometer data were collected from a site on the western side of the Medicine Bows Range, near the foot of the range at 2485 m above mean sea level. A ceilometer measures cloud base heights via the backscatter return of a laser beam. The vertical extent of the ceilometer data is from the ground to 3750 m, and the data are recorded in vertical For comparison with WRF-simulated cloud bases, the ceilometer data were partitioned into seven height categories ( Table 3). The frequency of height categories is then plotted for each hourly simulation in March 2008. Figure 6 shows the total cloud base frequency for each height category obtained using the hourly WRF simulations and ceilometer data and daily cloud cover frequency for each day in March 2008. Both comparisons show quite a good agreement between the model and observations. The Table 3 Ceilometer categories and corresponding cloud-base height intervals   Range  1  2  3  4  5  6  7 Height (m) <500 500-1000 1000-1500 1500-2000 2000-2500 2500-3000 3000-3700 ceilometer observations over the lowest heights drop off in frequency more sharply than the model simulations, possibly because of the lower resolution of the model at those heights in particular. There is also a tendency for a higher total frequency of cloud (obtained by integrating the curves of the hourly plots in Figure 6(a)) in the model output than in the observations. This tendency is also clear in the daily plots in Figure 6(b), with slightly higher highs and two periods in particular of significantly higher lows. The large changes in cloud cover on 16-17 March and 28-29 March were not entirely captured in the WRF output. However, in general, the trends of cloud cover are reasonably well represented by the model, even the almost clear days of March 10-11 and March 23-24.

Comparison with sounding data
Although the intent is to eventually incorporate the Saratoga sounding into the WRF data stream to nudge the model toward actual conditions, the model has been running without the sounding input. This allows for a comparison between sounding parameters and WRF output. It should be noted that all comparison soundings were carried out at Medicine Bow Range on case-days when snowfall cases were called. A total of 32 soundings (December 2007-February 2008) are used for the comparisons. Three variables are compared: winds (U and V) and temperature. The comparisons only cover the lowest 6 km to minimize displacement errors from horizontal drift of the balloon away from the release location. For example, if the horizontal wind is 30 km h -1 (16 kts), the drift distance is 10 km in the approximate 20 minutes it takes to rise to 6 km altitude. Figure 7 presents the differences in U and V wind components between the model output and the sounding. Warm colors (yellow to red) indicate that the model winds are more westerly or southerly than the observed winds, and cool colors (bluish) indicate the opposite. Except for a few instances, the modeled north-south component (V) agrees reasonably well with the observations. There is perhaps a slight bias towards more southerly winds in the model output above 2 km. However, the east-west component (U) comparison shows that the model had stronger westerlies below 1 km, particularly in the later cases. The differences are less obvious aloft but persist on some disparate days (e.g. 5th and 22th). If the model is simulating stronger westerlies, then the flow up slopes is likely to be stronger, leading to more condensate (e.g. more supercooled liquid water (SLW)) being simulated. This is somewhat reflected in the higher incidence of cloud cover in the ceilometer comparisons ( Figure 6).
In a few cases, the temperature differences between the model and observations are relatively small (1-2°C), as presented in Figure 8. The T plot shows that the differences, while small, are consistently negative below about 1 km, meaning that the model-simulated temperatures are cooler. Figures 7 and 8 show a large bias in wind and temperature mainly in the 1st, 5th, 19th and 28th cases, which correspond to cases from December in 2007 to February in 2008. To determine the reason for the large bias on these days, the wind speeds and directions at 700 hPa for 32 cases are analyzed (figure omitted). Almost all snowfalls in Wyoming and Colorado are influenced by moist and cold air from the western coast in winter. At 500 hPa, western flows predominate at the sounding location of the Medicine Bow Range. The stream near the ground on some days changes direction around the Medicine Bow Range because of a terrain block. On the No. 5th, 19th and 28th cases, there are strong wind shears between 700 and 500 hPa, and wind directions at these two levels change by about 85°, 60° and 75° respectively. Except the No. 6th case, three of four cases with big bias have the strongest shears among all 32 cases. The vertical grids of model have 37 levels in this simulation, and this may not be fine enough to represent a profile with strong wind shear. In addition, it is possible that the assimilated initial data cannot contain all the wind shear information, which would result in a large bias in the wind component simulation and thus influence the simulation of the temperature distribution.
This sounding data are not assimilated in the present model. A preliminary analysis of sensitivity studies in which sounding data are input into the WRF RT-FDDA (after the fact) supports the general characteristics of the difference plots in Figure 8. In addition, the vertical resolution sensitivity seems to improve the wind component simulations with finer resolution in the lower layers.

Conclusions
The WRF RT-FDDA model was used in the field experiment of the Wyoming Weather Modification Five-Year Pilot Project. The WRF RT-FDDA model assimilates a variety of synoptic and asynoptic observation data. The model has three nested domains with grid spacing of 18, 6 and 2 km. The model domains cover three target areas for orographic cloud seeding in Wyoming State: the Medicine Bow, Sierra Madre and Wind River ranges.
Several preliminary comparisons between the WRFmodel output and various observations were presented. Considering the comparisons in total, the WRF RT-FDDA generally provides good simulations but with some biases. Comparing the cloud base with ceilometer data and the cloud top temperature with satellite data show that clouds are simulated fairly well. There is also good agreement between sounding data and the simulated vertical profiles. Except in four cases, the modeled north-south component (V) compares reasonably well with observations below 2 km. However, the east-west component (U) comparison shows that the model has a stronger westerly below 1 km.
The interpretation is not simple however. While the comparisons suggest that the model is simulating cooler and hence moister conditions with the possibility of creating more SLW and cloud cover, these biases do not translate to more simulated precipitation.
The validations indicate that the WRF RT-FDDA model can be used for orographic cloud seeding in Wyoming. The hourly model outputs are put on a website for the pilot team and ground-based AgI generator team to decide the pilot line and the time to ignite AgI generator. The model simulations are also used to decide if a case is called.
From these evaluation efforts, a good basis has been established for further detailed analysis and sensitivity study, which should improve the modeling system and eventually provide accurate precipitation estimates for orographic cloud seeding.