1 Objectives

Though fog events highly affect transport safety with potentially large economical and human losses (Gultepe et al. 2007), accurately forecasting fog occurrence at an exact time and location still remains difficult for the current numerical weather prediction (NWP) model. This is due to the high number of non-linear fine scale processes taking part in the fog life cycle which are not yet all entirely understood. Fog is also known to be highly sensitive to initial conditions especially temperature and humidity initialization (Bergot and Guedalia 1994; Bergot et al. 2005; Hu et al. 2014). However, a significant observation gap is observed in the atmospheric boundary layer (ABL) which is undersampled compared to surface observations over continental regions or the free troposphere well sensed by satellite data. In this context, an un-precedented network of 8 ground-based microwave radiometers (MWR) has been deployed in the South-West of France during the SOuth west FOGs 3D (SOFOG3D) experiment for processes study (Martinet et al. 2020). MWRs can provide vertical profiles of temperature and absolute humidity at a very high temporal resolution (1 s to a few minutes) under all-sky conditions but rain up to 10 km altitude (Cimini et al.2006; Löhnert and Maier 2012; Martinet et al. 2015; 2017). Additionally, MWRs provide reference measurements for highly accurate integrated water vapor (IWV) contents and liquid water path (LWP) also provided in the database. Besides the usefulness of an accurate description of the ABL before fog formation, during fog mature phase and at fog dissipation to better understand physical processes driving the fog life cycle (Temimi et al. 2020), recent studies have highlighted the expected benefit of MWR observations for data assimilation experiments in view of improving fog forecasts (Martinet et al. 2020). This is why, supported by a strong European collaboration through COST Actions ES1303 TOPROF (Illingworth et al. 2019) and CA18235 PROBE (Cimini et al. 2020), an international network of MWRs has been deployed with the primary objective to conduct data assimilation experiments within the French convective scale model AROME (Application of Research to Operations at MEsoscale, Seity et al. (2011)). Assimilation experiments using 3D-Var and advanced 3D-EnVar data assimilation schemes are currently on-going.

2 Data description

MWRs can measure brightness temperatures (BT) in two spectral-bands: the K-band from 22.24 to 31 GHz to retrieve humidity (Q) profiles, integrated water vapor (IWV) content and liquid water path (LWP), and the V-band from 51 to 58 GHz to retrieve temperature (T) profiles. Elevation scans are also used to increase the vertical resolution of temperature profiles. This dataset comes from 7 multi-channel MWRs manufactured by Radiometer Physics GmbH (RPG) and a single-channel MWR manufactured by RPO Attex in cooperation and support with IfU GmbH. Optionally, a broadband infrared radiometer measuring the downwelling emission between 9.6 and 11.5 μm is attached to the RPG radiometer housing to improve LWP retrievals through an improved cloud base detection. During the SOFOG3D experiments, all temperature and humidity RPG MWRs were alternating boundary layer scans in 10 elevation anglesFootnote 1 every 10 min with zenith measurements at 1-s time resolution.

The Attex MTP-5 instrument is measuring BT at 56.6 GHz and is dedicated to the temperature profiling up to 1 km altitude with a time resolution of 5 min (Troitsky et al. 1993; Westwater et al. 1999). To improve the vertical resolution, the MTP-5 radiometer was scanning in 9 elevation angles.Footnote 2

The location, operating institution, MWR generation, and retrieved variables at each site are defined in Table 1. The position of the different MWR units is shown in Fig. 1. Figure 2 shows the data availability for each MWR unit through the whole period of the field campaign. Following the definition of fog events provided by Tardif and Rasmussen (2007), Table 2 provides a synthesis of the total number of fog events observed at each instrumented site. Please note that the total number of fog events detected depends on the availability of visibility measurements, which might differ from one site to another. The period of availability of visibility measurements is thus provided as an additional information in Table 2 as well as the type of visibility sensor deployed at each site. The database thus contains a total of approximately 424 fog events observed at different locations, the site of Agen being the most prone to fog occurrences with a total number of 103 fog events detected while only 15 fog events have been observed at the site of Biscarrosse.

Table 1 Station name (short-name), operating institution, location (latitude and longitude), altitude (Alt) above mean sea level, retrieved variables, MWR type for the 8 units deployed during SOFOG3D
Full size table
Fig. 1
figure 1

Surface altitude (above see level) extracted from the operational AROME model. Locations of the MWR sites are shown with the crosses (red indicates temperature and humidity profilers, yellow only humidity profilers, and cyan only temperature profilers.)

Full size image
Fig. 2
figure 2

Data availability of each MWR unit through the whole experimental campaign period

Full size image
Table 2 Number of fog events observed during the whole SOFOG3D experiment at each instrumented site
Full size table
Fig. 3
figure 3

Flow chart of data processing for the RPG HATPRO multi-channel radiometer. Pressure profile (p(z)), temperature profile (T(z)), humidity profile (Q(z)), and liquid water content profile (LWC(z)) are necessary to perform radiative transfer model (RTM) calculations. ANN corresponds to artificial neural network

Full size image
Fig. 4
figure 4

Flow chart of data processing for the Attex MTP-5 single-channel radiometer

Full size image

Level1 products correspond to raw BT measurements for each MWR site as well as the infrared radiometer brightness temperatures when this optional instrument is available and are described in Table 3. Examples of level1 data from the RPG and Attex instruments are shown in Figs. 5 and 7.

Table 3 Description of level1 dataset
Full size table
Fig. 5
figure 5

Example of level1 brightness temperature data observed on the 25th of December 2019 at the main site Charbonniere. a BT spectrum as a function of the channel frequency observed at 0 UTC, b time series of BT observed at 22.24 GHz and 90∘ elevation angle, c time series of BT observed at 52.28 GHz and 90∘ elevation angle, d time series of BT observed at 58 GHz and all elevation angles from 90 down to 4.2∘. Blue crosses show times when a rain event is detected by the instrument

Full size image

The MTP-5 single-channel radiometer has its own continuous self-calibration procedure ensuring a good stability of BT measurement. For HATPRO multi-channel radiometers, a liquid nitrogen calibration needs to be performed at the minimum twice per year in order to get the most optimal calibration coefficients to convert detected intensities into BTs. A liquid nitrogen calibration was thus performed for all radiometers on-site at the beginning of the experimental campaign. The quality of BT measurements from HATPRO MWRs, which accuracy depends on liquid nitrogen calibration, was thus monitored during clear-sky periods through the comparison of observations minus BT simulations from the AROME model. The method is described in De Angelis et al. (2017) and Martinet et al.(2015, 2017, 2020). Only channel 3 (23.84 GHz) of the MWR deployed at Charbonniere was found to be affected by a hardware failure. This channel has thus been removed in all level2 processing. For all the other HATPRO MWRs, only channel 5 (26.24 GHz) of the MWR deployed at Mont-de-Marsan was found to be affected by a large bias of 5 K. Contrary to the MWR at Charbonniere, this bias does not come from a hardware failure. It is roughly constant in time and can easily be corrected before the retrieval of level2 products. This first dataset release does not apply any bias correction of BT before level2 processing; the impact of a bias correction applied to BT measurements will be investigated in the future and may lead to a revised dataset release.

MWR measurements of BTs are then inverted into temperature and humidity profiles as well as IWV and LWP products. In this dataset, neural networks were developed for the RPG MWRs in collaboration with the manufacturer. Neural networks were trained from 3 years (25,000 profiles) of 1-h forecasts extracted from the french convective scale AROME model converted into the equivalent BT with a radiative transfer model taking into account an instrumental noise dependent on the instrument generation. The algorithm used is a feed-forward neural network with one hidden layer. The transfer function is tangent hyperbolic and the number of neurons in the hidden layer depends on the variable to be retrieved, the number of channel used, and the number of angles in the case of elevation scans (5 for LWP and IWV, 15 for temperature profiles with elevation scan, 15 for humidity profiles, 12 for temperature profiles zenith-only). Training of the neural network is done with standard backpropagation algorithm and adaptative learning parameters (self-written software, proprietary code).

Three temperature retrieval files are provided:

  • TABL files correspond to boundary layer temperature profiles every 10 min on 54 discrete levels up to 2500 m;

  • TAT files correspond to full troposphere temperature profiles every second on 93 discrete vertical levels up to 10 km altitude. This product only uses zenith measurements;

  • TACOMP files correspond to composite temperature profiles on 93 vertical levels up to 10 km altitude obtained by combining the TABL profiles with the TAT profiles.

Humidity profiles are retrieved on 93 vertical levels with a variable spacing from surface up to 10 km altitude.

For the LWP product, an offset correction has been included to improve the LWP accuracy. This correction is based on the identification of clear-sky periods during which a clear-sky LWP offset is calculated. To correct the original LWP values, a linear interpolation between the two closest LWP offset corrections is performed and this offset is subtracted from the original value.

For the Attex MTP-5 MWR, a statistical regularization method (Ilyin and Troitsky 2017) has been used to retrieve temperature profiles on 23 vertical levels. This retrieval method uses pre-calculated auto-correlation temperature matrices obtained from WMO radiosoundings depending on seasons. Figures 3 and 4 provide a flow chart synthetizing the data processing from raw BT measurements to retrieved products for both types of radiometers.

Table 6 provides a full description of all available level2 dataset with corresponding DOI (Fig. 5). Figures 6 and 7 provide an example of the main level2 products available for the RPG and Attex instruments.

The data quality of temperature and humidity products has been validated thanks to radiosonde comparison over 125 profiles launched at three sites: Charbonniere, Agen, and Toulouse (Fig. 8). Please note that radiosoundings are co-located with MWRs at Charbonniere and Agen while they are launched at a distance of 8 kms from the MWR at Toulouse.

MWR temperature retrievals agree well with RS measurements within 1.5 K RMSE below 2 km and 2.5 K below 6 km. Biases are below 0.5 K in absolute value for all sites below 2 km altitude but increase up to 1.5 K at Charbonniere and Toulouse above 4 km and 2 km altitude respectively. For absolute humidity, RMSE are below 1.8 g.m− 3 for all sites up to 6 km altitude with biases below 1.4 g.m− 3.

Table 4 shows that IWV can be retrieved with a RMSE below 0.6 kg.m− 2 at Charbonniere and Agen and below 1.4 kg.m− 2 at Toulouse.

The LWP accuracy has been validated in clear-sky conditions and has shown errors between 1 and 14 g.m− 2 depending on the site which is within the expected LWP product accuracy defined in the literature (Crewell and Löhnert 2003; Marke et al. 2016). The validation of LWP retrievals in cloudy-conditions is a much more complex task where in situ measurements from a cloud droplet probe have to be used. This is beyond the scope of this data paper and will be part of deeper investigation in the future.

The biases and uncertainties obtained from the SOFOG3D MWRs are consistent with those from other datasets for temperature profiles (Cimini et al. 2006; 2011; 2015; Martinet et al. 2015; 2020; Turner and Löhnert 2021). The uncertainty for humidity profiles is also consistent with other studies, while reported biases are either smaller (Cimini et al. 2006; 2011; 2015) or larger (Temimi et al. 2020; Turner and Löhnert 2021) than those obtained here for the SOFOG3D dataset. One possible contribution to the observed humidity bias in the lowest layers may be the use of the lowest V-band channels, which are affected by larger uncertainties in absorption models (Cimini et al. 2018). A bias correction of level1 data will be investigated in the future and may lead to a revised dataset (Tables 5 and 6).

Fig. 6
figure 6

Example of level2 products for the 25th of December 2019. From top left to bottom right: a boundary layer temperature profiles, b composite temperature profile, c absolute humidity profiles, d relative humidity profiles, e integrated water vapor, and f liquid water path

Full size image
Fig. 7
figure 7

Example of level1 and level2 data obtained on the 25th of December 2019 with the MTP-5 MWR deployed at the main site Charbonniere. a Time series of BT observed at 56.6 GHz and all elevation angles from 90 down to 1∘, b boundary layer temperature profile

Full size image
Fig. 8
figure 8

Vertical profiles of (left) temperature and (right) absolute humidity bias (solid line) and root-mean-square-error (dashed lines) collected by the RPG HATPRO radiometer (black line) or the Attex MTP-5 radiometer (red line). Statistics performed over 50 radiosonde profiles at Charbonniere (top), 49 radiosonde profiles at Agen (middle), and 26 radiosonde profiles at Toulouse (bottom)

Full size image
Table 4 Bias and root-mean-square-errors of IWV retrievals with respect to radiosonde measurements for the sites of Charbonniere, Agen, and Toulouse
Full size table
Table 5 Description of level2 temperature files
Full size table
Table 6 Description of humidity and LWP level2 files
Full size table

3 Limitations

As part of a research project, some MWR units had to face electrical failures as well instrumental breakdown during the experiment. Missing data are thus present within the database.