1 Introduction

If it did not have to condense to become visible, water vapor would fuel the fascination of many more scientists. Imagine seeing with the naked eyes how elevated layers of water vapor, and its radiative effects, engender shallow circulations, or how pockets of humidity surround and socialize cumulus convection. Imagination is indeed necessary because water vapor’s mysteries arise as much from its visible transparency as from the opulence of its infrared opacity (Stevens and Bony 2013).

Water vapor couples to atmospheric circulations directly, through water vapor’s thermodynamic effect on the development of clouds and convection, and indirectly, through water vapor’s non-local influence on infrared irradiances. The thermodynamic effect has been extensively studied over the past decades (Sherwood et al. 2010), the radiative effect—especially in the lower troposphere—has not. The purpose of this article is to review our understanding, and to evaluate our ability, to remotely sense important features of the lower-tropospheric water vapor distribution. We do so with the help of a series of flight campaigns in the tropical north Atlantic. In so doing we identify a new mechanism which, we hypothesize, contributes to the prevalence of mid-level clouds, near the triple-point temperature (\(T_*=0.01\,^{\circ}\)C).

This paper is organized as follows. In Sect. 2 we fix terminology and provide an overview of the field measurements and satellite data-products used in this study. To provide context for the measurements, in Sect. 3 we review how water vapor’s thermodynamic properties influence clouds and convection and how its radiative properties influence circulation. In Sect. 4 the ability of recent measurements, including state-of-the-art active and passive remote sensing, to delineate the humidity structure of the lower-troposphere, are evaluated. The measurements motivate a new hypothesis (Sect. 5) as to how water vapor radiative effects combine with barriers to ice initiation to help explain a preponderance of mid-level convection in the vicinity of deeper convection. The main points of the article are summarized in Sect. 6.

2 Data and Context

Terminology is fixed with the help of Fig. 1, which illustrates important features of the distribution of water vapor in the tropical troposphere. The planetary boundary layer, or PBL, is here defined as the layer below the lifting condensation level. It is assumed to be well mixed by turbulence, so that it is meaningful to talk about values of conserved quantities within the PBL in terms of single numbers, for instance the PBL specific humidity, denoted \(q_{\mathrm{PBL}}.\) A shallow cumulus layer, whose top is usually between 1.5 and 3.0 km, can often be identified not only by the mean height of the clouds that form there, but also by a slight temperature inversion or, more commonly, a pronounced lapse in the humidity (hydrolapse) at its top. In Fig. 1 this height is denoted by \(z_{\mathrm {i}}\) and placed at about 3 km. The layer between the average \(z_{\mathrm {i}}\) and the height, \(z_*,\) of the triple-point isotherm \(T_*\) is referred to as the lower free troposphere, LFT. We speak of the triple-point level rather than the ‘melting level’ or ‘freezing level’ as the latter are less well defined. In cases when the shallow cumulus layer is completely suppressed, the lower free troposphere will extend down to the top of the PBL. In disturbed conditions the shallow cumulus layer may extend to the triple-point level.

Fig. 1
figure 1

Conceptual diagram illustrating layers of moisture, and some of the terminology, discussed in this article. The height of the sub-cloud layer, which delineates the planetary boundary layer or PBL, is denoted by \(\eta ;\) the height of the trade-inversion or the hydrolapse delineating the shallow cloud layer is denoted by \(z_{\mathrm {i}};\) the height of the triple-point isotherm, \(T_*,\) is denoted by \(z_*,\) and the height of primary ice formation isotherm (\(T_{\mathrm{f}}\)) is denoted by \(z_{\mathrm {f}}.\) For the formation of ice by homogeneous nucleation \(T_{\mathrm{f}} = -38\,^{\circ}\)C. The term lower free troposphere is used to denominate the region below the height of \(T_*\) and the average height of the fair-weather cumulus layer. In the present-day tropics \(\eta\) is at about 600 m, the \(z_{\mathrm {i}}\) can be quite variable, but when evident is typically between 2 and 3 km. \(z_*\) locates at around 4.5 km

2.1 Airborne Measurements and the Barbados Cloud Observatory

The Next-Generation Airborne Remote Sensing for Validation Studies (NARVAL) field campaigns made use of the Germany Research Community’s HALO (High Altitude Long Range) research aircraft (Stevens et al. 2016) to make measurements over the northern tropical Atlantic. NARVAL has had two phases: Phase I took place in the downstream winter trades in December of 2013; phase 2 took place in and around the vicinity of the Atlantic ITCZ in August 2016. NARVAL-1 consisted of eight flights, with about 40 h of flight time over the North Atlantic trades abutting the ITCZ. NARVAL-2 consisted of roughly twice as many flight hours distributed over ten flights in, around, and across the ITCZ. For NARVAL-1, the configuration of HALO is described by Stevens et al. (2016). For NARVAL-2, the configuration was similar. Most relevant for this study is the extensive deployment of dropsondes and continuous water vapor profiling using a downward staring differential absorption lidar as elaborated upon below.

Airborne measurements were taken east of the Barbados Cloud Observatory (BCO), over the ocean between 40–60\(^{\circ }\)W and 5–20\(^{\circ }\)N. The BCO comprises an extensive collection of advanced ground-based remote sensing instruments located on Deebles Point, a promontory on the windward side of Barbados at 59.43\(^{\circ }\)W, 13.17\(^{\circ }\)N. Measurements at the BCO were initiated by the Max Planck Institute for Meteorology and the Caribbean Institute for Meteorology and Hydrology, with support from the Museum of Barbados, to advance understanding of processes influencing the distribution of convective clouds in the lower troposphere. The BCO has been in continuous operation since 2010 (Stevens et al. 2016). During NARVAL-1, the western Atlantic ITCZ was located around 10\(^{\circ }\)N; during NARVAL-2 it varied between 5–15\(^{\circ }\)N. Following the terminology introduced in Fig. 1, NARVAL-2 sampled the broader ITCZ region, NARVAL-1 the subsidence zone.

2.1.1 Dropsonde Humidity Measurements During the NARVAL Campaigns

During NARVAL-1, roughly forty Vaisala RD94 Dropsondes (hereafter sondes) were launched, most from an altitude of about 14 km. During NARVAL-2, many more (roughly 200) were sondes, most from an altitude of 9 km. The Vaisala RD94 Dropsonde uses twin thin-film capacitors to measure humidity with a stated accuracy of 2% (absolute relative humidity). To reduce sampling biases and help remove outliers associated with sondes launched from the ferry flights, only sondes with a near-surface air-temperature greater than a threshold (25 \(^{\circ}\)C for NARVAL-1 and \(27\,^{\circ}\)C for NARVAL-2) are analyzed. On those flights where more than ten sondes satisfied this criterion, the list of sondes was subsampled by taking a uniform stride through the sonde-list, to ensure a relatively homogeneous coverage of eight to ten sondes per flight. In Fig. 2 the location of all sondes analyzed in this paper is presented and demarcates the area of flight operations as indicated earlier. All dropsondes were processed using the ASPEN V3.3-270 software, and the data were further visually inspected for possible biases or malfunction before being incorporated into the analysis.

Fig. 2
figure 2

Location of NARVAL-1 (orange) and NARVAL-2 (blue) sondes (42 and 81 sondes, respectively) used in the analysis and the location of the Barbados Cloud Observatory (BCO) is indicated

Differences between the composite moisture sounding for NARVAL-1 versus NARVAL-2 measure differences between the winter trades (subsidence zone, Fig. 1) and the atmosphere near and around the ITCZ. In the NARVAL-1 soundings, the lower free troposphere is very dry, almost devoid of water vapor (Fig. 3). The shallow cumulus layer is well defined in terms of a moist layer extending from just above the top of a yet moister PBL to a height of 2 km, where it terminates in a strong hydrolapse. The principal mode of variability is thus expressed by variations of \(z_{\mathrm {i}},\) i.e., the height of the top of the shallow cumulus layer. This is evident in the large range of values of specific humidity, q,  around 2 km. During NARVAL-2, where sondes were dropped in and across the ITCZ as well as in dry regions in the vicinity of deeper convection, a shallow cumulus layer is less distinct. This is because the height of the shallow cumulus layer is more variable as is the humidity in the lower free troposphere, with median humidity values near 5 g kg\(^{-1}\). Surface temperatures are warmer in the air-masses sampled during NARVAL-2, as indicated by more elevated values of \(q_{\mathrm{PBL}}.\) In both NARVAL-1 and NARVAL-2, the PBL is relatively well mixed, as shown by the relative constancy in the humidity within the lowest two (NARVAL-2) to three (NARVAL-1) levels of the humidity distribution.

Fig. 3
figure 3

NARVAL-1 (left) and NARVAL-2 (right) humidity profiles composited from dropsondes shown in Fig. 2. The box and whisker plot shows the distribution of humidity values measured within a vertical height interval in terms of the values at the 10th and 90th percentile (whiskers), the interquartile (box) and the median (notch) of the distribution

2.1.2 WALES

Water vapor is continuously profiled by Water vapor Lidar Experiment in Space (WALES; Wirth et al. 2009), a differential absorption lidar designed to actively sense water vapor. WALES was operated on board HALO in a nadir viewing mode. Measurements at four wavelengths sample three water vapor lines of varying strength and allow WALES to measure water vapor over the enormous range of values found within the tropical troposphere. The water vapor molecule number density is proportional to the difference in absorption between the three “online” pulses and a reference “off-line” wavelength, as function of distance from the lidar. Weaker lines are better suited to measuring high-concentrations, where the signal from stronger lines would saturate, while stronger lines provide more signal to measure low concentrations. The differential absorption retrieval is applied with an effective vertical resolution of 200 m. To reduce instrument noise, the individual on- and off-line profiles are accumulated to 12 s averages, which results in a horizontal resolution of 2.5–2.9 km, depending on the aircraft speed. As a consequence, small clouds are detected only in the higher-resolved lidar backscatter (not shown here), and gaps \(>\simeq 3\) km between clouds permit water vapor lidar profiles down to 200 m above sea level. Comparisons with the dropsondes yield agreement within a few percent, which is on the order of the water vapor spectroscopic and sonde measurement uncertainties. Details on WALES and its NARVAL-1 measurements are found in Kiemle et al. (2017). More information on emerging technologies for better (tropical) water vapor measurements has been compiled by Nehrir et al. (2017).

2.2 SAPHIR and Megha-Tropiques

The Indo-French Megha-Tropiques satellite has been observing the tropical troposphere since October 2011. Megha-Tropiques carries three radiometers dedicated to the energy budget and the atmospheric water: Scanner for Radiation Budget (ScaRaB), Sounder for Atmospheric Profiling of Humidity in the Intertropics by Radiometry (SAPHIR) and Microwave Analysis and Detection of Rain and Atmospheric Structures (MADRAS). Because of mechanical issues, operation of MADRAS ceased after 15 months of observations (on January 26, 2013). A novelty of the Megha-Tropiques platform is its low inclination of 20 \(^{\circ}\) around the equator, which translates into a higher revisit frequency: The satellite orbits the tropics about fourteen times per day yielding between two to six observations per day at a given location between 25\(^{\circ }\)N and 25\(^{\circ }\)S. Further information about Megha-Tropiques is provided by Roca et al. (2015) and Capderou (2009).

We analyze humidity retrievals based on measurements by SAPHIR, which samples the 183.31 GHz water vapor absorption line with six channels. The use of microwave radiometry for water vapor profiling allows retrieval in cloud-covered scenes, within limits determined by the extent of scattering by large hydrometeors (Greenwald and Christopher 2002; Hong et al. 2005). The six channels all measure the same water vapor absorption line but with different radiometric sensitivity. Channel 1 measures the line within \(\pm 0.2\) GHz, Channel 6 within \(\pm 11\) GHz, (Eymard et al. 2002). Channel 1 is most sensitive to water vapor in the upper troposphere, near 300 hPa, Channel 6 is more sensitive to water vapor in the lower-troposphere, near 600 hPa (Roca et al. 2015). By virtue of their differences, signals from these channels can be used to estimate relative humidity (in % with respect to liquid water) in six unevenly spaced layers of the atmosphere, as indicated in Table 1. SAPHIR is a cross track scanning instrument with 130 non-overlapping (10 km diameter at nadir) footprints across a 1700-km scan.

Table 1 Layers in which relative humidity is retrieved based on six channels of the SAPHIR instrument aboard Megha-Tropiques

We use water vapor retrievals from a multivariate regression scheme that provides the parameters of conditional distributions chosen a priori (Brogniez et al. 2016). Compared to the relative humidity estimation scheme described by Brogniez et al. (2016), which relied on a normal (Gaussian) distribution, here a Beta distribution is used as its compact support and allowance for asymmetries about the mean are more favorable for describing the relative humidity. The estimation of parameters also provides a characterization of the retrieval uncertainty for every observing pixel of SAPHIR. Limits of such statistical schemes arise mainly from the training phase performed over a sample of tropical profiles, and from the signature of non-scattering clouds in the scene, as these cannot be accounted for in the radiative transfer computations that are a pre-requisite to the design of the retrieval technique. However, some studies have already shown that the three additional channels of SAPHIR compared to the configuration of the operational 183 GHz radiometers such as AMSU-B or MHS improve the estimation of the relative humidity of the upper part of the troposphere and in its lowest layers (Brogniez et al. 2013).

2.3 Infrared Atmospheric Sounding Interferometer (IASI)

The Infrared Atmospheric Sounding Interferometer (IASI) is a Fourier transform spectrometer based on a Michelson interferometer coupled to an integrated imaging system. IASI provides 8461 spectral samples, aligned in three bands between 645.00 and 2760.00 cm−1 (15.5 and 3.63 μm), with a spectral resolution of 0.50 cm−1 after apodization (filtering to remove diffraction rings) and a spectral sampling interval of 0.25 cm−1. IASI scans across-track with a ground-swath width of approximately 2200 km and a nadir footprint of about 12 km diameter. One IASI was launched in October 2006 onboard the Metop-A satellite and another in September 2012 onboard Metop-B. This enables twice daily (09:30 and 21:30 Local time) global coverage.

Both the temperature and moisture of the troposphere and lower stratosphere are derived under cloud-free conditions, at the pixel level, with a vertical resolution of 1–2 km in the lower troposphere, and an accuracy of 1 K and 10%, respectively. Roughly ten degrees of freedom are estimated as determining the moisture profiles. The IASI weighting functions for water vapor are generally maximum above 700 hPa, so that larger errors can be expected below 3 km (Chazette et al. 2014).

IASI spectra also contain information on the isotopic composition of water vapor (ratio of HDO to H2O) expressed as

$$\begin{aligned} \delta D =1000 \left( \frac{\mathrm {HDO}}{\left. \mathrm {H}_2\mathrm {O}\right| _\mathrm {Vsmow} }-1 \right) , \end{aligned}$$
(1)

where HDO denotes the relative amount of Hydrogen–Deuterium–Oxygen, and is normalized to the amount found in a standard, here taken to be Vienna Standard Mean Ocean Water. Because of the fractionation of the isotopologues during phase changes of water, δD observations are useful tracers of the hydrological cycle (Galewsky et al. 2016) and can provide additional constraints on the origin of water (e.g., González et al. 2016) and/or on the processes that give rise to its presence (e.g., Worden et al. 2007; Risi et al. 2008). δD can be retrieved after an inversion procedure specially adapted to reach the precision requirements of such observation (Schneider and Hase 2011; Lacour et al. 2012). In this study, we also present estimates of δD associated with the two NARVAL campaigns. We used the IASI δD retrievals developed at ULB/LATMOS (Lacour et al. 2012). The retrieved profiles have limited vertical information content (between 1 and 2 degrees of freedom) with a maximum of sensitivity in the free troposphere around 4.5 km, which is near \(z_*.\) These profiles have been cross validated with an error of about 38\({\permil}\). in the free troposphere on a single measurement basis (Lacour et al. 2015).

3 How Lower-Tropospheric Humidity Influences Clouds, Convection and Circulation

Lower-tropospheric humidity is important because it determines the strength and depth of convection and influences circulations and cloudiness, especially in the tropics. Water vapor’s influence can be described in terms of a few quantities. The specific humidity within the PBL, which we denote by \(q_{\mathrm{PBL}}\), the humidity within the lower free troposphere, \(q_\mathrm {LFT},\) as well as the height of the shallow (trade-wind, or fair-weather) cumulus layer, \(z_{\mathrm {i}}.\) A fourth piece of information, describing the bottom or top-heaviness of the relative humidity profile in the lower free troposphere, may also be important.

3.1 Humidity in the Planetary Boundary Layer

PBL humidity, \(q_{\mathrm{PBL}},\) sets the potential for deep convection, as well as influencing cloud amount and surface fluxes. The role of humidity variations in determining cloud amount is discussed by Vial et al. (2017), and our focus is on how \(q_{\mathrm{PBL}}\) influences convection. This can be understood with the help of the strict quasi-equilibrium ansatz for moist convection.

Strict quasi-equilibrium (Emanuel 1986) gives a name to the idea that convection instantaneously adjusts the temperature profile in the free troposphere to render convective clouds rising out of the PBL neutrally buoyant with respect to their environment. Roughly speaking, in the adiabatic limit the convective parcel rises isentropically from the surface, in which case neutral stability is associated with the saturated moist entropy (or the saturated moist static energy, \(h_{\mathrm {s}}\)) in the free atmosphere equalling the moist entropy (or moist static energy) within the well mixed sub-cloud layer (Fig. 4). Hence if the boundary layer moist static energy, \(h_{{\mathrm{PBL}}}\), rises to a value that is larger than \(h_{\mathrm {s}}\) in the free troposphere, then the atmospheric \(h_{\mathrm {s}}\) is immediately adjusted to \(h_{{\mathrm{PBL}}}\)—perhaps corrected for convective downdrafts, which reduce \(h_{{\mathrm{PBL}}}\). In strict quasi-equilibrium, the thermal structure of the tropical atmosphere everywhere is set by the value of \(h_{{\mathrm{PBL}}}\) wherever it maximizes.

Fig. 4
figure 4

Diagrammatic depiction of how lower-tropospheric humidity affects moist convective stability. On the left, the red lines show the saturated moist static energy for a parcel lifted isentropically from the PBL (dashed) or affected by cooling associated with lower-tropospheric mixing (dotted) versus the mean moist static energy (blue). The lower-tropospheric buoyancy sink is expected to be proportional to the environmental moisture deficit, which can be largest at the warmer temperatures of the lower troposphere. Right panel shows how reducing the moisture deficit in the lower troposphere reduces the mixing penalty and allows parcels rising from the PBL to more closely approach their saturated isentrope

The adjustment of the free-tropospheric \(h_{\mathrm {s}}\) is made manifest by fast-moving gravity waves that rapidly act to align density surfaces with geopotential surfaces over areas much larger than that associated with the convection itself. Hence, if convection arises in one region, it will suppress convection in neighboring regions with smaller values of \(h_{{\mathrm{PBL}}}\). An unstable density stratification (convective instability) can thus be generated by reducing the free tropospheric value of \(h_{\mathrm {s}}\), for instance by radiative cooling, or by increasing the value of \(h_{{\mathrm{PBL}}}\) where it maximizes, for instance through surface fluxes. During NARVAL-2, the variation of \(h_{{\mathrm{PBL}}}\) within and around the ITCZ region varied by about 8 kJ kg\(^{-1}\), variations in \(q_{{\mathrm{PBL}}}\) alone (which amount to 3 g kg\(^{-1}\), Fig. 3 ) contribute about 7.5 kJ kg\(^{-1}\) of variation. This type of thinking puts a premium on understanding processes that control, \(h_{{\mathrm{PBL}}}\), and by extension \(q_{{\mathrm{PBL}}}\). Despite its importance for deep convection, \(q_{{\mathrm{PBL}}}\) is poorly quantified over the tropics.

In practice, air does not rise isentropically, as \(h_{\mathrm {s}}\) is reduced by the mixing of convective parcels with environmental air as they ascend. Hence, the entire profile of h can influence convection, not just \(h_{{\mathrm{PBL}}}\). Because moisture anomalies are not communicated globally, values of h in the free troposphere vary more strongly on regional scales than does \(h_{\mathrm {s}}\), whereby the largest differences are concentrated in the lower free troposphere, where temperatures are warmer and differences between \(h_{\mathrm {s}}\) and \(h_{{\mathrm{PBL}}}\) can be larger. Mixing with low moist static energy air in the environment reduces \(h_{\mathrm {s}}\) of the convecting air, acting as a buoyancy sink. Importantly, increasing the ambient moisture in the lower free troposphere reduces the lower-tropospheric buoyancy sink (Fig. 4), and increases the value of \(h_{\mathrm {s}}\) for a test parcel. In such a situation, the temperature of the test parcel will more closely approximate the saturated isentrope, for which its temperature (and buoyancy) will be a maximum. This scenario (right panel of Fig. 4) emphasizes the importance of processes that control not just \(h_{{\mathrm{PBL}}}\) but also the moisture deficit of the lower troposphere (strength of the lower-tropospheric buoyancy sink), and explains why convection starting with the same value of \(h_{{\mathrm{PBL}}}\) is favored in a more humid lower free troposphere.

3.2 Column Water Vapor (Thermodynamic Effects)

There is a long literature discussing the very strong relationship between convective precipitation and column water vapor (see, for instance the discussion of the Kuo scheme in Anthes 1977). Bretherton et al. (2004) show that precipitation picks up exponentially with the increase in column relative humidity, defined as \(W/W_*\) where W is the water vapor path, and \(W_*\) is its saturated value given the temperature profile of the column. Initially found just over the tropical oceans this relationship has, with some caveats, been extended to other regions (Holloway and Neelin 2009; Schiro et al. 2016; Bergemann and Jakob 2016) and may also correlate with variations in shallow precipitation (Nuijens et al. 2009). The scale height of the saturation humidity following a moist adiabat is about 2 km. This means that column water vapor is largely determined by water vapor in the lower troposphere, between the top of the PBL, \(\eta ,\) and \(z_*\) (Raymond et al. 2003). In other words, the observed relationship between column water vapor and precipitation is really a relationship between lower free troposphere humidity and precipitation.

The humidity of the lower free troposphere influences convection because it encodes the potential of mixing to evaporatively cool saturated air—the buoyancy sink (Fig. 4). Such sensitivities are readily evident in cloud-resolving model simulations, but are not always evident in convective parameterizations (Derbyshire et al. 2004; Kuang and Bretherton 2006; Wu et al. 2009). The failure of convective parameterizations to adequately capture this coupling may explain many of the difficulties in representing large-scale circulation features, such as the Madden-Julian Oscillation (MJO) (Bony and Emanuel 2005), processes of convective aggregation (Craig and Mack 2013; Becker et al. 2017), and the structure of the tropical convergence zones (Möbis and Stevens 2012; Oueslati and Bellon 2013).

3.3 Column Water Vapor (radiative effects)

Column water vapor, W,  also strongly influences the distribution of radiative heating within the lower troposphere. In a nearly saturated atmosphere, the boundary layer cools at a rate of about 2 K d\(^{-1}\), commensurate with the cooling rate of the free troposphere.Footnote 1 But as the humidity in the free troposphere is reduced, the cooling rate increases roughly sixfold, inversely with the free troposphere humidity, but increasingly nonlinearly as the free troposphere becomes very dry.

The effect of column water vapor on cooling rates within the PBL is illustrated by considering only the clear-sky cooling rates within the PBL, as a function of free-tropospheric moisture. Cooling rates for three different profiles are calculated and plotted in Fig. 5 (three left panels). In addition, pairs of cooling rates, for the PBL and the free-troposphere, are calculated and plotted versus W (right-most panel) for two further scenarios: one with a PBL depth of 600 m the other with an 1800 m deep PBL, in both cases W is varied by varying the free tropospheric relative humidity. The dry (orange) and moist (blue) profiles in the left panel correspond to special cases of the first scenario and were chosen in analogy to the mean soundings during NARVAL-1 and NARVAL-2 (Fig. 3). Clear-sky cooling rates are peaked near the top of the PBL in each case, but are much stronger for the drier free troposphere as the downwelling infrared radiance is roughly proportional to the number of water molecules above a particular layer (e.g., the cooling to space approximation, cf., Mapes and Zuidema 1996; Thomas and Stamnes 1999). These effects, though less pronounced, remain in force also for the scenario of a deep (1800 m) boundary layer (Fig. 5, right panel). When clouds top the moist layer, they can be expected to increase the outgoing irradiance from the top of the layer, thereby enhancing its cooling rate, but not its sensitivity to the overlying water.

Fig. 5
figure 5

Potential temperature (a), and humidity (b), used to calculate the infrared cooling rates in (c). The three profiles illustrate a very dry free troposphere (orange), with a nearly saturated free troposphere (blue) and with an elevated moisture layer (gray) between the top of the boundary layer and the \(0\,^{\circ }\)C isotherm. d Cooling rates as a function of precipitable water, which is varied by varying the humidity above the boundary layer. Solid lines represent the radiative cooling rate (thermal-infrared only) averaged over the boundary layer and dashed lines the average cooling in the layer (with twice the depth of the boundary layer) above the boundary layer. Two pairs of lines are shown. The one with a larger dynamic range in precipitable water (and more cooling) corresponds to a boundary layer of 600 m in depth and the other to a layer of 1800 m in depth

Averaged over the PBL clear-sky cooling rates can be greater than 10 K d−1 or not differ substantially from the cooling rates within the free troposphere (Fig. 5). The degree of cooling can be expected to influence the state of the boundary layer, and hence surface fluxes, and may drive low-level circulations (Naumann et al. 2017). For instance, circulations associated with very strong radiative cooling rates are thought to be responsible for a net transport of moist entropy into convective regions and out of non-convective regions, thereby reinforcing convective regions at the expense of non-convective regions. Muller and Bony (2015) show that in cloud-resolving model simulations of radiative-convective equilibrium, cooling profiles similar to those shown for the case of the very dry free troposphere (blue line in Fig. 5), extract energy from the boundary layer, leading to the formation of radiatively driven cold pools. By neutralizing this cooling, their simulations no longer aggregate. The importance of these processes for the organization of convection is also emphasized in the review by Wing et al. (this issue). Naumann et al. (2017) recently developed a theoretical framework for studying how efficiently such cooling drives circulations in the lower troposphere. They show that infrared cooling driven by vertical gradients in lower-tropospheric moisture is as effective as differences in sea-surface temperatures, which following Lindzen and Nigam (1987) is the typical paradigm for understanding larger-scale circulations in the tropics.

3.4 Elevated Moist Layers

During NARVAL-2, there is the hint of a layer, or shelf, of constant absolute humidity within the lower free troposphere. Examples of an elevated moist layer are pronounced on individual NARVAL-2 research flights (RF), for instance, on RF03, which took place on August 12, 2016, and for which the moisture profile is plotted in Fig. 6. The sharp jump in \(q_\mathrm {ft}\) around \(z_*\) (which locates at about 4.5 km) may be associated with moisture originating from convective outflow that has been advected away from a more remote source or simply mark the remains of now dissipated local convection, something that an analysis of isotopologues might be able to differentiate. Regardless of its origins it has a strong influence on the infrared irradiance.

Fig. 6
figure 6

Moisture profile from fifty dropsondes released in the box, with corners (52\(^{\circ }\)W, 9\(^{\circ }\)N) and (55\(^{\circ }\)W, 14\(^{\circ }\)N), during the third research flight of NARVAL-2 (August 12, 2016)

Radiative heating responds not just to the total column water vapor, but also how it is distributed within the vertical. Unlike in the case of the dry-tongues analyzed by Mapes and Zuidema (1996), moisture shelves do not lead to a heating-cooling couplet across the top of the melting level, but rather the strong cooling at the top of the moist layer is offset by a reduction of the radiative cooling in the PBL (see Fig. 5). Whereas the cooling in the stably stratified free troposphere can be directly expected to induce subsidence, the reduction of cooling in the PBL will be partly compensated by a reduction in surface enthalpy fluxes (Naumann et al. 2017). However, the maximum of the cooling at the top of the moist layer is as strong as the PBL maximum in the very dry case (orange line) and may supplant very shallow circulations on the scale of the PBL by deeper circulation on the scale of \(z_*\) (Nishant et al. 2016). In contrast, the humid case (blue line) suggests that for a case with the same amount of precipitable water, but with q more bottom heavy so that it smoothly decreases with height, the structure of the resultant heating profile differs markedly. Thus, a fuller understanding of how water vapor in the lower troposphere affects clouds, convection and circulation also requires an understanding of its vertical distribution, as the gradient of absolute humidity is what determines the vertical profile of radiative heating.

4 Remotely Sensed Humidity Variations During NARVAL-1 and NARVAL-2

To better assess the structure of water vapor in the tropical atmosphere, we evaluate to what extent satellite retrievals capture important forms of water vapor variability as measured during the NARVAL series of campaigns. To do so, we compare water vapor retrievals from the SAPHIR and IASI instruments with dropsonde and water vapor lidar data collected during NARVAL-1 and NARVAL-2 research flights. Because of its better temporal sampling and its ability to extract water vapor information also in the presence of clouds, we focus on SAPHIR.

4.1 General Structure of Humidity Retrievals from SAPHIR

SAPHIR retrievals capture well the main, climatological, features of the humidity field as seen in the dropsonde data across the entire NARVAL-1 and NARVAL 2 periods. During NARVAL-1, an exceptionally dry free atmosphere (Fig. 7, left panel), with humidities well below 20%, is consistent with the dropsonde data presented in Fig. 3. A broader distribution of relative humidities between 750 and 800 hPa (SAPHIR layer 5) is consistent with the top of the shallow cumulus (trade-wind) layer being near this level—drier conditions being indicative of days with a shallower cloud layer. The August NARVAL-2 flights predominantly differ through a moister, and more variable, free troposphere as compared to the December NARVAL-1 flights. Figure 7 shows these differences to be evident at every level, but most pronounced between 400 hPa and 800 hPa, consistent with the dropsonde data in Fig. 3.

Fig. 7
figure 7

Vertical profiles of relative humidities from SAPHIR for the flight area of NARVAL-1 and NARVAL-2

Our further evaluation of SAPHIR focuses on two flights: one (RF07 on December 19, 2013) from NARVAL-1 and the other (RF03 on August 12, 2016) from NARVAL-2. The humidity retrievals over the six SAPHIR layers are presented for these two flights in Fig. 8. The area of flight operations, for the case of NARVAl-2, and the flight track, for the case of NARVAL-1, are overlain. The main features of the humidity field are the sharp decline of humidity with height on both swaths, and clear meridional excursions of moist tropical air in the lower free troposphere—easterly waves. On 19 December (NARVAL-1), an easterly wave is evident in the eastern third of the swath, and the humidity in the lower troposphere is in general confined to a shallower layer. On 12 August, two easterly waves are evident, one just east of the area of flight operations at 50\(^{\circ }\)W and 12\(^{\circ }\)N, the other extending further to the north and locating east of 30\(^{\circ }\)W. Neither of these meridional extended moist features is as deep, or as condensate laden, as the feature sampled on December 19, 2013 (NARVAL-1). To the extent convection can be associated with regions where retrievals (due to hydrometeor scattering) are not possible, for instance near (50\(^{\circ }\)W, 10\(^{\circ }\)N) it forms as expected in regions of exceptionally high near-surface humidities, not to mention a deeper layer of moisture, as anticipated by some of the arguments reviewed in Sect. 3.2 above. There is also some evidence of drier air near the surface in the vicinity of pixels where hydrometeor scattering resulted in failed retrievals. This would be consistent with downdrafts bringing entropy-poor (i.e., drier) air to the surface in the vicinity of deep convection. There also appears to be an imprint of the drier layers aloft also on the structure of the near-surface relative humidities; this is evident, for instance, in the large crescent-shaped features on December 19, 2013, and the dry feature aligned with the southern end of the track, features that are also evident in the form of the banded features in the cloud imagery from that day (Fig. 9).

Fig. 8
figure 8

Example of SAPHIR retrievals along orbit swaths coincident with NARVAL-1 RF07 (left) and NARVAL-2 RF03 (right). Shown are the relative humidities in the six layers over which humidity retrievals are performed. The HALO flight track and approximate time of the Megha-Tropiques overpass is indicated by the dashed line and circle in the left panel (NARVAL-1). In the case of NARVAL-2, the approximate HALO flight area is shown as a rectangle with a dashed perimeter

Fig. 9
figure 9

NASA Worldview MODIS (Aqua) visible imagery for NARVAL-1 RF07 (upper) and NARVAL-2 RF03 (lower). The swaths span the area from 20\(^{\circ }\)W–70\(^{\circ }\)W and from 0\(^{\circ }\)N to 20\(^{\circ }\)N identical to the bounding area for the SAPHIR swaths in Fig. 8

4.2 Evaluation of SAPHIR Retrievals by WALES

To get a better idea of what types of features can be reliably detected by the SAPHIR retrievals, and whether the apparent coincidence between cloudiness and features in the lower-tropospheric humidity field are real, the humidities in the five lower SAPHIR layers are compared to the WALES water vapor profiles. No comparison is made in the first (uppermost) layer because WALES at most sampled the lower 10 hPa of this level, and then only during NARVAL-1. Temperature fields from ECMWF analyses interpolated in space and time to the flight path are used to convert water vapor densities measured by the lidar into relative humidities. A comparison of the SAPHIR-retrieved relative humidities with those from WALES is presented along the cross section over the Atlantic for December 19, 2013 (NARVAL-1) in Fig. 10. Figure 11 shows the comparison of the SAPHIR humidity retrievals with WALES over the area of flight operations for August 12, 2016 (NARVAL-2).

Fig. 10
figure 10

Comparison of relative humidity profiles on December 12, 2013. Shown are measurements from the WALES instrument aboard HALO as part of NARVAL-1 and the Megha-Tropiques profiles along the same track. The time of the Megha-Tropiques overpass is indicated by the vertical dashed line. SAPHIR retrievals are shown (saphire-blue) bounded by the first and third interquartile uncertainty, WALES estimates show in ruby-red

Fig. 11
figure 11

Comparison of relative humidity profiles from the WALES instrument aboard HALO as part of NARVAL-2 RF03 (on August 12, 2016) and the Megha-Tropiques profiles along the same track. The overpass time of the Megha-Tropiques satellite was at 20:52 UTC. Color scale as in previous figure

For the 19 December comparison, SAPHIR retrievals of relative humidity agree well with WALES at most levels, especially in capturing the transition between the deep moist layer in the eastern part of the domain (Fig. 10) as compared to the much drier air to the west. Some differences are evident in fine-scale features, but these may arise due to a lack of coincidence. During NARVAL-1, SAPHIR also captures the vertical gradient in humidity, particularly when one compares layer 4 (relative humidities below 5%) with layer 6, where relative humidities are near 80%. Most remarkable, Fig. 10 suggests that the variations in the SAPHIR layer 6 retrievals, which correlate with patterns of cloudiness (Fig. 9), are also measured by WALES. Here our eye is drawn to what appears to be a long downward intrusion of dry air along a north–south-oriented line near 51\(^{\circ }\)W (Fig. 8) in the SAPHIR data and sampled at around 1840 UTC by WALES. A second, somewhat broader, feature is also apparent at the very end of the record, near 51\(^{\circ }\)W. These features are also evident in the layer 5 (2–2.6 km) retrievals, but much less pronounced than in WALES. These differences illustrate the challenges that retrievals from passive sensors face in the vicinity of a strong humidity gradient (80–10%), where the interpretation of the upwelling irradiances is not clear cut. In this case, the breadth of the sensitivity functions probably also leads to some of the moisture from the lower layers being interpreted as having arisen higher up. Earlier in the flight (before 1730 UTC), where the moisture is more homogeneous, the SAPHIR retrievals match the measurements by WALES quite well.

Considering their lack of coincidence, the SAPHIR humidity retrievals on 12 August also appear to capture the variability in relative humidity in the upper troposphere, and the gravest features of the vertical structure. In the lowest level (layer 6) near 1 km, across which the humidity profile varies as one expects climatologically, the humidities retrieved by SAPHIR are similar to that observed by WALES, with commensurate variability. On a layer-by-layer basis, particularly within the lower free troposphere, the retrievals have less skill. This appears to largely reflect the non-trivial vertical structure in the water vapor field, wherein a more humid layer, with relative humidities peaking near 4 km, overlays a relatively drier layer. The humidity structure measured by WALES is consistent with measurements from the dropsondes (Fig. 6), with absolute humidities constant or slightly increasing between about 1.5 and 4.5 km. These features are not well differentiated by the layer 4 and layer 5 SAPHIR retrievals (Fig. 11). SAPHIR under estimates the humidities (by about 15% absolute) around 3.4 km (its level 4) and compensates with commensurately higher humidities one kilometer lower (its level 5). Here again the broadness of the SAPHIR sensitivity functions makes it difficult to unravel the moist layer near 4 km from the absolutely drier layer aloft and a relatively drier layer below. When layers 4 and 5 are taken together, the SAPHIR retrievals capture the horizontal humidity variations seen in the WALES data below 3 km, but we interpret them in terms of a more climatological vertical structure, wherein relative humidity is strictly decreasing with height.

4.3 Comparison with IASI and the Added Value of Retrieving Isotopologues

Because the area of aircraft operations on August 12, 2016, was relatively cloud-free, we can explore whether IASI humidity retrievals are, by virtue of more independent information, better able to measure the vertical structure of water vapor in the lower free troposphere. Despite better vertical resolution, IASI retrievals are biased similarly to the retrievals using SAPHIR measurements. Neither instrument resolves the relative humidity inversion (relative humidity increasing with height between 2 and 4 km, Fig. 6) seen on August 12. In contrast to SAPHIR, IASI also provides retrievals within the PBL (below 1 km). But the IASI humidity retrievals are indicative of a substantially moister (10% absolutely) and shallower PBL as compared to what is measured in situ, by the sondes (Fig. 12).

Fig. 12
figure 12

Vertical profiles of relative humidities from SAPHIR and IASI (left) and from the dropsondes (right) for the flight area of RF03

Fig. 13
figure 13

Retrieved humidity (left) and δD anomaly (from domain mean) at 660 hPa for August 12, 2016, 21:30 Local Time IASI overpass

One would expect different isotopic signatures for water subsiding downward, versus mixing upward. The former, for instance following from convective detrainment near \(z_*\), would be associated with a subsiding moist layer and is consistent with the vertical humidity structure between 2 km and 4 km on 12 August. If this interpretation is correct, then the HDO/H2O ratio (as measured by δD, Eq. 1) should follow the Rayleigh distillation value associated with water detrained at \(T_*.\) In the latter case, where water is mixed upward out of the boundary layer, δD would be relatively enriched. Differences in the pattern of retrieved specific humidity near \(z_*\) (660 hPa) and δD are evident in the IASI retrievals (Fig. 13), allowing us to test this hypothesis, or at least investigate to what extent measurements of δD might help constrain the vertical structure of water vapor.

We examine co-variability between retrieved humidity, q,  and δD we focus again on the December 19, 2013, and August 12, 2016, cases, as shown in Fig. 14. In this example, different mixing processes are illustrated by lines in qδD space. We expect water originating from condensate evaporation aloft to follow the Rayleigh distillation curve, where as mixing of moister air from the lower atmosphere with much drier air that has subsided from a great altitude would be relatively enriched and follow the mixing curve (see also Risi et al. 2008; Galewsky et al. 2016). The August retrievals more closely follow the Rayleigh distillation curve than do the December profiles (Fig. 13), increasingly so as one approaches \(T_*\). The diagrams are consistent with the idea that the humidity profiles originate more from preferred detrainment near \(T_*\) in the August period, whereas in the December period the humidity profiles are more characteristic of mixing up of moist PBL air into much drier free tropospheric air. In this latter (December) case, one would not expect relative humidity reversals. Whether this information can be more effectively used to robustly constrain the retrievals merits further study.

Fig. 14
figure 14

\(q{-}\delta {D}\) diagram showing the variation of specific humidity, q and δD from retrievals at different heights. Left panel shows a schematic of how q and δD would vary following specific processes. Retrievals from NARVAL-1, for December 19, 2013, between 12\(^{\circ }\)N–20\(^{\circ }\)N and 40\(^{\circ }\)W and 60\(^{\circ }\)W, (middle) and NARVAL-2 for August 12, 2016, in the same area (right)

Despite a hint that retrieved isotopologues might help improve humidity retrievals, the overall impression from this analysis is that it is difficult for passive remote sensing to accurately capture non-trivial variability in the vertical distribution of water vapor in the lower troposphere. In the lower troposphere, above the PBL (where variations in water vapor are largest and also crucial for a variety of processes) the retrievals appear to capture some of the variability that is measured by both the dropsondes and the water vapor lidar (WALES), but misinterpret deviations from the climatological vertical structure encoded in the a priori estimates. The situation on August 12, 2016, is particularly challenging, whereby the water vapor retrievals from both the IASI and SAPHIR measurements yield a relative humidity profile that decreases, rather than increases, with height. This type of bias will cause a large error in the calculation of radiative heating rates (as shown in Fig. 5) which influence inferences as to the nature of circulations in the lower-troposphere. These biases are not as evident in the relative humidity profiles taken from the European Centre Analyses (not shown) for reasons that we can only speculate about. Nonetheless, the challenge in retrieving non-trivial profiles of humidity from passive measurements, such as those made by SAPHIR and IASI, could bias lower free troposphere humidity estimates to be bottom heavy.

5 A Hypothesis for the Preponderance of Melting Level Convection

There is a substantial literature documenting the prevalence of moisture and clouds in the middle troposphere, near \(z_*.\) Typically this is thought to be concentrated in the vicinity of deeper convection, similar to what was observed during NARVAL-2, where measurements took place in and around the ITCZ. In an analysis of data from the Tropical Ocean Global Atmosphere Coupled Ocean Atmosphere Response Experiment (TOGA-COARE), Johnson et al. (1999) accumulated a wide array of evidence from historical measurements dating back to the middle of the last century to make the case that there was a preferred mode of cloudiness near \(z_*\). To the extent that there is preferential detrainment near \(z_*\) it is usually attributed to enhanced stability at this level associated with the melting of ice at temperatures warmer than \(T_*\).

We hypothesize that the energy required for initial ice formationFootnote 2, which makes it difficult to form ice until ice-supersaturation is very high or temperatures are much colder than \(T_*\), combined with the radiative response to sharp moisture gradients (cf. Sect. 3.4) as may arise from a preferred level of convective detrainment, further favor convective outflow at temperatures between \(T_*\) and \(T_{\mathrm{f}},\) that required for primary ice formation (cf, Fig. 1).

Fig. 15
figure 15

Adaptation of Fig. 4 to illustrate how ice processes can act to stabilize the atmosphere near the melting level, so that the production of first ice inhibits remote convection at locations where ice must form by primary processes, from penetrating much above the melting level. Moisture gradients and radiative processes as the melting level reinforce this process

This hypothesis is illustrated with the help of Fig. 15. In the first panel, we envision a scenario in which, for some reason, moist convection reaches a height (or temperature) where ice is initiated. The freezing of condensate releases the enthalpy of fusion; this raises \(h_{\mathrm {s}}\) by allowing parcels to rise isothermally until all the existing condensate freezes, thereby augmenting the buoyancy of the rising air. In a second step (shown in the second panel of Fig. 15), secondary ice-formation processes (for instance by contact of supercooled water with descending ice crystals) contribute to the glaciation of the entire convective system down to \(T_*\). This further augments the convection, because the cloud may also be more condensate laden at the warmer temperatures near \(z_*\). Subsequent convection, in so far as it develops in a region where ice has not been initiated, will then encounter an environment more hostile to convective development for \(T_*> T > T_{\mathrm{f}}.\) This enhances the effective stability felt by subsequent convection reaching heights above \(z_*\) but below \(z_{\mathrm {f}}\). Hence for convection to penetrate through this region, it will have to have started from substantially more favorable conditions (higher \(h_{{\mathrm{PBL}}}\) or a moister lower troposphere) than the convection that first reached the ice-initiation level.

Enhanced detrainment from convection not strong enough to initiate ice will be associated with enhanced cloudiness and moisture gradients at \(z_*.\) This in turn initiates a radiative driven circulation feedback that further enhances the mid-level termination of convection. Clear-sky heating rates more negative than −20 K d\(^{-1}\) arise in response to strong moisture gradients near \(z_*\) (see Sect. 3.4). Even after averaging over a 1 km deep layer, the net cooling is many times larger than the background cooling that one expects in the absence of such a moisture gradient. As illustrated schematically in Fig. 16, this leads to greater detrainment at \(z_*\): both directly, as a result of the enhanced mass divergence arising from this cooling (Bony et al. 2016); and indirectly, because of the stabilization with respect to the layer above \(z_*\), which then makes it more difficult for convection to penetrate beyond this level. In analogy to the term used to explain the observed geographic preference for consumer brands that first occupy a market (Bronnenberg et al. 2009), we refer to this as the ‘first-mover’ advantage. We expect these ideas to be most pertinent in an atmosphere where primary ice formation is relatively difficult, so that \(z_{\mathrm {f}}- z_*\) is large, which could introduce a role for aerosol-cloud interactions.

Fig. 16
figure 16

Conceptual diagram illustrating how detrainment of moisture at the freezing level leads to moisture gradients, more radiative cooling from the top of this level, and a large-scale circulation that favors further detrainment

The NARVAL-2 dropsondes support the idea that there is a preferred moisture source at \(T_*.\) This is evident in Fig. 3 where the median humidity is nearly constant within the layer we refer to as the lower free troposphere, i.e., between the top of the shallow cloud layer and \(z_*.\) Another way to see this is to plot the saturation temperature of a given sonde measurement versus the temperature at which it is measured. This is shown in Fig. 17. If the humidity at a given temperature is controlled by the humidity that the sampled air had at last saturation (Sherwood et al. 2010), and there was no preferred temperature of the mid-tropospheric moisture source, we would expect \(T_{\mathrm {sat}}\) to be distributed between T and some minimum T characteristic of the lowest value of temperatures in the troposphere. Because tropospheric mixing is more efficient along isentropes (roughly isotherms), and air that was last saturated at much colder temperatures must have travelled a greater distance through the troposphere, \(T_{\mathrm {sat}}\) should be less than, but roughly follow T,  as for instance for \(T<0\,^{\circ }\)C in Fig. 17. A surprising feature in Fig. 17 is the pronounced range of temperatures, between approximately 2 and \(14\,^{\circ }\)C, over which \(T_{\mathrm {sat}}\) is roughly constant, with a value slightly below \(T_*.\) This is consistent with air in the lower free troposphere (\(z_\mathrm {i}< z < z_*\)) being associated with a preferred moisture source at temperatures at or slightly below \(T_*,\) as would occur for convective clouds preferentially terminating between \(z_*\) and \(z_{\mathrm {f}}.\)

Fig. 17
figure 17

NARVAL-2 dropsondes saturation temperature (dew-point) versus temperature

An analysis of data from the Barbados Cloud Observatory, where water vapor is profiled by a Raman lidar, supports the idea of a preferred moisture source at temperatures near or slightly below \(T_*\). This, we claim, can best be seen by looking at something we call the tropospheric dry point, i.e., the levels within the troposphere where the humidity minimizes. On many days there is an additional relative humidity maximum below this dry point, but above the climatological cloud-base relative humidity maximum. In Fig. 18, this humidity minimum varies with its base between \(z_*\) (located near 4.5 km) and 6 km. Humidity profiles in other months, for which episodes of deep convection are present, also show similar features. In addition to a more quantitative evaluation of these data, these ideas could be tested by cloud-resolving models, for instance by varying \(T_{\mathrm{f}}\) but allowing for secondary ice formation.

Fig. 18
figure 18

Blended Raman and sounding night-time moisture (relative humidity) over the Barbados Cloud Observatory during August 2016, the period of NARVAL-2 operations. The white and black dashed lines are drawn free-hand to emphasize the base of the middle tropospheric dry layer (white lines) below which humidity increases to a secondary maximum (in places) black lines

6 Conclusions

We reviewed mechanisms through which water vapor in the lower troposphere influences clouds, convection and circulation. For our purposes, the lower troposphere is defined as heights where the temperature is above the triple-point temperature, \(T_*.\) In the present-day tropics, this locates at a height \(z_* \approx 4.5\) km. Three aspects of the water vapor are concentrated on: (1) the amount within the PBL, roughly the well-mixed layer extending from the surface to the cloud-base height, \(\eta\) at about \(\eta \approx 0.6\) km; (2) the amount within the lower troposphere, between \(\eta< z < z_*\); and (3) the way in which the water vapor is distributed over the lower-troposphere. The latter factor can be quantified by the depth of the shallow cumulus layer, and the amount of water vapor between the top of that layer (at about 2.5 km) and \(z_*\). Boundary layer water vapor determines the convective potential, which can be thought of in terms of the potential energy that could be consumed by a convective overturning. The amount of lower-tropospheric water vapor determines the strength of the lower tropospheric buoyancy sink, which inhibits the vertical penetration of convection through the stabilizing effect of mixing with dry air. The amount and distribution of water vapor in the lower troposphere strongly influences the rate of radiative cooling of the PBL. Bulk cooling of the PBL can vary by a factor of three to five depending on the depth of the PBL and the amount of water vapor above. If the water vapor is distributed so that its relative humidity is relatively constant with height, the cooling within the lower free troposphere also varies little with height. Water vapor profiles for which the absolute humidity is approximately constant with height over a given layer are accompanied by large and sharp water vapor gradients at the top of this layer. These gradients concentrate the radiative cooling at the top of the moist layer, which may much more effectively drive circulations and help decouple the atmosphere in the vertical.

We analyzed in situ and remotely sensed measurements of lower-tropospheric water vapor measurements from two field campaigns (NARVAL-1 and NARVAL-2). The NARVAL missions were flown over the tropical Atlantic, between 45 and 60\(^{\circ }\)W and 5 and 20\(^{\circ }\)N. NARVAL-1 was flow in December 2013 and primarily sampled the winter trades north and adjacent to the ITCZ. NARVAL-2 flew in and around the ITCZ during August 2016. Water vapor was measured in situ by dropsondes, with more than 250 launched over the target region, and by a downward looking differential absorption lidar designed to measure water vapor throughout the troposphere. In terms of lower-tropospheric water vapor, the main difference between the campaigns is in the height of the shallow cumulus layer rising out of cloud base, and the amount of moisture above it, in what we call the lower free troposphere (Fig. 1). During the winter trades, the layer above the shallow cumulus layer is very dry and an extension of the free troposphere above. In air-masses within and near the ITCZ, the lower free troposphere is much moister and more continuous with the layers below. During NARVAL-2, most of the variation in boundary layer moist static energy can be explained by variations in boundary layer humidity, and elevated moisture layers, wherein absolute humidity was approximately constant.

We evaluated relative humidity retrievals from the SAPHIR instrument aboard the Megha-Tropiques satellite as well as from the IASI instrument aboard the Metop satellites. The SAPHIR retrievals are interesting because of their high temporal sampling of specific locations within the tropics, their ability to penetrate all but the thickest clouds, and their emphasis on the lower troposphere, with three of the levels retrieved corresponding to altitude ranges below \(z_*.\) In clear-sky conditions, the global coverage and richer spectral coverage of IASI better resolves the water vapor vertical structure. Despite an emphasis on the lower troposphere, the SAPHIR retrievals do not include a layer that overlaps with the PBL, leaving us relatively blind to variations of water vapor in this all important level. The SAPHIR retrievals capture the main differences between NARVAL-1 and NARVAL-2, and large-scale changes in the distribution of humidity associated with an Easterly wave. SAPHIR retrievals also show some skill in identifying large-scale dry filaments, interpretable as depressions in the height of the shallow cumulus layer (2–2.6 km) that are well correlated with meso-\(\alpha\) scale patterns in cloudiness.

Non-trivial variations in the vertical structure of the humidity field in the lower troposphere (SAPHIR layers 4 and 5) are less reliably retrieved. SAPHIR-based retrievals of humidity fluctuations at the base of the shallow cumulus layer show relatively little correspondence to the lidar measurements. Relative humidity reversals, whereby in a layer of fixed absolute humidity the relative humidity increases rather than decreases with height, prove challenging to identify in the SAPHIR retrievals, which instead places the moisture in the lower levels, more consistent with relative humidity being constant through the layer. IASI retrievals are characterized by similar biases, although here the retrievals of isotopologues should be explored as a basis for differentiating between top and bottom heavy humidity profiles. The biases in the passive remote sensing that this study identifies could result in a substantial underestimation of the humidity within the lower free troposphere (between 3 and 5 km).

The frequency with which elevated layers of nearly constant humidity are found in the soundings of the NARVAL-2 flights is used to advance a hypothesis for a microphysical control on mid-level cloudiness. Whenever there is a large difference between the temperature, \(T_{\mathrm{f}},\) at which primary ice formation occurs, and the triple-point temperature, \(T_*\), we hypothesize a ‘first-mover’ advantage for convection, whereby the first clouds sufficiently deep to form ice stabilize the atmosphere in a way that makes it more difficult for subsequent convection to reach the level at which primary ice forms. This leads to increased detrainment at levels above \(z_*\) but below the level of primary ice production. Detrainment or convective deposition of moisture at temperatures between \(T_*\) and \(T_{\mathrm{f}}\) is self-reinforcing as strong radiative cooling associated with this moisture increases the atmospheric stability across the top of the developing moist tongues, and supports stronger subsiding circulations, which also contributes to the termination of convection at heights below \(z_{\mathrm {f}}.\) The first-mover advantage for glaciating convection may help explain the preferred mode of cloudiness at temperatures near the triple point.