Automatic correction of barometric altimeters using additional air temperature and humidity measurements

Abstract

A GPS module is often combined with an atmospheric pressure sensor in onboard navigation systems. Thanks to this, using appropriate algorithms comparing the results obtained from GPS and from the barometric altimeter, it is possible to obtain much more accurate indications of the height above sea level. The authors analyzed the problem of correcting barometric altimeter errors caused by non-standard pressure and temperature conditions at sea level and the presence of water vapor in the air. The authors proposed methods of correcting the altimeter error using additional temperature and air humidity measurements. They also proposed a simple method for calibrating the altimeter prior to the mission to estimate the difference in pressure above the standard value. The paper discusses the influence of water vapor on the accuracy of the altimeter indications and the conditions for the applicability of the error correction method. The authors initially considered the possibility of using the method based on the concept of virtual temperature. However, this method was rejected because simulation studies showed that it did not give satisfactory results. Simulation tests were carried out in the MATLAB environment to assess the effectiveness of the proposed methods. The test results turned out to be satisfactory. They can be useful when designing low-cost navigation systems using simple sensors made in MEMS technology. The main envisaged application is in autonomous drone navigation systems.

Introduction

Drones are now gaining importance in both civil and military applications (Hassanalian and Abdelkefi 2017; Mogili and Deepak 2018; Hiebert et al. 2020). Since these are frequently autonomously flying devices, an efficient navigation system enabling precise positioning is very important. One of the important parameters of the flight is its height above sea level. Knowing the GPS coordinates and having a digital terrain model (DTM) at disposal, one can easily determine the height of the terrain above sea level and plan the cruising altitude (e.g., Matyja et al. 2022a, b). Unfortunately, the altitude itself given by civilian GPS systems is burdened with a significant error, up to about twenty meters (Kim and Park 2012). For this reason, two cooperating measuring devices are often used simultaneously: a GPS system and a barometric altimeter (e.g., Nakanishi et al. 2012; Zaliva and Franchetti 2014). There are also solutions that additionally use data obtained from the accelerometer (Jaewon et al. 2004; Tanigawa et al. 2008; Popowski and Dąbrowski 2008). The cost of such solutions is low, because cheap and sufficiently precise measuring sensors made in MEMS technology (Chin et al. 2011; Eswaran and Malarvizhi 2012) are available on the market. Other available height measurement techniques (radar, laser) require the use of much more expensive measuring devices and are therefore less often considered.

The barometric altimeter system converts pressure to altitude using the International Standard Atmosphere (ISA) model. If other than standard atmospheric conditions occur at sea level, then the indications are burdened with the so-called principal error, which is relatively easy to correct if, for example, current information from weather stations is available (Matyja et al. 2022a, b). The standard atmosphere does not take into account the presence of water vapor in the air, which will also be a source of error, classified as an error of external conditions (Bao et al. 2017). The inevitable and uncorrectable altimeter error is the drift error (Sabatini and Genovese 2013).

This paper analyzes the problem of correcting barometric altimeter errors caused by both non-standard pressure and temperature conditions at sea level and the presence of water vapor in the air. It is a natural extension of two other papers published by the authors (Matyja et al. 2022a, b; Stanik et al. 2022). The first of them discusses the classification of altimeter errors and gives the theoretical basis for the possibility of correcting these errors. Then, the height indications recorded by the bicycle computer were compared with the height obtained on the basis of data from the digital terrain model (DTM). This was possible because the computer also recorded the GPS coordinates of the route. Numerical analyses were carried out to separate the obtained total error into individual components. The second paper focuses on the error of the altimeter resulting from the measurement of dynamic pressure. Theoretical considerations of the influence of the movement of the pressure sensor relative to the air masses were carried out and a method of correcting such an error was also proposed. A simple experiment confirmed the usefulness of the method.

The drone measurement system can measure pressure and temperature simultaneously. The comparison of the measured temperature with the temperature calculated on the basis of the standard atmosphere model became the basis for the proposed method of correcting the altimeter error with the use of additional temperature measurements. The effectiveness of the method depends on a good estimate of sea level pressure conditions. For this reason, we propose a simple method of calibrating the altimeter before the drone takes off, enabling the estimation of the pressure increase above the standard value. Next, the influence of water vapor on the accuracy of the altimeter indications and the conditions of applicability of the possible method of error correction are discussed. A method of correcting the altitude error caused by air humidity in the low troposphere (additional humidity measurement) was proposed while rejecting the method using the concept of virtual temperature described in the literature (Guinn and Barry 2016). In the MATLAB environment, pressure and temperature distributions were generated depending on the geopotential altitude above sea level in humid, non-standard atmospheres. These distributions were used to calculate the value of the height error before and after the correction. Simulation tests performed in this way allowed us to positively assess the effectiveness of the proposed methods. The results of the presented analyses can be helpful when designing low-cost autonomous drone navigation systems. Particularly, sensors of atmospheric pressure, temperature and humidity, made in MEMS (MicroElectroMechanical System) technology, are now easily available and very cheap (Shuaibu Hassan et al. 2018).

Classification of barometric altimeter errors

A barometric altimeter converts pressure readings into altitude using a well-known pressure altitude equation (Diston 2009; Matyja et al. 2022a, b):

$$h=\frac{{T}_{0}}{{\lambda }_{0}}\left[{\left(\frac{p}{{p}_{0}}\right)}^{-\frac{R{\lambda }_{0}}{{g}_{0}}}-1\right]$$

(1)

where $h$ is the geopotential height, $p$ is the measured pressure, $R$ is the gas constant of dry air, ${\lambda }_{0}$ is the temperature gradient in the troposphere, and ${p}_{0},{T}_{0},{g}_{0}$ are standard pressure, temperature, and acceleration, respectively, due to gravity at sea level (U.S. Standard Atmosphere 1976).

The altimeter gives geopotential height, which differs from the actual (geometric) height according to the relation:

$$z=\frac{{r}_{0}h}{{r}_{0}-h}$$

(2)

where ${r}_{0}$ is the averaged radius of the earth. Due to the large radius of the earth, differences between geopotential and geometric height are very small and are usually neglected in the troposphere.

Some authors suggest using the density height equation (Guinn and Barry 2016).

$$h=\frac{{T}_{0}}{{\lambda }_{0}}\left[{\left(\frac{p}{{p}_{0}}\frac{{T}_{0}}{T}\right)}^{-\left(\frac{R{\lambda }_{0}}{{g}_{0}+R{\lambda }_{0}}\right)}-1\right]$$

(3)

Since the density of air is more difficult to measure, it can be replaced by the ratio of pressure and temperature using the ideal gas law. The measurement error of the geopotential height determined by the barometric altimeter can be divided into three components (Bao et al. 2017):

$$\delta h=\delta {h}_{p}+\delta {h}_{e}+\delta {h}_{d}$$

(4)

where $\delta {h}_{p}$ is the principal error, $\delta {h}_{e}$ is external disturbance error, and $\delta {h}_{d}$ is drift error. The principal error $\delta {h}_{p}$ occurs when at sea level there are non-standard conditions of pressure and temperature ${p}_{0}^{\prime}={p}_{0}+\varDelta p, {T}_{0}^{\prime}={T}_{0}+\varDelta T$. Such an atmosphere is called non-standard and can be denoted by $\mathcal{A}(\varDelta T,\varDelta p)$. The principal error can be effectively corrected:

$$\begin{aligned} h_{{{\text{corr}}_{{{\text{principal}}}} }} & = h + \frac{R}{{g_{0} p_{0} }}\left( {T_{0} + \lambda_{0} h} \right)\varDelta p \\ & \quad + \frac{h}{{T_{0} }}\varDelta T + \frac{R}{{g_{0} p_{0} }}\left( {1 + \lambda_{0} \frac{h}{{T_{0} }}} \right)\varDelta T\varDelta p \\ \end{aligned}$$

(5)

by adding the adjustment to the height indicated by the altimeter (Matyja et al. 2022a, b).

External disturbance error is caused by external factors interfering with the pressure sensor measurements. One of them may be the measurement of dynamic pressure instead of static in the case of the sensor movement relative to the air. The method of correcting such an error is discussed in detail in Matyja et al. (2022a, b) and Stanik et al. (2022). Another cause of the error $\delta {h}_{e}$ is the presence of water vapor in the air, which changes the value of the equivalent gas constant. The proposed method of correcting the error caused by air humidity will be discussed in point 5 of this article.

The last type of error, i.e., the drift error $\delta {h}_{d}$ , is difficult to describe theoretically. It is a periodically variable signal correlated with slow changes in pressure, which may result from weather phenomena. Drift is also observed in long-term measurements, and the drift trend can change over time. Studies indicate that the drift signal is non-stationary, but its first derivative has the characteristics of white noise. Therefore, the drift error can be modeled as a stochastic random walk process (Goff and Gratch 1946; Matyja et al. 2022a, b).

Simulation studies indicate that non-standard pressure and temperature conditions at sea level $\mathcal{A}(\varDelta T,\varDelta p)$ have the greatest contribution to the altimeter error. Figure 1 compares the errors to be expected using the pressure height formula and the density height formula under different non-standard temperature conditions (deviated by $\varDelta T$) at standard pressure $\mathcal{A}\left(\varDelta T,0\right)$ and vice versa under different non-standard pressure conditions (deviated by $\varDelta p$) at standard temperature $\mathcal{A}(0,\varDelta p)$. The black dashed straight line represents the zero error when the pressure and temperature conditions are standard.

Fig. 1

Altitude determination errors in non-standard conditions (on the left—from the formula for pressure height, on the right—from the formula for density height)

As shown in Fig. 1, only in standard conditions do the formulas for pressure height and density height give the same results. Another disadvantage of the density height formula is that in the case of non-standard sea level conditions, the errors in the calculation of the geopotential height can be much larger. This is the cumulative effect of pressure and temperature errors.

A method of correcting part of the principal error based on an additional temperature measurement

The general idea of the method is based on the assumption that air temperature decreases linearly with height above sea level. The measured temperature is compared with the temperature calculated from the altitude indicated by the barometric altimeter. The temperature difference allows us to correct the temperature conditions at sea level.

In a non-standard atmosphere $\mathcal{A}(\varDelta T,\varDelta p)$ , a distribution of pressure and temperature depending on the altitude can be generated. At the selected height $h$ in the non-standard atmosphere, the pressure $p$ and temperature $T$ prevail, and these values will be measured by sensors.

Let us assume that a barometric altimeter calibrated to standard conditions $\mathcal{A}(0,0)$ will measure the pressure height ${h}_{p}$. By comparing it with the actual height $h$ the altimeter error $\delta h={h}_{p}-h$, can be calculated, which will allow us to assess the usefulness of the method. Assuming a constant temperature gradient according to ISA and the standard sea level temperature ${T}_{0}$, the temperature ${T}_{{h}_{p}}$ corresponding to the height of ${h}_{p}$ (in atmosphere $\mathcal{A}(0,0)$) can be calculated. The measured temperature $T$ differs from the temperature ${T}_{{h}_{p}}$ by the value $\varDelta {T}_{1}=T-{T}_{{h}_{p}}$. Since ${h}_{p}$ is not determined exactly, this will be a first approximation of the value of $\varDelta T$. The procedure should be repeated iteratively until a good match between $\varDelta {T}_{1}$ and $\varDelta T$ is obtained. This is described in the following pseudocode:

• 1. Measure $p$, $T$

• 2. Set $\varDelta T=0$ and $\varDelta p=0$ (or $\varDelta p\ne 0$ if current sea level pressure is available)

• 3. In standard ISA condition calculate:

  $${h}_{p}=\text{Pressure Altitude}\,(p)$$
  $${h}_{p}={h}_{p}+\text{Principal Error Correction}\,({h}_{p},\varDelta T,\varDelta p)$$
  $${T}_{{h}_{p}}=\text{Temperature At Altitude}\,({h}_{p})$$

• 4. Calculate $\varDelta {T}_{1}=T-{T}_{{h}_{p}}$

• 5. If $\left|\varDelta {T}_{1}-\varDelta T\right|\le \varepsilon$ then stop else set $\varDelta T=\varDelta {T}_{1}$ end go to 3.

Table 1 shows exemplary results of the corrective algorithm. It is most effective in $\mathcal{A}(\varDelta T,0)$ atmospheres. Usually, a few iterations are enough to reduce the height measurement error to practically zero. The algorithm performs worse in $\mathcal{A}(\varDelta T,\varDelta p)$. If the $\varDelta p$ value can be given, for example, on the basis of weather station reports, then the algorithm partially corrects the principal error. With large deviations from the standard pressure $\varDelta p$ the algorithm overestimates the temperature increase $\varDelta T$ and, as a result, the error after correction increases. As it can be seen, correct pressure information will be crucial (compare the last two examples with Table 1). Since pressure is usually more stable than temperature, it can be assumed that the measurement system should be calibrated before takeoff based on meteorological information.

Table 1 Examples of correcting altitude with temperature [z (m) and h (m) are the geometric and geopotential altitude, respectively; $\varDelta {p}_{\mathrm{corr}}$ (Pa) and $\varDelta {T}_{\mathrm{corr}}$ (K) are the pressure and temperature correction, respectively; $\delta h$ (m) and $\delta T$ (K) are the geopotential altitude and temperature error, respectively]

In the atmosphere $\mathcal{A}\left(15,-2500\right)$ (last case in Table 1), a test was performed to simulate the error with increasing altitude. Cases were investigated: without corrections and when corrections were iteratively made while giving different $\varDelta p$ values in the range from zero to the exact value − 2500 Pa. The results shown in Fig. 2 (top) suggest that the more accurate the true pressure rise value $\varDelta p$ can be given, the smaller the altitude error will be. In the graph, the errors are not exactly straight lines. They are subject to slight fluctuations which reflect the iterative nature of the method, which ends when a satisfactory temperature difference $\delta T$ is reached, which does not mean that the correcting temperature $\varDelta {T}_{corr}$ has been enough precisely determined (Fig. 2 bottom).

Fig. 2

Simulation of error change with increasing height

Influence of air humidity on barometric altimeter errors

The amount of water vapor in air is limited by the current pressure and temperature conditions. Several measures are used to determine the amount of water vapor in the air (see Appendix). The most popular is the so-called relative humidity (RH), which is always between zero and one. Another measure used is the humidity mixing ratio (MR). It is convenient when humid air is treated as a mixture of dry air and water vapor. The disadvantage of MR is that not every assumed value has a physical meaning under given conditions (i.e., it is possible in real-world condition).

In the standard model of the atmosphere, pressure and temperature are related by a differential equation derived from the equations: static air equilibrium and the ideal gas law. The integration of the equation is carried out assuming that the value of the equivalent gas constant of air does not change (Matyja et al. 2022a, b). The formula for pressure height (also density height) follows directly from the integral of this equation.

The definition of non-standard atmosphere can be extended to the case of humid air—non-standard atmosphere $A(\varDelta T,\varDelta p,{R}_{h})$. However, in order to continue using the familiar form of the formula for pressure height, it must be assumed that the gas constant of humid air (${R}_{h})$ is independent of the height above sea level.

The relationship between the constants of humid (${R}_{h}$) and dry (${R}_{d}$) air is described by the equation (Appendix):

$${R}_{h}={R}_{d} \frac{1+X\frac{{M}_{d}}{{M}_{v}}}{1+X}$$

(6)

where $X$ is the mixing ratio of water vapor and dry air (MR) [kg/kg] and ${M}_{d},{M}_{v}$ are the molar masses of dry air and water vapor, respectively.

The amount of water vapor in the air does not exceed a few percent (Murry 1996). Figure 3 shows how the value of the equivalent gas constant of humid air will change depending on the value of MR. Figure 4 shows the maximum possible errors depending on the altitude, with RH is equal to100%. This graph was obtained as follows. In the standard atmosphere for a given geopotential height, the approximate MR was calculated, assuming that RH is equal to 100% at this height. Pressure and temperature distributions for dry air were used to convert RH to MR. Error analysis (Fig. 8 bottom) indicates that this is acceptable. Then, the parameters of the humid atmosphere were generated and the errors of height and pressure were determined. Since the error of height increases with altitude (Fig. 5), it will be the maximum error at a given height. The calculations were repeated for successive altitudes.

Fig. 3

Value of the gas constant of humid air as a function of the MR

Fig. 4

Maximum possible differences between dry and humid atmosphere parameters

Fig. 5

Altitude error depending on MR. On the top—standard conditions of temperature and pressure. On the bottom, the atmosphere is 15 °K warmer

Figure 5 shows what errors would be expected if there was a fixed amount of water vapor in the atmosphere regardless of altitude. The cut of certain lines of fixed MR values on the charts means that the maximum saturation of air with water vapor was reached (RH = 100%).

Constant value of ${R}_{h}$ is possible only when the value of the mixing coefficient MR is constant and independent of height. The question arises: is it possible to assume a constant value of the mixing coefficient? In order to answer them, the theoretical considerations presented below were carried out, and the results of experimental studies available in the literature were also referred to.

Graphs of the relationship between RH and height and MR—lines of constant MR value (Fig. 6) were obtained by generating temperature and pressure distributions depending on height in appropriate non-standard atmospheres, taking into account water vapor in the air. The lack of certain lines of fixed MR values on the charts means that under certain conditions of pressure and temperature (corresponding to water vapor saturation pressure), it is not possible to maintain such an amount of water vapor in the atmosphere. At lower altitudes under conditions of standard pressure and temperature, the MR will not exceed 0.01 (kg/kg) (Fig. 6 top). In a warmer atmosphere, there are conditions for the presence of more water vapor in the air (Fig. 6 bottom).

Fig. 6

Plots of relative humidity versus altitude at a fixed MR under various sea level temperature conditions. On the top—standard conditions of temperature and pressure. On the bottom, the atmosphere is 15 °K warmer

Experimental studies presented in the literature (Leblanc et al. 2008; Navas-Guzmán et al. 2014; Turgut and Usanmaz 2016) show an inverse tendency of RH change with height. RH and MR decrease with height, not increase. This applies primarily to the upper layers of the troposphere. This means that the assumption of a constant MR throughout the troposphere is difficult to maintain.

On the other hand, at low altitudes, up to about 2000 m, the observed daily and monthly fluctuations in the amount of water vapor in the air are very large (Navas-Guzmán et al. 2014; Turgut and Usanmaz 2016). In the case of the analyzed problem, however, this type of variability is unimportant. The ability to assume a constant, even approximately, MR value is more important. It should also be stipulated that the research results are local in nature, as they concern specific areas on earth.

In Fig. 7 (top), the black dashed line shows the RH values measured experimentally, based on the vertical profiles of humidity measured with the automatically operating small unmanned research aircraft M2AV (Meteorological Mini Aerial Vehicle) (Martin et al. 2011). The results of measurements made close to the ground surface using other methods (radiosonde, laser) were omitted, which indicated an initial slight decrease in the RH value with height. It should be noted that in (Martin et al. 2011) the height a.g.l. (above ground level) is given, whereas geometric altitude is measured a.s.l. (above sea level). The measurements were carried out close the Lindenberg Meteorological Observatory (near Berlin) at an altitude of about 40 m a.s.l. Thus, the height differences are not significant. Although the black dashed line (Fig. 7) has a slightly different slope than the other lines of constant MR, in the case of the lower troposphere, the assumption of constant MR in the air seems easier to accept.

Fig. 7

Altitude-dependent RH and lower troposphere MR. Top: lines of the constant MR value and the RH line drawn on the basis of the data presented in Martin et al. (2011) (dashed black). Bottom: error in determining RH when using temperature and pressure distributions for dry air

The error in determining RH when using temperature and pressure distributions in dry air (in standard conditions) is very small (Fig. 7 bottom). This means that in practice, using dry air models to calculate humidity is acceptable. In this way, plots of the inverse relationship between MR and RH were made (Fig. 8). Thus, experimental results and theoretical considerations indicate that the amount of water vapor in the air decreases with height.

Fig. 8

MR and lines of constant RH under different temperature conditions at sea level

All theoretical considerations so far lead to the conclusion that applying the formula for the pressure height with a modified value of the gas constant for moist air is permissible only in the low altitudes above sea level range. The range of applicability of the formula should be controlled by determining the limit height at which RH reaches 100%. On the other hand, the results of experimental studies suggest that the real limit is less than 2000 m.

A method for correcting an error caused by air humidity

Guinn and Barry (2016) have considered using the concept of virtual temperature to correct altitude in humid air. The formula for density height (3) was used and modified to measure pressure and temperature instead of density. A virtual temperature dependent on humidity was substituted in place of the measured temperature. The concept of virtual temperature is used to describe gas transformations of a mixture of dry air and water vapor (Wallace and Hobbs 2006). In the case of the density height formula, however, there are some doubts about the validity of using virtual temperature. The formula for the height of density is obtained analogously to the formula for the height of pressure. For this reason, also in this case, it is necessary to assume a constant value of the gas constant, which is equivalent with a constant value of MR regardless of altitude. The virtual temperature can be calculated by knowing the MR from the formula (see Appendix),

$${T}_{V}=T \frac{X+\frac{{M}_{v}}{{M}_{d}}}{(1+X)\frac{{M}_{v}}{{M}_{d}}}$$

(7)

if the current MR is known. The obvious disadvantage of formula (7) is that it does not protect against situations where maintaining a specific MR value under given pressure and temperature conditions is not physically possible. For this reason, it is better to use a different formula taking into account the water vapor pressure (Guinn and Barry 2016). Figure 9 shows the altitude errors when using virtual temperature in two cases of non-standard conditions. Based on the results obtained, the effectiveness of this method cannot be confirmed. The error can be relatively smaller only under standard pressure and temperature conditions and at high altitudes.

Fig. 9

Altitude error when using virtual temperature for two atmospheres: Top: $A({0,0},{R}_{h}\left(X=0.005\right))$. Bottom: $A({10,500},{R}_{h}\left(X=0.005\right))$. Legend: PA is pressure altitude, DA is density altitude, T_V(MR) is virtual temperature from MR, T_V(RH) is virtual temperature from RH)

Since virtual temperature did not yield correct results, another method was proposed to correct the error caused by water vapor in the air. It is a natural extension of the method of correcting the principal error (Kim and Park 2012; Matyja et al. 2022a, b).

In a humid atmosphere $\mathcal{A}(\varDelta T,\varDelta p,{R}_{h})$ , the pressure height formula can be expanded into a Taylor series around the standard values $({T}_{0},{p}_{0},{R}_{d})$, assuming that the increments of the variables are small. The expansion of the function will include, among others, a factor with the first partial derivative after ${R}_{d}$ corresponding to the influence of water vapor:

$$\begin{aligned} h\left( {T_{0} + \varDelta T,p_{0} + \varDelta p,R_{d} + \varDelta R} \right) & = h\left( {T_{0} ,p_{0} ,R_{d} } \right) \\ & \quad + \cdots + \frac{\partial h}{{\partial R_{d} }}\varDelta R + \cdots \\ \end{aligned}$$

(8)

where $\varDelta R={R}_{h}-{R}_{d}={R}_{d}\frac{X}{1+X}(\frac{{M}_{d}}{{M}_{v}}-1)$.

Hence, the approximate formula for the altitude correction in the case of a humid atmosphere:

$$\begin{aligned} \delta h_{v} & \approx \frac{\partial h}{{\partial R_{d} }}\varDelta R = \frac{X}{X + 1}\left( {\frac{{M_{d} }}{{M_{v} }} - 1} \right) \\ & \quad \times \frac{{T_{0} }}{{\lambda_{0} }}\left( {\frac{{\lambda_{0} h}}{{T_{0} }} + 1} \right)\ln \left( {\frac{{\lambda_{0} h}}{{T_{0} }} + 1} \right) \\ \end{aligned}$$

(9)

In the atmosphere $\mathcal{A}({0,0},{R}_{h})$ , formula (9) allows very effectively to correct the altitude error caused by water vapor (Fig. 10).

Fig. 10

Correcting the error of humidity in the atmosphere $A({0,0},{R}_{h}\left(X\right))$

In non-standard atmosphere $\mathcal{A}(\varDelta T,\varDelta p,{R}_{h}(X))$ , the correction according to formula (9) should be added to the right side of formula (5). This case will be considered later during simulation of the procedure for correcting altitude errors in a non-standard humid atmosphere.

Additional altimeter calibration at ground level

Based on temperature measurements, the previously discussed method shows how to correct the error caused by non-standard temperature conditions at sea level. Tests have shown that this method works very well when sea level pressure is standard or when a known deviation from a non-standard pressure can be given. For example, based on reports from weather stations. Otherwise, the method only partially corrects the height value and the height error itself remains large. At high-pressure drops at sea level, when we cannot give even an approximate value of $\varDelta p$, the method fails completely.

The question arises: is it possible to estimate the $\varDelta p$-value with sufficient accuracy from some additional measurements? In a non-standard atmosphere, described by increments of temperature $\varDelta T$ and pressure $\varDelta p$, the relationship between pressure and temperature can be written (Matyja et al. 2022a, b):

$$p=\left({p}_{0}+\varDelta p\right){\left(\frac{T}{{T}_{0}+\varDelta T}\right)}^{-\frac{{g}_{0}}{R{\lambda }_{0}}}$$

(10)

When the GPS coordinates of the aircraft takeoff site are known and the Digital Terrain Model (DTM) is available, it is easy to specify the height $H$ a.s.l. of this point with high accuracy). Assuming that the temperature gradient ${\lambda }_{0}$ is constant and standard, the temperature increase over the standard value will be equal to:

$$\varDelta T={T}_{H}-({T}_{0}+{\lambda }_{0}H)$$

(11)

and after transformations, the pressure increase formula is obtained:

$$\begin{aligned} \varDelta p & = p_{H} \left( {\frac{{T_{0} + \varDelta T}}{{T_{H} }}} \right)^{{ - \frac{{g_{0} }}{{R\lambda_{0} }}}} - p_{0} \\ & = p_{H} \left( {\frac{{T_{H} }}{{T_{H} - \lambda_{0} H}}} \right)^{{\frac{{g_{0} }}{{R\lambda_{0} }}}} - p_{0} \\ \end{aligned}$$

(12)

where ${T}_{H},{p}_{H}$ are temperature and pressure measured at ground level. An analogous formula can be obtained by converting the dependence to the pressure height.

The verification of the effectiveness of the formula (12) was carried out in the case of a non-standard atmosphere $\mathcal{A}\left({10,500},{R}_{h}(X=0.01)\right).$ As shown in Fig. 11, the determined value of the pressure increase slightly deviates from the current value. This is due, among other things, to adopting the gas constant as for dry air. Also, the accuracy of the temperature measurement at ground level ${T}_{H}$ affects the calculated $\varDelta p$. The $\varDelta p$ error also increases with the height for which the calculations are made (Fig. 12). The graphs show cases when the temperature ${T}_{H}$ is measured accurately and when it is deviated by ± 0.3°K. The use of the gas constant of humid air in the calculations significantly improves the results. This requires measuring the humidity, determining the MR and calculating the equivalent gas constant for moist air according to the relationship between the constants of wet (R_h) and dry (R_d) air.

Fig. 11

Pressure error $\varDelta p$ calculated from Eq. (12) and $\varDelta T$ calculated from Eq. (11). ${T}_{H}$ is measured with $\pm 0.3K\,\mathrm{precision}$ and H = 134 m above sea level

Fig. 12

Influence of the height of the terrain on the designated $\varDelta p$-value ($\varDelta {T}_{H}$ is the temperature measurement error, R_d denotes that the gas constant of dry air was assumed in the calculations, R_h denotes that a constant for moist air was assumed)

In summary, the altimeter calibration procedure could be as follows:

• (1) Before the start, the operator manually enters information about the height of the terrain or the navigation system autonomously determines the height based on the measurement of GPS coordinates and the recorded DTM data.

• (2) The system measures: pressure, temperature, and humidity. Using information about the height of the terrain, it estimates the value of the pressure increase above the standard pressure.

Simulation of the procedure for correcting altitude errors in a non-standard humid atmosphere

Simulation studies of the error correction method were carried out on the example of a non-standard atmosphere $\mathcal{A}\left({10,500},{R}_{h}(X=0.01)\right)$. The non-standard value of the pressure increase $\varDelta p$ was estimated by the method described in the previous section (the height of the starting point was assumed to be $h=134$ m a.s.l.). The non-standard temperature value was determined by the iterative method. As shown in Fig. 13, the correction of the principal error alone ($\varDelta T,\varDelta P$) significantly improves the height reading. The additional correction of the influence of humidity results in a further significant reduction of the error.

Fig. 13

Results of the method of correcting the altimeter error in a humid atmosphere

Presented simulation results do not take into account possible errors in the measurement of pressure, temperature, and humidity. Under real conditions, the accuracy of the altitude correction method should be expected to decrease. For example, a simulation was carried out when the systematic error of pressure measurement is + 5 Pa and at the same time the systematic error of temperature measurement is − 0.2 °K (Fig. 14). In this case, the error increased, but the increases are not dramatic. Random attempts to select different values of systematic errors show that the reverse situation is also possible i.e., the height error after correction will decrease. Figure 15 shows the corrected height error when the pressure increase $\varDelta p$ was estimated with an error of − 100 Pa. The error is slightly larger but still much smaller than if there was no correction at all.

Fig. 14

Altitude error after correction in the case of a systematic measurement error of + 5 Pa pressure and − 0.2 K temperature

Fig. 15

Altitude error after correction in the case of $\varDelta p$ estimation error − 100 Pa

Full assessment of the effectiveness of the method and the graphical presentation of the results is difficult due to the large number of variables and the significant range of their variability. It was decided to run a simulation experiment assuming discrete values of variables: geometric height $z\in \left\{{400,800,1200,1600,2000}\right\} m$; temperature increase $\varDelta T\in \left\{-30,-{15,0},15\right\} K$; pressure increase $\varDelta p\in \left\{-25,-{12.5,0},{12.5,25}\right\} hP$; MR factor $X\in \{{0,0.0025,0.005,0.0075}, 0.01\}$. At the same time, the height of the starting point was assumed to be $H=100$ m a.s.l. The values of $\varDelta T,\varDelta p,X$ were chosen so that there were also cases of standard conditions. The sea level temperature range corresponds to values from − 15 to + 30 °C, often found in Poland. Similarly, the range of pressures is within typical values often found in Poland. This is indicated by long-term atmospheric pressure measurements over the 1986–2007 period (Koźmiński and Michalska 2010). Extreme day-on-day shifts varied from 966.1 to 1049.4 hPa in winter and from 990.7 to 1031.5 hPa in summer.

The set of all cases was generated using the MATLAB “candgen” function, which first permutes the altitude, then successively the temperature and pressure and finally the MR factor. In total, 500 cases were generated, some of which had to be rejected because the adopted MR coefficient was unattainable under the given pressure and temperature conditions. Ultimately, 283 cases remained. Testing has shown that the number of cases can be further reduced using the D-optimal (Mitchell 2000) experiment design. It was assumed that the experiment results could be presented using the Multivariate Regression Model (MVRM) (Hair et al. 2018). However, this would be justified in the case of experimental field studies. In a simulation experiment, there are no obstacles to examining any number of cases. Appropriate calculations take a very short time.

Figure 16 shows the barometric altimeter reading errors before and after the proposed correction method. It can be seen that the method works well over a wide range of pressure, temperature and humidity.

Fig. 16

Comparison of altitude errors before (top) and after correction (bottom)

Table 2 presents the MVRM, which shows a very good fit to the results of the simulation experiment (Fig. 17). The variables ${x}_{1},{x}_{2},{x}_{3},{x}_{4}$ correspond to the model variables $z,\varDelta T,\varDelta p,X$ . It is not a full quadratic model, because some factors were rejected based on the preliminary regression results, due to too high probability of the hypothesis that a given factor may be statistically insignificant. Based on the regression model presented in Table 2, the error of the method can be estimated in other than selected previous conditions. Figure 18 compares the errors determined from the experiment with those calculated from the regression model. The result is very satisfactory. During the simulation experiment, all conditions were known, and the error of the method could be easily estimated. The MVRM model can be an additional tool for estimating the correction error when using the method in real-word conditions.

Table 2 Linear regression model (MATLAB function “fitlm” report: “Estimate”—coefficient estimates, “SE”—standard error of the coefficients, “tStat”—t-statistic to test the null hypothesis that the corresponding coefficient is zero, “pValue”—the null hypothesis is rejected if the p-value is less than or equal to a predefined threshold, usually 5%)

Fig. 17

Assessment of the fit of the regression model (residuals visualization, plot automatically generated by MATLAB for tested regression model, see MATLAB documentation)

Fig. 18

Difference between the altitude errors obtained from the simulation and those calculated from the MVMR

Conclusions

Unusual temperature and pressure conditions at sea level cause significant barometric altimeter errors. This is the so-called principal error because it results from factory calibration of the altimeter to standard conditions. The method of continuous temperature measurement and automatic correction of the altimeter principal error proposed in the paper may be an alternative to methods requiring obtaining information from meteorological stations. The effectiveness of the method depends on how accurately can be estimated non-standard pressure conditions and corrected part of the error resulting from this. It can be assumed that the pressure usually changes much slower during the day than the temperature. Hence, the concept of calibrating the altimeter system before the takeoff of the aircraft and the proposed method of estimating the pressure increase when the height of the terrain at the starting point is known. An alternative is to obtain pressure information from a weather station. In general, it is better to give the value of the pressure increase, even with some error, than to assume the standard pressure. The simulation tests carried out confirmed the usefulness of both methods: iterative temperature correction and pre-start calibration. Altitude correction using sea level temperature deviation is very effective, although the formula is approximate (Matyja et al. 20). The proposed method differs from others in that it allows real-time estimation of changes in sea level conditions. Of course, all assume that the air column heats up or cools down evenly and the temperature gradient remains standard.

Correcting the error caused by water vapor in the air is a more complex issue. Theoretical considerations and the results of experimental studies published in the literature clearly indicate that the amount of water vapor in the air decreases with height. The formula for pressure height can only be used assuming that the equivalent gas constant of humid air does not depend on height. This is equivalent to assuming a constant mixing ratio (MR). Such an assumption is acceptable only in the lower layer of the troposphere. The proposed method of correcting the error caused by air humidity works very well in a humid standard atmosphere $\mathcal{A}({0,0},{R}_{h})$. However, for the above reasons, it can only be used for low heights.

Even if the MR profile of the height (and consequently the ${R}_{h}$ profile of the height) was known, it seems impossible to derive an appropriate formula for the pressure height based on the equation of static air equilibrium. All that remains is numerical integration. However, such an effort is unprofitable because air humidity is characterized by high daily and seasonal variability. At the same time, measuring moisture profiles requires specialized equipment. Simulation studies have also shown that the combination of all the methods of correcting the error in a non-standard humid atmosphere $\mathcal{A}(\varDelta T,\varDelta p,{R}_{h})$ proposed in the work allows to significantly reduce the size of the error of the altimeter indications. The additionally developed regression model is a useful tool for evaluating the final height error after performing the correction using the method proposed by the authors. It should be noted, however, that many disturbing factors cannot or are difficult to include in simulations, such as specific atmospheric conditions and sensor measurement errors.

Practical verification of the correction methods would require field research using measuring equipment installed on a drone equipped with an additional, sufficiently accurate height measurement system, such as laser technology. Perhaps this can be done in future. In this case, the results of the multivariate regression model presented in last chapter may be useful, and the number of experiments can be reduced using the D-optimal design. According to the authors, the research results presented in the work can be used in the currently developed systems of autonomous drone flight.

Appendix: Water vapor in the air

Humid air, not under too much high pressure prevailing in the atmosphere, can be treated as an ideal gas, consisting of dry air and water vapors. Humid air pressure is the sum of the component pressures: $p={p}_{d}+{p}_{v}$. Since there is relatively little water vapor in the air, it is often assumed: ${p}_{d}\gg {p}_{v} \to p\approx {p}_{d}.$

The water vapor pressure in humid air cannot exceed the maximum value, which is equal to the saturated vapor pressure ${p}_{s}$ (water vapor is in thermodynamic equilibrium with water or ice, depending on the temperature). The temperature at which the vapor pressure equals the saturated vapor pressure ${p}_{v}\to {p}_{s}$ is the dew point. The saturated steam pressure was calculated in this paper using the Goff–Gratch empirical formula modified by Murray (Goff and Gratch 1946; Murray 1967): ${p}_{s}={p}_{s}(p,T,{H}_{2}O)$.

Three measures can be used to assess the amount of water vapor in the air: absolute humidity (AH), relative humidity (RH), or degree of mixing (MR). Relative humidity (RH) is most commonly used because it is related to the saturated vapor pressure and is always between zero and one. Relative humidity can be defined as the ratio of water vapor pressure to saturated vapor pressure:

$$\varphi ={\left(\frac{{p}_{v}}{{p}_{s}}\right)}_{T}$$

The mixing ratio (MR) is defined as the ratio of the mass of water vapor to the mass of dry air:

$$X=\frac{{m}_{v}}{{m}_{d}}$$

where ${m}_{d} , {m}_{v}$are the mass of dry air and mass of water vapor, respectively. In the case of the mixing ratio (MR), there is no guarantee that the assumed value is possible in the real world under given conditions of pressure and temperature. It can be seen that: ${p}_{d}V={m}_{d}{R}_{d}T$ and similarly ${p}_{v}V={m}_{v}{R}_{v}T$. Hence,

$$X=\frac{{m}_{v}}{{m}_{d}}=\frac{{p}_{v}}{{p}_{d}}\frac{{R}_{d}}{{R}_{v}}$$

Since the gas constants of dry air and water vapor depend directly on the universal gas constant ${R}^{*}$ ,

$${R}_{v}=\frac{{R}^{*}}{{M}_{v}} , {R}_{d}=\frac{{R}^{*}}{{M}_{d}} \to \frac{{R}_{d}}{{R}_{v}}=\frac{{M}_{v}}{{M}_{d}}$$

where ${M}_{v},{M}_{d}$ are the molar masses of dry air and water vapor, respectively. Also, it holds: ${p}_{d}={p-p}_{v}=p-\varphi {p}_{s}$. Thus, there is a relationship between MR and RH:

$$X=\frac{{p}_{v}}{{p}_{d}}\frac{{M}_{v}}{{M}_{d}}=\frac{{M}_{v}}{{M}_{d}}\frac{\varphi {p}_{s}}{p-\varphi {p}_{s}}$$

It is also easy to obtain the inverse relationship:

$$\varphi =\frac{p}{{p}_{s}}\frac{X}{X+\frac{{M}_{v}}{{M}_{d}}}$$

Water vapor can also be taken into account in the thermodynamic transformations of air by introducing the so-called virtual temperature ${T}_{v}$. Humid air density at temperature $T$:

$$\rho =\frac{p-{p}_{v}}{{R}_{d}T}+\frac{{p}_{v}}{{R}_{v}T}$$

After transformations, the formula for ${T}_{v}$ as a function of RH can be determined:

$$\begin{aligned} \frac{{\rho R_{d} }}{p}\frac{T}{{1 - \frac{{p_{v} }}{p}\left( {1 - \frac{{M_{v} }}{{M_{d} }}} \right)}} & = 1 \to \frac{{\rho R_{d} T_{v} }}{p} \\ = 1 \to T_{v} & = \frac{T}{{1 - \frac{{\varphi p_{s} }}{p}\left( {1 - \frac{{M_{v} }}{{M_{d} }}} \right)}} \\ \end{aligned}$$

The virtual temperature can also be displayed depending on the MR:

$${T}_{v}=T\frac{\frac{{M}_{v}}{{M}_{d}}+X}{\frac{{M}_{v}}{{M}_{d}}(1+X)}$$

Based on Dalton's law, the equivalent gas constant for humid air is:

$$R=\frac{{R}_{d}{m}_{d}+{R}_{v}{m}_{v}}{{m}_{d}+{m}_{v}}$$

The equivalent gas constant of humid air is most conveniently represented by MR:

$$\begin{aligned} R & = R_{d} \frac{{1 + \frac{{m_{v} }}{{m_{d} }}\frac{{R_{v} }}{{R_{d} }}}}{{1 + \frac{{m_{v} }}{{m_{d} }}}} = R_{d} \frac{{1 + \frac{{m_{v} }}{{m_{d} }}\frac{{M_{d} }}{{M_{v} }}}}{{1 + \frac{{m_{v} }}{{m_{d} }}}} \\ & = R_{d} \frac{{1 + X\frac{{M_{d} }}{{M_{v} }}}}{1 + X} \\ \end{aligned}$$

The attached appendix contained all the equations and explanations resulting from the calculation method.