Introduction

The production of hydrocarbons from underground reservoirs is associated with many challenging issues, due particularly to the complex characteristics of reservoirs. As more information regarding the reservoir properties is collected, the uncertainty associated with the recovery of hydrocarbons decreased. Reservoir characteristics have long been studied by means of a variety of techniques such as well log data (Dashtian et al. 2011), well test data (Braester and Zeitoun 1993; Vaferi et al. 2011), seismic data (Arora and Tomar 2010), and well drilling data (Weber 1993). Among these methods, recording transient well pressures created by flow rate alteration is a useful measure to obtain reservoir properties (Athichanagorn and Horne 1995; Gringarten 1987; Sanni and Gringarten 2008; Vaferi and Eslamloueyan 2015). Reservoir pressure is one of the key variables that is used to obtain reservoir properties, monitor reservoir condition, and forecast reservoir performance (Athichanagorn et al. 2002). In well pressure testing, wellbore pressure and flow rate are monitored to evaluate reservoir characteristics by matching a simplified reservoir model on pressure responses (Blasingame et al. 1989; Bourdet and Gringarten 1980; Ghaffarian et al. 2014; Horne and Reyner 1995; Jeirani and Mohebbi 2006).

Since 1990s, most of wells have been installed with permanent downhole gauges. These gauges are equipment that measures well pressure at small time interval, e.g., every second. As a result, the pressure monitoring generates large volumes of data. The recorded data are always associated with measurement errors due to various reasons such as sudden alteration in flowing temperature and problems in gauge that occur by physical changes in reservoir. Measurement errors are due to systematic and random deviations from the true value of the measured (recorded) data. While systematic errors are introduced by an inaccuracy inherent in the system such as imperfect performance (or calibration) of measurement gauges, random errors are caused by unpredictable fluctuations in the reading of gauges. The collected data are usually associated with noise. There is not a unified definition for noise since this term has been used in various disciplines. On one hand, noise is clearly related to measurement errors. Hence, it is possible to have systematic and random noises, although the latter is more common. Horne (1994) defined noise as a type of measurement error during a test which may be either unbiased (random noise) or biased (drift error). On the other hand, there are some anomalies in the recorded data which are distinguished from noise. For example, outliers are data points that lie away from the trend of data, while noise is considered as data points around the trend of overall data and in the same neighborhood as the true data (Athichanagorn et al. 2002). Either noise or other artifacts such as outliers may cause high degree of uncertainty in well test interpretations, and hence, it is essential to remove noise and other artifacts or reduce them to acceptable levels before further analysis.

In well test practice, the amount of collected pressure data is tremendous. Practically it is impossible to consider all data points in the interpretation due to limitation in computational resources. In addition, noisy pressure data may lead to wrong interpretation, and hence, large uncertainties would be expected in parameter estimation. As a result, noise removing is an essential part of analyzing well test data. In this regard, denoising of downhole gauge data has been the subject of several research papers. Osman and Stewart (1997) employed the Butterworth digital filter method to subtract noise in the data. Kikani and He (1998) showed that the Butterworth filter results in poor performance for some cases. Instead, they used a wavelet analysis to remove noise. Unneland et al. (1998) presented the use of decline curve analysis to describe production from a simple reservoir by pressure and rate measurement. The well flow rates were calculated by resolving the rate convolution integral. The rates that determined by convolution integral are rates which happen if pressure stays constant during production. Athichanagorn et al. (2002) proposed a seven-step procedure for processing of permanent downhole gauge data as follows: (1) outlier removal, (2) denoising, (3) transient identification, (4) data reduction, (5) flow rate history reconstruction, (6) behavioral filtering, and (7) data interpretation. They indicated that the wavelet thresholding method is one of the most effective methods to denoise data. An important application of wavelet in well testing is permuting a pressure signal to compress parameters that named wavelet coefficients (Daubechies 1990). The wavelet transform was also applied by Olsen and Nordtvedt (2005) to denoise pressure data. They proposed automatic filtering methods to estimate appropriate thresholds and pressure data reduction. In another study, Nomura (2006) described a smoothing algorithm based on derivative constraints suitable for pressure transient data. He compared the constraint smoother and unconstrained regression splines and concluded that the former can have better performance when the order of derivative constraints increases.

The aim of this paper is to evaluate the performance of regressive and autoregressive methods as well as various combinations of regressive/autoregressive methods for denoising pressure and pressure derivative transient data in comparison to the wavelet transform method which has widely been used in the petroleum industry. The paper is organized as follows: First, the basic concepts of the denoising/smoothing methods used in this study are described. Next, we explain how to generate clean and noisy dual-porosity model pressure data. Different levels of noise are added to the clean signal. Then, the smoothing techniques are applied to the noisy pressure and pressure derivative data, and the smoothing performance is evaluated using the mean square error (MSE) associated with each method for different levels of noise. The effect of time span as the most important parameter on smoothing is then investigated. Finally, the performance of noise reduction for various combinations of smoothing techniques is examined, and the most effective method is applied to the model and field noisy pressure derivative data.

Preprocessing techniques for enhancement of downhole pressure data

Reduction in massive data sets

With advances in pressure gauge technology, current gauges can measure pressure with small time interval (e.g., every second, ten seconds, …). In most instances, extremely large pressure data sets should be processed and analyzed. It is expected to use appropriate techniques to reduce large data sets to manageable size. Pressure thresholding is one of the most common methods to reduce the number of pressure data. Regarding noisy data, it was recommended to denoise data before reduction in their size (Athichanagorn 1999). In this study, we have used small data sets (with a few thousands data points), so data reduction was not needed.

Outlier removal

Outliers are defined those data that are far away from the main part or trend of a data set (Becker and Gather 1999). They are more obvious and easier to detect in comparison with noise. Outliers demonstrate a problem with the measurement equipment, human error or the data recording process (Hodge and Austin 2004). There are several methods to remove outliers from true data including statistical methods such as univariate and multivariate methods, Minkowski error, and signal processing methods such as wavelet, transform (Aggarwal 2013). If a data set contains a considerable amount of outliers, a combination of these methods should be tested. In this study, we focus on noise removal methods since noise is more common and difficult to detect. Besides, since we have added Gaussian noise to synthetic data, it is not needed to deal with outliers. In case of real field data, outliers should be removed from data set before noise removal techniques are applied. In this paper, we used an outlier-free field data to assess the effects of denoising methods on smoothing of pressure derivative data.

Denoising techniques

Regression-based smoothing methods

The smoothness of the acquired data is crucial for estimating the reservoir properties in well test analysis. In general, smoothing is a technique to create an approximate function that attempts to capture main patterns or trends in the data set. A variety of regression-based smoothing methods have been used among which four types of linear smoothers are considered in this study.

Simple moving average

Simple moving average (SMA) is a low-pass filter which replaces every point by unweighted mean of around points in a span (Johnston et al. 1999; Kirchgassner and Jurgen 2007; Mac Berthouex and Brown 1994; Wei 1994). The smoothing signal is given by:

$$\bar{x}(i) = \frac{1}{2N + 1}\left[ {x(i + N) + x(i + N - 1) + \cdots + x(i - N)} \right]$$
(1)

where \(\bar{x}(i)\) is the smoothed value, N denotes number of neighboring points on each side, and 2N + 1 represents the span. The (window) span is the time axis length over which the average is to be calculated. So, the span essentially determines the number of data points contributing to the moving average calculations. Obviously, the span controls the performance of the moving average method. SMA is the simplest smoother that has jagged appearance in some points such as boundaries.

Savitzky–Golay filter

Savitzky–Golay (S–G) filter is a least squares filter used to smooth a set of digital data points. This filter acts by fitting a polynomial of a low degree on a subset of data into a span by applying unweighted linear least squares. This method is based on mathematical technique published by Savitzky and Golay in 1964 (Luo et al. 2005; Savitzky and Golay 1964). A remarkable advantage of this filter is that it tends to keep features of the distribution such as extrema which are often flattened by other smoothing methods such as SMA (Azami et al. 2012).

The data consist of a set of n pairs (xi, yi) where xi is an independent variable (here is time) and yi is an observed value (here is pressure). The polynomial used to fit the data is given by:

$$p_{n} (x) = \sum\limits_{i = 0}^{n} {a_{i} x^{i} }$$
(2)

The S–G filter required that the sum of the squares of the discrepancies between the yi and pn(xi) be a minimum.

$$e_{\text{t}} = \sum\limits_{i = 0}^{N} {e_{i}^{2} } = \sum\limits_{i = 0}^{N} {\left[ {p_{n} (x_{i} ) - y_{i} } \right]^{2} }$$
(3)

where ei and et are the local error at xi and total error, respectively, and N + 1 is the total number of data points. In this work, various low-degree polynomials were tested and it was found that the polynomial of the second degree resulted in the best fit.

LOWESS and LOESS

The methods of “locally weighted scatterplot smooth,” known as “LOWESS” and “LOESS” employ locally weighted linear regressions to data smoothing. The difference between them is that LOWESS uses a first-degree polynomial to fit, whereas in the LOESS the second-degree polynomial is used. These methods, similar to the SMA and S–G filters, use neighboring data points to estimate smoothed value, and hence, they are local smoothing methods. In addition, a regression weight function is defined for each point inside the span, so the methods are weighted (Cleveland 1979, 1981; Cleveland and Devlin 1988; Cleveland and Loader 1996; Fan and Gijbels 1996; Hardle 1990).

In the LOWESS and LOESS methods, the least square criterion is modified by multiplication of the squared error at xi by a nonnegative weight factor, wi, before the sum of the squared error is minimized.

$$e_{\text{t}} = \sum\limits_{i = 0}^{N} {w_{i} e_{i}^{2} }$$
(4)

The weight function for each point in the span is calculated using Eq. (5) (Cleveland and Devlin 1988):

$$w_{i} = \left( {1 - \left| {\frac{{x - x_{i} }}{d(x)}} \right|^{3} } \right)^{3}$$
(5)

where x is the observed value we intend to be smoothed, xi is the nearest neighbor of x which is defined within the span, and d(x) is the maximum distance between x and xi. One of the characteristics of the weight function is that the target point to be smoothed has the maximum weight. Outside the span, data points have no impact on fitting, i.e., have zero weights. The regression weight function might not be symmetric about the target point. For end points and data points around the boundary for which the number of neighboring points are not symmetric, the weight function is also not symmetric. If the same numbers of neighboring data points on either side of the target point are available, then the weight function is symmetric (Cleveland 1979). After computation of the weight functions for each point within the span, a weighted linear least square regression is performed. The regression employs a linear polynomial, for LOWESS and a quadratic polynomial for LOESS. Span is a parameter that deeply affects the accuracy of smoothing. The chosen span should not change during the smoothing process.

Autoregressive moving average

A time series is a set of observations collected over evenly spaced intervals of some quantity of interest. Forecasting is a process that the value of a variable (or set of variables) is estimated at some future point in time. There are various models to time series forecasting. Three basic time series models are autoregressive (AR), moving average (MA), and a combined autoregressive moving average (ARMA). These methods have long been used to filter noisy data (Box et al. 2015; Huang and Shih 2003). In this paper, we use the ARMA model which includes the advantages of both AR and MA methods.

The ARMA model consists of an autoregressive (AR) and a moving average (MA) parts. AR describes time series as a linear function of previous observation values. The name autoregressive refers to the regression on self. AR (p) refers to autoregressive model of order p (Hannan and Deistler 1988). Given a time series of data xt, the autoregressive model of order p is written as:

$$x_{\text{t}} = \sum\limits_{i = 1}^{p} {\phi_{i} x_{{{\text{t}} - i}} } + \varepsilon_{\text{t}}$$
(6)

where φ1, φ2, …, φp are model parameters, εt is the white noise. The white noise refers to uncorrelated observations with zero mean and finite constant variance which indicates a weakly stationary condition (Montgomery et al. 2008, 1990). In addition, the noises (εt, εt−1,…) are supposed to be random variables in time and had normal distribution. It can be seen from Eq. (6) that the AR model is such a regression model in which xt is regressed on its past values.

The MA model regarded time series as a moving average (unevenly weighted) of a white noise series (εt). The moving average of order q is given by:

$$x_{\text{t}} = \varepsilon_{\text{t}} + \sum\limits_{i = 1}^{q} {\theta_{i} \varepsilon_{{{\text{t}} - i}} }$$
(7)

where θ1, θ2, …, θq are the model parameters, and εt, εt−1,… are the white noise terms.

The ARMA (p, q) is a model contains AR (p) and MA (q). The ARMA (p, q) has p terms of AR and q terms of MA models as expressed by Eq. 8 (Box et al. 2015):

$$x_{\text{t}} = \varepsilon_{\text{t}} + \sum\limits_{i = 1}^{q} {\theta_{i} \varepsilon_{{{\text{t}} - i}} } + \sum\limits_{i = 1}^{p} {\phi_{i} x_{{{\text{t}} - i}} }$$
(8)

The ARMA model was first developed by Whitle (1951). He employed the Laurent series and Fourier analysis along with statistical inference.

Wavelet transform

The wavelet transform (WT) is one of the main tools that is used to catch smooth functions (Donoho and Duncan 2000). In this study, we employ WT for denoising as a base method to compare with other smoothing methods. In this work, the WT noise removal algorithm proposed by Kopsinis and McLaughlin (2009) is used. This technique employs wavelet thresholding principle and empirical mode decomposition (EMD) for denoising signals. A sampled noisy signal can be shown as:

$$x(t) = \overline{x} (t) + \sigma m(t)$$
(9)

where x(t) is the noisy signal, \(\bar{x}\left( t \right)\) is the denoised signal, σ represents a noise variance, and m(t) denotes an independent random variable with Gaussian distribution. Discrete wavelet transform described below uses proper wavelet basis on the noisy signal. The wavelength coefficients are given by:

$$c\, = \,W\, \times \,x$$
(10)

where, x = [x(1), x(2), …, x(N)], W represents an orthogonal N × N matrix (Theodoridis and Koutroumbas 2009), and x = [c1, c2, …, cN] is the resultant wavelet coefficients. Because of orthogonality of wavelet components (W), wavelet coefficients (ci) have normal distribution with variance σ. The denoised signal is obtained by taking an average of wavelet coefficients.

Wavelet transform (WT) of noisy signal has small number of high signal-to-noise ratio (SNR) coefficients and large number of low SNR coefficients. Hence, signal energy is concentrated in a small number of coefficients. After removing low SNR coefficients, signal is reconstructed by taking inverse of WT. Figure 1 shows the procedure of wavelet transform denoising.

Fig. 1
figure 1

Flowchart of process of denoising by wavelet transforms adopted in this work

Full size image

To remove low SNR coefficient, wavelet-based thresholding is used by setting to zero all the coefficients that are lower than a threshold related to the noise level expressed by (Kopsinis and McLaughlin 2009):

$$T = \sigma \times \,C$$
(11)

where T is wavelet coefficients and C is a constant value. We then construct the denoised signal using the high amplitude coefficients. The common thresholding operators used in practice are hard thresholding and soft thresholding (Kopsinis and McLaughlin 2009) which are expressed by Eqs. (12) and (13), respectively.

$$\rho_{\text{T}} (y) = \left\{ {\begin{array}{*{20}l} {y,} \hfill & {\left| y \right| > T} \hfill \\ {0 ,} \hfill & {\left| y \right| \le T} \hfill \\ \end{array} } \right.$$
(12)
$$\rho_{\text{T}} (y) = \left\{ {\begin{array}{*{20}l} {\text{sgn} (y)\left( {\left| y \right| - T} \right),} \hfill & {\left| y \right| > T} \hfill \\ {0 ,} \hfill & {\left| y \right| \le T} \hfill \\ \end{array} } \right.$$
(13)

where sgn stands for the sign function. The above process can be applied for each level of wavelet transform which is called the interval-dependent thresholding method (Zhang et al. 2006). The high and low frequency parts of a signal may have different characteristics such as mean and variance. Hence, the threshold is obtained for each level of this method, and denoising process is applied for each interval.

Generation of synthetic well test data

Noisy pressure data

We used the PanSystem® well test analysis software to generate transient pressure data in a dual-porosity reservoir. The reservoir is assumed to be infinite acting. Table 1 shows the numerical values of reservoir fluid and rock properties, wellbore parameters, and operating conditions used in generating pressure data. Once the pressure transient data were generated, we added Gaussian noise with zero mean and unit variance to the synthetic data sets. To have a noise at an acceptable level, we added the random noise as small fractions of ± 2, ± 4, ± 6, and ± 8% to the original pressure signal. Figure 2 illustrates the original non-noisy (clean) pressure data and the simulated noisy pressure data generated by adding 2% noise to the clean pressure signal.

Table 1 Model properties for generation of synthetic pressure data
Full size table
Fig. 2
figure 2

Generated non-noisy (clean) pressure transient data and noisy pressure with 2% artificial normal noise

Full size image

Pressure derivative analysis

Using only pressure data for well test analysis may be insufficient and even may lead to misleading results. For example, the pressure data of Figure. 2 may be obtained from a homogeneous reservoir or a dual-porosity reservoir. Pressure derivative is hence useful in well test analysis, including for identification of reservoir model in particular. Pressure derivative curve is a graph in which the logarithm of the pressure derivative is plotted against the logarithm of the elapsed time. Using pressure derivative data is advantageous for accurate reservoir model recognitions (Onur and Reynolds 2001). While pressure versus time plot usually cannot provide enough information, derivative curves show useful information about reservoir and also reservoir characteristics (Booth et al. 2012). In this work, pressure derivative is calculated by the method proposed by Bourdet et al. (1989). This method, called three-consecutive points method, uses one point before and one point after the point of interest, point i, computes the corresponding derivatives, and calculates their weighted mean at the point considered. The formula is given by:

$$\left( {\frac{\partial p}{\partial \ln t}} \right)_{i} = \,\frac{{\frac{{p_{i} - p_{i - 1} }}{{\ln t_{i} - \ln t_{i - 1} }}\left( {\ln t_{i + 1} - \ln t_{i} } \right)\, + \frac{{p_{i + 1} - p_{i} }}{{\ln t_{i + 1} - \ln t_{i} }}\left( {\ln t_{i} - \ln t_{i - 1} } \right)}}{{\ln t_{i + 1} - \ln t_{i - 1} }}$$
(14)

Essentially, Eq. (14) is a weighted average of slopes calculated from data points on either side of data point i. A three-stage procedure for assessing the effectiveness of different smoothing techniques on pressure derivative data is adopted in this work as follows. First, a noisy pressure signal is created by adding a certain amount of Gaussian noise. Next, various single or combined smoothing techniques are employed to smooth pressure data. Finally, pressure derivative is calculated using Eq. (14).

Analysis of smoothing error

The criterion used in this study to compare the performance of different smoothing techniques is the mean square error (MSE) between the clean (non-noisy) and smoothed signals which is given by:

$${\text{MSE}} = \frac{{\sum\nolimits_{n = 1} {\left( {p_{\text{smoothed}} - p_{\text{clean}} } \right)^{2} } }}{n}$$
(15)

where n represents the numbers of data points of the bottom-hole pressure, pclean and psmoothed are the clean and smoothed pressure signals, respectively.

Results and discussion

To smooth data using this model, it is necessary for the data to be stationary or can be transformed to achieve stationarity. A stationary data has a constant mean with no temporal trend. A quadratic trend in the generated data (Fig. 2) was found when analyzing the data. We removed the trend by first applying a cubic spline fit on the data and second subtracting the original data from the fitted values. All the detrending process was performed using the ITSM time series analysis software (Brockwell and Davis 2013).

Pressure smoothing with the ARMA model

The first step in producing an accurate smoothing by the ARMA model is to find the best orders of the model for the pressure data. Autocorrelation function (ACF) and partial autocorrelation function (PACF) analyses are commonly used to determine the appropriate orders of ARMA. The ACF defines the correlation between series of data at different times. The general formula for ACF is given by Eq. 16 (Brockwell and Davis 2013):

$$\rho_{\text{k}} = \frac{{\text{cov} (R_{\text{it}} ,R_{{i,{\text{t}} - {\text{k}}}} )}}{{\text{var} (R_{\text{it}} )}}$$
(16)

where Rit is time series data at time level i and Ri,t-k is series data shifted back by a lag of k. The PACF describes as the partial autocorrelation of a time series that is the amount of correlation between a variable and a lag of itself which is not explained by correlations at all lower-order lags. The PACF is defined by Eq. 17 (Box et al. 2011; Enders 2008):

$$\pi_{\text{t}} = {{\left( {\rho_{\text{t}} - \sum\limits_{j = 1}^{t - 1} {\pi_{{{\text{t}} - 1,j}} .\rho_{{{\text{t}} - j}} } } \right)} \mathord{\left/ {\vphantom {{\left( {\rho_{\text{t}} - \sum\limits_{j = 1}^{t - 1} {\pi_{{{\text{t}} - 1,j}} .\rho_{{{\text{t}} - j}} } } \right)} {\left( {1 - \sum\limits_{j = 1}^{t - 1} {\pi_{{{\text{t}} - 1,j}} .\rho_{t - j} } } \right)}}} \right. \kern-0pt} {\left( {1 - \sum\limits_{j = 1}^{t - 1} {\pi_{{{\text{t}} - 1,j}} .\rho_{t - j} } } \right)}}$$
(17)

where ρ denotes the autocorrelation and π is partial autocorrelation. The pair plot of autocorrelation and partial autocorrelation versus time lag is named correlogram. Correlogram is a useful tool to provide optimum order of the model. The ITSM software was used to find the best degree of the ARMA model (Brockwell and Davis 2012) as illustrated in Fig. 3. The number of points with large spikes which cross the dashed lines in ACF and PACF plots is used to identify the order of AR and MA parts of the ARMA model, respectively (Friendly 2002). As can be seen from Figure. 3, there are two spikes for ACF and two spikes for PACF, and hence, the best degree for the ARMA model is ARMA (2, 2).

Fig. 3
figure 3

Correlograms of the noisy pressure data

Full size image

The coefficients of the ARMA model were estimated by the least square method. The resulting ARMA model was given by the ITSM as:

$$X(t) = - \, 0.06492X(t - 1) - 0.9664 \, X(t - 2) + Z(t) + 0.07060Z(t - 1) + 0.9834Z(t - 2)$$
(18)

Figure 4 shows the smoothed data points given by the ARMA model and the original data corrupted by 2% noise. It is evident from this figure that the main trend of the original data was kept using this model.

Fig. 4
figure 4

Smoothed pressure data using the ARMA model

Full size image

Pressure smoothing with nonparametric regressive smoothers

Four local regressive smoothers (i.e., SMA, S–G, LOWESS, and LOESS) are considered in this study. Figure 5 shows the noisy and the smoothed pressure data obtained by the methods for 2% noise addition. The calculated MSE for each regressive method is also reported in Fig. 5. The LOESS shows the least MSE. Span also controls the performance of regressive smoothing techniques. We calculated MSE for each smoothing technique at different spans. Figure 6 shows the calculated MSE as a function of span for different noise levels. The common trend which can be seen from all plots in Fig. 6 is that MSE decreases at first which implies that the performance gets better. MSE for SMA and S–G filters decreases with increasing span until they reach a minimum, then MSE starts to increase when span grows up. SMA gives equal weights to all the surrounding points, and error increases with span because data points that are far from the predicted point affect the estimation as much as the closer points. Regarding the LOWESS and LOESS methods, MSE decreases and then almost levels off. Because these two methods are weighted and also fit a polynomial on data points to estimate smoothed values, so as span increases, the number of data points which participate in fitting increases and yields better polynomial fit. As a result, the corresponding MSE decreases and better performance is acquired. From Figure. 6, one can conclude that the LOESS method is more efficient in smoothing than the other local methods.

Fig. 5
figure 5

Smoothed pressure data using a SMA, b S–G, c LOWESS, and d LOESS methods for pressure data with 2% noise addition

Full size image
Fig. 6
figure 6

MSE of various local regressive smoothers versus span for the noisy pressure signal corrupted by a 2%, b 4%, c 6%, and d 8% noise levels

Full size image

As can be seen from Figure. 6, an optimum span can be considered for each method and noise level. In this paper, the optimum span is one which minimizes MSE. The calculated optimum spans are given in Table 2 for each local smoother method for various noise levels. Figure 7 illustrates the optimum span as a function of noise levels for various local smoothing methods. The S–G filter shows small changes in optimum span with increasing noise levels. In other words, the S–G filter has a constant optimum span with different noises, and hence, this method is not affected by noise levels. For the LOESS method, a specific trend is not observed, while there is an approximately increasing trend for the LOWESS and SMA.

Table 2 Optimum spans for the regressive smoothing methods with different levels of noise
Full size table
Fig. 7
figure 7

Optimum spans versus noise percentages for various regressive smoothing methods

Full size image

Pressure smoothing with wavelet-based methods

In Figure. 8, denoised data for the soft and hard interval thresholding are illustrated. It is evident that the soft thresholding shows better performance with respect to the hard thresholding. In order to improve noise removal performance by WT, a number of denoised versions of the signal are obtained iteratively. Since the variance estimation in the first pass of the thresholding operation is based on the observed noisy coefficients Y (m, n), a considerable amount of unreliable estimates leads to visually annoying noise specks in the reconstruction. Hence, it is proposed to add some additional steps of thresholding, each based on a local variance estimation that uses already denoised coefficients of the previous step. Suppose that the denoised coefficients of the (i − 1)th iteration step is represented by x̂(i−1) (m, n), then the local variance of the ith iteration step and threshold is determined (Beck and Teboulle 2009; Bioucas-Dias and Figueiredo 2007; Daubechies et al. 2003; Fornasier and Rauhut 2008). Figure 9 illustrates the calculated MSE values for different WT methods. The iterative thresholding methods exhibit better performance than the interval thresholding ones. Overall, the soft iterative thresholding illustrates the minimum MSE.

Fig. 8
figure 8

Denoised signal for a hard interval thresholding and b soft interval thresholding

Full size image
Fig. 9
figure 9

MSE values for hard and soft interval thresholding (IT) and iterative interval thresholding (IIT)

Full size image

Figure 10 shows the smoothed data points given by the hard and soft iterative thresholding methods for 5 and 25 iterations. Denoising performance for hard and soft thresholding for different iteration number is shown in Fig. 11, in terms of MSE for different iteration numbers.

Fig. 10
figure 10

Denoised pressure for a soft thresholding with 5 iterations, b soft thresholding with 25 iterations, c hard thresholding with 5 iterations, and d hard thresholding with 25 iterations

Full size image
Fig. 11
figure 11

MSE for hard and soft iterative thresholding at different iteration

Full size image

Comparisons of different smoothing methods at different noise levels

Figure 12 compares MSE values of different smoothing methods used in this study for different levels of noise. It is evident that the lower the MSE, the more efficient the smoothing technique. Overall, the best results correspond to the WT and LOESS. For simulated pressure with 2% artificial noise, it can be seen from the results that the LOESS has minimum MSE and therefore best performance. For this noise level, the MSE of LOESS is lower than the MSE of WT by 21.5%. In addition, the maximum MSE was obtained for the SMA method. As more noise was added to the clean simulated signal, slight changes in the MSE values are observed. The SMA shows better performance at low noise level. For other methods, the results may alter by adding more noise to the pure signal. For 4% noise level, the WT shows better performance than the LOESS method by a small margin. SMA shows maximum MSE for all noise levels, and hence it is not recommended for denoising. For 6% noise level, LOESS has the best performance and its MSE is lower than the MSE of WT by 17.5%. When the clean pressure signal is corrupted by 8% noise level, the LOESS can reduce the noise level as much as the wavelet domain filter does.

Fig. 12
figure 12

MSE related to noises removal from pressure transient data by different denoising data for a 2%, b 4%, c 6%, and d 8% artificial noise addition

Full size image

Performance of combination of smoothing techniques for noise reduction

After having assessed the smoothing effects of various methods for different levels of noise, it may be of interest in this context to evaluate noise reduction using combination of smoothing methods. Table 3 reports the calculated MSE of the combined methods for the simulated pressure signal corrupted by adding 2% noise. From Table 3, it is concluded that the combined LOESS and ARMA method gives the best performance, namely a nearly 50% reduction in MSE. In this work, various combinations of the studied smoothing techniques have been examined for noise reduction. Unfortunately, a specific combination of smoothing techniques that works for all levels of noise was not found. However, this study shows that combination of smoothing methods can reduce MSE significantly in comparison with the results obtained by single smoothers.

Table 3 Calculated MSE of various combined smoothing methods for pressure signal with 2% noise
Full size table

Evaluation of smoothing efficiency for pressure derivative data

In the case of a noisy pressure signal with 2% added Gaussian noise, it was shown that the LOESS method led to the highest smoothing performance (Fig. 12). Figures 13 and 14 illustrate the noisy pressure derivative, calculated by adding 2 and 6% noise to the original non-noisy (clean) pressure signal, respectively, and the smoothed pressure derivatives, calculated by applying the LOESS method to the noisy pressure data before calculating the derivatives. The clean pressure derivatives calculated from the noise-free pressure data were also shown for comparison purposes. It is evident from Figures. 13 and 14 that, for the original pressure derivative data, the noise is so overwhelming that no pattern or trend can be observed. In contrast, the smoothed derivative data using the LOESS clearly shows a dual-porosity flow model in an infinite acting reservoir for 2% noisy pressure data (Fig. 13). However, the performance of the LOESS filter deteriorates when the level of noise increases as evident from Fig. 14. In a similar way, noise reduction from pressure derivative data was examined using various combined methods of smoothing. In all of these methods, denoising techniques were applied to pressure data and then the pressure derivative was calculated using the Bourdet pressure derivative formula (Eq. 14). In this work, it was not feasible to denoise pressure derivative data after its calculation by the Bourdet method since they did not show any trend. The smoothed pressure derivatives calculated using the combined ARMA and LOESS method are shown in Figs. 13 and 14. The most striking results to emerge from Figs. 13 and 14 are that the combined LOESS–ARMA makes the noisy signal much smoother, particularly for higher noise levels as evidenced by comparing the results shown in Fig. 13 with those in Fig. 14.

Fig. 13
figure 13

(Red) the calculated pressure derivatives from clean noise-free pressure data; (green) the calculated noisy pressure derivatives from 2% noisy pressure data; (magenta) the smoothed pressure derivatives, calculated by applying the LOESS filter; and (blue) the smoothed pressure derivatives, calculated by applying the combined LOESS and ARMA filters to the noisy pressure data

Full size image
Fig. 14
figure 14

(Red) the calculated pressure derivatives from clean noise-free pressure data; (green) the calculated noisy pressure derivatives from 6% noisy pressure data; (magenta) the smoothed pressure derivatives, calculated by applying the LOESS filter; and (blue) the smoothed pressure derivatives, calculated by applying the combined LOESS and ARMA filters to the noisy pressure data

Full size image

Application of the proposed denoising method to real field data

To demonstrate the reliability and applicability of the methods discussed in this paper for real field pressure data, we applied the denoising methods discussed in this paper to one of the pressure test data sets published by Bourdet et al. (1984). The available pressure transient test data in a naturally fractured reservoir were chosen for which the important reservoir and well properties are listed in Table 4. The pressure derivative versus time plot of the used data is shown in Fig. 15 by green color. As shown before, we have found that the combined LOESS–ARMA method showed the best performance for noise reduction and pattern recognition. As a result, this method was applied for the field data. The smoothed pressure derivatives are shown in blue in Fig. 15. It is evident from Fig. 15 that the applied method effectively smoothed the noisy field data and recovered the appropriate pressure derivative pattern for fractured reservoirs.

Table 4 Reservoir rock/fluid and well parameters of the pressure transient test (Bourdet et al. 1984)
Full size table
Fig. 15
figure 15

(Green) the pressure transient test data published by Bourdet et al. (1984); (blue) the smoothed pressure derivatives using the combined LOESS–ARMA method

Full size image

Conclusions

In this paper, we have compared the performances of regressive, autoregressive, and wavelet smoothing methods as well as various combinations of them to reduce noise from the synthetic well test data of a dual-porosity infinite reservoir. WT technique has been used in most previous studies. We show that the LOESS method can lead to the same or even better results than the widely used WT. Several key findings can be summarized as:

  1. 1.

    The LOESS and WT have shown the best performances in reducing noise. In 6% noise level, the MSE of the LOESS is less than the MSE of the WT by 17.5%.

  2. 2.

    SMA acts as a low-pass filter and the accuracy of this method decreases significantly by increasing the noise level.

  3. 3.

    In regression-based method, MSE decreases as span increases. Particularly, in the S–G and SMA methods, MSE first decreases, then reaches a minimum value and after that begins to increase. When time span increases in local regression, MSE is decreased. In LOWESS and LOESS methods, the same trend is observed with the difference that MSE remains unchanged when reaching the minimum point.

  4. 4.

    WT with soft thresholding shows better results than hard thresholding. It is found that iterative interval thresholding results in the minimum MSE.

  5. 5.

    The ARMA method significantly improves the reduction in noise as compared to the simple moving average.

  6. 6.

    The amount of noise added to the clean pressure signal affects the performance of smoothing techniques.

  7. 7.

    Regarding various combinations of smoothing filters, it was found that the combined LOESS–ARMA scheme yields a significantly better noise reduction performance. The combined method is more efficient when it is applied for smoothing the pressure derivative data.

  8. 8.

    The combined LOESS–ARMA method efficiently smoothed the field data with moderate noise levels and recovered the ideal pattern in the log–log diagnostic plot.