Real-time relative permeability prediction using deep learning

A review of the existing two- and three-phase relative permeability correlations shows a lot of pitfalls and restrictions imposed by (a) their assumptions (b) generalization ability and (c) difficulty with updating in real-time for different reservoirs systems. These increase the uncertainty in its prediction which is crucial owing to the fact that relative permeability is useful for predicting future reservoir performance, effective mobility, ultimate recovery, and injectivity among others. Laboratory experiments can be time-consuming, complex, expensive and done with core samples which in some circumstances may be difficult or impossible to obtain. Deep Neural Networks (DNNs) with their special capability to regularize, generalize and update easily with new data has been used to predict oil–water relative permeability. The details have been presented in this paper. In addition to common parameters influencing relative permeability, Baker and Wyllie parameter combinations were used as input to the network after comparing with other models such as Stones, Corey, Parker, Honapour using Corey and Leverett-Lewis experimental data. The DNN automatically used the best cross validation result (in a five-fold cross validation) for its training until convergence by means of Nesterov-accelerated gradient descent which also minimizes the cost function. Predictions of non-wetting and wetting-phase relative permeability gave good match with field data obtained for both validation and test sets. This technique could be integrated into reservoir simulation studies, save cost, optimize the number of laboratory experiments and further demonstrate machine learning as a promising technique for real-time reservoir parameters prediction.


Introduction
Relative permeability is the most important property of porous media to carry out reservoir prognosis in a multiphase situation (Delshad and Pope 1989;Yuqi and;Dacun 2004) and therefore needs to be as accurate and readily accessible as possible. Theoretically, it is the ratio of effective and absolute permeability. It is useful for the determination of reservoir productivity, effective mobility, wettability, fluid injection for EOR, late-life depressurization, gas condensate depletion with aquifer influx, injectivity, gas trapping, free water surface, residual fluid saturations, temporary gas storage amongst others (Fig. 1). It is well known that a significant variation in relative permeability data can have a huge impact on a macroscopic scale.
The oil and gas industries have a need for easily available and reliable relative permeability data, expense reduction on experiments and a more general model for the parameter judging by the pitfalls pointed out by several researchers (Table 1) after testing the existing two-and three-phase relative permeability models. Such workers 1 3 like Fayers and Matthews (1984) and Juanes et al. (2006), after testing non-wetting relative permeability interpolation models such as Baker and Stone's I and II, against Saraf et al. (1982), Schneider and Owens (1970), Saraf and Fatt (1967) and Corey et al. (1956) experimental data, presented the same conclusion that they give similar results for high oil saturations but are different as it tends towards residual oil saturation. Manjnath and Honarpour (1984) concluded that Corey gives higher values for non-wetting phase relative permeability after comparing against Donaldson and Dean data.
Based on the assumption that water and gas relative permeability depends only on their saturation and not on that of other phases, Delshad and Pope (1989) concluded after a comparative study of seven relative permeability models that Baker and Pope performed better but also stated the need for better models. Siddiqui et al. (1999) found Wyllie-Gardner and Honarpour to yield consistently better results at experimental condition after testing ten relative permeability models. Al-Fattah and Al-Naim (2009) found Honarpour regression model to be the best after comparing with five other models and also developed his own regression model. Since the coefficients of these regression models are not generalized, they are not suitable for real-time applications.
Furthermore, for wetting phase relative permeability in consolidated media, Li and Horne (2006) showed that the Purcell model best fits the experimental data in the cases studied by them provided the measured capillary pressure curve had the same residual saturation as the relative permeability curve which is sometimes not the case. Saraf and McCaffery (1985) could not recommend a best model due to scarcity of three-phase relative permeability data. The different relative permeability correlations have limitations and assumptions which no doubt have implications, thus increasing the uncertainty in reservoir simulation studies hence the need for a more generalized model. Therefore, the purpose of this study is to implement a Deep Neural Networks model for the prediction of relative permeability accounting for reservoir depletion, saturation and phase changes with time. Guler et al. (1999) developed several neural network models for relative permeability considering different parameters that affects the property and selected the best model to make predictions for the test set while Al-Fattah (2013) also used a generalized regression neural network to predict relative permeability. Getting better prediction for out-of-sample datasets (better generalization), performance flattening out with a certain amount of data (scalability) as well as requiring far more neurons (and hence an increased computational time) to achieve better results as deep learning models is an issue for such networks. Again most of the reviewed empirical models can hardly generalize (Du Yuqi et al. 2004) and are static but deep neural networks (with its advanced features), if appropriately tuned, can capture the Fig. 1 Schematic of oil-water relative permeability curve transients faster and more accurately throughout the reservoir life while also getting better as more data becomes available with time. Training can be done offline and the trained networks are suitable for on-board generation of descent relative permeability profiles as their computation requires a modest CPU effort hence not a concern to real-time application.

Methodology
The most commonly available factors influencing relative permeability such as porosity, ∅; viscosity, µ; permeability k; saturation s, together with Baker and Wyllie parameter combinations were used as inputs for the network. Baker gave correlation coefficients of 0.96 and 0.86 while Wyllie has correlation coefficients of 0.91 and 0.89 for Corey and Leverett-Lewis datasets, respectively ( Table 2). There were a total of 12 input parameters fed into the network as shown in Table 3 after testing the sensitivity of several parameter combinations.
Ten (10) sets of water-oil relative permeability data with 132 data points from a North Sea field with fourfifths used as training set and one-fifth as validation set. Another set of water-oil relative permeability data from a separate field were used as the testing set after data wrangling and normalization. A seed value was set to ensure the repeatability of the model. An optimised number of hidden layers was used to reduce the need for feature engineering. The best cross validation result in a fivefold arrangement was automatically used to train the DNN models until convergence using Nesterov-accelerated gradient descent (which minimize their cost function). The Rectifier Linear Units (ReLUs) were used in the DNN modelling to increase the nonlinearity of the model, significantly reduce the difficulty in learning, improve accuracy and can accept noise (Eq. 1). This allows for effective training of the network on large and complex datasets making it helpful for real-time applications compared to the commonly used sigmoid function which is difficult to train at some point.
where Y ∼ ℵ(0, (x)) is the Gaussian noise applied to the ReLUs.
Separate models were constructed for wetting and nonwetting phases as they have also been found to improve predictions (Guler et al. 1999). They were then validated and tested to check the generalization and stability of the models for out-of-training sample applications.
The developed Deep Neural Networks model could further be applied to predict other experimental data carried out based on Buckley and Leverett (1942) frontal advance theory (Fig. 2) and Welge (1952) method for average water saturation behind the water front using the saturation history to make predictions of relative permeability as a function of time.

Deep neural networks
Deep neural networks (sometimes referred to as stacked neural network) is a feed-forward, artificial neural network with several layers of hidden units between its inputs and outputs. One hundred hidden layers with twelve neurons each (100, 12) were used in this work. The ability of the model to transfer to a new context and not over-fit to a specific context (generalization) was addressed using cross validation which is described in detail below. All networks were trained until convergence with Nesterov-accelerated gradient descent which also minimizes the cost function. In addition, both 1 and 2 regularization (Eq. 2) were used to add stability and improve the generalization of the model. This regularization ability was further improved by implementing dropout. A copy of the global models parameters on its local data is trained at each computed node with multi-threading asynchronously and periodically contributes to the global model through averaging across the network.
Mathematically, where x are inputs, are parameters, is a measure of complexity by introducing a penalty for complicated and large parameters represented as l1 or l2 (preferred to l0 for convexity reasons). They are well suited for modelling systems with complex relationships between input and output which is what is obtainable in natural earth systems. In such cases with no prior knowledge of the nature of non-linearity, traditional regression analysis is not adequate (Gardner and Dorling 1998). It has been successfully applied to real-time speech recognition, computer vision, optimal space craft landing etc.

Cross validation
Overfitting which is the single major problem of prediction when independent datasets is used was reduced through cross validation by estimating out-of-sample error rate for the predictive functions built to ensure generalization. Other issues like variable selection, choice of prediction function and parameters and comparison of An extension of Purcell (1941) and Burdine (1953) Based on bundle of capillaries cut and rejoined along their axis with related entrapment of the wetting phase Considers irreducible water as part of the rock matrix Applied when water saturation is at irreducible level Honarpour et al. (1982) Water wet: Based on relative permeability, saturation-fluid pressure functional relationships with a flow channel distribution model in two-or three-phase flow subject to monotonic saturation path and to estimate effective mean fluid conducting pore dimensions Wettability takes the water > oil > gas sequence Irreducible fluid saturation is independent of fluid properties or saturation history No gas/water contact occurs in the three-phase region until the level where oil exists as discontinuous blobs or pendular rings Limited to cases where a satisfactory fit to the two-phase data is provided by the fitting equations m = 1 − 1∕n 1 3 different predictors were also addressed. A fivefold cross validation technique was used to split the data set into training and test set, build a model on the training set, evaluate on the test set and then repeat and average the errors estimated. A weight decay was chosen to improve the generalization of the model by suppressing any irrelevant component of the weight vector while solving the learning problem with the smallest vector. This also suppresses some of the effects of static noise on the target if chosen correctly and increases the level of confidence in the prediction (Fig. 3).

Results and discussion
Deep neural networks model has been validated using separate out-of-sample datasets not used for the training. The good agreement between experimental data and DNN's model predictions indicates that the complex, transient, non-linear behaviour of reservoir fluids can be effectively modelled as their saturation and phase changes with time. Figures 4, 5 and 6 give a comparison between actual experimental values and model predictions using neural networks without cross validation, neural networks with cross validation and the deep neural networks. The objective here was to see how deep learning out performs ordinary networks on new data. These cross plots show the extent of agreement between the laboratory and predicted values. For the testing set drawn from a different field from the training set, the deep neural networks for both the wetting and non-wetting phase relative permeability ( Fig. 6b and d) gives very close values to the perfect correlation line in all data points compared to the other models. Figure 4a and c representing neural networks without cross validation, gave an RMS value of 0.2484 and 0.0767 while neural net with cross validation gave an RMS of 0.0624 and 0.0765 ( Fig. 5a and c). The deep neural net gave an RMS value of 0.2517 and 0.065 (Fig. 6a and c) for both wetting and non-wetting relative permeability. It is clear that all the models did well for the validation set although the deep neural networks performed better than the other two models. The different models were then shown new data from a separate field to see how they performed. For the test set (which is an out-of-sample dataset) obtained from a different field, the RMS for neural network without cross validation is 0.9996 and 0.8483 ( Fig. 4b and d), 0.2295 and 0.8022 with cross validation (Fig. 5b and d) while DNNs gave 0.0759 and 0.15 ( Fig. 6b and d) for wetting and non-wetting relative permeability, respectively. The deep learning model used the fourth cross validation model which happen to be the best for the  As the saturation of a phase tends to zero, that of the other two-phase will dominate The end points of the three-phase relative permeability isoperms coincide with the two-phase relative permeability data Based on the mean hydraulic radius concept of Kozeny-Carman (bundle of capillaries model) Assumes that the whole spectrum of the relative permeability curve can be captured with the L, E, T parameters It exhibits enough flexibility to reconcile the entire spectrum of experimental data   wetting phase with a correlation coefficient of about 97% (Table 3) and the lowest training error of 0.0014 while the second cross validation model was used for the nonwetting phase relative permeability having 96% correlation coefficient and the lowest training error value of 0.030 (Table 4). Figures 7 and 8 display the trend comparing the different models using the standard relationship between saturation and relative permeability. The deep learning model clearly out performs the other models giving better predictions for both the wetting and non-wetting phases. Measurement error which causes input values to differ if the same example is presented to the network more than once is evident in the data. This limits the accuracy of generalization irrespective of the volume of the training set. The deep neural networks model deeply understands the fundamental pattern of the data thus able to give reasonable predictions than ordinary networks and empirical models (Figs. 9, 10). The curves show that significant changes in the saturation of other phases has large effect on the wetting phase ability to flow as observed from the less flattening of the water relative permeability curve and vice versa for the flattened curve. Although this flattening behaviour is usual in the secondary drainage and imbibition cycles but mainly in the wetting phase when flow is mainly through small pore networks. Again, the curve flattening of the oil relative permeability curve could from experience be from brine sensitivity and high rates causing particle movements resulting in formation damage. Figures 9 and 10 compares the deep neural network model with commonly used empirical relative permeability models like Baker, Wyllie, Honarpour, Stones, Corey, Parker. Despite the fact that some of these models were developed using lots of datasets way more than the amount Actual vs predicted value for neural networks without cross validation (cross validation not considered as part of the model formulation) with a wetting phase relative permeability for validation set, b wetting phase relative permeability for test set, c non-wetting relative permeability for validation set, d non-wetting relative permeability for the test set used for training the deep neural networks, it still out performed them showing that it is more able to capture the transients and eddies in real-time scenarios due to its ability to regularize and generalize using its robust parameters as discussed earlier. Figures 11 and 12 corroborate the earlier observation that the deep learning model predicts better compared to most of the relative permeability models used in reservoir modelling software. It is important to note here that the empirical models (Figs. 9, 10) have a problem of generalization especially as every reservoir is unique. Again, the assumptions associated with their formulation might not be practically true in all cases but this reservoir uniqueness or generalization is captured by the deep learning model bearing in mind that it will perform even better as more real-time data are added to the training set. Figures 13 and 14 describe the relative importance (sensitivity) of the variables used for the wetting and non-wetting deep learning relative permeability models. The wetting phase model was more sensitive to its saturation and relatively less Fig. 5 Actual vs predicted value for neural networks with cross validation technique used for its model formulation and it improved prediction ability of the network with a wetting phase relative permeability for validation set, b wetting phase relative permeability for test set, c non-wetting relative permeability for validation set, d nonwetting relative permeability for the test set Fig. 6 Actual vs predicted value for deep neural networks model with a wetting phase relative permeability for validation set, b wetting phase relative permeability for test set, c non-wetting relative permeability for validation set, d non-wetting relative permeability for the test set sensitive to that of the non-wetting phase while the non-wetting phase model was very sensitive to both its saturation and that of the wetting phase. Both models were also more sensitive to their own viscosities than the other. These models seem to obey the basic physics underlying relative permeability modelling. The least important variable still contributed above the median mark although in general, all variables show greater sensitivity in the non-wetting model than in the wetting relative permeability model. Table 5 shows the performance of the different variables combinations for both the wetting and non-wetting phase model.

Conclusion
A deep neural network methodology has been formulated for wetting and non-wetting phase relative permeability predictions taking into account phase and saturation changes hence its capability for real-time applications. This work has the following conclusions: 1. Deep neural network has shown to be a good predictive and prescriptive tool for relative permeability than ordinary networks. Its ability to generalize and regularize helped to stabilize and reduce the main problem of all predictive tools which is over fitting. 2. Different results were obtained from different relative permeability models for the same reservoir with some of the models giving better predictions at lower saturations but performs poorly at higher saturations and vice versa; hence, lots of uncertainty. Therefore, it is needful for practitioners to know the limitations of any correlation used for the prediction of wetting and non-wetting phase relative permeability. 3. In an industry where big data is now available, deep learning can provide the platform to systematically    forecast reservoir fluid and rock properties to drastically optimize the cost and time needed for laboratory experiments. Even with the amount of data used, the power of the deep neural networks is evident in that it gave reasonable predictions which will dramatically improve if more data were available.