Analysis of MCDM methods output coherence in oil and gas portfolio prioritization

Abstract

This research paper aims at developing a triplex multi-criteria decision model to achieve an optimum allocation of resources to a portfolio of oilfields based on a number of criteria that pertain to the oilfields’ technical and contextual characteristics. It also offers an approach for validating the prioritization results generated by the methods used through a two-dimensional test that compares the internal and external prioritization stability of three multi-criteria decision making (MCDM) methods against a conventional Intuitive approach. The MCDM methods used include Analytic Hierarchy Process (AHP), Preference Ranking Organization METHod for Enriched Evaluation (PROMETHEE), and Technique of Order Preference Similarity to the Ideal Solution (TOPSIS). The internal and external consistency testing results are outlined in nine possible scenarios through which a conclusion is derived confirming the competitive advantage of MCDM methods.

Introduction

The complexity of oil and gas (O&G) portfolio investment is continuously growing throughout the world. A typical E&P decision-making problem faced by national and international oil companies involves prioritization of projects in an investment portfolio in the face of limited financial resources. The prioritization is often complicated by the need to consider multiple competing and non-commensurable criteria, such as subsurface complexity, size of reservoir, plateau production, and needed infrastructure, in addition to other issues of strategic concern, such as socioeconomic, environmental, and fiscal policies, particularly in the case of governments or national oil companies. In many cases, however, these decisions are made through improvised Intuitive approaches or using decision-aiding tools that vary in focus and commonly lack the capacity to fully address the criteria involved and accurately incorporate the decision makers’ knowledge and preferences. The MCDM outranking methods have emerged as promising tools for project prioritization (Lopes and De Almeida 2013). However, they have been observed to perform differently in different applications (Cooper 2001), which also implies achieving inconsistent results by different methods if applied to the same decision problem. This research paper proposes measuring and comparing the consistency of results generated by three methods as a way to determine the validity of MCDM methods in E&P project prioritization and suggests an approach to prioritize the portfolio projects in the case of applying multiple methods. As an immediate application, the proposed approach is used to measure the percentage shares of six oilfields in Iraq, which together total more than 60 billion barrels of Iraq’s proven oil reserves. As illustrated in Fig. 1, three hypotheses speculate upon the outcome of the research study and a three-stage survey provides the data needed to test the hypotheses, which in turn involves three layers of data analysis, each with three discrete quantitative methods. The research is characterized by employing a “triplex” approach due to the three-pronged approach in which each of the main research elements splits into three components.

Fig. 1

Triplex approach

Literature review

E&P portfolio selection

The modern study of project portfolio selection and prioritization is associated with Modern Portfolio Theory (MPT) by Nobel Laureate Harry Markowitz. In his paper “Portfolio Selection,” he argued that an optimal portfolio is one that delivers the maximum expected return for a given level of risk and defined risk as the standard deviation of return (Markowitz 1952). The theory has been widely adopted in finance, but its application to project portfolio selection and risky industrial projects, as opposed to stocks, was first discussed by Hertz (1968). Since then, many others investigated the application of MPT on E&P projects, including Jiang et al. (2014), Xue et al. (2014) and (Mutavdzic and Maybee 2015). Numerous other methods and techniques have also been investigated to use for E&P portfolio selection. Rodriguez and Pádua (2005), for example, presented a model for the application of portfolio optimization with risk assessment to E&P projects that aimed at maximizing NPV (after tax) while accounting, investigating, and analyzing the inherent uncertainties. Chen (2011) studied modeling E&P project portfolio optimization through multistage stochastic programs with mixed integer programming (MIP) techniques. However, E&P portfolio selection decision problems are particularly challenging due to the high stakes and significant consequences, and selection becomes a trade-off decision, where certain objectives are met at the expense of others (Lopes and De Almeida 2013). Achieving an optimum trade-off requires evaluating all alternative projects based on the same predefined quantitative and qualitative criteria using a systematic methodology (Lopes and De Almeida 2013).Traditional capital budgeting tools, such as benefit–cost analysis, utility value, payback period, discounted cash flow, sensitivity analysis, preference theory, portfolio theory, and option theory are among those currently adopted by the industry. However, these techniques are mostly economics based and do not adequately consider other critical and strategic factors, such as social and environmental impacts, which cannot be captured by monetary valuations and in some cases not ethically acceptable, as argued by Cavallaro (2005). For a portfolio of oilfields to reach a total production target, each oilfield must be evaluated with respect to its individual optimum capacity. Consequently, the resources available for the development of the oilfields typically should not be distributed equally, nor should they be distributed based merely on the fields’ respective sizes. Instead, decision makers need to determine the optimum shares of each oilfield of the investment resources needed to achieve the total production target. Therefore, this problem is inherently multi-objective since various factors need to be considered simultaneously. Moreover, selecting the analytical tool that can synthesize the decision makers’ judgments is not an easy task considering the large number of methods available, which often makes it challenging to justify the choice. Despite the numerous available methods, none can be considered flawless or identified as the “super method” that is appropriate to use with all decision-making problems. They all have limitations, assumptions, and different characteristics and levels of sophistication (Ishizaka and Nemery 2013). Moreover, reviewing the available MCDM literature with respect to portfolio selection, it can be observed that it is: (1) primarily focused on suggesting specific MCDM methods without sufficiently justifying or validating their use, (2) inadequate in addressing E&P portfolio selection and prioritization relative to other fields, and (3) mainly concerned with green fields and not brown fields or a mix of both.

Application of multiple MCDM outranking methods in E&P portfolio selection

The advantages of using MCDM methods in project portfolio selection decision problems include the ability to address not only the two central criteria of risk and return on investment, but also any and all of the decision makers’ preferences and objectives (Lopes and De Almeida 2013). However, while there are numerous methods that can be used in such decision problems, there is no evidence that one method is most effective in all applications. A method that is most suitable for one group of projects might be unsuitable for another (Archer and Ghasemzadeh 1999). Roy and Bouyssou (1993) further argue that “a systematic axiomatic analysis of decision procedures and algorithms is yet to be carried out.” However, Cooper (2001) argued that using more than one method for these decision problems yields better outcomes given that individual methods perform differently in different applications, which also implies achieving inconsistent results by different methods if applied to the same decision problem. Examining the literature on the application of multiple MCDM outranking methods, the researchers have classified the studies into hybrid applications and parallel applications, as summarized below.

Hybrid applications

A typical MCDM process involves two main elements: (1) measuring criteria weights, and (2) ranking of alternatives (Wright and Loosemore 2001). Although most MCDM outranking methods address both elements, some research studies prefer using different MCDM methods for each. Some attribute the need to use a hybrid model to the inherent deficiencies of the methods used. For example, AHP requires a relatively large amount of computation and TOPSIS determines the criteria weights subjectively. For that reason, Chen et al. (2013) used AHP to derive the criteria weights of an MCDM model to address a portfolio problem in modern financial investments, while using TOPSIS to rank the alternatives. They argued that this approach helps in curtailing AHP’s lengthy computations and increasing objectivity in determining the criteria weights. Similarly, Lin et al. (2008) used AHP and TOPSIS to solve a customer-driven design problem, while Bakshi and Sarkar (2011) used AHP for analyzing the structure of a telecommunication project selection problem and to derive the criteria weights, and additive ratio assessment (ARAS) to obtain the final rankings of the alternative projects.

Parallel applications

In order to add an extra level of verification to the decision-making process and increase its reliability, some research studies have employed multiple MCDM methods to the same decision problem simultaneously. Some of these multi-method models yield similar results, while others do not. Research studies address variations between the ranking results differently. Some studies present these variations as an observation and highlight the strongest ranking differences and similarities produced by the methods employed. Kangas et al. (2001), for example, used Multi-Attribute Utility Theory (MAUT), ELimination Et Choix Traduisant la REalité (ELECTRE III), and PROMETHEE II for planning strategic natural resources, while Opricovic and Tzeng (2007) used VIse Kriterijumska Optimizacija kompromisno Resenje (VIKOR), TOPSIS, ELECTRE, and PROMETHEE for evaluating hydropower systems. Other studies have gone an extra step in attempting to explain the variations through a sensitivity analysis by modifying the values of certain elements (e.g., criteria weights, maximum, and minimum thresholds) and observing the changes that occur to the ranking results. For example, Gomes et al. (2010) used Portuguese acronym for Interactive and Multiple Attribute Decision Making (TODIM) and AlgoriTmo Híbrido de ApoiO MulticritéRio à Decisão para Processos Decisórios com Alternativas Discretas (THOR) for evaluating investment opportunities of natural gas reserves, while Martowibowo and Riyanto (2011) used AHP, ELECTRE II, PROMETHEE II, and TOPSIS for selecting municipal solid waste treatment technologies. However, there might be no way to verify the correctness of the methods used as there is neither a true best methodology nor a true ranking of the Pareto optima. The only ways available to assess the performance of the MCDM methods are: empirical studies and comparisons (Selmi et al. 2013). Therefore, as it is practically more challenging to conduct empirical studies of real decision problems (i.e., choosing multiple courses of action simultaneously), comparing the methods might be more practical. This research attempts to contribute to the literature by proposing an approach that compares the methods through measuring the consistency of their respective ranking results and suggesting nine scenarios that outline the consistency results, which can help derive the conclusion on the most reliable method.

Case study

Iraq’s oil and gas reserves are considered among the largest in the world. However, Iraq’s infrastructure lacks the capacity needed to take full advantage of this wealth of oil and gas. 45%t of Iraq’s GDP and 90% of the federal government’s revenue are based on oil exports (INES 2012). Iraq’s long-term prosperity will require diversifying the economy and mitigating its dependency on its fossil resources; but for the foreseeable future, a significant increase in the nation’s production and prudent utilization of the revenues it generates is vital to the country’s economic development.

Iraq has an estimated 143.1 billion barrels (Gbbl) of oil reserves that accounts for 8.7% of the world’s total reserves and ranks the fifth largest in the world (USGS 2013). Despite the high revenue this resource can generate, it is still prudent to develop different scenarios of production levels that respond to the dynamic oil market. This can be achieved by determining the optimum share of each oilfield of development resources needed to reach targeted production levels. Six of the major oilfields in Iraq were selected as a case study for this research. These fields are often referred to as the “Big Six” for having the largest oil reserves: (i.e., Rumaila: 17.8 Gbbl, West Qurna I: 8.6 Gbbl, West Qurna II: 12.9 Gbbl, Majnoon: 12.6 Gbbl, Zubair: 4 Gbbl, and Halfaya: 4.1 Gbbl)¹—more field characteristics are provided in Table 1. These oilfields are at different locations and levels of maturity and of varying technical attributes. This research paper does not attempt to offer a comprehensive evaluation of Iraq’s oilfields’ production potential, nor does it try to suggest a specific investment plan. The ranking results described herein are solely a reflection of subject matter expert preferences, structured through the MCDM methods. Moreover, while the researchers have made every effort to ensuring that a diverse, qualified, and unbiased group of experts participate in the study, the final results may still carry a certain level of subjectivity, as is the case with all similar investigations.

Table 1 Oilfield characteristics

Methodology

The research framework is composed of discrete and consecutive phases and activities. As shown in Fig. 2, the first phase (Data Formulation) includes activities through which the required data were formulated. These activities include: (1) identifying the subject matter experts (SMEs), (2) identifying the criteria, and (3) evaluating the criteria and alternatives through pairwise comparisons. After the data are acquired, the second phase (Calculations and Data Analysis) starts. This phase covers the main analysis and computation activities, which include: (1) testing the pairwise comparison matrices for rank reversals, (2) analyzing the priorities using three priority analysis methods (Eigenvalue, Approximate, and Geometric Mean), and (3) aggregating the priorities using three MCDM methods (AHP, PROMETHEE, and TOPSIS) to determine the overall alternatives’ priorities. The third phase (Hypotheses Testing) incorporates the activities that determine the optimum tools, which include: (1) measuring internal and external consistencies through using three correlation coefficients (Spearman’s, Pearson’s, and Kendall’s), (2) conducting a sensitivity analysis to measure the criteria’s effect on the final results, and (3) validating the methodology through eliciting the SMEs’ assessment of the results and methods used. The final phase (Conclusion) provides the research findings and conclusion, which include discussing the results in addition to offering recommendations for further research.

Fig. 2

Research methodology

Phase 1: data formulation

Identification of SMEs

A total of 42 SMEs from industry and academia were invited to participate in the study, with prerequisite qualifications, including a minimum of bachelor’s degree in engineering, management, or economics, 25 years of relevant experience in O&G and in-depth knowledge of the six oilfields, and 14 SMEs agreed to commit to the study. The Delphi technique was used to elicit the SMEs’ perspectives and preferences as it allows for remote interactions through multiple stages (i.e., Delphi rounds), with each stage building on the results of the previous one. Turoff (1970) defines a “suitable group size” for the Delphi technique to be anywhere between ten and fifty participants. However, many others have suggested an optimum panel size of up to twelve people (Cavalli-Sforza and Ortolano 1984; Phillips 2000). Furthermore, the SME survey involved three consecutive questionnaires (i.e., Questionnaire 1 to identify criteria—filled by 14 SMEs, Questionnaire 2 to pairwise compare criteria and alternatives—filled by 10 SMEs, and Questionnaire 3 to evaluate the methodology—filled by 10 SMEs).

Identification of criteria

The criteria addressed by this research paper can be defined as the factors that apply when prioritizing alternative brown and green oilfields in an upstream portfolio investment with the goal of increasing production under resource constraints for policy purposes. The process of identifying the criteria was iterative. It initially involved identifying and constructing a hierarchy of the goal, objectives, and attributes and then deriving the criteria (combination of objectives and attributes) that fed into the decision-making model. Psychological studies have repeatedly proven that the human brain is limited in both its short-term memory capacity and its determination ability to seven to nine “things” (Forman and Shelly 2001). Saaty (1980) also argued that maintaining a reasonable consistency when deriving priorities from paired comparisons requires the number of criteria being considered to be equal or less than nine. Therefore, this research paper has chosen to use eight criteria as the basis for prioritizing the oilfields. By filling out Questionnaire 1, each SME initially provided a list averaging 12 items (minimum of 5 and maximum 32). The researchers produced an aggregate list and sent it back to the SMEs for further review and comment. Multiple Delphi rounds followed, resulting in a hierarchy that included around 90 items classified as a goal, objectives, and attributes, which was constructed according to Kidd et al. (1977) composition of decision problems. A list of 38 criteria was then derived from the hierarchy and sent back to the SMEs to prioritize i.e., assignment of a score between 0 and 100 to each criterion. The researchers calculated the arithmetic mean of each criterion’s scores provided by all SMEs and the following eight criteria, which scored the highest, were selected: (1) size of proven reserves (recoverable oil), (2) costs, (3) plateau production period, (4) distance from existing infrastructure, (5) size of reservoir (original oil in place), (6) plateau production, (7) known reservoir characteristics (historical data), and (8) need for secondary recovery.

Evaluation of alternatives and criteria

As shown in Fig. 3, the decision problem involves three levels. The top level represents the decision goal, while the second includes the criteria based on which the alternatives on the lowest level are evaluated. To employ the MCDM methods, the alternatives and criteria need to be first evaluated through pairwise comparison. Therefore, each lower level is evaluated based on its performance with respect to its immediate upper level. For example, in pairwise comparing criteria 1 and 2 on level 2, the SME would decide which criterion is more important to achieve the decision goal. The criteria pairwise comparison matrix is referred to as the criteria priorities, while the alternatives pairwise comparison matrices are referred to as the alternatives local priorities (Ishizaka and Nemery 2013).

Fig. 3

Decision hierarchy

This research paper uses Saaty’s (1980) pairwise comparison structuring approach, which is based on a 1–9 fundamental scale whereby 1: equal importance, 2: weak or slight, 3: moderate importance, 4: moderate plus, 5: strong importance, 6: strong plus, 7: very strong or demonstrated importance, 8: very, very strong, and 9: extreme importance. In cases where numerical values are not available to prioritize the elements, subjective preferences can be determined using the above scale. The SMEs were requested to fill out Questionnaire 2 through which pairwise comparison matrices we generated for the criteria and alternatives as shown in the example provided in Table 2, which illustrates pairwise comparisons of three alternatives with respect to an element of the above level—a criterion. All the comparison values are positive and those on the diagonal always have the value of 1, because the alternative is compared with itself, and the matrix is reciprocal (i.e., the upper triangle is the reverse of the lower triangle).

Table 2 Example of pairwise comparison matrix with respect to a criterion

The number of required comparisons for each matrix can be represented by the equation:

$$\# \;{\text{of}}\;{\text{comparisons}} = \frac{{n^{2} - n}}{2}$$

where n is the number of alternatives/criteria that are pairwise compared. The reciprocity rule is represented by the equation below:

$$a_{ij} = \frac{1}{{a_{ji} }}$$

where i and j are any alternatives of the matrix below:

$$A = \left[ {\begin{array}{*{20}c} {a_{11} } & {a_{21} } & {a_{13} } \\ {a_{12} } & {a_{22} } & {a_{23} } \\ {a_{13} } & {a_{i32} } & {a_{33} } \\ \end{array} } \right]$$

Phase 2: calculations and data analysis

Testing for rank reversals

When the criteria and alternatives pairwise comparison matrices are complete, a consistency test can be performed to check for possible contradictions in the entries. Assuming the decision making is normative, entry contradictions refer to the inconsistencies in the values of successive pairwise comparisons (e.g., A is better than B, B is better than C, but C is better than A). These contradictions can be attributed to a variety of reasons, including unclearly defined problems, uncertain or lack of sufficient information or concentration. Generally, the pairwise comparison matrix consistency can be checked by examining the transitivity and reciprocity rules for all values: Transitivity Rule:

$$a_{ij} = a_{ik} * a_{kj}$$

where i, j, and k are three alternatives, and a _ij is the comparison of alternative i with j. Reciprocity Rule:

$$a_{ij} = \frac{1}{{a_{ji} }}$$

Many matrix consistency testing methods were developed, such as matrix determinant (Pelaez and Lamata 2003), ratio of calculated priorities and given comparison (Grawford and Williams 1985), transitivity rule (Salo and Hamalainen 1997; Ji and Jiang 2003), and normalized column of the comparison matrix (Stein and Mizzi 2007). However, Saaty’s (1997) consistency ratio (CR), which uses the Eigenvalue prioritization method, is most commonly used in the literature.

$${\text{CI}} = \frac{{\lambda_{ \hbox{max} } - n}}{n - 1}$$

where CI is the consistency index, $\lambda_{ \hbox{max} }$ is the maximal Eigenvalue, and n is the number of alternatives/criteria.

$${\text{CR}} = \frac{\text{CI}}{\text{RI}}$$

where RI is the random index. If CR is <10%, the matrix is said to be consistent. Researchers have proposed different methods to calculate RI, which were mostly based on simulations (Alonso and Lamata 2006; Lane and Verdini 1989; Tummala and Wan 1994) or incomplete matrices (Forman, 1990). However, this research paper uses Saaty’s (1977) approach, which calculates RI based on the average CI of 500 randomly filled matrices, as shown in Table 3.

Table 3 Saaty’s random indices

The researchers calculated CR for all criteria and alternatives pairwise comparison matrices and for all SMEs as shown in Table 4. It was observed that most of the CR values exceeded 0.1, indicating a matrix inconsistency concern.² New pairwise comparison values were not obtained from the SMEs due to impracticality (i.e., 1) the 10 SMEs were remotely and disparately located, (2) the SMEs were senior experts with limited time to dedicate to the study; therefore, their contribution had to be optimized, particularly as the study involved multiple other questionnaires and Delphi rounds, (3) the number of pairwise comparison matrices were rather large-one criteria matrix and eight alternatives matrices, and (4) the pairwise comparison matrices were relatively sizeable—8 × 8 for criteria matrix and 6 × 6 for alternatives matrix.

Table 4 CR values for all matrices

Priority analysis and derivation

Priority analysis and derivation is a key step in converting pairwise comparisons into priority values. Different methods have been proposed to derive priorities from a pairwise comparison matrix (Lin and Ji 2007). Choo and Wedley (2004), discussed 18 methods for deriving preference values from pairwise comparison matrices under a common framework of effectiveness, which is distance minimization and correctness in error-free cases. This research paper uses three common methods (i.e., Eigenvalue, Approximate, and Geometric Mean) to derive the preference values from the pairwise comparison matrices (see Annex for computation details).

Aggregating priorities using three MCDM methods

Selecting a MCDM method for a specific decision problem is a challenging task as numerous methods exist, but no one method is suitable for all decision problems. This research paper uses comparisons of results to measure method performance, which dictates the use of multiple methods, hence the question of “how many and which methods to use?” This research paper encompasses three MCDM methods: AHP, PROMETHEE, and TOPSIS, because: (1) a single off-the-shelf method cannot provide an effective and realistic analysis due to limitations and inherent assumptions, and (2) three is the minimum required number of ranking sets that can be compared against each other (Ishizaka and Nemery 2013).

Analytic hierarchy process (AHP)

Developed by Thomas Saaty (1977), AHP is an MCDM method that helps decision makers to structure and analyze complex decisions and harmonize their goal and understanding of the problem. When all the local priorities are derived using the Eigenvalue, Approximate, and Geometric Mean methods, they are then synthesized to determine the global priorities (i.e., alternatives priorities with respect to goal or final rankings). The AHP method uses an additive aggregation with normalization of the sum of the local priorities, which is referred to as distributive mode and is expressed as:

$$P_{i} = \mathop \sum \limits_{i} w_{j} * p_{ij}$$

where P _i is the global priority of alternative i, p _ij is the alternative’s local priority with regard to criterion j, and w _j is the criterion j with regard to the goal. Table 5 shows the global priorities using AHP with the Eigenvalue method. Similar sets were generated for AHP using the Approximate and Geometric Mean methods.

Table 5 Analytic hierarchy process

Preference Ranking Organization METHod for Enriched Evaluation (PROMETHEE)

Developed by Jean-Pierre Brans (1982), PROMETHEE is a family of outranking methods, which was first introduced in the form of a partial ranking tool (PROMETHEE I) and extended by Brans and Vincke (1985) to a full ranking technique (PROMETHEE II), which is used by this research paper. Other modifications were introduced in subsequent versions, including PROMETHEE III, IV, V, VI, and GDSS. Compared with AHP, PROMETHEE overcomes the trade-offs between good scores on some criteria and bad scores on others where detailed and often important information can be lost. Similar to AHP, PROMETHEE is based on pairwise comparisons. ROMETHEE compares two alternatives (a _i and a _j ) at a time based on a unicriterion preference degree P _k (a _i , a _j ), which reflects how strongly alternative a _i is preferred over a _j with respect to criterion f _k . P _k (a _i , a _j ) is a function of the difference between the evaluations f _k (a _i )−f _k (a _j ): the higher the difference, the stronger the unicriterion preference degree. A preference degree is a score between 0 and 1 that states how one alternative is preferred over another with respect to a specific criterion by each SME. A preference degree value of 1 indicates a total preference, whereas 0 means no preference at all. If there is some degree of preference, however, the value is between 0 and 1. Table 6 shows the calculation of the difference between two alternatives priorities. While the preference degree can be expressed by two types of functions: linear and Gaussian, this research uses the linear function, which requires two parameters to determine each preference: an indifference threshold (q) and the preference threshold (p). The indifference threshold used for this research is 0.075 and the preference threshold is 0.15 for all criteria, as shown in Table 7, as used commonly in the literature. However, these thresholds can be different for each criterion if they were of different numerical scales.

Table 6 PROMETHEE—difference between alternatives priorities

Table 7 PROMETHEE—criteria indifference and preference thresholds

If the difference between the preference degrees of two alternatives is smaller than the indifference threshold (q), it can be concluded that no preference exists. However, if the difference is greater than preference threshold (p), it indicates a strong preference. Other values of the difference provide some degree of preference for one alternative or the other as expressed by the function below:

$$P_{ij}^{k} = \left\{ {\begin{array}{*{20}c} {0 } & {if\;f_{k} \left( {a_{i} } \right) - f_{k} \left( {a_{j} } \right) \le q} \\ {\frac{{\left[ {f_{k} \left( {a_{i} } \right) - f_{k} \left( {a_{j} } \right) - q} \right]}}{{\left[ {p - q} \right]}}} & { if\; q < f_{k} \left( {a_{i} } \right) - f_{k} \left( {a_{j} } \right) < p} \\ 1 & {if\; f_{k} \left( {a_{i} } \right) - f_{k} \left( {a_{j} } \right) \ge p} \\ \end{array} } \right.$$

where $(P_{ij}^{k} )$ is the unicriterion preference degree, which expresses how alternative (a _i ) is preferred over (a _j ) according to the decision maker. $(P_{ij}^{k} )$ and $(P_{ji}^{k} )$ are not symmetric values, but follow the condition $0 \le (P_{ij}^{k} ) + (P_{ji}^{k} ) \le 1$. Table 8 shows the unicriterion preference degrees. When all unicriterion preference degrees are calculated, the unicriterion net flows are determined, as shown in Table 9. The global preference degree (π _ij ) can then be calculated while considering the weights associated with the criteria. The global preference degree is expressed by the following equation and shown in Table 10:

$$\pi \left( {a_{i} , a_{j} } \right) = \pi_{ij} = \mathop \sum \nolimits w_{k} * P_{ij}^{k}$$

where (w _k ) is the weight associated with criterion (f _k ). The preference degree (π _ij ) represents the global preference of alternative (a _i ) over (a _j ) with respect to all criteria and lies between 0 and 1 adhering to the constraint: 0 ≤ π _ij  + π _ji  ≤ 1.

Table 8 PROMETHEE—unicriterion preference degrees

Table 9 PROMETHEE—unicriterion net flows

Table 10 PROMETHEE—global preference degrees

Technique of order preference similarity to the ideal solution (TOPSIS)

Developed by Huang and Yoon (1981) and Yoon (1987), with further development by Yoon (1987) and Hwang et al. (1993), TOPSIS is a compensatory aggregation MCDM method. The criteria are assumed to be monotonically changing and typically need to be normalized as they are often of incompatible dimensions in complex multi-criteria decision problems (Yoon and Hwang 1995; Zavadskas et al. 2006). TOPSIS evaluates the alternatives based on the geometric distance between each alternative and the ideal alternative. The best alternative has the longest geometric distance from the negative ideal solution and the shortest geometric distance from the positive ideal solution (Hwang and Yoon 1981; Lai et al. 1994; Yoon 1980). As shown in Fig. 4, alternative A is preferred over B for being closer to the ideal solution. TOPSIS permits trade-offs between the criteria, whereby a weak result of one criterion is compensated by a strong result of another. The positive ideal solution is assumed to be maximizing benefit and minimizing cost, while the negative ideal solution is maximizing cost and minimizing benefit (Wang and Chang 2007).

Fig. 4

TOPSIS method

TOPSIS involves the following five computation steps:

• 1. Gathering the alternatives’ performance values (i.e., priorities) with respect to all criteria. The performances of (n) alternatives (a) with respect to (m) criteria (i) are collected in a decision matrix X = (x _ia ), where i = 1, …, m and a = 1, …, n. The priorities are normalized as they might be of different units and scales. The distributive normalization requires that the priorities be divided by the square root of the sum of each squared element in a column as shown in the equation below and Table 11:

  $$r_{ia} = \frac{{x_{ia} }}{{\sqrt {\mathop \sum \nolimits_{a = 1}^{n} x_{ia}^{2} } }} \quad \, {\text{for}}\;a = 1,\; \ldots, n \quad {\text{and}}\quad i = 1,\; \ldots , \;m.$$

  Table 11 TOPSIS—priority normalization

• 2. The normalized scores are weighted and the distances to an ideal and anti-ideal points are computed. A weighted normalized matrix is calculated through multiplying the normalized scores (r _ai ) by their corresponding weights (w _i ) as follows:

  $$v_{ai} = w_{i} * r_{ai}$$

• 3. The weighted scores are used to compare each alternative to an ideal and anti-ideal virtual solution. This is achieved through collecting the best and worst priority with respect to each criterion of the normalized matrix. For the ideal alternative, this is represented by:

  $$A^{ + } = \left( {v_{1}^{ + } , \; \ldots \;, v_{m}^{ + } } \right)$$

  while for the anti-ideal alternative, it is represented by:

  $$A^{ - } = \left( {v_{1}^{ - } ,\; \ldots , \;v_{m}^{ - } } \right)$$

  where $v_{1}^{ + } = \max_{a} \;(v_{ai} )$ if criterion (i) is to be maximized and $v_{1}^{ - } = \min_{a} \;(v_{ai} )$ if criterion (i) is to be minimized. The weighted normalized matrix and ideal solution are shown in Table 12.

  Table 12 TOPSIS—weighted normalized matrix and ideal solution

• 4. The distance for each alternative to the ideal alternative is calculated as follows and shown in Table 13:

  $$d_{a}^{ + } = \sqrt {\mathop \sum \limits_{i} (v_{i}^{ + } - v_{ai} )^{2} } ,\quad a = 1,\; \ldots ,\; m$$

  and to the anti-ideal alternative:

  $$d_{a}^{ - } = \sqrt {\mathop \sum \limits_{i} (v_{i}^{ - } - v_{ai} )^{2} } ,\quad a = 1,\; \ldots , \;m$$

  Table 13 TOPSIS—distance to ideal and anti-ideal solution

• 5. The relative closeness coefficient of each alternative is calculated as follows:

  $$C_{a} = \frac{{d_{a}^{ - } }}{{d_{a}^{ + } + d_{a}^{ - } }}$$

The closeness coefficient is always between 0 and 1, where 1 is the preferred action. If an alternative is closer to the ideal alternative than the anti-ideal, the value of (Ca) will approach 1, but if it is closer to the anti-ideal alternative than to the ideal, (Ca) will approach 0 (Ishizaka and Nemery 2013). The relative closeness values of the alternatives to the ideal solution are shown in Table 14.

Table 14 TOPSIS—relative closeness to ideal solution

Phase 3: hypotheses testing

The combination of the above MCDM and priority analysis methods yields nine MCDM ranking sets per SME: (1) AHP-Eigenvalue, (2) AHP-Approximate, (3) AHP-Geometric, (4) PROMETHEE-Eigenvalue, (5) PROMETHEE-Approximate, (6) PROMETHEE-Geometric, (7) TOPSIS-Eigenvalue, (8) TOPSIS-Approximate, and (9) TOPSIS-Geometric). Each ranking set comprises six priority values corresponding to the six oilfields. In addition to requesting the SME to fill out the pairwise comparison matrices, Questionnaire 2 elicits a direct oilfield ranking set from each SME as well (i.e., numerical values ranging from 0 to 100 representing the SME’s subjective judgement of each oilfield’s relative performance with regard to the criteria, and referred to as the Intuitive ranking set). Therefore, each SME will produce a total of ten ranking sets (i.e., nine MCDM ranking sets + one Intuitive ranking set). As mentioned earlier, this research paper aims to determine the optimum method(s) and optimum ranking set through measuring the ranking stability or output coherence of the above methods. To achieve that, we measured the internal and external consistencies of the ten ranking sets. The internal consistency refers to the similarities between the ranking sets produced by the same method for all SMEs, while the external consistency refers to the similarities between the ranking sets produced by all methods for one SME. The internal and external consistencies are addressed by testing Hypotheses 1 and 2, while testing Hypothesis 3 validates the methodology.

Hypotheses 1 and 2

Hypothesis 1

The MCDM methods yield internally more consistent results compared with the Intuitive ranking method.

• H₀: MCDM_(i) ≤ Intuitive_(i)

• H₁: MCDM_(i) > Intuitive_(i)

where MCDM_(i) and Intuitive_(i) are the internal consistencies of the MCDM and Intuitive methods.

Hypothesis 2

The MCDM methods yield externally more consistent results compared with the Intuitive ranking method.

• H₀: MCDM_(e) ≤ Intuitive_(e)

• H₁: MCDM_(e) > Intuitive_(e)

where MCDM_(e) and Intuitive_(e) are the external consistencies of the MCDM and Intuitive methods.

Three correlation coefficient methods (Spearman’s rank correlation coefficient, Pearson’s product–moment correlation coefficient, and Kendall’s coefficient of concordance) are used to measure the internal consistencies (intra-consistencies) and external consistencies (inter-consistencies) of the MCDM methods and Intuitive approach (see Annex for computation details). However, the ranking sets and SMEs first need to be organized as shown in Table 15 and normalized (as each method produces results on different scales) as shown in Table 16.

Table 15 Organized ranking results

Table 16 Normalized ranking results

Hypothesis 1 testing

Spearman’s, Pearson’s, and Kendall’s coefficients were calculated for the ten ranking sets (i.e., nine MCDM sets and one Intuitive set). Tables 17, 18 and 19 show the final coefficients for all methods (i.e., one coefficient per method for two SMEs at a time). It was observed that the Intuitive coefficient was neither smaller nor larger than all corresponding MCDM coefficients. Therefore, testing the Null Hypothesis was accomplished by comparing the Intuitive coefficient against each MCDM coefficient across all SMEs (two SMEs at a time for 10 SMEs, hence 45 combinations for each MCDM method; and since there are 9 MCDM methods, there will be 405 comparisons of the Intuitive coefficient against the corresponding MCDM coefficients). Table 20 summarizes the results for all three correlation coefficient methods. It can be observed that the total number of times where the Null Hypothesis holds (i.e., MCDM coefficient ≤ Intuitive coefficient) = 591/1215 = 48.64%, against 624/1215 = 51.35% for the Alternative Hypothesis (i.e., MCDM coefficient > Intuitive coefficient). Therefore, although based on a small difference, the Null Hypothesis can be rejected.

Table 17 Summary Spearman’s rank correlation coefficients

Table 18 Summary Pearson’s product–moment correlation coefficients

Table 19 Summary Kendall’s coefficients of concordance

Table 20 Summary of hypothesis 1 tests

Hypothesis 2 testing

Spearman’s, Pearson’s, and Kendall’s coefficients were calculated for the Intuitive-MCDM and MCDM–MCDM combinations (i.e., one coefficient per SME for two methods at a time), as shown in Tables 21, 22 and 23, respectively. It was observed that the MCDM–MCDM coefficients were not necessarily smaller than all corresponding Intuitive-MCDM coefficients. Therefore, the Null Hypothesis was tested through comparing the Intuitive-MCDM coefficient against the corresponding MCDM–MCDM coefficient (e.g., (Intuitive-AHP Eigen) against (AHP Eigen-AHP Approximate)) leading to 72 comparisons for each SME, and since there are 10 SMEs, there will be a total of 720 comparisons.

Table 21 Summary Spearman’s rank correlation coefficients

Table 22 Summary Pearson’s product–moment correlation coefficients

Table 23 Summary Kendall’s coefficients of concordance

Table 24 summarizes the results for all three correlation coefficient methods. It can be observed that the total number of times where the MCDM–MCDM coefficient > Intuitive-MCDM coefficient) = 1749/2160 = 80.97%, against 411/2160 = 19.02% for the number of times where the MCDM–MCDM coefficient ≤ mean Intuitive coefficient. Therefore, the Null Hypothesis can be rejected.

Table 24 Summary of hypothesis 2 tests

Hypothesis 3

The SMEs find the MCDM methods to be useful, practical and easy to use.

• H₀: the SMEs find the MCDM methods useless, impractical, and uneasy to use.

• H₁: the SMEs find the MCDM methods useful, practical, and easy to use.

This research paper used two methods to validate the decision-making approach. One was through applying the consistency test, which helps verify the reliability of the method(s), and the other was through testing Hypothesis 3, which is based on the SMEs’ evaluations of the final results and MCDM methods’ practicality and usefulness for portfolio oilfield selection and prioritization purposes, elicited through Questionnaire 3’s Likert scale assessments, as shown in Table 25. Testing the above Null and Alternative Hypotheses depends on the overall assessment score given by the SMEs. The maximum score per SME = 6 arguments * 5 points = 30 points, and the minimum score per SME = 6 arguments * 1 point = 6, hence the average score per SME = (30 + 6)/2 = 18. Therefore, if the final score per SME for all SMEs was greater than 18, it would be concluded that the Null Hypothesis could be rejected and that the SMEs find the MCDM methods useful, practical and easy to use. 10 SMEs (out of 10) filled out Questionnaire 3, and their scores are summarized by Table 26. It can be observed that the total scores assigned by all SMEs are greater than 18 (minimum is 21 by SME 10 and maximum is 30 by SME 8). Thus, the Null Hypothesis could be rejected. Moreover, it could also be seen that the SMEs were in almost full agreement that structuring the decision process yields better results as argument 5 scored the highest among the SMEs (i.e., total score = 48/50 = 96%), while they were in less agreement over the MCDM methods results being more realistic than the Intuitive Approach results, although still at a high rate of agreement (i.e., total score = 39/50 = 78%).

Table 25 Questionnaire 3

Table 26 Summary of Questionnaire 3 scores

Deriving the final ranking sets

As the SMEs were not collocated, it was difficult to reach consensus on the criteria and alternatives priorities. The literature suggests using a weighted arithmetic mean for determining the final ranking sets. The weights that are assigned to the arithmetic mean represent the relative significance of the SMEs and/or methods. The SMEs could either assign a weight to each other (Ramanathan and Ganesh 1994), or SMEs with lower matrix inconsistency ratios could be assigned higher weights (Cho and Cho 2008), or one SME could act as a supra decision maker and assign weights to others based on their merits. However, as these conditions were not applicable for this research, equal weights were assigned to all SMEs and methods. Therefore, the final ranking set was determined by the arithmetic means of the nine MCDM ranking sets generated for each alternative. As 10 SMEs returned Questionnaire 2, the final priority value of each oilfield is the summation of all of its priorities (generated by all methods and SMEs) divided by 90 (i.e., 10 SMEs * 3 MCDM methods * 3 priority analysis methods). The final priorities are shown by Table 27 and Fig. 5, which represent the percentage shares of each oilfield of the investment resources, as perceived by the SMEs.

Table 27 Final priorities

Fig. 5

Final priorities

Sensitivity analysis

To carry out a sensitivity analysis, certain input data were altered and the effect on the final results was observed, which allowed for different scenarios to be generated and may consequently call for further discussion. If the results remain unchanged, they are said to be robust, otherwise they are sensitive (Ishizaka and Nemery 2013). This research used a one-way sensitivity analysis through which the value of each criterion is modified by 10 and −10% (while keeping other criteria constant) and the new oilfield priorities recorded. The original criteria priorities are considered the baseline scenario, as shown in Table 28. This was followed by creating variation scenarios that reflect the percentage modification to the criteria, one at a time. Tables 29 and 30 show the +10% modification to the value of criterion recoverable oil and the effect it has on the ranking sets, respectively. All criteria are modified twice (i.e., +10 and −10%) and the effects are reflected on the ranking sets. However, it was observed that these criteria modifications did not cause any significant changes in the alternatives ranking sets.

Table 28 Baseline scenario

Table 29 Variations scenario

Table 30 Criteria percentage modifications

Discussion of results and conclusion

If both Hypotheses 1 and 2 tests were to show more internal and external consistencies (i.e., higher coefficients) for MCDM methods over the Intuitive approach, it may be safe to argue that structuring decision problems using MCDM methods leads to more consistent results, and perhaps also more reliable and informed decisions. However, if either the MCDM internal or external coefficients are lower or equal than those of the Intuitive approach, the conclusion would be different, as shown in Table 31 and explained below:

Table 31 Possible consistency scenarios

• Scenario 1: using multiple MCDM methods is optimal as their consistency coefficients are higher in both tests;

• Scenario 2: using multiple MCDM methods is preferred over the Intuitive approach given that the internal consistency test yielded higher coefficients for the MCDM methods versus equal coefficients for the external consistency test;

• Scenario 3: using multiple MCDM methods is preferred, because: (1) the internal consistency test yielded higher coefficients for the MCDM methods while the external consistency test yielded lower coefficients, (2) However, Hypothesis 2 results (i.e., all three correlation coefficient methods results) showed a high degree of consistency between the MCDM methods when the same SME set of priorities are analyzed, which could lead to arguing that the internal consistency test (i.e., testing the results of the same MCDM method across all SMEs) may carry more weight than the internal consistency test; hence the Scenario 3 recommendation. Nevertheless, this argument calls for further research;

• Scenario 4: using multiple MCDM methods is preferred given that one test favors the use of multiple MCDM methods while the other is neutral;

• Scenario 5: using multiple MCDM methods is not recommended as both tests yielded equal coefficients for both MCDM methods and Intuitive approach;

• Scenario 6: using multiple MCDM methods is not recommended as the internal consistency test yielded equal coefficients for both MCDM methods and Intuitive approach while the external consistency test yielded lower coefficients for the MCDM methods;

• Scenario 7: using multiple MCDM methods is not recommended because while the external consistency test yielded higher coefficients for MCDM methods than the Intuitive approach, the internal consistency test yielded lower coefficients for the MCDM methods, and if the argument made under Scenario 3 is verified (i.e., internal consistency test carries more weight than external consistency test), this conclusion holds. However, this is also subject to further research, which may also consider how higher or lower the coefficients are;

• Scenario 8: using multiple MCDM methods is not recommended as the internal consistency test yielded lower coefficients for the MCDM methods while the external consistency test yielded equal coefficients; and

• Scenario 9: using multiple MCDM methods is not recommended as both the internal and external consistency tests yielded lower coefficients for the MCDM methods.

Testing both Hypotheses 1 and 2 resulted in rejecting the Null Hypothesis as the MCDM coefficients were greater than the Intuitive coefficients. This situation fits into the definition of Scenario 1 above. However, as the difference between the number of times the Null Hypothesis could not be rejected and the number of times it could be rejected was marginal (i.e., 33/1215 = 2.71%), it could be argued that the MCDM coefficients may be considered equal to those of the Intuitive approach. In this case, the combination of Hypotheses 1 and 2 testing results (i.e., MCDM coefficients = Intuitive coefficients for internal consistency test and MCDM coefficients > Intuitive coefficients for external consistency test) would fit into the definition of Scenario 4 where the use of multiple MCDM methods is also recommended; hence, the final oilfield ranking sets discussed under “Hypothesis 1 Testing” section could be adopted. This may also be supported by the result of testing Hypothesis 3, where the total score achieved by all SMEs for all arguments was 224/270 = 83%. Therefore, as both methods used by this research to validate the decision-making approach yielded positive results, it could be concluded that using MCDM methods would help address portfolio oilfield selection and prioritization decision problems.

In conclusion, as depicted by Table 26 and Fig. 5, the final oilfields priorities are as follows: (Rumaila: 55%, West Qurna 1: 12.6%, West Qurna 2: 11.3%, Majnoon: 11.4%, Zubair: 0.7%, and Halfaya: 0.2%). These priorities represent the percentage shares of each oilfield of the investment resources and may be helpful in informing policy decisions with respect to future investment plans. Moreover, the triplex approach and consistency test could also contribute to the research literature on MCDM method selection and validation, with an immediate application in E&P investment portfolio prioritization.