An examination of performance relations among selected consistency measures for simulated pairwise judgments

Abstract

A review of contemporary literature devoted to decision making support systems draws attention to the Analytic Hierarchy Process (AHP). At the core of the AHP are various prioritization procedures which elicit priorities for alternative solutions for complex decisional problems. Certainly, the procedures coincide when decision makers’ preferences of alternative solutions are cardinally transitive, otherwise the results differ. This is why consistency measurement of human judgments is so important. It has been scientifically proven that a high inconsistency of decision makers’ preferences concerning alternative solutions of decisional problems may lead to fallacious choices. Research verifies the thesis that consistency measures derived from different prioritization procedures are interrelated. It turns out that one of the independent consistency measures is extremely closely related to the consistency index embedded in original approach of the AHP. The main objective of this study is realized through the novel and sophisticated simulation algorithm designed for the AHP and executed within its exemplary decisional framework for three levels. The outcome of the research proves that consistency can be measured in various ways, but recently devised concepts can indicate better solutions as a result of significantly improved methodology.

1 Introduction

Very recent developments in the field of ‘decision making support systems’ devoted to the ranking system commonly known as the Analytic Hierarchy Process (AHP) (Chen et al. 2015; Pereira and Costa 2015; Moreno-Jiménez et al. 2014; Aguarón et al. 2014; Lin et al. 2013; Bozóki et al. 2013) exposed the necessity for further analysis in the area. The ranking system was devised by Saaty (1980) in order to support decision makers in a process of systematization and prioritization of their preferences regarding various alternative solutions embedded in different decisional models (multiple criteria and/or scenarios). The primary methodology of AHP applies the right eigenvector approach (REA), used for ranking alternatives; and strictly associated with the approach, the consistency index $(\hbox {CI}_{\mathrm{REA}})$, used for consistency measurement of decision makers’ preferences concerning alternative solutions for a problem. However, the well-defined Saaty (1990) mathematical theory of pairwise comparison matrices (PCMs), and their associated principal right eigenvector’s ability to generate true ranking (when PCMs are consistent), or approximate weights together with a consistency measure (when PCMs are inconsistent), currently constitutes the primary source of its criticism (see e.g. Kazibudzki and Grzybowski 2013; Grzybowski 2012; Kazibudzki 2012 and references therein). It should be noted that the AHP—as a decision making support system—has also received severe criticism with regard to the background of more fundamental issues (Bana e Costa and Vansnick 2008; Farkas 2007; Barzilai 2005). However, facing sustainable growth of the AHP applications number (see e.g. Grzybowski 2012 and references therein), as well as its stable position among other theories of choice, that discussion remains beyond the scope of this examination. Instead, the identified research gap is the essence of this examination i.e. performance relations analysis of selected consistency measures for various prioritization procedures available within AHP methodology (Brunelli et al. 2013).

It seems that a significant number of research papers devoted to various issues associated with prioritization procedures within AHP and thereby derived consistency measures, focus on determining—roughly speaking—how efficacious those measures are, and/or the degree of relation to the original approach i.e. the REA and $\hbox {CI}_{\mathrm{REA}}$ (see for instance Grzybowski 2012 or  Aguaron and Moreno-Jimenez 2003). This method of analysis attracts attention, especially in the face of contemporary research outcomes (Kazibudzki and Grzybowski 2013; Kazibudzki 2012).

The following sections of this examination will scrutinize a number of fundamental relations between already established prioritization procedures from the perspective of their original consistency indices.

2 Methodology

2.1 Scope of research

This research adopts the simulation scenario (in detail exemplified in the Sect. 2.2) that had been already published (Kazibudzki and Grzybowski 2013) with minor modifications. Thus, the approach to the main objective of the paper is from the perspective of a real-life decision problem i.e. the simulations assume that a problem can be structured as a hierarchy with three levels: goals, criteria and alternatives. Within the framework, a number of distinct scenarios were generated and examined related to possible outcomes of hypothetic problem analysis i.e. relations between consistency measures of simulated human judgments regarding alternative solutions of hypothetic problems were observed within a decisional hierarchy.

The scenario took into account four prioritization techniques and their related consistency indices rated as the best within AHP methodology (Kazibudzki and Grzybowski 2013; Lin 2007; Choo and Wedley 2004) i.e. the logarithmic utility approach (LUA) (Kazibudzki 2012); the Sum of squared relative differences (SRDM) (Grzybowski 2012); the REA (Saaty 1980); and the Logarithmic Least Squares Method (LLSM) (Crawford and Williams 1985). The general formulae of the considered prioritization procedures, together with their respective consistency indices, are presented in Table 1.

All of the procedures presented in Table 1 can operate under Reciprocal PCMs (RPCMs) and Nonreciprocal PCMs (NRPCMs), but only three consistency indices originated from them have application in nonreciprocal cases. The consistency index originating with the REA for NRPCMs is meaningless. Therefore, in order to compare volatility and relations among all consistency measures devised for selected prioritization procedures, it was decided to limit the scope of research and restrict the simulation scenarios to cases exclusively based on RPCMs. It should be noted that the traditional AHP methodology excludes the application of nonreciprocal PCMs. Thus, in the original AHP prioritization procedure, only the elements above the PCM diagonal are taken into consideration. The remaining are ignored and replaced by the inverse of their corresponding symmetric elements above the PCM diagonal (Kazibudzki 2012). Certainly the algorithm itself is the reason of prospective estimation distortions (Linares et al. 2014) leading directly to significant errors labeled as technical. Scale errors are the other facet causing distortions. However, in opposition to the former, their presence is inevitable, and guaranteed by the existence of various preference scales that are used by decision makers to express in some way the intensity of their judgments.

Table 1 Formulae for the prioritization procedures and their related consistency indices

2.2 Exemplary simulation framework

The best way to introduce the idea of a simulation framework that was implemented in the research scenario is to exemplify; thus, minor adaptations were purposely introduced to the model already described in Kazibudzki and Grzybowski (2013). Furthermore, the exemplary research scenario was restricted to Saaty’s preferences scale and his methodically genuine prioritization procedure (REA) that entails the implementation of RPCMs. Thus, the following ‘errors free’ model of the AHP framework with three levels (four criteria and four alternatives):

where $\mathrm{PV}_{\Xi }^{c}$ and $\mathrm{PV}_{\Xi }^{a}$ denote proposed known, normalized priority vectors of criteria and alternatives, respectively.

Further, for this model Saaty’s consistency index $(\hbox {CI}_{\mathrm{REA}})$ is calculated for the entire hierarchy scenario i.e. the $\hbox {CI}_{\mathrm{REA}}$ is determined for a given set of alternatives under every criterion (one by one) and then, their values known under all criteria, the weighted mean is calculated using criteria ranks as the weights. In this particular example, $\hbox {CI}_{\mathrm{REA}}=0$ for all sets of alternatives under every criterion (‘errors free’ case). In this situation, the total $\hbox {CI}_{\mathrm{REA}}$ $(\hbox {TCI}_{\mathrm{REA}})$ value for the hierarchy equals zero. Obviously, the model can be transformed in a way to reflect a real-life situation by the introduction of a number of distortions. Only two kinds of distortion (perturbation errors) are applied for exemplification purposes. Each entry of a PCM is rounded to Saaty’s numerical preference scale, and next the particular PCM is made reciprocal. Then, the REA is applied for the criteria’s priority vectors calculation $(\hbox {PV}^{\mathrm{C}}_{\mathrm{REA}})$ and the $\hbox {CI}_{\mathrm{REA}}$ for alternatives under respective criteria. Finally, the $\hbox {TCI}_{\mathrm{REA}}$ is determined for the illustrative model of the hierarchy. Thus, after all transformations, the model can be presented as below:

$\hbox {TCI}_{\mathrm{REA}}$ for the particular hierarchy can be calculated as follows:

$$\begin{aligned} \mathrm{TCI}_\mathrm{REA}= & {} w_{c1(\mathrm REA)} \times \mathrm{CI}_\mathrm{REA}^{a(c1)} +w_{c2(\mathrm{REA})} \times \mathrm{CI}_\mathrm{REA}^{a(c2)} +w_{c3(\mathrm{REA})}\\&\times \mathrm{CI}_\mathrm{REA}^{a(c3)} +w_{c4(\mathrm{REA})} \times \mathrm{CI}_\mathrm{REA}^{a(c4)} \\= & {} 0.304999\times 0.0113226+0.276859\times 0.0113226+0.113143\times 0.0068734 \\&+\,0.304999\times 0.0068734=0.00946220261 \end{aligned}$$

where $w_\mathrm{ci(REA)}$, $\mathrm{CI_{REA}}^{a(ci)}$ respectively denote—computed in accordance with the REA—the weight of $i\mathrm{th}$ criterion, and the consistency index of the decision makers’ judgments concerning the alternatives under $i\mathrm{th}$ criterion.

2.3 Preliminary simulation outcomes

Upon reflection, the following question seems reasonable: Are different consistency indices, having their genesis in different prioritization procedures, somehow related to each other from the perspective of decisions based upon the entire decisional hierarchy?

To answer the above question, various situations were simulated related to preselected sources of PCMs inconsistency (see e.g. Grzybowski 2012). Recapitulating, the inconsistency is usually a result of distortions caused by the nature of human judgments and errors being the consequences of the technical realization of comparison procedures i.e. errors resulting from the forced reciprocity requirement and those concerned with the application of the chosen scale (rounding-off). Obviously, all of them may be imbedded in a simulation scenario. Therefore, this scenario encompassed three preferences scales: Saaty’s, geometric and numerical, as well four probability distributions for a perturbation factor $(e_{ij})$: uniform, gamma, truncated normal and log-normal (Kazibudzki 2012).

Additionally, every scenario assumes that criteria priorities for a given hierarchy model are calculated by that ranking procedure which designate its examined consistency index. Therefore, if the $\hbox {TCI}_{\mathrm{LLSM}}$ is calculated, criteria priorities are calculated according to the LLSM procedure. When computing the $\hbox {TCI}_{\mathrm{LUA}}$, criteria priorities are determined applying the LUA procedure.

Scenario 1

The uniform probability distribution for a perturbation factor, Saaty’s scale for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.5, 1.5]$, and the singular hierarchy model with 12 criteria and 12 alternatives, set randomly (uniform distribution) just once and perturbed 500 times. Figure 1a–f present outcomes for all distorted cases within one hierarchy model i.e. 500 distinct snapshots taken altogether.

Fig. 1

Fluctuation of a $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, b $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}$, c $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, d $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, e $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, f $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$ within the first simulation scenario. Pearson’s correlation coefficient between indices: $r_{1/1}=0.99228$ (a), $r_{1/2}=0.999593$ (b), $r_{1/3}=0.999908$ (c), $r_{1/4}=0.999841$ (d), $r_{1/5}=0.99468$ (e), $r_{1/6}=0.993756$ (f)

Scenario 1a

The gamma probability distribution for a perturbation factor, Saaty’s scale for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.5, 1.5]$, and the singular hierarchy model with 12 criteria and 12 alternatives, set randomly (uniform distribution) just once and perturbed 500 times—in order to save space, since the particular scenario is a variation of the former, only Pearson’s correlation coefficients are given for the respective indices.

• Pearson’s correlation coefficient between indices:

  • $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}: r_{1a|1}= 0.989069$;

  • $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}: r_{1a|2}= 0.999246$;

  • $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}: r_{1a|3}= 0.999783$;

  • $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}: r_{1a|4}= 0.999748$;

  • $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}: r_{1a|5}= 0.993037$;

  • $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}: r_{1a|6}= 0.991681$.

Scenario 1b

The log-normal probability distribution for a perturbation factor, the geometric scale for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.01, 1.99]$,and the singular hierarchy model with 12 criteria and 12 alternatives, set randomly (uniform distribution) just once and perturbed 500 times—in order to save space, since the particular scenario is also just a variation of the first, only Pearson’s correlation coefficients are given for the respective indices.

• Pearson’s correlation coefficient between indices:

  • $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}: r_{1b|1}= 0.976036$;

  • $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}: r_{1b|2}= 0.996563$;

  • $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}: r_{1b|3}= 0.998829$;

  • $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}: r_{1b|4}= 0.999006$;

  • $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}: r_{1b|5}= 0.987822$;

  • $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}: r_{1b|6}= 0.984522$.

Recapitulating this section of the research, it may be concluded that from the perspective of consistency measures originating with prioritization procedures selected for examination purposes, all consistency indices are rather closely related to one another.

3 Primary simulation results

One can appreciate that the best results in genuine priority vectors estimation for variously perturbed PCMs are obtained with the application of the geometric scale. It seems the above conclusion is valid for most appreciated prioritization procedures (Kazibudzki and Grzybowski 2013). Therefore, the geometric scale is utilized for most of the further research analysis.

Scenario 2

The uniform probability distribution for a perturbation factor, the geometric scale for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.5, 1.5]$, hierarchy models drawn randomly (the uniform distribution) for the prescribed number of alternatives $n_{a}=4$ and the randomly chosen number of criteria from the set $\{3, 4, 5, \ldots , 15\}$. The simulation scenario encompasses 500 various hierarchy models, each perturbed just once. Figure 2a–f present scores for all cases i.e. 500 distinct situations depicted one by one.

Fig. 2

Fluctuation of a $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, b $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}$, c $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, d $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, e $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, f $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$ within the second simulation scenario. Pearson’s correlation coefficient between indices: $r_{2/1}=0.999531$ (a), $r_{2/2}=0.999386$ (b), $r_{2/3}=0.999993$ (c), $r_{2/4}=0.999419$ (d), $r_{2/5}=0.999749$ (e), $r_{2/6}=0.999608$ (f)

Scenario 2a

The uniform probability distribution for a perturbation factor, the geometric scale for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.5, 1.5]$, hierarchy models drawn randomly (the uniform distribution) for the prescribed number of alternatives $n_{a}=15$ and the randomly chosen number of criteria from the set $\{3, 4, 5,\ldots , 15\}$. The simulation scenario encompasses 500 various hierarchy models, each perturbed just once. Figure 3a–f present scores for all cases i.e. 500 distinct situations depicted one by one.

Fig. 3

Fluctuation of a $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, b $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}$, c $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, d $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, e $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, f $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}${Scenario 2a}. Pearson’s correlation coefficient between indices: $r_{2a/1}=0.992902$ (a), $r_{2a/2}=0.999486$ (b), $r_{2a/3}=0.999917$ (c), $r_{2a/4}=0.9998$ (d), $r_{2a/5}=0.99547$ (e), $r_{2a/6}=0.994216$ (f)

Scenario 2b

The gamma probability distribution for a perturbation factor, the geometric scale for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.5, 1.5]$, hierarchy models drawn randomly (uniform distribution) for the prescribed number of alternatives $n_{a}=15$ and the randomly chosen number of criteria from the set $\{3, 4, 5,\ldots ,15\}$. The simulation scenario encompasses 500 various hierarchy models, each perturbed just once. Figure 4a–f present scores for all cases i.e. 500 distinct situations depicted one by one.

Fig. 4

Fluctuation of a $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, b $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}$, c $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, d $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, e $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, f $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$ {Scenario 2b}. Pearson’s correlation coefficient between indices: $r_{2b/1}=0.989469$ (a), $r_{2b/2}=0.999568$ (b), $r_{2b/3}=0.999874$ (c), $r_{2b/4}=0.999844$ (d), $r_{2b/5}=0.99241$ (e), $r_{2b/6}=0.991455$ (f)

Because of outcome similarities for the remaining research studies prescribed in the second principal simulation scenario, it was decided to withhold their presentation.

Scenario 3

The uniform probability distribution for a perturbation factor, Saaty’s scale for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.75, 1.25]$, hierarchy models drawn randomly (uniform distribution) for the randomly chosen number of criteria and alternatives from the set $\{8,9,\ldots ,12\}$. The simulation scenario encompasses 1000 various hierarchy models, each perturbed just once. Figure 5a–f present scores for all cases i.e. 1000 distinct situations depicted one by one.

Fig. 5

Fluctuation of a $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, b $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}$, c $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, d $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, e $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, f $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$ within the third simulation scenario. Pearson’s correlation coefficient between indices: $r_{3/1}=0.986178$ (a), $r_{3/2}=0.978378$ (b), $r_{3/3}=0.998231$ (c), $r_{3/4}=0.964632$ (d), $r_{3/5}=0.997227$ (e), $r_{3/6}=0.975627$ (f)

Scenario 4

The uniform probability distribution for a perturbation factor, the geometric ratios for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.01, 1.99]$, hierarchy models drawn randomly (uniform distribution) for the randomly chosen number of criteria and alternatives from the set $\{8, 9,\ldots ,12\}$. The simulation scenario encompasses 1000 various hierarchy models, each perturbed just once. Figure 6a–f present scores for all cases i.e. 1000 distinct situations depicted one by one.

Fig. 6

Fluctuation of a $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, b $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}$, c $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, d $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, e $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, f $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$ within the fourth simulation scenario. Pearson’s correlation coefficient between indices: $r_{4/1}=0.915488$ (a), $r_{4/2}=0.925365$ (b), $r_{4/3}=0.991935$ (c), $r_{4/4}=0.89004$ (d), $r_{4/5}=0.976455$ (e), $r_{4/6}=0.898704$ (f)

Scenario 5

The uniform probability distribution for a perturbation factor, the geometric scale for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.5, 1.5]$, hierarchy models drawn randomly (uniform distribution) for the prescribed number of criteria $n_{c}=2$ and the randomly chosen number of alternatives from the set $\{3, 4, 5,\ldots , 15\}$. The simulation scenario encompasses 500 various hierarchy models, each perturbed 10 times. Figure 7a–f present scores for all cases i.e. 5000 distinct situations depicted one by one.

Fig. 7

Fluctuation of a $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, b $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}$, c $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, d $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, e $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, f $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$ within the fifth simulation scenario. Pearson’s correlation coefficient between indices: $r_{5/1}=0.788192$ (a), $r_{5/2}=0.939231$ (b), $r_{5/3}=0.98646$ (c), $r_{5/4}=0.873884$ (d), $r_{5/5}=0.908351$ (e), $r_{5/6}=0.682681$ (f)

Fig. 8

Fluctuation of a $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, b $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}^{0}$, c $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, d $\hbox {TCI}_{\mathrm{LUA}}^{0}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, e $\hbox {TCI}_{\mathrm{LUA}}^{0}$ and $\hbox {TCI}_{\mathrm{LLSM}}$, f $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LLSM}}$ within the sixth simulation scenario. Pearson’s correlation coefficient between indices: $r_{6/1}=0.810917$ (a), $r_{5/6}=0.998936$ (b), $r_{6/1}=0.987145$ (c), $r_{5/6}=0.985027$ (d), $r_{6/1}=0.814528$ (e), $r_{5/6}=0.713087$ (f)

4 Discussion

The bibliography contains numerous prioritization procedures devoted to the AHP e.g. 18 ranking methods were studied by Choo and Wedley (2004). There are procedures based on constrained approximation models (e.g. Grzybowski 2012; Hovanov et al. 2008), grounded on certain statistical methodologies (e.g. Basak 1998; Dong et al. 2008; Lipovetsky and Tishler 1997), and others, based on various fuzzy preference description algorithms (e.g. Xu 2004). Certainly, the original approach continues to function with its own methodology (Saaty 1980). Nevertheless, few of the prioritization procedures deal with the problem of consistency control. Apart from the original consistency measure devised by Saaty (and those presented earlier in this paper), there have been a number of other propositions regarding consistency e.g. Koczkodaj’s consistency index (Koczkodaj 1993; Bozóki and Rapcsák 2008).

For this examination, four consistency indices were selected, designated for prioritization procedures that were recently ranked as the best (Kazibudzki and Grzybowski 2013; Lin 2007; Choo and Wedley 2004). Effort was then made to establish some fundamental relationships between the indices from the perspective of a complex decisional problem. The problem was structured as an AHP three level hierarchy model. Applying sophisticated computational software and advanced mathematical algorithms introduced into simulation scenarios, the following was determined.

On the bases of simulations executed within the first primary scenario i.e. in Scenario 1, 1a, and 1b, it is noticeable that when dealing with hierarchies variously perturbed but singular (PCM for criteria and PCMs for alternatives are derived from their respective particular priority vectors which are drawn only once), all of the chosen consistency indices are highly correlated. Especially noticeable are the linear relations between $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{REA}}$, or $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LUA}}$.

Very similar relationships between the above consistency measures are also observed within the second primary simulations scenario i.e. Scenario 2, 2a, and 2b. It can be stated that regardless of the probability distribution applied for a perturbation factor, or how a particular hierarchy is structured (many criteria and few alternatives, or few criteria and many alternatives, or many criteria and many alternatives), or what kind of preference scale is applied, the relations between consistency indices are very apparent (especially noticeable are very high and positive correlations between $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{SRDM}}$, $\hbox {TCI}_{\mathrm{LUA}}$ and $\hbox {TCI}_{\mathrm{REA}}$, or $\hbox {TCI}_{\mathrm{SRDM}}$ and $\hbox {TCI}_{\mathrm{LUA}}$). The situation changes when we introduce to the simulation scenario the possibility of fluctuations for alternative numbers within the hierarchy model.

Results of the third simulation scenario demonstrate that in situations when perturbation errors are relatively small, and the number of alternatives and criteria in the model relatively similar, minor dispersions among indices performance exist. Still, strong and positive correlations are seen between all consistency measures, especially visible when perturbation errors are relatively small.

When perturbation errors become stronger and the number of alternatives and criteria in the model remain unchanged (the fourth simulation scenario), similar relations between consistency indices occur; however, the dispersion among them becomes more visible. Nevertheless, the relations are highly positive and statistically significant on the significance level equal to 0.01.

Finally, in the fifth simulation scenario, the set for the number of alternatives is widened (it ranges now from 3 to 15), the average interval is set for perturbation errors and only two criteria are imposed within the model. Consequently, we observe noticeable dispersions among consistency measures; nevertheless correlation coefficients are still positive and statistically significant. This latter scenario, allows answering the fundamental question stated at the beginning of this research—that selected consistency measures designed for the AHP applications are related. It seems the only difference among them is their different sensitivity to the PCM distortions because of perturbation errors in relation to the PCM size (the number of alternatives). In other words, different consistency ratios display more or less vulnerability on a distortion’s strength dependent on the number of alternatives in a particular decisional problem. It confirms the supposition that the consistency of decision makers’ judgments concerning alternative solutions for a problem can be measured in various ways, but recently devised concepts (listed in the references) are more reasonable and improve the quality of the consistency control.

Verifying this notion can be done in the following way; by transforming the original $\hbox {CI}_{\mathrm{LUA}}$ to a new formula:

$$\begin{aligned} \mathrm{CI}_{LUA}^{O} ={\mathrm{CI_{LUA}} \ln (n)\ln (n+0.75745)}/{\ln (n-0.75745)} \end{aligned}$$

(1)

and present the relations between the transformed consistency index with the remaining consistency measures in simulation Scenario 6.

Scenario 6

The uniform probability distribution for a perturbation factor, the geometric scale for the preferences expression (rounding-off), the perturbation factor ($e_{ij}$) drawn from the interval $e_{ij}\in [0.5, 1.5]$, hierarchy models drawn randomly (uniform distribution) for the prescribed number of criteria $n_{c}=2$ and the randomly chosen number of alternatives from the set $\{3,4,5,\ldots ,15\}$. The simulation scenario encompasses 500 various hierarchy models, each distorted 10 times. Figure 8a–f present scores for all cases i.e. 5000 distinct situations depicted one by one.

It is apparent that the correlation coefficients between consistency indices during the last simulation scenario are different than those obtained in Scenario 5. Very high correlations now appear between $\hbox {TCI}_{\mathrm{LUA}}^{0}$, $\hbox {TCI}_{\mathrm{SRDM}}$, and $\hbox {TCI}_{\mathrm{REA}}$. Current literature devoted to AHP applications describes functional relations between, for example, $\hbox {CI}_{\mathrm{LLSM}}$ and $\hbox {CI}_{\mathrm{REA}}$ (Aguaron and Moreno-Jimenez 2003), or $\hbox {CI}_{\mathrm{SRDM}}$ and $\hbox {CI}_{\mathrm{REA}}$ (Grzybowski 2012). Therefore, a similar relation was sought between $\hbox {TCI}_{\mathrm{LUA}}^{0}$ and $\hbox {TCI}_{\mathrm{REA}}$. Then, 2500 different hierarchy models were simulated, each perturbed 5 times which gives 12,500 cases, encompassing the randomly chosen number of criteria from the set $\{2,3,4,\ldots ,10\}$ and the randomly selected number of alternatives from the set $\{3,4,5,\ldots ,15\}$. Applied were the uniform probability distribution for the perturbation factor, the geometric scale for the preferences expression (rounding-off), and the perturbation factor interval $e_{ij}\in$ [0.5, 1.5].

Applying the standard procedure of regression analysis, the relation between $\hbox {TCI}_{\mathrm{REA}}$ and $\hbox {TCI}_{\mathrm{LUA}}^{0}$ is expressed. Because $\hbox {TCI}_{\mathrm{REA}}$ = 0, if and only if $\hbox {TCI}_{\mathrm{LUA}}^{0}=0$, exclusively the models with an intercept equal to 0 are considered. Thus the established relation may be presented in the following form:

$$\begin{aligned} \hbox {TCI}_{\mathrm{REA}} (\hbox {TCI}_{\mathrm{LUA}}^{0}) = 1.4884 \cdot \hbox {TCI}_{\mathrm{LUA}}^{0} \end{aligned}$$

(2)

Table 2 and Fig. 9 present some basic selected estimation characteristics of the relation.

Table 2 The regression statistics

Fig. 9

The distribution of the approximation residuals

It is apparent that the linear function presented above explains more than 99.5 % of the $\hbox {TCI}_{\mathrm{REA}}$ volatility and approximates the true value of the $\hbox {TCI}_{\mathrm{REA}}$ with a standard regression error equal to 0.00128, which means that a 5 % level of significance can be expected in designating the true value of the $\hbox {TCI}_{\mathrm{REA}}$ with an accuracy of plus/minus 0.00256. Additionally, the mean relative prediction error for the model does not exceed 1.5 % of the true value of the $\hbox {TCI}_{\mathrm{REA}}$.

In order to introduce a point of reference for a consistency measurement of judgments made by decision makers during a real-life problem analysis, a random simulation was performed (uniform distribution) of 10,000 reciprocal transitive PCMs (RTPCMs) and transitive PCMs (TPCMs) of different values from $n=3$ to $n=10$. For a given size of the PCMs and their respective type (TPCMs or RTPCMs), the random $\hbox {CI}_{\mathrm{LUA}}^{0}$ was computed [denoted as $\hbox {RCI}_{\mathrm{LUA}}^{0}$(n)]. Then, the mean, maximum and minimum levels were computed, along with a few basic quantiles of order p. The outcome is presented in Tables 3 and 4.

Table 3 Statistical characteristics of $\hbox {RCI}_{\mathrm{LUA}}^{0}$(n) for uniform empirical distribution of 10,000 random RTPCMs

It bears attention that the outcomes are similar when taking into account the relation (2) and the compared mean values of $\hbox {RCI}_{\mathrm{LUA}}^{0}$(n) for random RTPCMs (Table 3) with the average levels of Saaty’s consistency index (generated for the same kind of random PCMs (RTPCMs) as presented in Table 11 printed in Grzybowski (2012).

Table 4 Statistical characteristics of $\hbox {RCI}_{\mathrm{LUA}}^{0}(n)$ for uniform empirical distribution of 10,000 random TPCMs

Concluding, it can be asserted that the consistency of decision makers’ preferences regarding alternative solutions for decisional problems can be measured in various ways. Nevertheless, there are procedures that assure high quality consistency control in situations where Saaty’s procedure fails (nonreciprocal PCMs). This research indicates the $\hbox {TCI}_{\mathrm{LUA}}^{0}$ is highly correlated with the $\hbox {TCI}_{\mathrm{REA}}$ in situations where reciprocal PCMs are considered. It gives the $\hbox {TCI}_{\mathrm{LUA}}^{0}$ the advantage over the $\hbox {TCI}_{\mathrm{REA}}$ because the $\hbox {TCI}_{\mathrm{LUA}}^{0}$ can also operate when nonreciprocal PCMs are considered. As is known, reciprocal PCMs significantly increase estimation errors of decision makers’ true priorities concerning alternative solutions for a problem (Linares et al. 2014; Grzybowski 2012).

Thus, the consistency index derived from the LUA may not only replace the consistency index conceived with the REA (when reciprocity is forced) but it can also function when the latter is meaningless i.e. under nonreciprocal conditions. In consequence, the possibility exists to significantly improve the quality of consistency control and ranking accuracy in the AHP.

Furthermore, the $\hbox {TCI}_{\mathrm{LUA}}^{0}$ prevails the $\hbox {TCI}_{\mathrm{REA}}$ not only because it allows better consistency control from the perspective of a decisional problem (it complies with statistics guidelines (Brunelli et al. 2013; Grzybowski 2012; Lin et al. 2013; Kazibudzki and Grzybowski 2013) but also gives an opportunity to designate the accuracy of priorities estimation. This feature of the $\hbox {TCI}_{\mathrm{LUA}}^{0}$ is seminal and has not been indicated yet. Thus, if $\Phi$ is given by the relation (3):

$$\begin{aligned} \Phi =\sqrt{\frac{1}{n}\min \sum \nolimits _{i=1}^n {\ln ^{2}\left( {\sum \nolimits _{j=1}^n {\frac{a_{ij} w_j }{nw_i }} } \right) } } \end{aligned}$$

(3)

then, the mean standard estimation error (MSE) and mean relative estimation error (MRE) for the particular priority $w_{i}$ can be given by the formulae (4) and (5), respectively:

$$\begin{aligned} \mathrm{MSE}(w_i)= & {} w_i (e^{\Phi }-1) \end{aligned}$$

(4)

$$\begin{aligned} \mathrm{MRE}= & {} (e^{\Phi }-1)\times 100\,\% \end{aligned}$$

(5)

The formulae (4) and (5) give the opportunity to establish an accuracy of estimation results i.e. having the particular priority $w_{i}$, decision maker realizes that the particular priority true value can deviate from the obtained outcome plus/minus MSE ($w_{i}$). This is the revolution in a consistency measurement within the AHP because in a case when decision makers priorities differ insignificantly, it is very easily to realize that a true order of their preferences can deviate from the obtained one. In consequence, this knowledge provides decision makers a unique opportunity to prevent prospective fallacious choices caused by their judgments’ inconsistency. Thus, the LUA and the $\hbox {TCI}_{\mathrm{LUA}}^{0}$ deliver not only better priorities approximation methodology but also increase the quality of decisions, allowing for their precision and legitimacy control.

5 Final remarks

It has been scientifically proven that high inconsistency may lead to fallacious choices of alternative solutions for a problem. Research verifies the thesis stating that inconsistency measures derived from different prioritization procedures are interrelated. In order to answer the research question, relations among various consistency indices derived from different prioritization procedures devised for AHP were analyzed. Correlations were sought in decisional problems structured as three level hierarchies. It was believed that only this approach reflects relations among different indices during simulated decisional problems. A close relation between the $\hbox {TCI}_{\mathrm{LUA}}^{0}$ and the $\hbox {TCI}_{\mathrm{REA}}$ was found which indicates that the latter index can be simply replaced by the $\hbox {TCI}_{\mathrm{LUA}}^{0}$. It bears emphasis that the approximation was established independently for a number of alternatives in a randomly distorted hierarchy model that was created from known priorities. Therefore, the relation is distinguished from the others that were established for randomly generated and single PCMs of particular size as seen by e.g.  Pereira and Costa (2015), Brunelli et al. (2013), Grzybowski (2012), Aguaron and Moreno-Jimenez (2003). Proposed; that the consistency of decision makers’ judgments should be measured from the perspective of the entire hierarchy as opposed to a single PCM since the simulations based on the entire decisional hierarchy as the model of a true situation better reflects reality.

To conclude; in conjunction with other seminal research papers e.g. Chen et al. (2015), Pereira and Costa (2015), Linares et al. (2014), Moreno-Jiménez et al. (2014), Aguarón et al. (2014), Lin et al. (2013), Kazibudzki and Grzybowski (2013), Brunelli et al. (2013), Grzybowski (2012), this present examination enriches the knowledge about the true value of the AHP which is widely recognized as an applicable decision making support system. Hopefully, the examination will encourage better quality of human prospective choices.