Heuristic rating estimation: geometric approach

Heuristic rating estimation is a newly proposed method that supports decisions analysis based on the use of pairwise comparisons. It allows the ranking values of some alternatives (herein referred to as concepts) to be initially known, whilst ranks for other concepts have yet to be estimated. To calculate the missing ranks it is assumed that the priority of every single concept can be determined as the weighted arithmetic mean of the priorities of all the other concepts. It has been shown that the problem has an admissible solution if the inconsistency of the pairwise comparisons is not too high. The proposed approach adopts heuristics according to which a weighted geometric mean is used to determine the missing priorities. In this approach, despite increased complexity, a solution always exists and its existence does not depend on the inconsistency or reciprocity of the input matrix. Thus, the presented approach might be appropriate for a larger number of problems than previous methods. Moreover, it turns out that the geometric approach, as proposed in the article, can be optimal. The optimality condition is presented in the form of a corresponding theorem. A formal definition of the proposed geometric heuristics is accompanied by two numerical examples.


Introduction
The first written evidence about pairwise comparisons (PC) method dates back to the thirteenth century, when Ramon Llull from Majorca wrote a seminal piece "Artifitium electionis personarum" (The method for the elections of persons) about voting and elections [4,3], followed by the two consecutive works being a practical study on the election processes 1 .Nowadays PC as a voting method is a way of deciding on the relative utility of alternatives used in decision theory [19] and other fields like economy [16], psychometrics and psychophysics [20] and so on.The PC theory is developed by many research teams representing different fields and approaches.One can point out some characteristic approaches like fuzzy PC relation developed by Kacprzyk et al. and Mikhailov [7,15], data inconsistency reduction methods proposed by Koczkodaj and Szarek [10] and issue of incomplete PC relation by Koczkodaj and Orłowski [8] and Bozoki and Rapcsak [1], problem of non-numerical rankings addressed by Janicki and Zhai [6] or using PC in Data Envelopment Analysis [14].
Currently, the Heuristic Rating Estimation (HRE) method which enables the user to explicitly define the reference set of concepts, for which the ranking values are a priori known, is being developed [11,12].The base heuristics used in HRE proposes to determine the relative values of a single non-reference concept as a weighted arithmetic mean of all the other concepts.This proposition leads to the linear equation system defined by the matrix A and the strictly positive vector of constant terms b .
In this work, the authors show that using a geometric mean to determine the relative priorities of concepts instead of arithmetic one in some cases may be more convenient.The main benefit of the proposed solution stems from the guarantee of solution existence.Hence, unlike the original proposal, the ranking list can always be created.This guarantee is paid with the increase in computational complexity.The presented solution is accompanied by two numerical examples.
The presented work is a follow-up of research initiated in [11,12].It redefines the main heuristics of HRE and the method of calculating the solution.The HRE approach as proposed in the previous articles is briefly outlined in (Sec.2).There are also a short summary of a few important properties of M-matrices (Sec.2.3), which are essential to the properties of the presented method.The next section (Sec.3) describes the proposed solution and discusses two important properties: solution existence (Sec.3.2) and optimality (Sec.3.3).Theoretical considerations are accompanied by two meaningful examples showing how the presented method can be used in practice (Sec 4).A brief summary is provided in (Sec.5).

Basic concepts of pairwise comparisons method
The input to the PC method is the PC matrix M = (m i j ), where m i j ∈ + and i , j ∈ {1, . . ., n }.It expresses a quantitative relation R over the finite set of concepts C df = {c i ∈ and i ∈ {1, . . ., n }} where is a non empty universe of concepts, and R(c i , c j ) = m i j , R(c j , c i ) = m j i .The values m i j and m j i represent subjective expert judgment as to the relative importance, utility or quality indicators of concepts c i and c j .Thus, according to the best knowledge of experts should holds that c i = m i j c j .Definition 1.A matrix M is said to be reciprocal if for all i , j ∈ {1, . . ., n } holds m i j = 1 Since the data in the PC matrix represents subjective opinions of experts, thus they might be inconsistent.Hence, it may exist a triad m i j , m j k , m k i of entries in M for which m i k • m k j = m i j .This leads to the situation in which the relative importance of c i with respect to c j is either m i k • m k j or m i j .This observation underlies two related concepts: a priority deriving method that transform even an inconsistent matrix M into consistent priority vector, and an inconsistency index describing how far the matrix M is inconsistent.There are a number of priority deriving methods and inconsistency indexes [2,5].For the purpose of the article the Koczkodaj's inconsistency index is adopted.

Definition 2. Koczkodaj's inconsistency index
of n × n and (n > 2) reciprocal matrix M is equal to where i , j , k = 1, . . ., n and i The result of the pairwise comparisons method is ranking -a function that assigns values to the concepts.Formally, it can be defined as follows.

Definition 3. The ranking function for C (the ranking of C
) is a function µ : C → + that assigns to every concept from C ⊂ a positive value from + .
Thus, µ(c ) represents the ranking value for c ∈ C .The µ function is usually defined as a vector of weights µ df = µ(c 1 ), . . ., µ(c n ) T .According to the most popular eigenvalue based approach proposed by Saaty [19] the final ranking µ ev is determined as the principal eigenvector of the PC matrix M , rescaled so that the sum of all its entries is 1, i.e.
where µ ev -the ranking function, where It can be shown that for the fully consistent matrix M both ranking vectors µ ev and µ gm are identical.A more completely overview including other methods can be found in [2,5].

Pairwise comparisons method with the reference set
Usually when using the pairwise comparisons method the ranking values µ(c 1 ), . . ., µ(c n ) are initially unknown.Hence they are need to be determined by the priority deriving procedure.In some cases, however, there are concepts for which the priorities are known from elsewhere.Hence, the decision makers may have additional knowledge about the group of elements C K ⊆ C that allow them to determine µ(c ) for C K in advance.
For example, let c 1 , c 2 and c 3 represent oil paintings that an auction house plans to put for auction.The sequence of paintings during the auction should correspond to their approximate valuation.In order to determine the indicative price of paintings the auction house asked experts to evaluate them in pairs taking into account that two other paintings from the same period of time were previously auctioned for µ(c 4 ) and µ(c 5 ).
The situation as described above prompted the first author [11,12] to propose a Heuristic Rating Estimation (HRE) model.According to HRE the set of concepts C is composed of unknown concepts C U = {c 1 , . . ., c k } and known (reference) concepts C K = {c k +1 , . . ., c n }, where C U ,C K = and C U ∩ C K = .The values µ(c i ) for c i ∈ C K are known, whilst the values µ(c j ) for elements c j ∈ C U need to be calculated.Following the heuristics of averaging with respect to the reference values [12] solution proposed by HRE is to adopt as µ(c j ), for every c j ∈ C U , the arithmetic mean of all the other values µ(c i ) multiplied by factor m j i : If the experts judgments gathered in the matrix M were fully consistent (Def.1), then every component of the sum (5) in the form m j i µ(c i ) would equal µ(c j ).Because, it is generally not, then every component is only an approximation of µ(c j ).Thus, the arithmetic mean of the individual approximations has been adopted as the most probable value of µ(c j ).To determine unknown values µ(c j ) for c j ∈ C U the problem formalised as ( 5) can be written down as the linear equation system Aµ = b , where: and The solution µ = µ(c 1 ), . . ., µ(c k ) T determines the values of µ for elements from C U .Together with known µ(c k +1 ), . . ., µ(c n ) the vector µ forms the complete result list, which after sorting can be used to build ranking.Although the values µ(c ) for c ∈ C are called priorities, they usually have a specific meaning.In the case of previously mentioned example they represent the expected price of paintings.According (Def.3) the ranking results must be strictly positive, hence only strictly positive vectors µ are considered as feasible.It can be shown that the equation Aµ = b has a feasible solution if A is strictly diagonally dominant by rows [12].It has recently been shown that the equation has a feasible solution when the inconsistency index (M ) is not to high [13].

M-matrices
Very often the real life problem can be reduced to the linear equation system Aµ = b , where the matrix A has some special structure.Frequently the matrix A has positive diagonal and nonpositive off-diagonal entries.Due to their importance to the practice this type of matrix was especially thoroughly studied by researchers [17,18].To define it formally a few more notions and definitions are needed.Let (n ) be a set of n × n matrices over , and (n ) the set of all A = [a i j ] ∈ (n ) with a i j ≤ 0 if i = j and i , j ∈ {1, . . ., n }.Furthermore, assume that for every matrix A ∈ Following [17] some of the M-matrix properties are recalled below in the form of the Theorem 1.
Theorem 1.For every A ∈ (n ) each of the following conditions is equivalent to the statement: A is a nonsingular M-matrix.

1.
A is inverse positive.That is, A −1 exists and A −1 ≥ 0 2.There exists a positive diagonal matrix D such that AD has all positive row sums.
It is worth to note that for every matrix equation in the form Aµ = b , where A is a nonsingular M-matrix, holds

Heuristics of the geometric averaging with respect to the reference values
Most often the pairwise comparisons method is used to transform the PC matrix into the ranking list of mutually compared concepts.During the transformation to each concept a priority is assigned.Therefore, this transformation is often called a priority deriving method.There are many priority deriving methods.Besides the eigenvalue based method (2), where the ranking values µ(c i ) are approximated as the arithmetic means of m i j • µ(c j ), also the geometric mean of rows is used (3).This may suggest that also for the ranking problem with the reference set [12], the arithmetic mean (5) might be replaced by the geometric mean.This observation prompted the author to formulate and investigate the geometric averaging with respect to the reference values heuristics.According to this proposition to determine the unknown values µ(c j ) for c j ∈ C U the following non-linear equation is used: After rising both sides to the n −1 power the geometric averaging heuristics equation ( 8) leads to the non-linear equation system in the form: Of course, since the ranking values for c k +1 , . . ., c n ∈ C K make the reference set where the values µ(c j ) are known and fixed, some products in the form m j i µ(c i ) are initially known constants.Let us denote: for j = 1, . . ., k as the constant part of each equation (9).Thus, the non-linear equation system can be written as: = log ξ m i j and g j df = log ξ g j for some ξ ∈ + .It is easy to see that the above non-linear equation system is equivalent to the following one:  (11) By grouping all the constant terms on the right side of each above equation we obtain the linear equation system where b i df = k j =1,j =i m 1,j + g i for i = 1, . . ., k , which can be easily written down in the matrix form where: Therefore, the solution µ of the linear equation system (13) automatically provides the solution to the original non-linear problem as formulated in (9).Indeed the ranking vector µ can be computed following the formula: Importantly, as it is shown below a feasible solution of (13) always exists.Hence, the heuristics of the averaging with respect to the geometric mean always provides the user an appropriate ranking function.

Existence of solution
The form of A is specific.The positive diagonal and the negative off-diagonal real entries cause that A ∈ (k ) (see Sec. 2.3).Let us put: and D ∈ (k ).Of course D is positively dominant matrix.Thus, the product A • D = A. The sum of each row in A equals k is greater than 0. This means that the sum of each row of A • D is positive.Hence, due to the Theorem 1, A is a nonsingular M-matrix (Def.4).Thus, A −1 exists (i.e.µ = A −1 b ) and always the equation ( 13) has a solution in k .Due to the form of the solution of the main problem (16) µ is a vector in k + , i.e. every its entry is strictly positive.In other words unlike the original proposition [12] the heuristics of the geometric averaging with respect to the reference values always provides a feasible ranking result to the user.

Optimality condition
One of the reasons for introducing the geometric mean method (3) is minimizing the multiplicative error e i j [5] defined as: m i j = p i p j e i j (17) In the case of the geometric averaging heuristics the multiplicative error equation takes the form: The multiplicative error is commonly accepted to be log normal distributed (in the same way the additive error would be assumed to be normally distributed).Let e : n + → be the sum of multiplicative errors (see [5]) defined as follow: As it is shown in the Theorem below very often the heuristics ( 8) is optimal with respect to the value of multiplicative error function e .

Theorem 2. The geometric averaging with respect to the reference values heuristics minimizes the sum of multiplicative errors e (µ(c
Proof.To determine the minimum of ( 19) let us forget for a moment that µ(c k +1 ), . . ., µ(c n ) are constants (the reference values), and let us treat them as any other arguments of e .In order to determine the minimum of ( 19) the first derivative need to be calculated.Thus, for i = 1, . . ., n .Due to the reciprocity of M , i.e. m i j = 1 /mji , the equation ( 21) can be written as: The function e reaches the minimum if ∂ e /∂ µ(ci ) = 0.This leads to the postulate that which is directly equivalent to (8).In other words any solution to the equation system ( 9) is a good candidate to be a minimum of (19).It remains to settle the matrix H of second derivative of e .When H is positive definite then the solution of ( 9) actually minimizes the function e .As a result of further differentiation is determined that the diagonal elements of H are Proof.where i = 1, . . ., n , and the other elements for which i = j and i , j = 1, . . ., n take the form: Since the matrix H is considered for e in the point µ(c 1 ), . . ., µ(c n ) such that (8) holds, thus the first derivative of e is 0. Therefore, the Hessian matrix H takes the form: According to [18, p. 29] if H is strictly diagonally dominant by rows, symmetric, and with positive diagonal entries then it is also positive definite.To meet the first strict diagonal dominance criterion (other are satisfied) it is required that: for i = 1, . . ., n .Thus, Since every µ(c i ) > 0, then it is easy to verify that the above equation is equivalent to the desired condition (20).

Numerical examples
The HRE method can be useful in many situations in which, based on the expert subjective opinions and the actual data, the new concepts, objects or entities need to be assessed.In order to show how the method may work in practice the following two numerical examples are presented.The first one, more abstract, discusses the method for solving the non-linear equation system.The second one, more complex, tries to put the method into the actual business context, where it can be successfully used.
In both examples the set of concepts consists of C K -the reference (known) and C U -the initially unknown elements.To solve an intermediate linear equation system (13) the Gaussian elimination method is used.

Example I (Scientific entities assessment)
Let c 1 , ..., c 5 represent the scientific entities 2 , where two of them c 2 , c 3 ∈ C K are the reference entities.Their values were arbitrarily set by experts to µ(c 2 ) = 5 and µ(c 3 ) = 7.The analysis of the scientific achievements of the entities c 1 , c 4 and c 5 leads to the following PC matrix: To calculate the rank using HRE with the geometric averaging heuristics, the following system of non-linear equations (compare with 9) need to be solved: (31) 2 Actually the official ranking of the scientific entities in Poland compares the entities in pairs [9].thus, after rising both sides of the equations to the power, Substituting the logarithm of both sides of the equations, we get the following system: Then, according to the procedure proposed in (Sec.3.1) the linear equation system (13) where the unknown values μ(c i ) df = lg µ(c i ) for i = 1, 4, 5 takes the form:

Example II (Choosing the best TV show)
Certain TV broadcaster wants to produce a new entertainment TV show in one of the European countries.It considering a purchase the license for one of the five entertainment shows produced in the United States.So far in Europe three similar programs were broadcasted.Through the market research there are known approximate size of their European audience.They are respectively 5, 500, 000, 4, 500, 000 and 4, 950, 000 persons for programs c 6 , c 7 and c 8 correspondingly.The production costs of these programs are similar.In order to select possibly the most profitable TV show the station hires a few seasoned media experts.During the expert panel they prepared the following PC matrix M representing a relative attractiveness of all the considered programs.
In the matrix M every entry m i j corresponds to the ratio describing attractiveness of the TV show c i with respect to the attractiveness of TV show c j .Since the values of attractiveness for c 6 , c 7 and c 8 are known (they are approximated by the number of people watching the given TV show), thus the appropriate ratios m i j for i , j = 6, 7, 8 are not the subject of the expert judgment.Instead, they are calculated based on data from the market research.For example: The other entries of M represent the subjective judgements of experts.Similarly as before, to find a solution with the help of HRE supported by the geometric averaging heuristics, the system of equations ( 9) must be solved.The desired values µ(c i ) for i = 1 . . ., 5 will be derived from the formula µ(c i ) = log µ(c i ).Because |C U | = 5, the dimensions of matrix A are 5 × 5.The linear equation system need to be solved is as follows:  Hence, following the rule µ(c i ) = ξ µ (ci ) , where ξ = 10 is the logarithm base, the final result vector is calculated.Thus, according to the expert judgments and the market research the TV show number 5 (denoted as c 5 ) has a chance to gather in front of TVs near 6.8 million people, whilst the second one in line "only" 4.8 million of people.Based on this estimate the board of directors representing the broadcaster has decide to recommend the purchase of the license for the fifth presented TV show.

Summary
The presented geometric HRE approach is another solution to the problem of rankings with the reference set.It proposes to use a geometric mean instead of arithmetic one used in [11,12].The advantage of this approach is the robustness of the procedure.As has been shown in (Sec.3.2) the proposed solution works for arbitrary set of input data producing admissible vector of weights.The resulted ranking very often turns out to be optimal in sense of the magnitude of multiplicative errors.According to the formulated and proven condition (Sec.3.3), this happens when the differences between the resulted priorities are not too large.
The HRE approach may be useful in many different situations including, ranking creation, valuation of goods and services, risk assessment and others.Due to the lack of restrictions on the input PC matrix (method with the geometric mean always produces an admissible result), the scope of the applicability of the HRE method increases.Thus, the presented method covers cases which can not always be dealt with using the arithmetic mean heuristics.
Despite the encouraging results, much remains to be done.In particular, the role of the inconsistency in the input matrix M should be more deeply investigated.Of course, the more studied examples, the better.Thus, further development of the method will be particularly focused on the study and analysis of use cases.

Definition 4 .
(n ) and vector b ∈ n the notation A ≥ 0 and b ≥ 0 will mean that every m i j and b k are non-negative and neither A nor b equals 0. The spectral radius of A is defined as ρ(A) df = max{|λ| : det(λI − A) = 0}.An n × n matrix that can be expressed in the form A = s I − B where B = [b i j ] with b i j ≥ 0 for i , j ∈ {1, . . ., n }, and s ≥ ρ(B ), the maximum of the moduli of the eigenvalues of B, is called M-matrix.