OWA-based multi-criteria decision making based on fuzzy methods

Abstract

One of the most important challenges in Multi-Attribute Decision Making (MADM) problems is how can the optimal weights of the criteria can be determined properly by the decision maker. In the relevant research literature, various methods based on the requirements and assumptions of the problem were introduced to determine the weights of the criteria. In this regard, in particular, the Yager’s OWA operator is one of the most significant and widely used approaches to evaluate the weights of criteria. But there is a drawback, that is, the results of Yager’s OWA operator depend only on the level of decision-maker’s risk and the number of the criteria. Therefore, in this paper, using a multi-objective decision making approach, we try to express this MADM challenge in the form of a generalization of the Yager’s OWA operators and Ahn’s method. One of the advantages of this generalization is that the proposed method uses all the information in the decision matrix compared to the methods proposed by Yager’s OWA operators and the Ahn’s method. The proposed approach is also able to enter various types of preferences considered by the decision maker for the criteria calculations as crisp or fuzzy quantities. Numerical examples and real dataset analysis based on a survey of students’ opinions on teaching activities are provided.

1 Introduction and motivation

Suppose a teacher wants to evaluate a number of students. How can s/he do such a thing by distributing 100 marks to n questions in such a way that:

• 1. The students get the most benefit,

• 2. The teacher provides an optimal comparison between them,

• 3. His/her initial judgments about the value of each question are considered.

It is clear that simply considering the initial judgments of each question may not satisfy the first and the second goals. This is why in many cases teachers are forced to change the initial scores assigned to the questions after taking tests in order to meet the benefit of the most of their students. This example is a special case of the general important challenge of determining the weights of criteria in the Multiple Attribute Decision Making (MADM) problems, in which the aim is to provide an optimal comparison between alternatives based on the criteria (Xu, 2015). In literature, different methods have been proposed by researchers to estimate the weights of the criteria of an MADM problem.

For instance, Simple Multi-Attribute Rating Technique (SMART) is a method of multi-criteria decision making in which each alternative consists of some criteria that have values and each criterion have a weight that illustrates how important the other criteria are (Edwards & Barron, 1994). In the SWING technique (Danielson & Ekenberg, 2019) criteria are compared with the least important criterion, and in each comparison, an importance percentage value is assigned. In AHP, the relative importance of all criteria is compared pairwise with each other, and then the weights of the criteria are estimated using these ratios (see also paired comparison matrix (Johnson et al., 2019), including the matrix of pairwise comparisons in terms of Saaty spectrum (Sáa et al., 2015)). In the Best-Worst Method (BWM), with the aim of reducing the number of pairwise comparisons, all criteria are compared with both the least important criterion and the most important criterion (Yücel & Taşabat, 2019). In the Stepwise Weight Assessment Ratio Analysis (SWARA) technique (Zolfani et al., 2018) first the criteria are ranked in descending order of importance, and each criterion is compared with the first criterion after it. Here, like the BWM method, the number of pairwise comparisons is less compared to AHP. In the Linear Programming Technique for Multidimensional Analysis of Preference (LINMAP) technique (Hatami-Marbini et al., 2013; Wang & Liu, 2013), based on a set of initial pairwise judgments that may be insufficient or inconsistent, the weights of the criteria are estimated using of a linear programming problem. In the Entropy technique (Lai & Yang, 2008), the weights of the criteria are estimated based on the information of the decision matrix. In this method, the data related to each criterion are transformed into a probability distribution, and then the entropy associated with it is calculated. Finally, the weights of the criteria are calculated as complements to the level of entropy. There is a type of well-known widely used method, called as Ordered Weighted Aggregation (OWA) operator (Yager, 1988), which plays a key role in this paper (see also Sect. 2 bellow). In the short time since its first appearance, the OWA operators (Yager, 1988; Yager & Kacprzyk, 1997) have been used in an astonishingly wide range of applications in variety of fields (Chachi et al., 2021; D’Urso & Leski, 2020; Kazemifard & Chachi, 2022). The main reason for this is their great flexibility to model a wide variety of aggregations, as their nature is defined by a weighting vector, and not by a single parameter. By appropriately selecting the weighting vector, it is possible to model different kinds of relations among the criteria aggregated in group decision-making problems with multiple assessments (Ahn, 2008, 2017; Chachi et al., 2021; D’Urso & Leski, 2020). Therefore, Yager’s OWA operators (Yager, 1988; Yager & Kacprzyk, 1997) in a constrained optimization problem are considered in the literature as well as the present paper to provide formidable tools that can be employed to provide proper weight values in group decision-making problems (Kacprzyk et al., 2019; Kazemifard & Chachi, 2022; Medina & Yager, 2021). Its success to a great extent depends on proper determination of the associated weights that characterize the operators (Chaji 2017; Chaji et al., 2018; Yari & Chaji 2012a; b). Therefore, OWA operator has received much attention in literature and the properties of this operator was investigated as well.

In Yager (1988)’ operator which was concerned with the problem of aggregating multicriteria to form an overall decision function the data is merged after modulating them by means of some weights, but in such a way that the weight affecting to each datum only depends on the place it takes in the descending chain of the arranged data. Hence Yager’s OWA operators are symmetric, i.e., the global value that they obtain from a collection of data does not depend on either the expert or the resource that has provided each datum or even the recorded value of each datum or the information contained in the decision matrix. An efficient method proposed by Yager to solve the constrained OWA optimization problem was mainly based on the maximization of entropy value of the weights considering an orness value (risk level). An OWA operator is a mapping $OWA:{\mathbb {R}}^n\longrightarrow {\mathbb {R}}$ as follows

$$\begin{aligned} OWA(a_1,\dots ,a_n)=\sum ^n_{{j=1}}{w_j a_{(j)}}, ~~ a_{(1)}\ge \cdots \ge a_{(n)}. \end{aligned}$$

where $\textbf{w}=[w_1,\dots ,w_n]^t$, such that $w_j\in [0,1]$ and $\sum ^n_{j=1}{w_j}=1$ (Yager, 1988; Yager & Kacprzyk, 1997). The weighting vector $\textbf{w}$ determines the preference between ordered argument values in the aggregation which needs to be determined properly (Ahn, 2008; 2017; Chaji, 2017; Chaji et al., 2018; García-Zamora et al., 2022; Harmati et al., 2022; Yari & Chaji, 2012a; b). In fact, if decision maker risk level is $\alpha$ then the optimized weights are obtained by running the following optimization problem

$$\begin{aligned} \max _\textbf{w}{} & {} -\frac{1}{\log (n)}\sum _{j=1}^{n} w_j \log (w_j) \nonumber \\ s.t.{} & {} orness(\textbf{w})=\sum _{j=1}^{n} \frac{n-j}{n-1} w_j = \alpha \nonumber \\{} & {} \sum _{j=1}^{n} w_j =1, \text { and } w_j\ge 0,\text { for }j=1,\ldots ,n. \end{aligned}$$

(1)

Remark 1

In the optimization problem, the feasible region is not empty and the objective function has finite optimum, so the problem has an analytical solution. In addition to the alternation of the Entropy objective function, which was an efficient method proposed by Yager to solve the constrained OWA optimization problem, other objective functions were proposed by some researchers. Depending on the goal to attain, some of them are listed below, but are not limited to (Beliakov, 2017):

• 1. minimize: $\sum _{j=1}^{n}(w_j-w_{j+1})^2$ for a minimum variance approach (Wang et al., 2007),

• 2. minimize: $\max _j|w_j-w_{j+1}|$ for a minimax disparity approach (Wang & Parkan, 2005),

• 3. minimize: $\sum _{j=1}^{n}(w_j-\frac{1}{n})^2$ for a least square OWA approach (Fullér & Majlender, 2003),

• 4. minimize: $\max _j w_j$ for another expression of a measure of entropy (Yager, 1993).

As Yager’s approach shows, evaluating the weights of criteria depends on only two things:

• 1. The number of criteria, and

• 2. Decision maker’s risk level (orness).

Table 1 Decision matrix $D_1$

Table 2 Decision matrix $D_2$

We question Yager operator for not paying attention to the information provided by the decision matrix in determining the weights. We are going to address the problem by providing a new approach to determine the weights. For example, suppose that in the Yager method with an $orness=1$ we want to calculate the weights of three criteria. In this case, the Yager method results in weights of $\textbf{w}=(1, 0, 0)$. Now suppose we have two decision matrices $D_1$ and $D_2$ as given in Table 1 and Table 2, respectively. the Yager’s OWA operator for the matrix $D_2$ leads to $\mathcal {A}_1=\mathcal {A}_2=\mathcal {A}_3$. In fact, the Yager’s OWA operator is not able to rank $\mathcal {A}_1$, $\mathcal {A}_2$, and $\mathcal {A}_3$ in the decision matrix $D_2$. In order to overcome the problem, Ahn (2008) investigated an approach to obtain the values of the OWA weights in the decision matrix criteria while considering a set of preferences among some alternatives. But there are situations in which the approach proposed by Ahn (2008) is not applicable, including:

• 1. There maybe cases in which there is no such the preferences set between alternatives;

• 2. The decision maker may have considered not only the binary preferences between the alternatives, but also the degree of preferences that may be also fuzzy;

• 3. There may be inconsistencies between the preferences adopted by the decision maker, i.e. the preferences may not be transitive, that is, they are intransitive.

Now, in this article we want to improve and generalize Yager (1988) approach as well as Ahn (2008) approach to a more general situation. Aiming to overcome the limitations of the preceding methods, this article attempts to put forward L-p metric method and Goal Programming (GP) in the context of Multi-Objective Decision Making (MODM) framework. To this end, a weighted aggregation of the following items will be optimal:

• 1. Maximizing the desirability of alternatives;

• 2. Possibility of comparison between alternatives;

• 3. Involvement of the risk level of the decision maker;

• 4. The possibility of interfering with the “decision-maker’s preferences”;

• 5. Involvement “the degree of decision-makers’ preferences” in relation to alternatives;

• 6. Involvement the decision maker’s degree of preferences in the items 1 to 5 above.

Finally, we perform our proposed method to choose the optimal weight values on several numerical examples. We also implement a comparative analysis based on a real datatset.

The remaining sections are arranged as follows. In Sect. 2 the literature review is provided. MODM is considered in Sect. 3. In Sect. 4, our proposed approach of derivations of the OWA operator weights in the context of MODM is presented. Several numerical examples will be illustrated to solve the constrained OWA aggregation problems in Sect. 5. In Sect. 6, we analyze real data collected in a survey of students’ opinions on teaching activities. Concluding remarks follow in Sect. 7.

2 Literature review

The OWA operator has been used to structure the optimization problem to derive the optimal weight values of the criteria in an MAMD problem (Kazemifard & Chachi, 2022; Yager, 2020). Obtaining the weights associated with the OWA operator has been investigated by several researches in the literature, in particular when data on the arguments and the aggregated value are already taken. To do so, one of the main concerns is how to generate OWA operator weights and, as a result, numerous weight generation methods have been advanced, for instance, Ahn (2008, 2017) proposed approaches to solve the constrained OWA aggregation problem. García-Lapresta et al. (2011) generate OWA weights from individual assessments. Beliakov (2017) proposed a method that introduced weights into OWA operators and other symmetric functions. De Miguel et al. (2017) proposed an algorithm for group decision making using n-dimensional fuzzy sets, admissible orders and OWA operators. Zarghami and Szidarovszky (2009) revised the OWA operator for multi criteria decision making problems under uncertainty. Liu et al. (2021) investigated the ranking range comparisons for the selected seven popular MADM approaches while the attribute weights were manipulated. Zhou and Chen (2020) investigated multiple criteria group decision analysis using a Pythagorean fuzzy programming model for multidimensional analysis of preference based on novel distance measures. Ji et al. (2021) developed an induced ordered weighted averaging operator for expert opinions aggregation. He et al. (2021) proposed an induced OWA operator for group decision making which dealt with extended comparative linguistic expressions with symbolic translation. By considering the partitioning around medoids approach in a fuzzy framework, D’Urso and Leski (2023) proposed fuzzy clustering models for multivariate time series. In order to neutralize the negative effects of outlier time series in the clustering process, they proposed robust fuzzy c-medoids clustering models for time series based on the combination of Huber’s M-estimators and Yager’s OWA operators. Srivastava et al. (2023) present a novel representative of the existing family of ordered weighted aggregation (OWA) operators with constant orness (optimism/pessimism level). Gagolewski et al. (2023) explored the relationships between the famous Lance-Williams update formula and the extended OWA-based linkages with weights generated via infinite coefficient sequences. D’Urso et al. (2023) proposed a new model based on the use of the entropy as a regularization function in the fuzzy clustering criterion. D’Urso et al. (2023) proposed a robust fuzzy C-medoids method based on entropy regularization which used an appropriate exponential transformation of the dissimilarity based on Dynamic Time Warping, that can be computed also for time series of different length. Maldonado et al. (2023) proposed a novel adaptive loss function for enhancing deep learning performance in classification tasks. Pérez-Fernández (2023) introduced a unifying perspective by presenting under a common framework different classes of multivariate OWA functions and discussed the main fulfilled properties by each of these classes.

3 MODM

Multi-Objective Decision Making (MODM) is a procedure targeting at supporting decision makers faced with conflicting evaluations (Zavadskas et al., 2019). The procedure aims at highlighting these conflicts and deriving a way to come to a compromise in a more transparent manner. Evaluation criteria in MODM are derived or interpreted subjectively as indicators of the strength of various preferences. Multi-Criteria Decision Making (MCDM) naturally involve several competing objectives that are required to be optimized simultaneously, i.e.

$$\begin{aligned} \text {optimize}{} & {} f_1(\textbf{x}),\ldots ,f_N(\textbf{x}), \\ \text {s.t.}{} & {} \textbf{x}=(x_1,\ldots ,x_k)\in \mathbb {D}\subseteq \mathbb {R}^k, N\ge 2\text { and }k\ge 1. \end{aligned}$$

In literature, there are many methods that have been introduced and suggested to deal with MODM issues (Kubler et al., 2016). In the following two methods known as L-p metric method and GP method will be used to deal with MODM issues.

3.1 L-p metric

L-p metric method is among the most popular algorithm whose objectives are the minimization of deviations of the objective/target functions from ideal solution(s) (Brezis, 2011; Royden & Fitzpatrick, 2017). Here, it is supposed that we have a multi-objective optimization problem as follows

$$\begin{aligned} \min{} & {} f_1(\textbf{x}),\ldots ,f_N(\textbf{x}), \\ \text {s.t.}{} & {} \textbf{x}\in \mathbb {D}\subseteq \mathbb {R}^k, ~ N\ge 2, ~ k\ge 1. \end{aligned}$$

In this case effective solutions induced in the method L-p metric are obtained by solving the following optimization problem which is also a weighted p-mean

$$\begin{aligned} \min{} & {} \left[ \sum _{l=1}^N \delta _l \left( \frac{f_l^+ - f_l}{f_l^+-f_l^-} \right) ^p\right] ^\frac{1}{p}\\ \text {s.t.}{} & {} \textbf{x}\in \mathbb {D},~ \delta _l\ge 0 \text {~for~} l=1,\ldots N, ~ \sum _{l=1}^{N}\delta _l=1, \end{aligned}$$

where $1\le p \le \infty$ and for each $l=1,\ldots ,N$

$$\begin{aligned} f_l^+=\max _{\textbf{x}\in \mathbb {D}} f_l(\textbf{x}),\text {~and~}f_l^-=\min _{\textbf{x}\in \mathbb {D}} f_l(\textbf{x}), \end{aligned}$$

and $\delta _l$ is the relative importance of the objective function $f_l$.

3.2 GP method

Suppose that $f_l$, $l=1,\ldots ,N$, are objective functions and $L_l$, $l=1,\ldots ,N$, denote the target or goal set by decision maker for $l^{th}$ objective function $f_l$ and $\mathbb {D}$ represents the feasible region from which the choices of vector $(f_1,\ldots ,f_N)$ must be effected. A goal-programming model can be stated as follows:

$$\begin{aligned} \min{} & {} \sum _{l=1}^{N} \delta _l(d_l^+ + d_l^-)\\ \text {s.t.}{} & {} \textbf{x}\in \mathbb {D}\subset \mathbb {R}^k,\\{} & {} { f_l + d_l^- - d_l^+ = L_l,~d_l^+\ge 0, ~ d_l^-\ge 0},\\{} & {} \delta _l\ge 0, \text {~for~}l=1,\ldots N, ~ \sum _{l=1}^{N} \delta _l=1. \end{aligned}$$

The parameters $\delta _l$, $l =1, \ldots , N$ are positive weights that reflect the decision maker’s preferences regarding the relative importance of each goal, and $d^+_l$ is the slack and $d^-_l$ is the “surplus” variables, respectively. The criterion, then, is to minimize the sum of the absolute values of the differences between target values and actual achieved values (Oliveira et al., 2021).

4 Derivation of the OWA operator weights in the context of MODM

Let $\mathfrak {A}=\{\mathcal {A}_1, \ldots , \mathcal {A}_m\}$ be a finite collection of alternatives and, let $\mathfrak {C}=\{\mathcal {C}_1, \ldots , \mathcal {C}_n\}$ be a collection of criteria (attributes), so that their weight information is totally unknown or in some situations part of it is known. Here, an expert evaluates the alternative $\mathcal {A}_i\in \mathfrak {A}$ with respect to the criteria/attribute $\mathcal {C}_j\in \mathfrak {C}$ by the so-called decision matrix $\textbf{D}=[d_{ij}]_{m\times n}$, $i=1,\ldots ,m$ and $j=1,\ldots ,n$. In the general design matrix $\textbf{D}$ of an MADM, $d_{ij}$ is the attribute value of the alternative $\mathcal {A}_i\in \mathfrak {A}$ with respect to attribute $\mathcal {C}_j\in \mathfrak {C}$ (Xu, 2015).

4.1 Notations and entropy measure

Without loss of generality, suppose the matrix $\textbf{D}=[d_{ij}]_{m\times n}$ is scaleless. Then we set $\textbf{D}^{\text {owa}}=[\zeta _{ij}]_{m\times n}$, where for each $i=1,\ldots ,m$, $\zeta _{ij}$ be the jth largest value of $d_{i1},\ldots ,d_{in}$, i.e. $\zeta _{i1}\ge \ldots \ge \zeta _{in}$.

In the following, we recall Entropy measure (Shannon, 1948) that can be used for determining weight values. The entropy weight values can then be used to modify the optimal weight values obtained from the optimization problems introduced in this paper, in order to introduce new modified weight values. To obtain entropy weight values, the following steps 1, 2, 3 are done for matrix $\textbf{D}^{\text {owa}}$.

(1) Transform matrix $\textbf{D}^{\text {owa}}$ into matrix $\mathcal {E}=[\varepsilon _{ij}]_{m\times n}$ by using the normalization formula

$$\begin{aligned} \varepsilon _{ij}=\frac{\zeta _{ij}}{\sum _{i=1}^m \zeta _{ij}}, \quad i=1,\ldots ,m, \text {~and~} j=1,\ldots ,n. \end{aligned}$$

Notice each column can be considered as a probability distribution. Delete the column jth, for some $j=1,\ldots ,n$, in case $\sum _{i=1}^m \zeta _{ij}=0$. Here, all of the alternatives are evaluated the same, therefore, such a criterion makes no distinct values among the alternatives.

(2) Calculate the information Entropy of each column as

$$\begin{aligned} E_j=-\frac{1}{\ln (m)}\sum _{i=1}^m \varepsilon _{ij} \ln (\varepsilon _{ij}), \quad j=1,\ldots ,n. \end{aligned}$$

Note that $\varepsilon _{ij} \ln (\varepsilon _{ij})$ is defined as 0 if $\varepsilon _{ij}=0$, for some i and j (Shannon, 1948).

(3) Derive the weight vector $\textbf{e}=[e_1,\ldots ,e_n]_{1\times n}$, where

$$\begin{aligned} e_j=\frac{1-E_j}{\sum _{l=1}^n (1-E_l)}, \quad j=1,\ldots ,n. \end{aligned}$$

4.2 Set of alternatives preferences

Ahn (2008) presented a method for determining the OWA weights when:

• 1. The preferences of some subset of alternatives over other subset of alternatives are specified in a holistic manner across all the criteria, and

• 2. The consequences (criteria values) are specified in one of three different formats: precise numerical values, intervals and fuzzy numbers.

The OWA weights are to be estimated in the direction of minimizing deviations from the OWA weights implied by the preference relations, thus as consistent as possible with a priori preference relations. Assume a decision situation in which the consequence $a_{ij}$, $i = 1,\ldots ,m$, $j = 1,\ldots ,n$ is specified in precise numerical values and a priori ordered pairs on the subset of alternatives are obtained. Let us define an optimal solution $W^*$ to be a set of the OWA weights $\{w_k\}$ for $k = 1,\ldots ,n$. The solution would be consistent with the decision-maker’s holistic judgments between alternatives if $f(A_i)-f(A_j) > 0$ for every a priori ordered pair $(i, j)\in \Theta$ and for all feasible values of $W = \{(w_1, \ldots , w_n): \sum _{k=1}^n w_k = 1, w_k\ge 0, k = 1,\ldots ,n\}$. Here $f(A_i)$ and $f(A_j)$ denote the aggregated value of input arguments of alternatives $A_i$ and $A_j$, respectively. We can state this as, for all $(i, j)\in \Theta$,

$$\begin{aligned} \sum _{k=1}^{n} \left( b_{ik}-b_{jk}\right) w_k \qquad \text { for }w_k\in W, \end{aligned}$$

in which $b_{ik}$ and $b_{jk}$ are the reordered arguments of the arguments $a_{i1},\ldots ,a_{in}$ and $a_{j1},\ldots ,a_{jn}$ respectively. Thus, the goal of the analysis is to determine the solution $W^*$ for which the conditions such as $\sum _{k=1}^{n} \left( b_{ik}-b_{jk}\right) w_k> \epsilon$ for every priori ordered pair $(i, j)\in \Theta$ are violated as minimally as possible in which $\epsilon$ is a small arbitrary positive number to make the problem tractable by linear program. To attain the objective “as minimally as possible”, we use auxiliary variables $d_{ij}$ in $\sum _{k=1}^{n} \left( b_{ik}-b_{jk}\right) w_k+ \delta _{ij}\ge \epsilon$, for every ordered pair $(i, j)\in \Theta$ and minimize the sum of auxiliary variables in the objective as shown

$$\begin{aligned} \text {Minimize }&~~&\sum _{(i, j)\in \Theta } \delta _{ij}\\ \text {s.t. }&~~&\sum _{k=1}^{n} \left( b_{ik}-b_{jk}\right) w_k + \delta _{ij}\ge \epsilon , \qquad \text { for } (i, j)\in \Theta \\{} & {} w_k\in W, ~ k = 1,\ldots ,n, \\{} & {} \delta _{ij}\ge 0 \text { for all } (i, j)\in \Theta ,\\{} & {} \epsilon >0. \end{aligned}$$

The preference relations other than the strictly ordinal paired orders can be included in the model. Weak ordinal relations between alternatives (e.g., $A_i$ is at least as preferred as alternative $A_j$) or preferences with ratio comparisons of some paired alternatives (e.g., $A_i$ is $a_{ij}$ times more important than alternative $A_j$) are some examples that can be included in the incomplete holistic judgments. These holistic judgments are then used to determine the OWA weights as the system of constraints restricting the feasible region of the weights.

In the following we are going to extend Ahn (2008)’ method for determining the OWA weights in the case in which the set of preferences $\Theta$ is an empty set or in the form of fuzzy quantities. To do this, suppose that $\Theta \subseteq \mathfrak {A}\times \mathfrak {A}$ is a set of preferential relationships that are decided by the decision maker. We define the fuzzy set of preferences $\widetilde{\Theta }$ as follows

$$\begin{aligned} \widetilde{\Theta }_1=\left\{ \left( (A_s,A_t),\wp _{(s,t)}\right) | (A_s,A_t)\in \Theta \text { and } \wp _{(s,t)}\in [0,1] \right\} , \end{aligned}$$

where $\wp _{(s,t)}$ indicates the degree of preference of $A_s$ over $A_t$. Also, in order to take into account the prioritize options from the decision maker perspective, the following fuzzy set is defined

$$\begin{aligned} \widetilde{\Theta }_2=\left\{ \left( A_i,\nu _i\right) | A_i\in \mathfrak {A} \text { and } \nu _i\in [0,1] \right\} , \end{aligned}$$

in which the bigger value of $\nu _i$, means, the decision maker’s more emphasis on the closeness of the value of $\sum _{j=1}^{n} w_j \zeta _{ij}$ to $\zeta _{i1}$. Notice the degree of preferences can also be defined as such as Likert scale (Likert, 1932), Saaty’s spectrum (Sáa et al., 2015), and so on.

4.3 The L-p metric for derivation of the weight in OWA operators

To model the problem in approach L-p, we are considering the following goals, simultaneously:

• 1. Maximum Entropy of $w_1,\ldots ,w_n$ must be achieved.

• 2. The maximization of $\sum _{j=1}^{n}w_j\zeta _{ij}$ for each $i=1,\ldots ,m$.

• 3. The minimization of $\frac{\sum _{j=1}^{n}w_j\zeta _{tj}}{\sum _{j=1}^{n}w_j\zeta _{sj}}$ for each $\left( (A_s,A_t),\wp _{(s,t)}\right) \in \widetilde{\Theta }_1$.

• 4. Considering the decision maker’s risk level.

Therefore, for any fixed value of $0\le \alpha \le 1$, L-p metric approach to evaluate $w_1,\ldots ,w_n$ is formulated as follows:

$$\begin{aligned} \min _{\textbf{w}}{} & {} \left[ \delta _1\left( 1+\frac{1}{\log (n)}\sum _{j=1}^{n}w_j\log (w_j) \right) ^p + \delta _2\sum _{(A_s,A_t)\in \widetilde{\Theta }_1}\wp _{(s,t)}\left( \frac{\sum _{j=1}^{n}w_j\zeta _{tj}}{\sum _{j=1}^{n}w_j\zeta _{sj}}\right) ^p \right. \nonumber \\{} & {} \qquad \left. + \delta _3 \sum _{A_i\in \widetilde{\Theta }_2}\nu _i\left( \frac{\zeta _{i1}-\sum _{j=1}^{n}w_j\zeta _{ij}}{\zeta _{i1}-\zeta _{in}}\right) ^p\right] ^{\frac{1}{p}}, \nonumber \\ \text {s.t.}{} & {} orness(\textbf{w})=\sum _{j=1}^{n}\frac{n-j}{n-1}w_j=\alpha ,\nonumber \\{} & {} w_j\in [0,1] \text {~for~} j=1,\ldots , n, \qquad \sum _{n=1}^{n}w_j=1,\nonumber \\{} & {} \delta _l\in [0,1] \text {~for~} l=1,2,3, \qquad \delta _1+\delta _2+\delta _3=1,\nonumber \\{} & {} p \ge 1, ~ \nu _i\in [0,1] \text {~for~} i=1,\ldots , m. \end{aligned}$$

(2)

Remark 2

If $\textbf{w}^*=[w_1^*,\ldots ,w_n^*]$ are the optimal values for $\textbf{w}=[w_1,\ldots ,w_n]$, then $w_j^{'*}=\frac{e_jw_j^*}{\textbf{e}\textbf{w}^{*t}}$, for $j=1,\ldots ,n$ is the modified weight value of the optimal weight values $w_j^*$ by the entropy weight vector $\textbf{e}$.

Remark 3

Note that $\widetilde{\Theta }_1$ and $\widetilde{\Theta }_2$ are closely related. In fact, if $\Theta =\mathfrak {A}\times \mathfrak {A}$ and Saaty spectrum are considered, and we set $\varvec{\wp }=[\wp _{(i,j)}]_{m\times m}$, then

$$\begin{aligned} \lim _{k\rightarrow \infty } \frac{\varvec{\wp }^k\textbf{I}}{\textbf{I}^t\varvec{\wp }^k\textbf{I}} = \varvec{\nu }, \end{aligned}$$

where $\varvec{\nu }=[\nu _1,\ldots ,\nu _m]^t_{m\times 1}$, and $\textbf{I}=[1,\ldots ,1]^t_{1\times m}$.

Remark 4

(1) If in the above optimization problem, $\delta _1=1$ and $p=1$ then the proposed method coincides with the OWA operator proposed by Yager (1988).

(2) Ahn (2008)’s approach is not applicable when $\Theta =\emptyset$.

(3) If in the above optimization problem, $\delta _2=0$, $p=1$ and $\wp _{(s,t)}=1$ for each $(A_s,A_t)\in \widetilde{\Theta }_1$, then the proposed method coincides with the method proposed by Ahn (2008).

(4) We can consider the decision-level risk deviation from $\alpha$ as a goal instead of a constraint in the optimization problem. This makes it possible to achieve optimal weights for an acceptable amount of tolerance of orness value. So, it can be used when the decision maker may consider the amount of tolerance about orness into the optimization problem. Therefore, the optimization problem (4) can be rewritten as

$$\begin{aligned} \min _{\textbf{w}}{} & {} \left[ \delta _1\left( 1+\frac{1}{\log (n)}\sum _{j=1}^{n}w_j\log (w_j) \right) ^p +\,\delta _2\sum _{(A_s,A_t)\in \widetilde{\Theta }_1}\wp _{(s,t)}\left( \frac{\sum _{j=1}^{n}w_j\zeta _{tj}}{\sum _{j=1}^{n}w_j\zeta _{sj}}\right) ^p \right. \nonumber \\{} & {} \qquad \left. +\, \delta _3 \sum _{A_i\in \widetilde{\Theta }_2}\nu _i\left( \frac{\zeta _{i1}-\sum _{j=1}^{n}w_j\zeta _{ij}}{\zeta _{i1}-\zeta _{in}}\right) ^p\right] +\delta _4 \left| orness(\textbf{w})-\alpha \right| ^p \Big ]^{\frac{1}{p}},\nonumber \\ \text {s.t.}{} & {} w_j\in [0,1] \text {~for~} j=1,\ldots , n, \qquad \sum _{i=1}^{n}w_j=1,\nonumber \\{} & {} \qquad \delta _l\in [0,1] \text {~for~} l=1,\ldots ,4, \qquad \sum _{l=1}^{4}\delta _l=1.\nonumber \\{} & {} \qquad p \ge 1, ~ \nu _i\in [0,1] \text {~for~} i=1,\ldots , m. \end{aligned}$$

(3)

5) The values of p, $\delta _1,\ldots ,\delta _4$, and $\nu _1,\ldots ,\nu _m$ can be priori determined by the decision maker.

Remark 5

Any decision makers paired judgment on the alternatives such as $(A_s,A_t)\in \Theta$ can be appeared as the constraint $\sum _{j=1}^{n}w_j\zeta _{tj}\le \sum _{j=1}^{n}w_j\zeta _{sj}$ in the model. This point of view leads to the following optimization

$$\begin{aligned} \min _{\textbf{w}}{} & {} \left[ \delta \left( 1+\frac{1}{\log (n)}\sum _{j=1}^{n}w_j\log (w_j)\right) ^p+(1-\delta )\sum _{A_i\in \widetilde{\Theta }_2}\nu _i\left( \frac{\zeta _{i}^+-\sum _{j=1}^{n}w_j\zeta _{ij}}{\zeta _{i1}-\zeta _{in}}\right) ^p\right] \nonumber \\ \text {s.t.}{} & {} orness(\textbf{w})=\sum _{j=1}^{n}\frac{n-j}{n-1}w_j=\alpha ,\nonumber \\{} & {} \sum _{j=1}^{n}w_j\zeta _{tj}<\sum _{j=1}^{n}w_j\zeta _{sj}, \quad \forall (A_s,A_t)\in \Theta ,\nonumber \\{} & {} w_j\in [0,1] \text {~for~} j=1,\ldots , n, \qquad \sum _{j=1}^{n}w_j=1,\nonumber \\{} & {} \delta \in [0,1], ~ p\ge 1,~\nu _i\in [0,1] \text {~for~} i=1,\ldots , m. \end{aligned}$$

(4)

Remark 6

Instead of the maximal entropy objective function in the above optimization problem, the other objective functions given in Remark 1 can be considered as well.

4.4 The goal programming (GP) approach for derivation of the weight in OWA operators

Considering the decision matrix $\textbf{D}=[d_{ij}]_{m\times n}$, the GP approach for derivation of the weights is formulated as follows

$$\begin{aligned} \min _{\textbf{w}}{} & {} \left( \delta _D D+ \delta _\alpha ^+d_\alpha ^+ + \delta _\alpha ^- d_\alpha ^- +\sum _{(A_s,A_t)\in \widetilde{\Theta }_1}\wp _{(s,t)}d^+_{(s,t)} +\sum _{A_i\in \widetilde{\Theta }_2} \nu _i d^+_i\right) ,\nonumber \\ \text {s.t.}{} & {} D-\frac{1}{\log (n)}\sum _{j=1}^{n}w_j\log (w_j)=1,\nonumber \\{} & {} \sum _{j=1}^{n}\left( \frac{n-j}{n-1}w_j\right) + d_\alpha ^- -d_\alpha ^+ =\alpha ,\nonumber \\{} & {} d^+_{(s,t)}+ \sum _{j=1}^{n}w_j\zeta _{sj}\ge \sum _{j=1}^{n}w_j\zeta _{tj}, \quad \forall (A_s, A_t)\in \widetilde{\Theta }_1,\nonumber \\{} & {} d^+_1+ \sum _{j=1}^{n}w_j\zeta _{1j}\ge \zeta _{11},\nonumber \\{} & {} \vdots \nonumber \\{} & {} d^+_m+ \sum _{j=1}^{n}w_j\zeta _{mj}\ge \zeta _{m1},\nonumber \\{} & {} {\delta _D, D, \delta _\alpha ^+, \delta _\alpha ^-, w_j, d_\alpha ^+, d_\alpha ^-, d^+_i, d^+_{(s,t)}\ge 0,} \nonumber \\{} & {} ~~ \forall i=1,\ldots ,m, ~ \forall A_i\in \widetilde{\Theta }_2, ~ \forall (A_s,A_t)\in \widetilde{\Theta }_1, \end{aligned}$$

(5)

where

• 1. D is the slack variable with respect to optimal measuer of entropy (i.e. 1);

• 2. $\delta _D$ is the importance of D;

• 3. $d_\alpha ^+$ and $d_\alpha ^-$ are the slack and surplus variables with respect to orness measure, respectively;

• 4. $\delta _\alpha ^+$ and $\delta _\alpha ^-$ are also the importance of $d_\alpha ^+$ and $d_\alpha ^-$, respectively;

• 5. $d^+_{(s,t)}$ is the slack variable of $\sum _{j=1}^{n}w_j\zeta _{sj}$ in comparison to $\sum _{j=1}^{n}w_j\zeta _{tj}$;

• 6. $d_i^+$ is the slack variable of $\sum _{j=1}^{n}w_j\zeta _{ij}$ in comparison to $\zeta _{i1}$;

• 7. $\nu _i$ is the importance of $d_i^+$.

Remark 7

(1) If $\textbf{w}^*=[w_1^*,\ldots ,w_n^*]$ are the optimal values for $\textbf{w}=[w_1,\ldots ,w_n]$, then $w_j^{'*}=\frac{e_jw_j^*}{\textbf{e}\textbf{w}^{*t}}$, for $j=1,\ldots ,n$ is the modified weight value of the optimal weight values $w_j^*$ by the entropy weight vector $\textbf{e}$.

(2) The weights expressed in the objective function (5) can be numerical or they indicate the priority of each summand.

5 Numerical examples

5.1 Example I

Consider the decision matrix given in Table 3. In order to provide a comparison between our introduced method and other approaches, a collection of crisp preferences are taken. Now, by applying the proposed method in the present paper, we obtain the following results.

Let orness $\alpha =0.7$, then:

• (1) By employing Yager OWA operator (1) for $n=3$ the weight values are obtained as $w_1^*=0.5539$, $w_2^*=0.2920$, $w_3^*=0.1539$.

• (2) By employing the L-p method given in Eq. (2), with $\delta =p=1$, and $\Theta =\emptyset$ the ordinary results of the Yager OWA operators are obtained as item 1 above.

• (3) By employing the GP method given in Eq. (5), with $w_D=1$, $w_\alpha ^+=w_\alpha ^-=d_\alpha ^+=d_\alpha ^-=0$, and $\widetilde{\Theta }_1=\widetilde{\Theta }_2=\emptyset$ the ordinary results of the Yager OWA operators are obtained as item 1 above.

• (4) If in Eq. (4), $\delta =p=1$ and $\Theta \not =\emptyset$ and without considering the constraint about orness, then the results of Ahn’s approach will be obtained.

• (5) Let $\delta =0.5$, $p=1$ and $\Theta =\emptyset$ then by employing optimization problem given by (4) the weight values are obtained as $w_1^*=0.420$, $w_2^*=0.031$, $w_3^*=0.548$.

• (6) Let $\delta _1=\delta _2=\delta _3$, $p=1$, $\Theta =\{(A_3,A_1),(A_2,A_1)\}$, $\wp (3,1)=\wp (2,1)=1$ then applying model in Eq. (2) the weight values are obtained as $w_1^*=0.137$, $w_2^*=0.074$, $w_3^*=0.788$ for the criteria.

• (7) Let $\widetilde{\Theta }_1=\{(A_3,A_1),(A_2,A_1)\}$ and $\widetilde{\Theta }_2=\emptyset$, $w_D=1$ then applying GP-model (5), we conclude that as $w_1^*=0.508$, $w_2^*=0.098$, $w_3^*=0.392$. Note that $d^+_{(2,1)}=0.1295$ shows that the performance of $A_2$ over $A_1$ can not be satisfied. Note that the results of the weights obtained by our proposed approach depend on the information of the decision matrix, i.e. the criteria weights are influenced by the decision matrix. Let the decision matrix given in Table 3 be replaced by the one given in Table 4, then under the conditions of the item 5 above, i.e. $\delta =0.5$, $p=1$ and $\Theta =\emptyset$, the criteria weight values are obtained as $w_1^*=0.640$, $w_2^*=0.159$, $w_3^*=0.200$.

Table 3 Decision matrix in Example I

Table 4 Decision matrix in Example II

5.2 Example II

In this section, we analyzed an example investigated and designed by Ahn (2008). The decision matrix under consideration consisted of five alternatives characterized by three criteria as shown in Table 5. Further, Ahn (2008) assumed that a decision-maker indicated paired judgments/preferences on the alternatives such as $\Theta = \{(A_2,A_3),(A_2,A_4),(A_3,A_4),(A_4,A_1),(A_1,A_5)\}$.

Table 5 Decision matrix designed by Ahn (2008)

Table 6 Solutions obtained by L-p metric with no preference set of alternatives ($\Theta =\emptyset$) on decision matrix 5

Table 7 Solutions obtained by GP approach and Yager OWA operators, with no preference set of alternatives ($\Theta =\emptyset$) on decision matrix 5

Table 8 Solutions obtained by L-p metric, GP approach and Yager, with preference set of alternatives $\Theta$ on decision matrix 5

Table 9 Solutions obtained by L-p metric, GP approach, with preference set of alternatives given by of Ahn (2008)

Tables 6 and 7 shows the results of L-p metric method for $p = 1,2,\infty$, GP method, and Yager method for $\alpha \in \{$0.0,0.1,...,0.9,1$\}$. If $\alpha$ is known, then among a finite number of values of p’s, a situation with a lower target value than the others is more desirable. For example in Table 8, in which we did the same process for the decision matrix given in Table 5 with the Ahn’ set of preferences $\Theta$, if $\alpha =0.6$ then among $p = 1,2,\infty$, the minimum target value is 0.9349 obtained for $p=\infty$. In fact, we suggest that, assuming $\alpha$ is being fixed, then the optimal value of p can be determined by the L-p method. These calculations are listed in Table 9. For example, in Table 9, for $\alpha = 0.6$ the minimum value of the target function occurs for $p = 555.0$. If the decision maker does not want to include the value of $\alpha$, then according to Eq. (3) with $\delta _4=0$, we will have the results of the first row of Table 9, in which the optimal value for p is 3414.7. If for any reason the decision maker is hesitant to determine the values of $\alpha$ and the values of p, their optimal values can also be assigned based on the optimization model. Also, based on the decision matrix given in Table 5, using L-p method (2) the optimal value for $\alpha$ is obtained as $\widehat{\alpha }=0.707$ and the optimal value for p is obtained as $\widehat{p}=1108.66$. The optimal weight values for $\widehat{\alpha }$ and $\widehat{p}$ are $w^*_1=0.4900$, $w^*_2=0.4343$, $w^*_3=0.0755$.

6 A survey of students’ opinions on teaching activities

The evaluation of the teaching activities of Italian universities by attending students, introduced by law, is a consolidated activity in all universities, which periodically (annually) collect data on the various courses through the administration of questionnaires. In order to make the various universities comparable, the Italian National Agency for the Evaluation of Universities and Research Institutes has defined a common set of questions, in order to guarantee a homogeneous survey on a national scale. The statistical unit of reference is the teaching course. The questions are organized into sections. The sections considered are:

Section 1 Organization of teaching: Opinions are collected on the work required by the teaching course of the questionnaire, on the definition of the methods and rules for taking the exam, on the actual availability of teachers to meet students to provide explanations and clarifications.

Section 2 Teaching activities and study: Opinions are collected on the prior knowledge possessed by the student, the interest aroused and the clarity of the teacher, the usefulness of teaching materials and supplementary teaching activities, as well as the sustainability of the required study load.

Section 3 Interest and satisfaction: Finally, opinions on personal interest in the teaching course and the degree of overall satisfaction with the teaching course are noted.

Student evaluations of the various aspects covered by the survey are expressed through the following rating scale:

• 1. Definitely Yes.

• 2. More yes than no.

• 3. More no than yes.

• 4. Definitely no.

The satisfaction is assessed by comparing the frequency of positive evaluations with the frequency of negative evaluations, where positive evaluations are defined as “More yes than no” and “Definitely yes”, while negative evaluations are defined as “More no than yes” and “Definitely not”.

The percentual frequency of positive and negative evaluations for each question is computed at the teaching course level and at the Academic Programme level (on all the teaching courses of the Academic Programme). 25 Academic Programmes of an Italian university for the academic year 2020/21 are considered. For each question, the percentual frequency of positive evaluations by all the attending students of the Academic Programme is considered. The questions are the following:

Fig. 1

The diagram of the values of the weights obtained by the 48 comparative multi-criteria decision making methods. The shading in the main panel goes from dark red (higher values) to very light yellow (lower values). (Color figure online)

Fig. 2

The dendrogram and diagram of the values of the weights obtained by the 48 comparative multi-criteria decision making methods. The shading in the main panel goes from dark red (higher values) to very light yellow (lower values). (Color figure online)

Section 1 Organization of teaching:

• Q1 Are the examination procedures clearly defined?

• Q2 Are the timetables for lectures, tutorials and other teaching activities respected?

• Q3 Is the teaching staff available for clarification and explanation?

Section 2 Teaching activities and study:

• Q4 Is there sufficient prior knowledge to understand the topics covered?

• Q5 Does the teacher stimulate/motivate interest in the discipline?

• Q6 Does the teacher present the topics clearly?

• Q7 Is the study load required by this course proportionate to the credits assigned?

• Q8 Are the teaching materials (indicated or provided) adequate for the study of the subject?

• Q9 Are the supplementary teaching activities (exercises, laboratories, seminars, etc.) useful for learning?

• Q10 Was the teaching carried out in a manner consistent with what was stated on the course website?

Section 3 Interest and overall satisfaction:

• Q11 Are you interested in the topics covered in the teaching?

• Q12 Are you overall satisfied with the way the course was delivered?

6.1 Computation of competitive methods

In the following, a statistical study will be conducted to make a coherent comparative study between the method proposed in this paper and TOPSIS method, SAW method and OWA operator (Chakraborty, 2022; Yager, 1988), which are well-known and widely used methods in multi-criteria decision making. Notice, there are a variety of multi-criteria decision-making methods that can be employed to rank such the problems. Here Yager OWA with different values of orness $\alpha =0.1,0.2,\ldots ,0.9$, (shown as $OWA(\alpha )$ in the results), our proposed L-p method and GP method with different values of orness $\alpha =0.1,0.2,\ldots ,0.9$ and $p=1,2,\infty$, (shown as L-$p(p,\alpha )$ and GP$(\alpha )$ in the results), as well as our proposed L-p with no value of orness $\alpha$ (shown as L-p), TOPSIS method, SAW method will be employed to rank the Academic Programmes. One of the most important steps in this project was to determine the weights of the criteria. Figures 1 and 2 show the results of approximate weights of the criteria based on the proposed method in this paper along with the results of TOPISIS, SAW and OWA. Light colors indicate smaller amounts of weight values and dark colors indicate larger amounts of weight values. For visual comparison, the figures show the graph values of the weights of the different methods from different perspective to better see their differences and similarities.

Remark 8

In this example, there exists no priority between alternatives, which means that the

Ahn (2008)’s method cannot be used here due to $\Theta =\emptyset$.

Table 10 Decision matrix

Now, using these approximated weights, the alternatives in Table 10 are ranked as depicted in Fig. 3. The shading in the main panel goes from dark red (higher rank values) to very light yellow (lower rank values). In order to sort the alternatives, i.e. Academic Programmes, two methods of average ranks and hierarchical clustering are used. The sorted average ranks of the 48 methods obtained for each Academic Programme are shown in the second column of Table 11. The results show that Academic Programme 9, Academic Programme 18 and Academic Programme 14 have obtained the first top three ranks. Their corresponding preference values as well as their ranks are given in Table 11.

Fig. 3

The diagram of the values of the ranks obtained by different methods. The shading in the main panel goes from dark red (higher values) to very light yellow (lower values). (Color figure online)

Table 11 Mean of ranks of 47 methods as well as the final ranks

The clustering results are shown in Fig. 4. Figure 4 shows the dendrogram of rank values of alternatives. The column dendrogram (top) in Fig. 4 shows the relationship between the various methods in the competitive study. The row dendrogram (on left of diagram) shows the relationship between the various rank values of alternatives, which shows that the Academic Program 9, Academic Program 18, Academic Program 14, Academic Program 21, Academic Program 19, Academic Program 7, and Academic Program 1, get the top ranks by most of the competitive methods. It can also be concluded that the Academic Program 12, Academic Program 13 and Academic Program 20 get the worst ranks by most of the competitive methods (see also Table 11). The other Academic Programs get the middle rank values by of the competitive methods.

6.2 Rank correlation coefficient tests as measures of comparison

As we know both Spearman’s rank correlation coefficient ($\rho$) and Kendall’s rank correlation coefficient ($\tau$) are nonparametric measures of rank correlation (statistical dependence between the rankings of two variables) (Wasserman, 2006). These measures assess how well the relationship between two variables can be described using a monotonic function. The Spearman’s rank correlation coefficient test results as well as Kendall’s rank correlation test results for different comparative methods are shown in Figs. 5, 6, 7 and 8, respectively. The results are testing the following hypotheses

$$\begin{aligned}{} & {} H_0: \text {correlation coefficient is 0} \nonumber \\{} & {} H_1: \text {correlation coefficient is not 0}. \end{aligned}$$

(6)

Fig. 4

The dendrogram and diagram of the values of the different ranks obtained by different methods. The shading in the main panel goes from dark red (higher values) to very light yellow (lower values). (Color figure online)

Fig. 5, shows the diagram of the values of the Spearman Correlation matrix between ranks obtained by different methods. Figure 7, shows the diagram of the values of the Kendall Correlation matrix between ranks obtained by different methods. These figures are constructed from the correlation between the 48 competitive methods used to rank the academic programmes given in Table 10. The big size of the rectangular bars indicate the strength/size of the correlation, i.e. the larger the bar, the higher the absolute correlation between two methods. P-values for testing hypothesis (6) are color-coded in Figs. 6 and 8, for Spearman’s rank correlation coefficient ($\rho$) and Kendall’s rank correlation coefficient ($\tau$), respectively. By these results, it can be concluded that almost everywhere there is a significant correlation between the rank values obtained by the competitive methods.

Fig. 5

The color-coded diagram of the values of the Spearman Correlation matrix between ranks obtained by different methods. The shading in the main panel goes from warm colors (higher values) to cold colors (lower values). (Color figure online)

Fig. 6

The color-coded diagram of the p-values of the Spearman Correlation matrix between ranks obtained by different methods. (Color figure online)

Fig. 7

The color-coded diagram of the values of the Kendall Correlation matrix between ranks obtained by different methods. The shading in the main panel goes from warm colors (higher values) to cold colors (lower values). (Color figure online)

Fig. 8

The color-coded diagram of the p-values of the Spearman Correlation matrix between ranks obtained by different methods. (Color figure online)

Fig. 9

Dendrogram for Single linkage function and Euclidian distance, which involves creating clusters that have a predetermined ordering from top to bottom

Fig. 10

Dendrogram for Complete linkage function and Euclidian distance, which involves creating clusters that have a predetermined ordering from top to bottom

Fig. 11

Dendrogram for Average linkage function and Euclidian distance, which involves creating clusters that have a predetermined ordering from top to bottom

Fig. 12

Dendrogram for Centroid linkage function and Euclidian distance, which involves creating clusters that have a predetermined ordering from top to bottom

6.3 Clustering

Hierarchical clustering is a group of statistical techniques that measure the similarity among a group of entities (Wilkinson & Friendly, 2009). These methods start with the calculation of the distances of each entity from all the other entities in a dataset. Following measures of the distance between entities are typically calculated using between vector variables $\textbf{x}=(x_{1},\ldots ,x_{n})$, and $\textbf{y}=(y_{1},\ldots ,y_{n})$:

• 1. Euclidean distance: $d(\textbf{x},\textbf{y})=\sqrt{\sum _{i=1}^{n}(x_{i}-y_{i})^2}$.

• 2. Manhattan distance: $d(\textbf{x},\textbf{y})=\sum _{i=1}^{n}|x_{i}-y_{i}|$.

• 3. Maximum distance: $d(\textbf{x},\textbf{y})=\max _{i=1,\ldots ,n}|x_{i}-y_{i}|$.

• 4. Correlation distance: $d(\textbf{x},\textbf{y})=1 - \rho (\textbf{x},\textbf{y})$ where $\rho (\textbf{x},\textbf{y})$ is the Spearman or Kendall correlation.

• 5. Absolute Correlation distance: $d(\textbf{x},\textbf{y})=1 - |\rho (\textbf{x},\textbf{y})|$.

R software has a function that computes distances between the columns of matrices and offers many different distance functions (R Core Team, 2017). There are many choices for the linkage function that tells you how to measure the distance between clusters. Given $\textbf{x}$ and $\textbf{y}$ are in the same cluster, some linkage function are as follows:

• 1. Single linkage function: $\min (d(\textbf{x},\textbf{y}))$.

• 2. Complete linkage function: $\max (d(\textbf{x},\textbf{y}))$.

• 3. Average linkage function: $\text {average}(d(\textbf{x},\textbf{y}))$.

• 4. Centroid linkage function: $d(\text {average}({\textbf {X}}),\text {average}({\textbf {Y}}))$ where we take the average over all items in each cluster.

Clusters are then formed usually by a process of agglomeration or division, distance measure and linkage function being fixed. A dendrogram is a tree diagram used to display the groups formed by hierarchical clustering (Wilkinson & Friendly, 2009). Therefore, in order to provide a comparative study, dendrogram is used to show the differences and similarities of the proposed method in contrast to TOPSIS, SAW and OWA methods. Here, the ranks assigned to the alternatives (academic programs) in different 48 methods are the segmentation variables. So when two different methods create similar (or relatively identical) rankings, they are placed in the same cluster. Figures 9, 10, 11 and 12 are dendograms with Euclidian distance and Single linkage function and Average linkage function, Complete linkage function Centroid linkage function, respectively. The other dendrograms are depicted by Figs. 13, 14, 15 and 16, for different linkage functions as well as different distance measures in Appendix A. In spite of different algorithms of these competitive methods, it is seen that our proposed method ultimately produces the same rankings as TOPSIS and SAW, while there is a difference between the rankings of OWA and the others.

7 Conclusion remarks

The approach we proposed in this article is a generalization of the approach investigated by Ahn (2008) which itself is a generalization of OWA operators. The Ahn (2008)’ approach is executable only in case a set of preferences between alternatives is available, also, in practice, there may be contradictions between the elements in the preferred set, but unrecognizable by the model investigated by Ahn (2008). So, we extended the approach from proposed by Ahn (2008) from two points of view: First, whether the preference set is available or not. Second, if there are some contradictions in the elements of preferences set.

• 1. Calculating the percentage of similarity between the top rankings,

• 2. A pairwise comparison between the correlation coefficients between the obtained rankings,

• 3. Clustering of the obtained rankings.

In the above three cases, the similarity and closeness between the methods proposed in this paper with TOPSIS and SAW methods were concluded. The results indicated that there was a significant difference between the OWA method and the others.

Clustering results: dendrogram

The other dendrograms which are depicted by Figs. 13, 14, 15 and 16, are given here for different linkage functions and different distance measures.

Fig. 13

Dendrogram results for Single linkage function and Manhattan distance, Maximum distance, Correlation distance $\rho$, which involves creating clusters that have a predetermined ordering from top to bottom

Fig. 14

Dendrogram results for Complete linkage function and Manhattan distance, Maximum distance, Correlation distance $\rho$, which involves creating clusters that have a predetermined ordering from top to bottom

Fig. 15

Dendrogram results for Average linkage function and Manhattan distance, Maximum distance, Correlation distance $\rho$, which involves creating clusters that have a predetermined ordering from top to bottom

Fig. 16

Dendrogram results for Centroid linkage function and Manhattan distance, Maximum distance, Correlation distance $\rho$, which involves creating clusters that have a predetermined ordering from top to bottom