Introduction

Enterprise performance issue is one of the main concerns in the business area. Therefore, companies make performance analyses to follow their status. In this direction, they develop a performance system including key performance indicators (KPI) for departments. Following critical factor results helps the decision makers to understand strengths and weaknesses. Basu and Wright [1], Letza [8], and Kaplan and Norton [9] discussed that the companies should determine critical KPIs reflecting the departments’ success to measure the accurate company performance. Furthermore, many systems developed to meet the performance evaluation system (PES) requirement such as Strategic Measurement Analysis and Reporting Technique (SMART), the Balanced Scorecard Systems, and Performance Measurement System. These systems use classical evaluation that measure the KPIs and determine the success scores. Next, the total score of the all KPIs accepted as department final success. However, they are not capable of showing overall performance scores, and they ignore department’s differences and do not take KPI’s prioritization issue into consideration [26].

The output of performance evaluation systems (PES) is used as the input of the multi attribute decision-making (MADM) models. To obtain overall performance scores for all departments and a company, various MADM steps implemented. In our study, we have two main requirements, determining of KPI weights and department rankings. KPIs should have exact weight scores. Moreover, department rankings show the success orders and dominancy. In this direction, we searched the literature to satisfy our requirements. As a result, we decided two respective MADM methods to use. Firstly, we use analytical hierarchical process (AHP) because AHP method has a great success to determine rank values according to pairwise comparisons. In this point, performance KPIs considered as attributes in the AHP model to obtain accurate values. On the other hand, we suggest grouping similar KPIs and turning them into a strong critical factor as an attribute of the AHP model. Since fewer attributes give more certain results for AHP method. Secondly, Elimination and Choice Expressing the Reality (ELECTRE) approach applied to rank departments since ELECTRE method based on pair comparison and shows dominancy among departments. At the end, we obtain overall scores that present a reliable and systematic evaluation result. As a result, even if departments have different job missions, overall performance scores give ranks and high-performing or low-performing departments are easily selected. In this paper, we present a case study within a company that uses balance scorecard performance evaluation system to show the implementation of these methods. We show model correctness with the implementing sensitivity analyses and comparisons among other MCDM methods.

With this paper, our major focus is to change performance management viewpoint of the pilot company. We hope other companies will benefit from our proposal and use MADM techniques while they consider performance issues. In the next stages, literature review for MADM methods and related PES presented. After that, AHP and ELECTRE methods introduced to explain the methodology. To illustrate the working mechanism of the developed model, a case study given. Model outputs discussed at the conclusion.

Literature

We covered over 30 research published during the previous 15 years in this chapter. We investigate many facets of the problem and synthesize our findings to establish a solution technique. Sehhat et al. [21] propose ways to enhance PES. In the past, financial measurements and divisions were regarded the most essential variables in determining a company's performance, however, these no longer accurately reflect an organization's performance today. According to the writers, all types of KPIs and departments have a critical influence on a business's success. According to Yurdakul and Ic [26], the existing PES are insufficient to demonstrate a department's or company's overall performance. They assert that using MADM techniques can help assure the evaluation's correctness. The majority of research on department ranking or PES are mainly undertaken in the financial sector. Onder and Hepsen [11] conduct a comparative analysis of two MADM techniques, AHP and TOPSIS. The authors classify 17 banks into three categories (state, private, and international) and utilize the AHP technique to calculate the realized weight of qualities for the TOPSIS strategy. Similarly, Tunay and Akhisar [23] employ the TOPSIS technique to order banks according to the weights assigned to AHP’s criterion and sub-criteria in their work. Percin and Aldalou [13], on the other hand, examine two MADM stages: weighting qualities and rating options. Çağıl [5] uses the ELECTRE technique to conduct a performance study in the finance sector.

Multiple MADM approaches are frequently used to address issues. Rouyendegh et al. [15, 16] demonstrate how AHP and fuzzy TOPSIS approaches may be integrated. Rouyendegh [14] employs fuzzy MADM algorithms to identify high-performing university departments. Although the majority of research employ the AHP and TOPSIS methodologies to tackle ranking issues, Roy [18] developed the ELECTRE method for sorting departments (Benayoun and Bilsberry [2]. Omurbek et al. [10] demonstrate that the AHP-TOPSIS and AHP-ELECTRE approaches are equivalent. Ozdemir [12] solves the supplier selection problem using fuzzy AHP and ELECTRE algorithms. Attribute weights are determined at the first stage and then ranks calculated for the alternatives using ELECTRE method at the second step.

Bouri et al. [4] applies both ELECTRE and PROMETHEE methods for the portfolio selection. Additionally, Rouyendegh and Erkan [17] employ the fuzzy ELECTRE methodology as an MCDM method for selecting academic personnel. Scarelli and Narula [20] appoint referees to football matches using the ELECTRE technique.

Pérez-Domínguez et al. [6] conducted a study on lean manufacturing performance. They noted that businesses employ lean manufacturing but must evaluate its effectiveness. They addressed this issue through the use of MADM algorithms, the Technique for Order Preference by Similarity to the Ideal Solution (TOPSIS), and hesitant fuzzy linguistic term sets (HFLTS).

Şenel et al. [22] conducted more research on PES. They advocate for a comprehensive performance management methodology. Additionally, they employed AHP to score critical performance metrics. Following that, they rate the departments based on their data envelopment analysis. Additionally, they advocate for the clustering of ranking departments.

Table 1 contains a summary of the literature review. Within the table, the materials and studies are summarized per research area. PES and MADM research can be viewed independently or in combination.

Table 1 Literature review
Full size table

Overall, after literature review, we found out that PES should be strengthened using MADM approaches. In this point, AHP is one of the most appropriate method for KPI weight determination. Its primary features are ease of computation, pairwise calculation, and the ability to solicit decision maker input. Additionally, TOPSIS, ELECTRE, and PROMETHEE methodologies, as well as other fuzzy approaches, may be suitable for comparison to identify weak and strong departments. We chose ELECTRE because it provides more straightforward results for each pair of departments. Other MADM techniques can also be used to conduct comparisons in the study.

Methodology

Analytical hierarchy process (AHP)

Saaty's [19] AHP is the most often used approach for determining the best option based on their relative performance criteria. According to Yucel and Gorener [25], the AHP technique quantifies subjective judgments by converting them to quantifiable numeric values based on the component's relative relevance.

The AHP method establishes a framework for decision-making that is hierarchical in nature in terms of decision layers (Erkan and [15, 16]. The decision goal is established at the top of the hierarchy, followed by categorization of qualities and selection of options at the bottom, as seen in Fig. 1.

Fig. 1
figure 1

Hierarchy of the three-level MADM problem

Full size image

Rouyendegh et al. [15, 16] outlines this procedure as follows:

  • A non-structured issue is described.

  • Detailed descriptions of attributes and alternatives.

  • Comparisons between choice factors are made on a pair-by-pair basis.

  • The eigenvalue approach is used to determine the weights of decision elements.

  • The matrix's consistency qualities are found;

  • Weighted decision elements are determined.

Pairwise comparisons are performed on the qualities one by one using Saaty's 1–9 scale as specified in Table 2. They are then gathered in a pairwise comparison matrix after the comparisons. Following that, the eigenvector is utilized to determine the attribute weights. Each column's element is subdivided by column sums. Then, by averaging the row items, the attribute weights are computed. Cheng and Li [7] discuss these computing procedures in their article.

Table 2 Saaty’s 1–9 Scale
Full size table

After calculations, it is needed to measure consistency of the AHP procedure. The next step is to measure consistency of the AHP procedure. Consistency Rate (CR) is the ratio of Consistency Index (CI) to Random Index (RI). If consistency rate is less than 0.1, it is accepted as consistent. Otherwise, the model is needed to be revised. Formulas are given below formula (1) to (3).

$$ {\text{CI }} = (n_{\max } - n)/(n - 1), $$
(1)
$$ {\text{RI }} = \left( {\left( {n - 2} \right) \times 1.98} \right)/n, $$
(2)
$$ {\text{CR}} = {\text{CI}}/{\text{RI}}{.} $$
(3)

After completing the AHP processes, the weights of decision elements are established. These weights are then utilized as inputs in the subsequent stage of calculating departments' overall ranking scores.

Elimination and Choice Expressing the Reality (ELECTRE)

Elimination and Choice Expressing Reality method, abbreviated to as ELECTRE, came up by Roy [18]. The method is based on the concept of ranking by paired comparisons between alternatives on the attributes. Steps of ELECTRE method are given below.

Step 1: we create decision matrices. Columns in the matrix represent the attributes (n), whereas rows represent the alternatives (m). A standard decision matrix is given in 4.

$$ A_{ij} = \left[ {\begin{array}{*{20}l} {a_{11} } &\quad {a_{12} } &\quad {...} &\quad {a_{1n} } \\ {a_{21} } &\quad {a_{22} } &\quad {...} &\quad {a_{2n} } \\ . &\quad {} &\quad {} &\quad . \\ . &\quad {} &\quad {} &\quad . \\ . &\quad {} &\quad {} & \quad . \\ {a_{m1} } &\quad {a_{m2} } &\quad {...} &\quad {a_{mn} } \\ \end{array} } \right] $$
(4)

Step 2: normalized decision matrix are built using the formula given in 5 for non-cost attributes.

$$ x_{ij} = \frac{{a_{ij} }}{{\sqrt {\sum\nolimits_{k = 1}^{m} {a_{kj}^{2} } } }} $$
(5)

After calculation, new matrix (X) is given as;

$$ X_{ij} = \left[ {\begin{array}{*{20}l} {x_{11} } &\quad {x_{12} } &\quad {...} &\quad {x_{1n} } \\ {x_{21} } &\quad {x_{22} } &\quad {...} &\quad {x_{2n} } \\ . &\quad {} &\quad {} &\quad . \\ . &\quad {} &\quad {} &\quad . \\ . &\quad {} &\quad {} &\quad . \\ {x_{m1} } &\quad {x_{m2} } &\quad {...} &\quad {x_{mn} } \\ \end{array} } \right] $$
(6)

Step 3: we generate weighted normalized decision matrices. The Weights obtained from AHP are multiplied by the normalized matrix. The weighted normalized decision matrix (Y) is given in 7. The total of the weights should equal one.

$$ Y_{ij} = \left[ {\begin{array}{*{20}l} {w_{1} x_{11} } &\quad {w_{2} x_{12} } &\quad {...} &\quad {w_{n} x_{1n} } \\ {w_{1} x_{21} } &\quad {w_{2} x_{22} } &\quad {...} &\quad {w_{n} x_{2n} } \\ . &\quad {} &\quad {} &\quad . \\ . &\quad {} &\quad {} &\quad . \\ . &\quad {} &\quad {} &\quad . \\ {w_{1} x_{m1} } &\quad {w_{2} x_{m2} } &\quad {...} &\quad {w_{n} x_{mn} } \\ \end{array} } \right] $$
(7)

Step 4: the concordance and discordance sets are determined.

The weighted normalized decision matrix data for each pair is compared to the data for the other pair. Equal or better alternative of pair means under concordance set. Otherwise, the pair is classified as being in discordance. Formula 8 returns the collection of concordances.

$$ C.C_{kl} = \left\{ {j,y_{kj} \ge y_{lj} } \right\} $$
(8)

Step 5: concordance matrix are calculated.

By summarized the weights, Concordance matrix is generated for concordance set members.

$$ c_{kl} = \sum\limits_{{j \in C_{kl} }} {w_{j} } $$
(9)

Step 6: discordance matrix are calculated.

The discordance matrix is calculated by dividing the maximum gap values of discordance set elements by the maximum gap values of the entire set. Calculations are performed using Formula 10.

$$ d_{kl} = \frac{{\mathop {\max \left| {y_{kj} - \left. {y_{lj} } \right|} \right.}\nolimits_{{j \in D_{kl} }} }}{{\mathop {\max \left| {y_{kj} - \left. {y_{lj} } \right|} \right.}\nolimits_{j} }} $$
(10)

Step 7: outranking Relationships are determined.

When comparing two alternatives, concordance and discordance indices are utilized to determine dominance [3]. If one alternative has a higher concordance index and a lower discordance index than another, the first alternative is superior to the second.

Step 8: ranks are determined.

Using outranking relationships, alternatives are ranked.

A case study

A case study is used to demonstrate how these strategies are used inside a company that utilizes the balance scorecard performance evaluation system. As a pilot, five upper hierarchical departments selected. There are between 13 and 16 key performance indexes (KPI) in total at the scorecards, but similar metrics are combined and under eight KPIs which are attributes of the model. High number of metrics decrease the comparison strength and KPI importance. As a result, the case study applied to rank 5 department with respect to 8 metrics. We do not have permission to share performance metrics information because of the case company requirements. However, we can say that they are related to Finance, Delivery Performance, Non-Conformity Issues, Customer Relations, and Employee Relations generally. All data taken from the case company performance evaluation specialist, and processed on Microsoft EXCEL.

AHP application

First of all, pairwise decision matrix are created using Saaty’s 1–9 scale. Past Balanced scorecard weights of the metrics are compared and pairwise decision matrix values are determined to use in the AHP. In the Table 3, K is the abbreviation of metric.

Table 3 Pairwise comparison matrix
Full size table

AHP procedure is performed and consistency rate is 0.03 which means that model is consistent. Model results are given in the Table 4.

Table 4 Attributes weights obtained by AHP
Full size table

ELECTRE application

At first, decision matrix that is shared at the Table 5 is prepared in terms of a past data. D is the abbreviation of department. Table shows KPI results.

Table 5 ELECTRE decision matrix
Full size table

After some procedure, concordance and discordance matrix are created.

Higher concordance index is calculated as 0.47. On the other hand, lower discordance index value are 0.66. Tables 6, 7, and 8 shows concordance matrix, discordance matrix, and outranking relationships.

Table 6 Concordance matrix
Full size table
Table 7 Discordance matrix
Full size table
Table 8 Outranking relationships
Full size table

When the outranking relationships table are examined, it is seen that D3 (Department 3) is the best alternative and D2, D5, D1 and D4 follows it, respectively.

We can use these results to determine how a department can change its rank. For example, when we analyze the sensitivity of the performance, if we increase 40 percentage (%) of the D5 performance scores, it is observed the dominancy of D2 to D5 removed and the ranking changes as D3, D5, D2, D1, D4.

Another sensitivity analyze was made on Department 1 KPI 2 value. We want to watch if we increase the KPI 2 performance to other successful departments’ results for this KPI how the ranks are differed. The analyze result shows that Department 1 went to rank 3. This finding says that the suggested model also indicates the weak point of department and when they solve the cause of low-performance KPI, then they increase the general performance of department.

Moreover, when we examine the weight value determination stage, applied with AHP, effects on the ranks, we apply ELECTRE step directly. In this direction, we supposed all KPIs have same weight value. Results demonstrate that department success orders completely changed as D2, D5, D1, D3 and D4, respectively. Although the highest performing department D3, it placed at the rank 4. It shows the weight determination stage importance.

On the other hand, when we compare the results, we examine the classical method which is used by company before case study. We calculate the ranks. We shared department KPI results at the Table 5, and KPI weights, which are determined by performance management specialists, are shared at the Table 9. After, when KPI results and weights matrixes multiplication is applied we obtained scores 0.72, 0.83, 0.91, 0.53, 0.78, respectively. Therefore, ranking is obtained as D3, D2, D5, D1, and D4. Ranks shared at the Table 10.

Table 9 Classical method KPI weights obtained
Full size table
Table 10 Suggested method ranks
Full size table

Furthermore, to compare the results we applied TOPSIS and PROMETHEE methods to validate our model. TOPSIS application outputs are shared at the Table 11. When the results are examined, ranking results are same as our proposed solution. Therefore it could be said that the model is valid. Next, PROMETHEE application also prove the model accuracy. For PROMETHEE analyze, criterion V is used. In addition, p and q values are accepted 50% and 1%, respectively. In terms of PROMETHEE ranks, there is a difference order between Department 1 and Department 5. When the results are examined, we can say that small difference occurred. Other comparison results of Department 1 and Department 5 have also close values. Therefore, it does not affect the suggested model accuracy. PROMETHEE results are shared at the Table 12

Table 11 TOPSIS results
Full size table
Table 12 PROMETHEE results
Full size table

All in all, we share all comparisons at the Table 13.

Table 13 Comparison rank results
Full size table

Conclusion and future work

We propose two step methodologies for performance evaluation in this study: AHP and ELECTRE. We present the methodology for companies that makes classical performance evaluation. Instead of only evaluation of metric results, we propose metric rankings and pair comparison among departments in terms of overall success. It also gives the systematic approach that ensures fair evaluation. In the model AHP determine the weight value of performance metrics and then ELECTRE ensures the ranking departments in terms of dominancy relationship.

We indicate a shift in managerial viewpoint in order for decision makers to perceive relative strengths and weaknesses. On the other hand, because these MADM procedures are extremely beneficial for decision makers, we expect that this study will result in an increase in the usage of MADM by companies, particularly case companies.

Sensitivity analyses demonstrate that the model provides showing department’s performance and also their weakness and strengths. Even if they focus on one low-performing KPI, they can replace the department ranks. Additionally, establishing weight values results in a more rational evaluation, as each KPI does not contribute equally to the company's performance.

We used the Classic, TOPSIS, and PROMETHEE techniques in place of the ELECTRE method in the second stage and saw that the ranking results were consistent with the recommended model. It demonstrates the model's accuracy. However, we test these models with a small number of departments in order to demonstrate the model's usefulness.

As a consequence of the study, it can be concluded that the proposed model is suitable for companies with comparable job-focused divisions or departments, such as banks or service organizations. However, when departments have distinct job descriptions, performance review is necessary to ensure that inputs and outputs are comparable. As a result, data envelopment analysis may be utilized to achieve a more precise and equitable solution. On the other hand, the model is only applicable to a limited number of departments due to case limitations. As a result, it may be prudent to examine a large variety of departments. For the future studies, developed model will be applied for bigger cases. A decision support system will be built to aid in the usage of the model by the decision maker.