1 Introduction

Multi-Criteria Decision Analysis (MCDA) is one of the solutions that can be used to assess decision variants under multiple conditions (Marttunen et al. 2017). The flexibility of operation makes the MCDA methods highly applicable to many decision problems from various fields (Colapinto et al. 2017). Decision models support decision-makers in the problems concerning healthcare (Stević et al. 2020), sustainable transportation (Biresselioglu et al. 2018), sport (Qader et al. 2017), management (Moridi et al. 2023), risk management (Gupta et al. 2023), and renewable energy sources (Kumar et al. 2017), among others. The variety of determined solutions shows that they play an important role in information systems, allowing for the selection of rational decision variants. With the application of such systems in decision problems, it is possible to identify the order of alternatives and justify the choice based on the obtained results (Goulart Coelho et al. 2017). Moreover, with a comprehensive assessment approach, these systems prove reliable and robust outcomes based on which decision-makers can rely.

Due to the customized purposes of the decision systems, specialized knowledge regarding particular problems is needed in many situations. For this purpose, domain experts are needed to extract their professional opinion, which is then used as input data represented as criteria weights in the MCDA methods (Fei et al. 2019; Phulara et al. 2024). To gather expert knowledge, subjective weighting methods are used, allowing for establishing criteria weights in a structured manner (Odu 2019). The expert needs to define the relationship between corresponding criteria, and based on that, the criteria weights are established. Techniques like Analytical Hierarchy Process (AHP) (Kumari and Pandey 2020), Best-Worst Method (BWM) (Gigović et al. 2019), and Decision-Making Trial and Evaluation Laboratory (DEMATEL) (Jeong and Ramírez-Gómez 2018) are used, among others. Another way for experts to express their opinions is through preference functions, which are used in the Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) approach (Nassereddine et al. 2019; Amponsah et al. 2012). Belonging functions allow experts to accurately determine the degree to which a criterion is satisfied, allowing a more precise evaluation and comparison of different options. Another option is using fuzzy numbers, which are used in fuzzy normalization (Kizielewicz et al. 2022). An expert can determine the expected value with fuzzy numbers by providing a central (core) value and boundaries, for example, in the form of a triangular fuzzy number. The domain experts are valuable due to their professional knowledge in the given field, which can provide additional information to be considered while handling a particular problem. To this end, in many cases, their involvement is required and highly useful.

However, sometimes involving domain experts in developing a multi-criteria model can prove challenging (Xiao 2018). It is connected mainly to their availability and salary costs. In addition, other problems can relate to establishing criteria weights repeatably and with demanded consistency (Shao et al. 2020). Despite the professional expertise in the given field, expert knowledge can be distorted when using subjective weighting methods (Sodenkamp et al. 2018). Besides, with complex problems, performing criteria comparison is highly time-consuming. Those factors may be the biggest challenges limiting expert-knowledge-driven systems’ potential. To provide more scalable and independent solutions, artificial experts and theoretical decision-makers can be used. The methods from the area of Artificial Intelligence (AI) (Hafezalkotob et al. 2019) and Machine Learning (ML) (Khosravi et al. 2019) algorithms are applied to prepare the solutions trained to compare criteria regarding their importance based on the available knowledge from the problems solved earlier. It allows for employing an artificial expert for the given multi-criteria problems when a domain expert is unavailable. Moreover, the adjustments that are used in the training process enable to obtain more and more efficient and precise solutions that reflect experts’ knowledge.

One of the fields in which domain experts are highly valuable is Sustainable Development Goals (SDG) (Fuso Nerini et al. 2019). With professional knowledge regarding sustainable solutions and experience gained when solving multiple problems with similar characteristics, these experts can provide useful information about the importance of subsequent factors that should be considered while solving particular problems (Purcell et al. 2019). However, it makes systems mainly based on those experts’ knowledge highly dependent on the availability and efficiency of those experts. For this purpose, it is worth studying the possibilities of using artificial experts and theoretical decision-makers in similar decision problems to provide more independent solutions.

In this paper, we propose the INtelligent Characteristic Objects METhod (INCOME) based on the k-Nearest Neighbour (kNN) and Characteristic Object METhod (COMET) methods. The specified approach aims to create a theoretical decision maker, which will then be used to compare the Characteristic Objects (CO) in the COMET method. The main novelty present in this approach is the use of a machine learning technique based on the possibility of non-linear model identification. As has been shown in research (Piegat and Sałabun 2012; Biswas et al. 2024) the form of expert knowledge can be non-linear which is a challenge in MCDA modelling. Combining the kNN algorithm with COMET makes it possible to use the advantage of kNN in terms of adaptation to non-linear patterns in the data, leading to a more precise evaluation of Characteristic Objects. In addition, the model itself is identified from previously assessed alternatives by which there is no need to re-engage a decision expert to identify the model. The crucial aspect of the proposed technique is that the artificial expert is first adjusted based on the comprehensive dataset, making it fluent and reliable in the particular field. To verify the performance of the proposed technique, the INCOME method is used to assess the gas power plants based on four criteria that were taken into consideration. The main contributions of the study are:

  • An approach using the k-nearest neighbors method to identify a continuous decision model

  • A robust framework for assessing power plant performance

  • An approach resistant to Rank-Reversal phenomenon

  • The possibility of replacing the decision-making expert in the process of comparing Characteristic Objects with an artificial expert

The rest of the paper is organized as follows. Section 2 presents an overview of the works related to artificial decision experts and theoretical decision-makers and the importance of multi-criteria methods in the area of sustainable development. Section 3 shows the main assumptions of the kNN and the COMET methods, which are used to build the decision model in the study. Section 4 describes the proposed approach of employing an artificial expert for the multi-criteria decision analysis process. Then, based on the proposed approach, in Sect. 5, the study case is presented, and the practical problem of gas power plants is used to verify the performance of the determined model in the Sustainable Development Goals field. Section 6 presents a discussion of the obtained results. Finally, Sect. 7 includes the summary and indicates the future direction of the research.

2 Literature review

2.1 Expert knowledge in multi-criteria decision analysis

Many real-world decision-making problems are linked to sets of quantifiable criteria, and the information corresponding to them is challenging to acquire in the quantitative form Petrov (2022). Therefore, the extraction of expert knowledge is closely related to aspects of subjective nature based on which the expert has gained experience. For this reason, subjective approaches are used in decision-making processes, which provide mechanisms to convert information from domain experts into numerical values used to determine the relevance of criteria. Several subjective methods, such as:

  • Analytical Hierarchical Process (AHP): A hierarchical decision support method based on a pairwise comparison mechanism (Saaty 1980). The decision maker can use the pairwise comparison mechanism to identify the criterion weights using the Saaty scale in the preference matrix. To ensure the evaluations’ reliability, the pairwise comparisons’ consistency is checked using a consistency coefficient, also known as the Consistency Ratio (CR) (Franek and Kresta 2014). The consistency ratio helps to measure the degree of consistency in the judgments made. The judgments are considered acceptably consistent if the CR is less than 0.1. If the CR is higher, it indicates inconsistency and the decision maker may need to revise the comparisons.

  • Best-Worst Method (BWM): An approach that identifies the criteria weights, which works on the optimization principle (Rezaei 2015). The decision maker in this method determines the preferences of the best criterion relative to other criteria and the worst criterion relative to other criteria. The given preferences become the input data for the optimization through which the criterion weights are determined. With the BWM approach, pairwise comparisons are made only in the two representative vectors associated with the best and worst criteria. This limits the number of pairwise comparisons to 2n-3 comparisons (Liang et al. 2020).

  • Full Consistency Method (FUCOM): An approach based on a pairwise comparison mechanism and linear programming. This technique determines the criteria weights at a certain level of hierarchy while maintaining the consistency of the comparisons (Pamučar et al. 2018). This method reduces the possibility of errors due to the small number of comparisons, subject to the constraints imposed during optimization. Verification of the weights in this method is performed when determining the Deviation from Full Consistency (DFC).

  • Level Based Weight Assessment (LBWA): An approach based on the logic of mutual comparison of criteria, where criteria are grouped according to their level of importance (Žižović and Pamucar 2019). This approach eliminates inconsistencies in expert preferences. The elasticity coefficient of the LBWA model allows, once the criteria have been compared, additional adjustments to the values of the weighting factors depending on the preferences of the decision-makers. This feature of the LBWA model makes it possible to analyse the sensitivity of the MCDA model by examining the impact of changes in the values of the criteria weights on the final decision. The LBWA model determines the criterion weights, which allows for more precise and consistent decision-making.

  • Opinion Weight Criteria Method (OWCM): The approach developed for determining criterion weights uses a Likert scale, which facilitates identifying weights through quantitative analysis (Ahmed et al. 2024). The Likert scale assesses the degree of preference variation for each criterion, enabling a detailed understanding of the decision-maker’s preferences. The OWCM method assigns weights to criteria based on their preference values, ensuring that the sum of the weights is one, which maintains consistency and coherence in the decision-making process.

  • RANking COMparison (RANCOM): A pairwise comparison logic approach using a three-value scale (Więckowski et al. 2023). RANCOM is designed for less experienced experts, is characterized by robustness to inconsistencies in criterion relationships, is intuitive, time-efficient for solving complex problems, and copes with inaccuracies in expert assessments while ensuring high repeatability of results. The method involves defining criterion rankings, creating a MAtrix of ranking Comparison (MAC) based on pairwise evaluations, calculating Summed Criteria Weights (SCW), and deriving final criterion weights.

  • Simple Multi-Attribute Rating Technique (SMART): Multi-criteria decision analysis approach based on evaluating alternatives for each criterion (Edwards 1977). With this method, it is possible to determine criterion weights based on evaluating alternatives on a scale of 0-100. Once the scoring values have been determined using the SMART approach, the weights are calculated using a utility function based on min-max normalization (Siregar et al. 2017).

present the processed expert knowledge as a vector of weights that stores the relevance for the criteria under consideration. The mostly subjective methods use comparison mechanisms to determine the vector of weights; however, there are also other approaches, such as reference point determination. Among examples of methods that use reference points to express subjective expert insights are Characteristic Object METhod (COMET) (Sałabun 2015), Reference Ideal Method (RIM) (Cables et al. 2016), or Stable Preference Ordering Towards Ideal Solution (SPOTIS) (Dezert et al. 2020). The COMET method is based on assigning preference values to characteristic objects, which are reference points. These values are determined by comparing the characteristic objects with each other. In the SPOTIS approach, the expert identifies the most relevant reference point, called the Expected Solution Point (ESP), by which the evaluations of the alternatives are determined using distance. In the RIM approach, on the other hand, a reference point called Reference Ideal is determined, which represents the maximum importance or relevance within a given range. Using this reference point, the alternatives are normalized to consider the expert’s preferences.

Because of the properties of subjective methods, they are often combined with other Multi-Criteria Decision Analysis approaches. Wang et al. (2020) used the AHP-TOPSIS approach to evaluate the security features of IoHT-based devices in a healthcare environment. In the proposed framework for security evaluation, the AHP approach was used to determine attribute weights, which were fed into the TOPSIS method. Through this approach, the security evaluation of alternatives was performed. Pamučar et al. (2021) developed a D-number approach using the Best-Worst method to process the knowledge of four experts and MABAC to evaluate an infectious waste treatment facility. In addition, the paper also used TOPSIS and VIKOR approaches based on D-numbers to validate the results. Badi et al. (2022) proposed the FUCOM-MARCOS approach to establish and evaluate aspects of green innovation. This work considered sustainability performance indicators, where social performance (SPI-3) was chosen as the most sustainable and vital indicator. Prajapati et al. (2019) used the SWARA-WASPAS approach to assess barriers to the implementation of feedback logistics. SWARA was used to assess the relative impact of barriers, while WASPAS was used to prioritize solutions to barriers.

Experts also face challenges related to the uncertainty in the available data. Therefore, approaches are used to extract expert knowledge in problems with uncertainty. In work (Božanić et al. 2024), the authors used the DIBR II method to rank lean organization system methods and techniques in the technical maintenance process. In paper (Biswas et al. 2023), the authors used SPC methodology for group decision-making to determine the criterion weights in evaluating SMEs’ preparation for quality 4.0. In this case, the uncertain data were represented as spherical fuzzy sets. In paper (Radovanović et al. 2023), the authors used subjective approaches such as DIBR or FUCOM to select unmanned aerial vehicles (UAVs) using grey numbers. Expert knowledge is also applied to uncertainty problems in areas such as risk assessment (Zhou et al. 2022), digitalization (Monek and Fischer 2025), transport (Badi et al. 2023), business (Kazemi et al. 2024), and climate (Bouraima et al. 2024).

Expert knowledge can be fundamental in the decision-making process, but sometimes decision experts are not available to support the creation of a decision model. Their lack of availability can be due to several reasons, such as time, geographic or logistical constraints, or confidentiality. In addition, expert knowledge is constantly updated for the sake of the realities of an ever-changing world. Therefore, some identified decision-making models may already need to be updated. In addition, the input data may no longer be available, which raises the problem of rebuilding the decision-making tool (Hafez et al. 2019). Another problem associated with expert knowledge is the difficulty of aggregating the knowledge of multiple experts. There may be a conflict between the knowledge of multiple experts caused by disparities in their opinions due to their motivations, attitudes, or insights (Tang et al. 2020).

One solution to the problem associated with decision-making experts’ availability may be machine learning methods. These techniques are used in many fields, such as healthcare (Letham et al. 2015; Devarakonda et al. 2022), marketing (Moro et al. 2014), credit risk assessment (Baesens et al. 2003), energy (Watróbski et al. 2022; Saidi et al. 2019), and flood vulnerability assessment (Rahman et al. 2019). They provide reliable results and predictions. In particular, machine learning is used in cases where experts are unavailable, and the decision model is challenging to identify due to the non-monotonic nature of the criteria. The preference learning technique is an approach that deals with the possibility of using machine learning mechanisms for expert tasks. It makes it possible to induce predictive preference models from empirical data (Aggarwal and Fallah Tehrani 2019). An example of applying this technique is the diagnosis of thyroid nodules (Fu et al. 2021). There are also other approaches based on machine learning algorithms that are used in decision-making processes. One of them is NN-MCDA, or Neural Network-based Multiple Criteria Decision Aiding, which allows accurate prediction of ratings (Guo et al. 2021). Another approach is PROAFTN, a method belonging to the supervised learning algorithms stream (Al-Obeidat and Belacel 2011). PROAFTN is an approach by which generalizing indicators can determine fuzzy relationships. Another approach is Stochastic Multicriteria Acceptability Analysis (SMAA), through which it is possible to explore the space of weights for multiple decision-makers (Pelissari et al. 2020). Re-identification approaches such as Stochastic Identification of Weights (SITW) (Kizielewicz et al. 2022) or Stochastic Triangular Fuzzy Number (S-TFN) (Kizielewicz and Dobryakova 2023) are also dedicated to dealing with the lack of expert availability. The SITW approach is based on the identification of criterion weights based on the ranking of alternatives. This approach solves an optimization problem using a stochastic approach, finding weights that cause a ranking as close as possible to the reference ranking when combined with a method for evaluating alternatives. The S-TFN approach, on the other hand, determines the cores for the triangular fuzzy numbers, which are used to normalize the decision matrix. These cores are also determined using stochastic optimization techniques and reference ranking. In the context of the lack of a decision expert to identify a multi-criteria model, a subfield of machine learning called Preference Learning (PL) is also used to offer practical solutions (Martyn and Kadziński 2023; Guo et al. 2021). In contrast to traditional Multi-Criteria Decision Analysis (MCDA) methods, PL has minimal or no user interaction. Preference Learning, in combination with MCDA, is used to solve many problems in fields such as medicine (Sobrie 2016) or recruitment (Liu et al. 2023).

Due to the vast possibilities associated with machine learning mechanisms that can act as an artificial expert in the decision-making process, this paper proposes a new approach with the re-identification of the decision model based on the k-Nearest Neighbors (kNN) algorithm and the Characteristic Objects METhod (COMET). Due to the need associated with the evaluation of Characteristic Objects by the domain expert, this method can only evaluate decision options with his knowledge. Therefore, based on the evaluated samples, implementing an artificial expert based on the k-nearest neighbor algorithm was investigated. With this approach, it is possible to re-identify the decision model and update it under new standards based on the characteristic values of random criteria. In addition, this approach can provide much better results because the domain expert, with the high dimensionality of the expert matrix, can make a mistake in comparisons and identify completely different models in subsequent samples. In contrast, an artificial domain expert in the form of kNN based on extracted knowledge will always respond similarly.

2.2 The importance of MCDA in sustainable development goals

Since a significant number of countries in the world today are struggling with problems related to environmental degradation, among others, the United Nations (UN) established the Sustainable Development Goals (SDGs) in 2015. There are 17 established SDG goals related to education, poverty, gender equality, and energy. Their implementation is expected to lead to the stability of countries and the ability to deal with the problems they face. Due to the vital interconnectedness of the established SDG goals, all of them are essential links in the chain of sustainable development (Vlachokostas et al. 2021). The idea of sustainable development is crucial in order to achieve ecological balance or prevent the depletion of natural resources (Sahabuddin and Khan 2021).

An essential part of the SDG chain is the 7th goal of creating more affordable energy for humanity and being reliable and sustainable for everyone. The 7th SDG goal is vital because sustainability assessment related to the energy sector has the highest priority (Khan 2021). Energy development touches many necessary social, economic, and environmental layers (Onat et al. 2020). Therefore, sustainable energy development can improve the quality of human life or economic growth. In addition, introducing sustainable energy development policies significantly mitigates climate change and reduces dependence on fossil fuels (He et al. 2022). For this purpose, new developments related to renewable energy sources, such as wind energy, hydropower, solar energy, and hydroelectric power, are being introduced. Employing renewable energy sources also reduces changes associated with global warming (Chary 2021).

Many decision-making problems related to SDG Goal 7, such as assessing the implementation of sustainable energy policies, often need specialized tools such as MCDM/MCDA approaches (Bhardwaj et al. 2019). Multi-Criteria Decision Analysis (MCDA) was used to evaluate a stand-alone photovoltaic and battery electric system (PV-BES) (Salameh et al. 2022). This problem is strongly linked to sustainability goals due to the need for a continuous power supply. Another example of the application of the MCDA/MCDM approach by Ongpeng et al. (2022) is the AHP-VIKOR approach used for sustainable energy retrofitting of existing buildings. With this study, it is possible to increase the efficiency of retrofitting activities in existing buildings, which will improve the quality of life of residents, as well as protect the environment. Alao et al. (2022) used fuzzy MCDM approaches such as Fuzzy-AHP, Fuzzy-Entropy, and Fuzzy-MULTIMOORA to select the appropriate waste-to-energy (WtE) conversion technology. Waste treatment challenges are a significant problem for many countries. Practical evaluation of waste technologies is essential in achieving the Sustainable Development Goals (SDGs), including minimizing environmental risks and producing cleaner, renewable energy.

Some approaches in multi-criteria problems related to SDG Goal 7 topics require expert knowledge to create a decision model. While these approaches can map the expert’s preferences to some extent, extracting the expert’s knowledge is often done through time-consuming mechanisms. MCDA/MCDM methods such as AHP or COMET work on the principle of pairwise comparison, where the number of pairs increases with the number of criteria and additional parameters in the form of sub-criteria or characteristic values. Therefore, re-identifying continuous decision-making models based on these approaches can be complex. In addition, the resulting model may differ from the reference model due to the continuous adaptation of the expert’s knowledge to the current needs of his field. In addition, due to the dynamically changing ranges of values in which the criteria are located, there is a problem with adapting a model based on old boundaries to current needs.

Therefore, this study will use a k-Nearest Neighbor algorithm to take the role of an artificial expert in an energy-related field. The artificial expert will be learned from evaluated samples, through which the knowledge of the reference expert can be extracted. The possibility of re-identifying a continuous decision-making model will be investigated based on the artificial expert’s knowledge of electric power prediction. The study (Chary 2021) showed that the use of inference based on samples that have already been evaluated once, which relate to the prediction of electric power at full load of the base plant, makes it possible to increase the benefit of available megawatt-hours. This topic is closely related to the 7th Sustainable Development Goal (SDG), making this issue relevant to sustainable energy. Therefore, the study related to identifying a continuous decision-making model for evaluating power plant performance is significant. However, this research can contribute to increasing energy use efficiency and promoting clean energy sources.

3 Preliminaries

3.1 kNN: k-nearest neighbor algorithm

K-Nearest Neighbor algorithm (kNN) are one of the most intuitive machine-learning algorithms in existence. The idea of the algorithm is based on memorizing the entire training data set and then performing regression or classification based on a group of k training objects which are closest to the object at the moment (Wu et al. 2008).

To perform the classification, it is required to define training set D and an object \(x = ({\textbf{x}}^\prime , y^\prime )\). Next, the distance (or similarity) should be calculated between x and all the training objects \(({\textbf{x}}, y) \in D\). This way, we can determine the set \(D_x\) of nearest neighbors for object x. Once \(D_x\) is determined, we can predict the class \(y^\prime \) for object x according to (1):

$$\begin{aligned} y^\prime = \textit{argmax}_v \sum _{({\textbf{x}}_i, y_i) \in D_x} I(v = y_i), \end{aligned}$$
(1)

where v is one of possible class labels, \(y_i\) is the class label for the i-th nearest neighbours computed for object x, and function \(I(\cdot )\) returns 0 if argument is false, e.g. \(v \ne y_i\) or 1 if argument is true, e.g. \(v = y_i\).

To perform regression we can use similar approach. To predict real value \(y\prime \) for object \(x = ({\textbf{x}}^\prime , y^\prime )\), we can use formula (2) to calculate the predicted value as an average value for nearest objects.

$$\begin{aligned} y^\prime = \frac{1}{k} \sum _{({\textbf{x}}_i, y_i) \in D_x} y_i, \end{aligned}$$
(2)

where k is a number of nearest neighbours, \(y_i\) is a value of i-th nearest neighbour for object x.

Equation (2) could be modified to include weights of the nearest neighbors. Weights are usually based on normalized values of distances to whose objects (3).

$$\begin{aligned} y^\prime = \frac{1}{k} \sum _{({\textbf{x}}_i, y_i) \in D_x} w_i \cdot y_i, \end{aligned}$$
(3)

where \(w_i = 1 - \frac{d_i}{\sum _{i=1}^{k} d_i}\) is weight based on distance \(d_i\) between the nearest object \(({\textbf{x}}_i, y_i) \in D_x\) and evaluated object x.

In this paper, we mainly use the kNN regression algorithm, which could also be described with the following Algorithm 1. The algorithm requires several things to be defined: training dataset D, number of nearest neighbors k, distance or similarity function \(dist(\cdot , \cdot )\), which will be used to measure the distance to the new point, as well as aggregation function \(agg(\cdot )\) which will be used to predict regression value for the new object based on k nearest objects. The last thing we need is the object \(x = ({\textbf{x}}^\prime , y^\prime )\), in which value \(y^\prime \) is unknown at the moment and should be predicted. When all data is prepared, we should calculate distances to the new object. In lines 1-4 of pseudo-code, we create an empty array and then fill it with distances between the new object and every other object in the training dataset. In line 5, we define \(D_x\) as a set of k objects with the smallest distance value, and in line 6, we calculate predicted value \(y^\prime \) as the average or weighted average of \(y_i\) values from \(D_x\).

Algorithm 1
figure a

kNN regression algorithm.

Full size image

3.2 The COMET method

The Characteristic Objects Method (COMET) is an approach designed for multi-criteria decision analysis. Its main advantage is its complete resistance to the rank reversal paradox (Sałabun 2015). This method makes it possible to identify a decision-makers preferences by comparing reference points called Characteristic Objects. Characteristic Objects represent the relevant points of the model through which it is possible to evaluate the given decision options. The method also has the advantage of being adaptable to non-monotonic problems (Kizielewicz et al. 2021). The algorithm of the COMET method can be presented in the following steps:

Step 1. Define the Space of the Problem – the expert determines the dimensionality of the problem by selecting the number r of criteria, \(C_1, C_2,\ldots , C_r\). Then, the set of fuzzy numbers for each criterion \(C_i\) is selected (4):

$$\begin{aligned} \begin{array}{l} C_r = \{{\tilde{C}}_{r1}, {\tilde{C}}_{r2},\ldots , {\tilde{C}}_{rc_r}\} \end{array} \end{aligned}$$
(4)

where \(c_1,c_2,\ldots ,c_r\) are numbers of the fuzzy numbers for all criteria.

Step 2. Generate Characteristic Objects – The characteristic objects (CO) are obtained by using the Cartesian Product of fuzzy numbers cores for each criteria as follows (5):

$$\begin{aligned} \begin{array}{l} CO = C(C_1) \times C(C_2) \times \ldots \times C(C_r) \end{array} \end{aligned}$$
(5)

Step 3. Rank the Characteristic Objects – the expert determines the Matrix of Expert Judgment (MEJ). It is a result of pairwise comparison of the COs by the problem expert. The MEJ matrix contains results of comparing characteristic objects by the expert, where \(\alpha _{ij}\) is the result of comparing \(CO_i\) and \(CO_j\) by the expert. The function \(f_{exp}\) denotes the mental function of the expert. It depends solely on the knowledge of the expert and can be presented as (6). Afterwards, the vertical vector of the Summed Judgments (SJ) is obtained as follows (7).

$$\begin{aligned} \alpha _{ij} = \left\{ \begin{array}{ll} 0.0, &{} f_{exp}(CO_i)<f_{exp}(CO_j)\\ 0.5, &{} f_{exp}(CO_i)=f_{exp}(CO_j)\\ 1.0, &{} f_{exp}(CO_i)>f_{exp}(CO_j) \end{array} \right. \end{aligned}$$
(6)
$$ \begin{array}{*{20}l} {SJ_{i} = \sum\limits_{{j = 1}}^{t} {\alpha _{{ij}} } } \hfill \\ \end{array} $$
(7)

Finally, values of preference are approximated for each characteristic object. As a result, the vertical vector P is obtained, where \(i-th\) row contains the approximate value of preference for \(CO_i\).

Step 4. The Rule Base – each characteristic object and value of preference is converted to a fuzzy rule as follows (8):

$$\begin{aligned} IF~ C\left( {\tilde{C}}_{1 i}\right) ~AND~ C\left( {\tilde{C}}_{2 i}\right) ~AND~ \ldots ~THEN~ P_{i} \end{aligned}$$
(8)

In this way, the complete fuzzy rule base is obtained.

Step 5. Inference and Final Ranking – each alternative is presented as a set of crisp numbers (e.g., \(A_i=\{a_{1i}, a_{2i},\ldots , a_{ri}\}\)). This set corresponds to criteria \(C_1, C_2,\ldots , C_r\). Mamdani’s fuzzy inference method is used to compute preference of \(i-th\) alternative. The rule base guarantees that the obtained results are unequivocal. The bijection makes the COMET a completely rank reversal free.

3.3 Ranking similarity coefficients

To compare the effectiveness of MCDA methods, comparing the rankings obtained after evaluating a set of alternatives is useful. For this purpose, ranking similarity coefficients can often be used in the literature to compare the obtained rankings. In our study, we chose to use the weighted Spearman correlation coefficient (\(r_w\)) (Pinto da Costa and Soares 2005), which allows for a comparison of rankings with greater significance for the alternatives rated highest, and the WS rank similarity coefficient (Sałabun and Urbaniak 2020), whose primary assumption is that the positions at the top of the rankings have a more significant impact on similarity. The equations for calculating both coefficients are shown below: equation (9) for the weighted Spearman correlation coefficient and equation (10) for the WS rank similarity coefficient.

$$\begin{aligned}{} & {} r_{w}=1-\frac{6 \cdot \sum _{i=1}^{n}\left( x_{i}-y_{i}\right) ^{2}\left( \left( n-x_{i}+1\right) +\left( n-y_{i}+1\right) \right) }{n \cdot \left( n^{3}+n^{2}-n-1\right) } \end{aligned}$$
(9)
$$\begin{aligned}{} & {} W S=1-\sum _{i=1}^{n}\left( 2^{-x_{i}} \frac{\left| x_{i}-y_{i}\right| }{\max \{\left| x_{i}-1\right| ,\left| x_{i}-n\right| \}}\right) \end{aligned}$$
(10)

where \(x_i\) and \(y_i\) denote the sequential positions in the two rankings being compared, and n denotes the number of lengths of the ranking vector.

4 Proposed approach

This paper focuses on the proposed INtelligent Characteristic Objects METhod (INCOME) approach for solving MCDA problems when an expert is unavailable. The approach combines the kNN algorithm with the COMET method, where the kNN algorithm learned to solve a regression problem provides ratings of the Characteristic Objects (COs), which are then compared pairwise. With the kNN algorithm, as opposed to linear regression, it is also possible to capture non-linear trends in the modeling of the decision model. The COMET approach itself has the possibility of non-linear modeling using characteristic objects. The proposed approach can be illustrated in Fig. 1.

Fig. 1
figure 1

Flowchart of the proposed INCOME approach

Full size image

The process starts with initialization, where the MCDA problem, sets of alternatives, kNN hyperparameters, and decision criteria are defined (Step 0). Next, the dataset is split into training and test sets. An artificial expert is created by learning the kNN algorithm on the test set. Once the artificial expert model has been created, validation is carried out (Step 1). Subsequently, a decision model with triangular fuzzy numbers for each criterion is initialized (Step 2). In Step 3, characteristic objects are generated based on these numbers. In Step 4, pairwise comparisons of these objects are performed, a MEJ matrix is created, and estimated preference values are calculated. A rule base is generated based on the characteristic objects (Step 5), which is then used for inference and final classification (Step 6).

Instead of engaging a domain expert in Step 3 of the COMET, it is possible to use collected data to determine the Matrix of Expert Judgement. Instead of the \(f_{exp}\) function, we can use the kNN procedure described in Sect. 3.1 with Algorithm 1. This procedure is described with Algorithm 2. In the first two steps of the COMET, we define the set of t characteristic objects \(CO_i\), which will be evaluated with the kNN regression procedure. In line 1, we define mej as an empty matrix (two-dimensional array). Next, in lines 2-3, we use nested for loops to iterate over all indices in the created matrix. Conditions in lines 4-12 corresponds to formula (6) in Step 3 of COMET algorithm. However, instead of the expert function \(f_{exp}\), we use the \(knn(\cdot )\) procedure described previously. That way, we can evaluate characteristic objects based on collected data and therefore compare them.

Algorithm 2
figure b

Determining MEJ using kNN regression.

Full size image

5 Study case

5.1 Data description

In this paper, we will focus on investigating the proposal of the INCOME method in evaluating the performance of a combined cycle power plant. For research purposes, we used the Combined Cycle Power Plant Data Set datasetFootnote 1 referring to 9568 samples related to a power plant operating at full load. These samples correspond to the states of the power plant, which, in the case of the decision-making process, we can represent by decision options that will be evaluated. The authors of the selected dataset performed statistical and shuffling tests on it, which made it decided not to perform data preprocessing on its own in this study. The criteria under which the decision variants will be considered are ambient temperature (\(C_1\)), ambient pressure (\(C_2\)), relative humidity (\(C_3\)), and outlet vacuum (\(C_4\)). In addition, due to the learning mechanism associated with the INCOME approach, the set used also has an evaluation of power plant states represented by hourly net electricity production (P). An example of the first ten decision variants is presented using Table 1.

Due to the required input data for the INCOME approach, an analysis of the correlation between the alternative values for each criterion and evaluating alternatives was conducted. For criterion one, a high negative correlation was observed between its value and evaluating alternatives. Pearson’s correlation coefficient for this relationship was \(-\)0.94812, indicating a strong negative relationship between these variables. A similar phenomenon was observed for criterion two, where the Pearson correlation value was \(-\)0.86978. Such a negative correlation suggests that the lower the value of criterion two for an alternative, the higher its rating is. A low Pearson correlation was observed between criterion three’s value and evaluating alternatives (0.51842). This result indicates a stronger correlation between these variables, meaning that the value of criterion three only affects the evaluation of alternatives as much as criteria one and two. However, for criterion four, a low correlation with evaluating alternatives was also observed (Pearson’s coefficient was 0.38979). This indicates that the value of criterion four has less influence on evaluating alternatives compared to criteria one and two. Depends between the criteria and the score are presented using Fig. 2.

Table 1 An example of 10 alternatives derived from a set of 9568 alternatives
Full size table
Fig. 2
figure 2

Dependence between target function and values of individual criteria along with selected characteristic values

Full size image

In addition to the relationships between the criteria and the evaluation, the distribution of the values of each criterion was also subjected. With the information about the distribution of the criteria values, it was possible to select the characteristic values needed to initialize the INCOME model. In the case of criterion \(C_1\), in addition to the criteria boundary values [1, 40], the values of [14, 18, 25] were selected. Referring to criterion \(C_2\), in addition to the edge values of [25, 82], [40, 54,68] were selected. For criterion \(C_3\), the characteristic values are [992, 1003, 1011, 1019, 1026, 1035], and for criterion \(C_4\), the characteristic values are [25, 60, 80, 95, 101]. Information related to the selected decision criteria is presented by using Table 2.

Table 2 Description of selected decision-making criteria
Full size table

5.2 Study of model stability

After selecting characteristic values for each criterion, the INCOME model was initialized. Using the selected characteristic values, characteristic objects corresponding to the Cartesian product of the characteristic values of the selected criteria were created. 750 Characteristic Objects were created for comparison in the INCOME model. The characteristic objects in the INCOME approach are responsible for evaluating the selected decision options. In addition, characteristic objects perform a function related to storing extracted expert knowledge. The expert knowledge, on the other hand, is extracted with the help of comparisons of Characteristic Objects among themselves by the expert, who assigns to the given comparison the values 0, 0.5, and 1. The explanation related to comparing characteristic objects is presented in Step 3, related to the operation of the COMET method. Referring to the methods used during the study, they are taken from a library related to multi-criteria decision analysis called pymcdm (Kizielewicz et al. 2023).

For comparisons of Characteristic Objects, the INCOME approach will use a so-called artificial expert, whose role will be assumed by the k-Nearest Neighbor algorithm. This algorithm will be based on reference evaluations of decision options through which it can identify the decision model. This study will focus on investigating the accuracy of the INCOME approach proposal. Therefore, at the outset, the effect of the number of neighbors of the k-Nearest Neighbor algorithm on the accuracy of matching the INCOME model’s rankings with the reference model was examined. The focus was on 5 values related to the number of neighbors for the kNN algorithm, i.e., 1,5,10, and 15. An experiment was set up using a 10-fold cross-validation mechanism for randomly selecting 90% of the alternatives for the learning set and 10% for the testing set. This activity was repeated 5 times for selected numbers of neighbors of the kNN algorithm. The weighted Spearman correlation coefficient was chosen as a measure related to the accuracy of the INCOME approach for this experiment. For one-neighbor neighborhoods, the correlation between rankings is low, while it increases for more significant numbers of neighbors. The best-obtained result for the INCOME approach was the number of 15 neighbors. The determined correlation values of 5 draws and selected numbers of neighbors of the kNN algorithm are presented using Table 3.

Table 3 Values of the \(r_w\) coefficient for 5 random sampling of 10% of the test set for a selected number of kNN neighbors
Full size table

The effect of distance metrics was examined after studying the effect of the number of neighbors in the kNN algorithm. In this study, three classical distance-related metrics were considered, i.e., Manhattan, Euclidean, and Chebyshev. For the study, 1,000 models were created based on 90 % samples from the dataset and 2,3,4,5,10,15 number of kNN neighbors. After the INCOME models were taught, the models were tested on 10% of the samples derived from the dataset. Then the similarity of the created models with selected distance metrics with the reference model was measured on selected created test sets. The similarity was measured using weighted Spearman correlation coefficient \(r_w\).

The Fig. 3 shows the results from this study. For the Chebyshev metric, the mean value of the \(r_w\) coefficient for each test set with the considered number of neighbors is about 0.983, and the standard deviation is relatively low, oscillating around 0.01. The minimum and maximum values indicate that the results are well concentrated around the mean. The Manhattan distance metric has similar characteristics. The mean value of the \(r_w\) coefficient for each test set with the considered number of neighbors is about 0.981, and the standard deviation is slightly larger than for the Chebyshev metric, oscillating around 0.012. The minimum and maximum values indicate that the results concentrated are around the mean. The results of the Euclidean distance metric are similar to the previous two metrics. The mean value of the \(r_w\) coefficient for each test set with the considered number of neighbors is about 0.983, and the standard deviation is similar to the Manhattan metric, oscillating around 0.014.

Based on these results, it can be concluded that for the tested model and dataset, the Chebyshev, Manhattan, and Euclidean distance metrics produce similar results in the context of the \(r_w\) coefficient. The mean values of the measure are high (close to 1), indicating a strong correlation between the created models and the reference model. The low standard deviation suggests the stability of the results. The choice of a particular metric does not significantly impact the prediction results of this model.

Fig. 3
figure 3

Impact of selected distance metrics on similarity with reference model with selected number of INCOME model neighbors

Full size image

5.3 Comparative study

After research on the stability of the proposed INCOME approach, a comparison to existing approaches such as TOPSIS and T-COMET (TOPSIS-COMET) was made. Initially, 10-fold cross-validation was conducted on 10% of the test data. In the case of cross-validation, the weighted Spearman correlation coefficient (\(r_w\)) and the rank similarity coefficient (WS) were used to measure the similarity between the reference model and the INCOME, TOPSIS, and T-COMET models. For this study, equal weights were used in the TOPSIS and the TOPSIS-COMET approaches.

Table 4 presents the results for each part from the 10-fold cross-validation for the INCOME, TOPSIS, and COMET models. The most similar models were obtained using the INCOME approach for the present study. The mean value of the \(r_w\) coefficient (0.96182) and WS (0.98344) indicate a strong correlation between the created models and the reference model. Referring to the TOPSIS approach, the mean value of the coefficient \(r_w\) (0.90767) and WS (0.84082) shows a strong correlation between the created models and the reference model. However, this correlation is lower than that for the INCOME model, which means that INCOME can better adapt to the present problem. The last model considered was TOPSIS-COMET, where the mean value of the \(r_w\) coefficient is 0.90361, and the mean value of the WS coefficient is 0.83115. These values approach the TOPSIS model result, which is logical due to the hybrid combination of the TOPSIS and COMET approaches. The obtained values of the similarity metrics indicate a strong correlation. However, it is generally lower than that of the INCOME approach.

Table 4 10-fold cross-validation of the 10% test data for INCOME, TOPSIS, and T-COMET models
Full size table

After a comparative analysis based on 10-fold cross-validation examined how the models imitate the actual rankings for a smaller number of samples. Models were trained on 90% of the data (the training set) and tested on 10% (the test set). For this purpose, a test was created 1000 times, randomly selecting 5, 10, and 20 alternatives from the test set. The alternatives were ranked, and, using the \(r_w\) and WS coefficient, the similarity of the created rankings with the reference rankings was examined. As in the previous study, equal weights were selected for the TOPSIS and TOPSIS-COMET methods.

Figure 4 shows the distributions of \(r_w\) and WS values for the obtained similarity rankings between the reference model and the INCOME, TOPSIS, and TOPSIS-COMET models for 5 testing alternatives from the test set. Based on the results obtained, it can be seen that for N = 5, the INCOME model has the highest mean \(r_w\) value (0.890610), followed by the TOPSIS-COMET model (0.798944), and then the TOPSIS model (0.785944). The standard deviations for all three models indicate some variability in results, with TOPSIS-COMET having the highest standard deviation. All models’ minimum and maximum values show that the \(r_w\) coefficient ranges from negative values (for TOPSIS-COMET) to perfect correlation (1.000000). Overall, the INCOME model performs best in terms of similarity to the reference model, followed by TOPSIS-COMET and TOPSIS. The WS measure noted that the INCOME model achieves the highest mean value (0.890610) for the WS coefficient. The TOPSIS-COMET model, with a mean value of 0.798944, takes second place, and in third place is the TOPSIS model, with a mean value of 0.785944. Based on the WS coefficient, it can be concluded that INCOME performs best in terms of similarity to the reference model.

Fig. 4
figure 4

Similarity distribution of the reference ranking with the rankings of the resulting models at 5 randomly selected alternatives from a test set of 1000 times

Full size image

Figure 5 illustrates the distributions of \(r_w\) and WS values for the similarity rankings obtained between the reference model and the INCOME, TOPSIS, and TOPSIS-COMET models for the 10 test alternatives from the test set. Analyzing the results on the \(r_w\) coefficient, it can be seen that the INCOME model shows the highest mean value (0.916083) among the three models. This suggests a strong correlation between the INCOME model and the reference model. The TOPSIS-COMET model follows with a mean value of 0.848832, while the TOPSIS model has a slightly lower mean value of 0.834952. The relatively small standard deviations for all three models indicate consistent results. The range of the \(r_w\) coefficient extends from lower values to a perfect correlation of 1.000000, indicating that the models’ rankings are aligned to varying degrees with the reference model. Analysis of quartiles further reveals the distribution of \(r_w\) coefficient values. The 25th percentile for the INCOME model is 0.890909, indicating that at least 25% of the alternatives have an \(r_w\) value higher than this threshold. In addition, the TOPSIS and TOPSIS-COMET models obtain smaller mean values than this threshold, indicating that the INCOME approach can replicate the reference model much better.

The WS coefficient’s mean value for the INCOME model is 0.916083, indicating a strong fit with the reference model. The TOPSIS-COMET model follows with an mean WS value of 0.848832, while the TOPSIS model has a slightly lower mean value of 0.834952. The quartiles show that most of the alternatives in all models have high similarity rankings. The INCOME model shows the highest mean for the similarity coefficient of WS rankings, indicating its ability to capture the characteristics and preferences of the tested alternatives accurately. The TOPSIS-COMET model also performs well, while the TOPSIS model lags slightly behind in similarity ranking.

Fig. 5
figure 5

Similarity distribution of the reference ranking with the rankings of the resulting models at 10 randomly selected alternatives from a test set of 1000 times

Full size image

Figure 6 illustrates the distributions of \(r_w\) and WS values for the similarity rankings obtained between the reference model and the INCOME, TOPSIS, and TOPSIS-COMET models for the 20 test alternatives from the test set. For the INCOME model, the mean \(r_w\) value is 0.937335, indicating a high level of correlation with the reference model’s rankings. The standard deviation is relatively low (0.039384), suggesting consistent and precise results across the tested alternatives. The minimum and maximum values (0.700680 and 0.992410) indicate that the \(r_w\) metric ranges from lower to nearly perfect correlation. The quartiles (25th percentile: 0.923702, 50th percentile: 0.946845, 75th percentile: 0.962916) provide further insight into the distribution of the \(r_w\) metric, showing that the majority of the alternatives have high similarity rankings.

For the TOPSIS model, the mean \(r_w\) value is 0.865253, which is slightly lower than that of the INCOME model. The standard deviation is also relatively low (0.065100), indicating consistent and reliable results. The quartiles (25th percentile: 0.836932, 50th percentile: 0.880845, 75th percentile: 0.910150) suggest a range of \(r_w\) values, with the majority of the alternatives still demonstrating a substantial similarity with the reference model.

Similarly, the TOPSIS-COMET model has a mean \(r_w\) value of 0.871544, which is comparable to that of the TOPSIS model. The standard deviation (0.063839) is also similar, indicating consistent and precise results. The quartiles (25th percentile: 0.843144, 50th percentile: 0.885929, 75th percentile: 0.914894) show a distribution pattern similar to that of the TOPSIS model, with a significant portion of the alternatives exhibiting high similarity rankings.

Fig. 6
figure 6

Similarity distribution of the reference ranking with the rankings of the resulting models at 20 randomly selected alternatives from a test set of 1000 times

Full size image

The INCOME model shows the highest mean value for the similarity coefficient of WS rankings among the three models for the 20 samples from the test set. The resulting mean WS value is 0.937335, and it shows a solid fit to the reference model’s rankings, as indicated by the low standard deviation (0.039384) and quartiles that highlight most of the high similarity rankings. On the other hand, the TOPSIS model has a slightly lower mean WS value of 0.865253 and a relative standard deviation (0.065100). Its quartiles also suggest a significant similarity. Similarly, the TOPSIS-COMET model achieves a mean WS value of 0.871544, with a low standard deviation (0.063839) and quartiles resembling the TOPSIS model. In summary, the INCOME model best captures the characteristics and preferences of the tested alternatives, while the TOPSIS and TOPSIS-COMET models also show substantial similarity in rankings.

6 Discussion

In classical multi-criteria decision analysis methods such as TOPSIS, a significant limitation is observed in identifying a decision model for data, especially reference data. In algorithmic terms, this task is made more difficult because the TOPSIS approach is based on two reference points: a Positive Ideal Solution (PIS) and a Negative Ideal Solution (NIS). It is not easy to model complex functions based on multiple criteria in this situation. These points are determined based on the maximum and minimum values of the normalized decision matrix, which can further complicate the adaptation process as they strongly depend on the occurrence of outliers. The solution with the largest or smallest values is not always the best. Often the optimal value for a given criterion is non-monotonic. For example, the human body, when it cools down too much, can fall into hypothermia, and when the temperature rises too much, there is the danger of hyperthermia. This is a significant challenge in proposing solutions for non-monotonic modeling in the literature.

Among the methods using reference points, one can also identify a group of new approaches such as SPOTIS, RIM, and COMET. Each of these methods has its own limitations and advantages, but from the point of view of the number of reference points used, COMET requires the most information to create them. This involves the curse of dimensionality but, on the other hand, provides the possibility of better modeling of decision functions (Sałabun 2014). The COMET method assigns preference values to reference points to combine the approach with data-based algorithms such as k-Nearest Neighbor (kNN) algorithms. With this combination, the curse of dimensionality that the decision expert is subjected to is eliminated due to the fact that the comparisons themselves are made using the kNN algorithm. This combination uses data-based knowledge instead of knowledge extracted directly from the expert. This combination can model decision functions based only on evaluated decision options. Therefore, it is possible to apply this approach to problems where we have adequate data about the problem being modeled and expert knowledge is unavailable or limited. In addition, the kNN algorithm is used instead of the expert, allowing the possibility to consider more reference points for identifying the decision model. However, using the kNN algorithm is associated with certain limitations, such as selecting an appropriate number of neighbors. The parameter related to the number of neighbors must be chosen carefully and tailored to the problem. Otherwise, it may adversely affect the results obtained.

Depending on the number of Characteristic Objects, the kNN algorithm may be more efficient than the domain expert in identifying the model. With many Characteristic Objects, the domain expert may make mistakes in comparisons. In this study, the number of characteristic objects was 750, which means 280,875 pairs for comparisons in the model. With such a large number of comparisons, the expert could be inefficient. However, it should be noted that the efficiency of comparisons performed by the kNN algorithm mostly depends on the quality of the learning set. With a small amount of data, it will be challenging to identify a decision model comparable to the one that an expert could identify. On the other hand, with fewer comparisons, it may be more efficient to engage an expert to identify a decision model.

Referring to the results related to the case study survey, it can be observed that the considered TOPSIS, TOPSIS-COMET, and INCOME approaches obtain high correlations with the reference model. The TOPSIS approach using limited knowledge can model similar results, as is the TOPSIS-COMET approach. The results of the INCOME approach show a slightly higher similarity with the reference model. However, this fit is related to the decision options evaluated earlier. In contrast, in the case of TOPSIS and TOPSIS-COMET, the evaluation is obtained by considering the variants evaluated. This means that the identification of TOPSIS and TOPSIS-COMET models is limited to knowledge based on the input data, and it is not possible to model the decision function from the knowledge of previously evaluated data. The final result in these two methods depends directly on the values of PIS, NIS, and weights. Changing one of these values can result in a significant distortion of the ranking. In the case of INCOME data, the model is based on all the data provided, making it more stable.

7 Conclusions and future works

This paper proposes the INCOME approach to multi-criteria decision analysis. INCOME is a new approach based on the knowledge extracted from data using the COMET method and the k-nearest neighbor algorithm. Unlike classical approaches such as TOPSIS, the proposed approach can effectively model decision functions based on evaluated samples. Therefore, it is more flexible and can be used when expert knowledge is unavailable. The study observed that INCOME has a slightly better correlation with the referral rankings than the TOPSIS and TOPSIS-COMET approaches. However, in the case of the TOPSIS and TOPSIS-COMET approaches, the possibilities for model identification are limited due to the acquisition of expert knowledge. It is represented only in PIS, NIS values, and criterion weights, where a change in one can result in a distorted ranking. Therefore, using data knowledge and more reference points, the INCOME approach can create a more stable decision-making model. In addition, by testing the proposed approach, its high stability was demonstrated using a 10-fold cross-validation of 10% of the test data.

Future research directions could consider further studies related to the effectiveness of the proposed approach. In addition, research related to the effective selection of parameters for the INCOME approach would need to be considered. Testing the INCOME approach in other decision-making problems, such as sustainable supplier selection and healthcare, would also be appropriate. The INCOME approach would also need to be extended to handle uncertain data represented as fuzzy numbers and their generalization. Given the numerous emerging decision-making models operating on uncertain data (Rana et al. 2023; Tešić and Marinković 2023), this research direction is promising. In addition, it would be appropriate to extend the research directions in terms of the kNN algorithm itself, in which the effectiveness of the classical algorithm with various weight-based variations could be studied. In addition, it would also be appropriate in the future to use the triad algorithm to reduce comparisons of the number of characteristic objects in INCOME (Sałabun et al. 2021).