1 Introduction

Project portfolio selection is the process of selecting a specific number of projects out of a wider pool, taking into consideration the “value” of each project, as well as policy constraints and other limitations like e.g. budget availability, interdependencies etc. (Archer and Ghasemzadeh 1999). Decision-making involves allocation of funding among different sectors and regions to achieve the greatest possible “value”, combining high impact, balanced and efficient distribution of the limited resources. Usually, the “value” of a project is of multidimensional character due to the multiple points of view that should be taken into account. In this context, the evaluation of the projects is performed using multiple criteria evaluated by experts, while the adaptation of the policy constraints for the final portfolio is implemented through appropriate constraints in a Mathematical Programming (MP) model that has as 0–1 decision variables, the acceptance or not of a specific project in the final portfolio. The combination of Multiple Criteria Decision Analysis (MCDA) with MP and more specifically Integer Programming (IP) is often used to address the “portfolio problematique” in MCDA, where a set of alternatives (projects) that fulfill specific constraints must be selected under the prism of multiple criteria (Belton and Stewart 2002). This combination of MCDA and MP is used in many problems of Operations Research that have to do with evaluation and optimization (e.g., see Raed and Khorramshahgol 2020 for Supply Chain or McKenna et al. (2018) for energy projects in small communities.

In project portfolio selection, the combination of the two phases i.e., first evaluation of the projects using multiple criteria and then using this information in the objective function of an integer programming model that includes the policy constraints, has been widely used (Abu-Taleb and Mareschal 1995; Golabi et al. 1981; Mavrotas et al. 2003, 2006, 2008; Salehi et al. 2022). Other methods based also on Mathematical Programming like Goal Programming (Badri et al. 2001; Zanakis et al. 1995) and Data Envelopment Analysis (Cook and Green 2000; Oral et al. 2001) have been implemented for the same problem. More recently, Portfolio Decision Analysis (PDA) i.e., the application of decision analysis to the problem of selecting a subset or portfolio from a large set of alternatives has been developed Salo et al. (2011), along with a version incorporating explicitly the multicriteria character of the problem in Morton et al. (2016). A comprehensive review of recent developments and future prospects in the field of portfolio decision analysis we meet in the article of Liesiö et al. (2021). The article discusses various approaches and techniques used in portfolio decision analysis, including multi-criteria decision analysis, decision trees, optimization, and simulation.

One of the major challenges in project portfolio selection is to deal with the inherent uncertainty of project evaluation. Especially when a multiple criteria evaluation is performed, the importance of criteria must be clearly defined. The subjective character of the importance of criteria provides the “value” of each project with a degree of subjectivity. In order to control the subjectivity in the final portfolio we propose a novel version of the Iterative Trichotomic Approach (ITA) that uses the so-called “converging weights”.

ITA (Mavrotas and Pechak 2013; Mavrotas and Makryvelios 2021) is a method especially designed for project portfolio selection and works iteratively in rounds. As also stated by Marla et al. (2020) finding robust solutions is an iterative and interactive process. In each round, ITA perform multiple optimizations and classifies the projects into three sets (thus the term “Trichotomic”), according to their inclusion or not in the derived optimal portfolios. In the present version of ITA “converging weight” from round to round the subjectivity increases as the importance of criteria needs to be more specific in order to draw conclusions about which projects enter the optimal portfolio of the specific round. In Mavrotas and Makryvelios 2021 the uncertainty was related to the scores of the projects and a Monte Carlo simulation approach was designed to deal with this uncertainty.

The classification in three sets is not new in the literature. Liesio et al. (2007) used a similar approach in the framework of robust portfolio modeling using the concepts of “core” (= sure) projects and “borderline” (= ambiguous) projects. However, the way the projects are assigned to each set is different between the two approaches. Also, Mavrotas and Rozakis (2009) used similar trichotomic concepts in a student selection problem for a post graduate program.

After a predetermined number of decision rounds the final portfolio is derived. The final portfolio provides information about which projects are selected so that the “value” of the final portfolio is maximized and the constraints and limitations are satisfied. However, several times between planning and applying the decision for project selection, the conditions may change. In these situations, it is very useful to have additional information regarding the portfolio and the projects associated with it. Having information about which projects are “stable” regarding fluctuations on the importance of criteria and which projects can be considered as ambiguous or “borderline” is very helpful for the decision maker for the post optimization phase. Without creating and running the model from the beginning, she/he has enough information to draw conclusions about e.g. which projects may exit the final portfolio if the available budget is reduced.

In the proposed version of ITA the “stable” projects are identified as being those that are selected in the final portfolio, independently of the comparative importance of criteria and they are actually the common projects of the portfolios derived from the individual optimizations (without criteria weights). In other words, the “stable” projects are those that objectively belong to the final portfolio, as they do not need the subjective opinion regarding the importance of criteria in order to draw conclusions about their selection or not. On the other hand, the “borderline” projects are those that we need to be very specific regarding the importance of criteria in order to draw conclusions about their entrance or not in the final portfolio.

The current version of ITA actually studies how subjectivity of the weights imposed by the decision maker is reflected in the projects that are selected or rejected from the final portfolio. In relation to the conventional MCDA-IP approach that provides as final output just the selected and rejected projects, without anymore information, the proposed method delivers to the decision much more fruitful information. Although the obtained portfolios of projects derived from the two methods (the conventional MCDA-IP and the proposed ITA version) may differ only slightly, the richness of qualitative information for the selected projects by the latter method is valuable for the decision maker that has additional information regarding the sensitivity of the final portfolio and the projects themselves on the criteria weights.

The rest of the paper is organized as follows: In Sect. 2 we describe the proposed methodology of the novel ITA version. In Sect. 3 we describe the application of the proposed methodology to a project selection problem with 540 R&D projects using real data and decision parameters. In Sect. 4 we discuss the results and finally in Sect. 5 we present the main conclusions of our work.

2 Methodological approach

In project portfolio selection, we usually have two phases: (a) in the first phase we evaluate the projects and (b) in the second phase we apply the policy constraints using an Integer Programming model where the objective function that drives the optimization is the aggregate evaluation of the projects. We study the cases where the evaluation of the projects is performed using multiple criteria analysis and the score of each project is the weighted sum of the project’s performances in each one criterion, where the weights express the importance of each criterion:

$$s_{i} = \mathop \sum \limits_{k = 1}^{K} w_{k} \times p_{ik}$$
(1)

where si is the score of the i-th project, wk is the weight of the k-th criterion (k = 1…K) and pik is the performance of the i-th project to the k-th criterion.

Subsequently, the score of the projects are used as the objective function coefficients in an IP problem that incorporates the policy requirements as constraints, like e.g. segmentation, sectoral, geographical, economical, mutually exclusive, pre-requisites etc.

$$\begin{array}{*{20}l} {\max Z = \mathop \sum \limits_{i = 1}^{P} s_{i} \times X_{i} } \hfill \\ {{\mathbf{X}} \in S} \hfill \\ {X_{i} \in \left\{ {0,1} \right\} } \hfill \\ \end{array}$$
(2)

where Z is the value of the objective function that expresses the aggregate score of the project portfolio, P is the total number of candidate projects, S is the feasible region defined by the constraints, Xi is the binary variable that expresses if the ith project participates in the project portfolio (Xi = 1) or not (Xi = 0) and X is the set of projects.

The Iterative Trichotomic Approach is a method designed to deal with various kinds of uncertainty in project portfolio selection problems. Uncertainty regards the evaluation of the projects, the constraints, the budgets etc. In the present paper, the uncertainty has to do with the multicriteria evaluation of the projects and more specifically with the uncertainty imposed by the weights of the criteria. The iterations start from the greatest possible differentiation of weights and gradually converge to the preferred set of weights. In the meantime, during the iterative process, the decision maker can see which projects are “surely” in or out of the final portfolio, independently of the weights of importance to the criteria. As the weights of criteria become more specific, more projects participate in the final portfolio, until the final round when the criteria weights are fully specified and the final portfolio is fully determined. In other words, with ITA we can draw conclusions about the degree of certainty of the projects in the final portfolio in relation to the unavoidable uncertainty that characterizes criteria weights in a decision-making process.

In Fig. 1, the darker the color of the projects, the more certain we are about their inclusion (green projects) or exclusion (red projects) from the final portfolio. The borderline projects that are either light green or light red are those that we need to be very specific about the criteria weights in order to decide if they are in or out of the final portfolio. On the contrary, for the dark color projects we are more sure, because the decision to let them in or out of the final portfolio is almost independent of the criteria weights as it is explained in the next paragraphs.

Fig. 1
figure 1

Graphical illustration of the ITA method (color figures online)

Full size image

The proposed version of ITA with the “converging weights” is applied as follows: Assume that we have K criteria for evaluating the projects and the assigned weights to the K criteria are w1, w2, … wK Also assume that we use R rounds (r = 1…R) in order to converge from maximum uncertainty with actually no information about the weights, to the final portfolio with a very specific set of weights. In each round r, we use K different sets of weights for calculating the scores of the projects and subsequently calculate the optimal portfolio. We start by considering each criterion separately and every project is evaluated only based on this sole criterion. In other words, for the first round (r = 1) the K sets of weights are:

$$\begin{array}{*{20}l} {\varvec{w}^{{\left( {1,1} \right)}} = {\mkern 1mu} \left( {1,{\mkern 1mu} 0, \ldots ,{\mkern 1mu} 0} \right){\text{ }}} \hfill \\ {\varvec{w}^{{\left( {1,2} \right)}} = {\mkern 1mu} \left( {0,{\mkern 1mu} 1, \ldots ,{\mkern 1mu} 0} \right){\text{ }}} \hfill \\ { \ldots \ldots \ldots \ldots \ldots \ldots \ldots \ldots .} \hfill \\ {\varvec{w}^{{\left( {1,K} \right)}} = {\mkern 1mu} \left( {0,{\mkern 1mu} 0, \ldots ,{\mkern 1mu} 1} \right){\text{ }}} \hfill \\ \end{array}$$
(3)

where w(r,kk) is the set of weights used in rth round for the kkth optimization.

For each set of weights, a project portfolio is calculated. The projects that are present in all K portfolios are the green projects, the projects that are in some of the K portfolios are the gray projects and the projects that are absent from all the portfolios are the red projects. In ITA’s terminology, the green projects are those that eventually selected in the final portfolio, while the red projects are those that are rejected.

In each subsequent round the set of weights is calculated as:

For k = kk (diagonal elements) we have:

$$w_{k}^{{\left( {r,kk} \right)}} = 1 - \frac{{{\text{r}} - 1}}{R - 1} \times (1 - w_{k} )$$
(4)

For k ≠ kk we have:

$$w_{k}^{{\left( {r,kk} \right)}} = \frac{{{\text{r}} - 1}}{R - 1} \times w_{k}$$
(5)

In this way, after R rounds the weights converge to w1, w2, …, wK. For example, if we assume K = 3 criteria with weights w1 = 0.2, w2 = 0.3 and w3 = 0.5 and the number of rounds for the converging weights R = 3 then the weights for the k-criterion in r-round for the kk-optimization are shown in Table 1:

Table 1 The sets of criteria weights in the decision rounds
Full size table

In the final round the criterion weights for the 3 optimizations are identical, as the process leads to full convergence to the original set of weights, which means that in the last round only one optimization is performed. It must be noted that the first round corresponds to a non-compensatory approach as each criterion is considered independently, while the final round, corresponds to a fully compensatory approach with only one optimization using the original set of criteria weights. The “green” projects of the first round can be considered as the objectively chosen projects. As we proceed and we supply more specific criteria weights information, the subjectively chosen projects are emerged.

In the first round, K optimization problems (k = 1…K) are solved that differ only in the objective functions. Combining (1) and (2) we have:

$$\begin{array}{*{20}l} {\max Z_{1.k} = \mathop \sum \limits_{i = 1}^{P} s_{i} \times X_{i} = \mathop \sum \limits_{i = 1}^{P} p_{ik} \times X_{i} } \hfill \\ {{\mathbf{X}} \in S} \hfill \\ {X_{i} \in \left\{ {0,1} \right\} } \hfill \\ \end{array}$$
(6)

From these K optimization problems K optimal portfolios are produced. The projects that are present (Xi = 1) in all K project portfolios are defined as “green” projects. Those projects that are absent (Xi = 0) from all the project portfolios are defined as “red” projects. The remaining projects that are present in some of the K portfolios are defined as “gray” projects and they are actually the subject of the subsequent optimizations. In the subsequent optimizations with the new set of weights the green projects are fixed to Xi = 1 and the red projects are fixed to Xi = 0.

Therefore, the model for the subsequent rounds is model (7):

$$\begin{array}{*{20}l} {\max Z_{r.kk} = \mathop \sum \limits_{i = 1}^{P} s_{i}^{{\left( {r,kk} \right)}} \times X_{i} } \hfill \\ {{\mathbf{X}} \in S} \hfill \\ {X_{i} \in \left\{ {0,1} \right\} } \hfill \\ {X_{i} = 1 \, if\;i \in {\text{green set}}} \hfill \\ {X_{i} = 0 \, if\;i \in {\text{red set}}} \hfill \\ \end{array}$$
(7)

where si(r,kk) is the score of the i-th project if we use the weights wk.(r,kk)

Finally, in the last round, the weights for the K models have been converged to the original set of weights and only one optimization is performed in order to produce the optimal portfolio. In this last optimization we can identify the “borderline” projects which are those that became “green” or “red” in the final round, as also illustrated in Fig. 1. It must be noted that if no new green or red projects are detected in one round, we continue the process using the new, more converged weights, without freezing any 0–1 variable that corresponds to a green or red project.

Conclusively, the additional information acquired with this version of ITA is the degree of certainty of the participation or not of each project to the final portfolio in relation to the unavoidable subjectivity in criteria weights. The degree of certainty is actually associated with the decision round in which a project is either selected or rejected from the final portfolio. A project which is green from the first round declares that it is always selected whatever criterion we consider as more important (an objective or non-compensatory view), therefore we are more certain about its inclusion in the final portfolio. This is illustrated with the dark green projects on the top of the project-column in the last round (rightmost column) of Fig. 1. On the contrary, the light green and light red projects are those that are revealed in the last iteration (“borderline” projects), when the subjective weights of importance in the criteria must be fully determined.

Therefore, the proposed version of ITA, provides additional, fruitful information for the decision maker given that he/she will not have simply a “yes/no” decision for each project (as it would be the outcome of a conventional optimization), but also a degree of certainty in relation to possible fluctuations in the weighting of criteria. This information for the projects’ degree of certainty can be exploited in order to make quick but justified decisions if some projects should exit from the final portfolio due e.g. to alterations in budget or some projects should enter in the final portfolio, without re-running the whole model. The flowchart of the method is depicted in Fig. 2.

Fig. 2
figure 2

The flowchart of the proposed version of ITA (color figures online)

Full size image

3 Application

The described method is applied to an example with real data that refers to the selection of the most appropriate R&D projects submitted for funding by European and National resources to the General Secretariat of Research and Technology (GSRT). The projects’ data, the policy constraints and the preference parameters have been taken from the call of proposals of the funding action "Cooperation 2011" (can be found at: http://www.eyde-etak.gr/central.aspx?sId=119I490I1266I646I491153&JScript=1) of the Operational Programme “Competitiveness and Entrepreneurship” (EPAN-II), of the Partnership Agreement for the Development Framework, for the 2007–2013 programming period operated by the Management and Implementation Authority for Research, Technological Development and Innovation Actions (MIA-RTDI). For the successful implementation of the Single Action "Cooperation 2011", the MIA-RTDI follows the following three steps in absolute priority order as shown in Fig. 3.

Fig. 3
figure 3

Key implementation steps of action “Cooperation 2011” (color figures online)

Full size image

The research proposals for funding were selected following an evaluation procedure initiated by the preliminary audit, in which the proposals are checked for completeness and their formal specifications in accordance with the requirements set out in the call for proposals and the Implementation Guide of the action, the main evaluation and the hierarchy of proposals. In this case study, we focus on the second and third step, thus in the evaluation and selection of the most appropriate research projects for funding.

3.1 Description of the problem

The funding action "Cooperation 2011" aimed to improve competitiveness, business extroversion, and quality of life, strengthen the link between research and production, the interdisciplinary approach, specialise research potential, and international cooperation through networking cooperation with actors from European and other countries. Enterprises and research institutions (Universities, Technological Education Institutes (TEIs), Research Centres, Institutes) are the beneficiaries of the funds. Within the framework of the action, applications for R&D project funding were submitted by individual enterprises, by groups of enterprises and by partnerships of enterprises with research institutions. The main features of the partnerships are presented in the following (Table 2):

Table 2 Features of partnerships of beneficiaries of the “Cooperation 2011” operation
Full size table

Research and development activities (industrial research, experimental development, feasibility studies) as well as actions to promote innovation (patent acquisition/validation/protection, posting of staff from research organizations and knowledge dissemination) and support actions (e.g., participation in trade fairs, consultancy services) are necessary activities for each proposed R&D project. The budget of the action (public expenditure), which will be allocated to the research projects’ beneficiaries, amounts to € 107,900,000. The action is co-financed by the Operational Programme "Competitiveness and Entrepreneurship" (OPCE-II) and by the Regional Operational Programmes (R.O.P.)—to which the 5 Regions of transitional support of the National Strategic Reference Framework (NSRF) 2007—2013 belong and from National Resources.

The allocation of public resources to the beneficiaries is based on: (a) the thematic area (sector) to which the scientific object of the research proposal falls into, (b) the geographical area-region where the beneficiary is established. The research projects are classified to one of the following sectors: (a) Pharmaceutical-Cosmetics, (b) food-beverages, (c) agriculture, fisheries, animal husbandry and biotechnology, (d) chemical processes in the industry, (e) advanced materials, (f) information technology, telecommunications and automation (g) energy, (h) environment, (i) security, and (j) services (health-tourism-finance-transport-primary production, environment and cities). For each sector, there is specific public funding available to the beneficiaries, which is detailed in Table 3. Each project proposal may have beneficiaries from various regions. The regions are classified into the following five categories: (1) EPAN-II (Eastern Macedonia and Thrace, Epirus, Thessaly, North Aegean, Crete, Ionian Islands, Peloponnese), (2) Attica, (3) Central Makedonia, (4) Western Macedonia and (5) Central Greece– “Sterea Ellada”. Each category has a specific funding ceiling which is shown in Table 3.

Table 3 Submitted budget and public funding for the R&D proposals of the funding action “Cooperation 2011” per region and per thematic area
Full size table

The 540 research proposals were submitted by 1,098 funding beneficiaries established in different regions of the Greek Territory. With their research proposals, the beneficiaries requested funding of 422.4 million euros, when the total available public expenditure of the action amounts to 107.8 million euros. As mentioned above, within the framework of the action, research proposals were submitted by beneficiaries that were either groups of enterprises or partnerships of enterprises with research institutions (case of cooperative projects), regardless of whether their members are established in different institutions, i.e., they belong to a different region. In these cases, the total budget of the project is composed of the sum of the individual budgets of each member, but the public funding attributable to each member of the cooperative project (partner) is derived from their region, which makes allocation of public funding to beneficiaries even more complicated. In addition, it should be noted that for the funding of cooperative projects the funds should be sufficient for all potential beneficiaries participating in the application for funding, otherwise the application for funding is proposed to be rejected.

The evaluation of the proposals was carried out by Evaluation Committees by “thematic/sectoral” area, consisting of three (3)—five (5) expert members with research and/or industrial experience related to the subject of the project to be evaluated, as well as the assistance of international reviewers/evaluators. The evaluation examined the relevance of the subject matter of each proposal to the scientific and technological priorities of the call, the completeness of each proposal from the perspective of scientific, technical and economic impact and the activities of the proposed project. The evaluation was comparative and the evaluation criteria for each research proposal were as follows:


Criterion Α: Quality—credibility of the partnership, with a weighting factor of 30%


Criterion Β: Scientific and technical excellence of the proposed project, with a weighting factor of 30%


Criterion C: Contribution to the country's economy and productivity and impact on the operation and progress of the participating enterprises, with a weighting factor of 40%

Each criterion was scored on a scale of 0 to 4. The total score for each proposal is the sum of the scores of the three criteria, multiplied by the corresponding weighting coefficient. Given that the scores are mostly extracted from specific measurable characteristics in each criterion, we assume that the scores of the projects reflect an objective preference although they may include subjective assessments. After evaluation, the funding proposals are classified in tables in descending order per topic and are proposed for funding, in order of priority according to (a) the total score of the funding proposal, (b) the indicative distribution of allocated funds by thematic sector and (c) the available funds for the region, from which the potential beneficiary may draw funds.

3.2 Model building

According to ITA methodology, the decision model is a series of optimization models and more specifically Integer Programming models that iteratively converge to the generation of the final portfolio. The policy constraints are expressed as constraints in the IP model and the objective function is the sum of the multi-criteria scores of the projects. The objective function is given by the following equation:

$$\max Z = \mathop \sum \limits_{i = 1}^{540} s_{i} \times X_{i}$$
(8)

where Z is the value of the objective function that expresses the aggregate score of the project portfolio, si is the multi-criteria score of the i-th project, and Xi is the binary variable that expresses if the i-th project participates in the project portfolio (Xi = 1) or not (Xi = 0).

The score si of the i-th project is calculated as:

$$s_{i} = \mathop \sum \limits_{k = 1}^{3} w_{k} \times p_{ik}$$
(9)

where wk is the weight of importance of the k-th criterion and pik the performance of the i-th project in the k-th criterion.

The constraints of the problem define the sectoral and geographical distribution of the projects:(a) sectoral constraints

$$\mathop \sum \limits_{i = 1}^{P\left( s \right)} b_{i} \times X_{i} \le tb_{s}\; for\; s = 1,...,10$$
(10)

P(s) is the number of projects that belong to the s-sector bi is the budget of the i-th project and tbs is the total budget for s- sector taken from Table 1 (s = 1.0.10).(b) geographical constraints

$$\mathop \sum \limits_{i = 1}^{P\left( r \right)} b_{ir} \times X_{i} \le tb_{r}\; for\; r = 1,...,5$$
(11)

P(r) is the number of projects that have beneficiaries (subprojects) in the rth geographical region, bir is the budget of the of the ith project that is attributed to the r-region through the rth subproject and tbr is the total budget for r- geographical region taken from Table 2 (r = 1.0.5). It is noted that a project proposal may be composed by sub-projects from several regions with a specific budget and beneficiary.

The source of uncertainty in the present case is the weights of importance of the criteria which affect the coefficients of the objective function in Eq. (9). According to the ITA version described in Sect. 2, we proceed iteratively in rounds in order to converge to the final projects’ portfolio. In each round we perform a number of optimizations that equals the number of criteria, which is 3 in the specific case. Moreover, from round to round a number of projects is selected or discarded (green and red projects), a condition which is reflected in the model by fixing the corresponding binary variables either to Xi = 1 or Xi = 0 respectively.

In the present case we use 4 rounds in order to reach the final projects’ portfolio. Given that the weights in the three criteria are w1 = 0.3, w2 = 0.3 and w3 = 0.4, the criteria weights that we use in each round for the 3 models of each round are according to Sect. 2 as shown in Table 4:

Table 4 Criteria weights for the 4 decision rounds
Full size table

Therefore, for the first objective function of the first round, the criteria weights that are used to calculate the multicriteria score of each project si are those presented in the first row, next to Ζ1,1. Accordingly, for the second and the third objective function of the first round. In other words, in the first round, we consider as each criterion being the only one that counts, we calculate the projects’ scores and we perform the optimization. Those projects that appear in the final portfolio in all three optimizations of the first round (Xi = 1 for all three optimizations) are considered as being “stable” in relation to the criteria importance and they are the first “green” projects. No matter which criterion prevails, they are in the final portfolio because their performance in all three criteria is better than their competitors’ performance in the specific decision situation, which means that the policy constraints are also taken into account. On the other hand, the projects that are not present in none of the three portfolios (Xi = 0 for all three optimizations) are considered as inferior under all circumstances and are characterized as the “red” projects of the first round. In the next round the green projects are fixed to Xi = 1 and the red projects to Xi = 0 and actually, in the subsequent rounds, we explore the set of gray projects, that appear in one or two of the final portfolios. Therefore, in the r-th round the three optimization models that are solved are expressed as follows for kk = 1…3:

$$\begin{array}{*{20}l} {\max Z_{r.kk} = \mathop \sum \limits_{i = 1}^{540} s_{i}^{{\left( {r,kk} \right)}} \times X_{i} } \hfill \\ {\mathop \sum \limits_{i = 1}^{P\left( s \right)} b_{i} \times X_{i} \le tb_{s}\; for\; s = 1,...,10 } \hfill \\ {\mathop \sum \limits_{i = 1}^{P\left( r \right)} b_{ir} \times X_{i} \le tb_{r}\; for\; r = 1,...,5 } \hfill \\ \end{array}$$
(12)
$$\begin{array}{*{20}l} {X_{i} \in \left\{ {0,1} \right\}} \hfill \\ {X_{i} = 1 \, if\;i \in {\text{green set}}} \hfill \\ {X_{i} = 0 \, if\;i \in {\text{red set}}} \hfill \\ \end{array}$$

where si(r,kk) is the score of the i-th project if we use the weights wk.(r,kk)

4 Results and discussion

The models that implement ITA were developed in GAMS-General Algebraic Modeling System- platform GAMS (2010) available in https://www.gams.com. They have 540 binary variables (one for every project) and 15 constraints that express the policy constraints from Eqs. (10) and (11). The optimization of the MP model has been contacted with the GUROBI solver provided with GAMS. The runs were made in a machine with core i5—64bit at 2.5 GHz and the solution time for the optimization problems varied from 2.3 to 7.5 s. Regarding the projects, the following table presents the results, per round:

As it is derived from the summation of the projects in the “Green” column, 189 projects were finally selected after round 4. The final output of ITA i.e., the final portfolio with the degree of certainty for the inclusion or exclusion of each project can be depicted in the following graph of Fig. 4. The 540 projects are characterized according to the degree of certainty for their inclusion (green projects) or exclusion (red projects) from the final portfolio. The shading of green and red projects, express the degree of certainty. The darker the color, the more certain we are about the inclusion or exclusion from the final portfolio in relation to the subjectivity in the criteria weights. The light color projects express the “borderline” projects that can change status (in or out of the final portfolio) with slight modifications in the criteria weights.

Fig. 4
figure 4

Graphical representation of the final output of ITA as a colored matrix (color figures online)

Full size image

It must be noted that the solution of the problem without the ITA methodology, using a conventional way of a single step optimization with the original set of weights, provides 190 projects in the final portfolio (the last row of Table 5). However, in the output, there is no information regarding the degree of certainty of the projects that are in or out of the final portfolio. In other words, with the “one-off” decision process used in the conventional approach, we cannot recognize the “stable” projects from the “borderline” projects which is crucial information in the decision making process.

Table 5 Projects classification by round
Full size table

In addition, there are slight variations in the selected and not selected projects. More specifically, the difference between the two approaches is in 9 projects (4 projects are in the final portfolio with ITA which are not included in the final portfolio with the conventional approach and 5 projects are in the final portfolio with the conventional approach that are not included in the final portfolio with ITA). It is worth noticing that the average score of the 4 projects that are included with ITA is 17% higher than the average score of the 5 projects that are included with the conventional approach. However, although the difference in the final set of projects between the two methods is not significant, it is significant the delivered information regarding the projects and the final portfolio in terms of the sensitivity to the weights of criteria (“stable’ and “borderline” projects).

5 Conclusions

Project portfolio selection is a common problem that is dealt with operations research techniques. In the proposed approach, we combine multi-criteria analysis with mathematical programming in order to provide additional, fruitful information. The proposed version of the ITA method overcomes the subjectivity in the importance of criteria, providing also information emerged from the non-compensatory approach. The decision process moves gradually from the “no information about the criteria weights” situation, to the “full information about the criteria weights”, extracting information about the “stability” of the selected or not selected projects. It must be noted that for the time being, the proposed approach has been tested with additive value functions.

The basic idea of the method is that some projects are constantly in the final portfolio (“stable” projects), independently of the criteria weights that are used in the formation of the multicriteria score of the projects that form the objective function and actually drive the optimization. These projects are extracted as those being common in the individual criterion optimizations without any compensation implied by the weights of criteria (non compensatory approach). Subsequently, in the next rounds the amalgamizing of criteria is performed gradually in order to converge to the imposed set of weights. In the final round of the ITA version, the “borderline” projects, which are the more sensitive in variations in criteria weights, are revealed. In other words, the “stable” projects are the objectively chosen projects while the “borderline” projects are the subjectively chosen ones. The information that characterizes a project as “stable” or “borderline” is crucial for the decision maker because he/she can draw conclusions on the projects’ prioritization if the conditions change (available budgets, geographical quotes etc.) without re-running the model. Therefore, the obtained information from the proposed approach is much more fruitful than the information obtained from the conventional MCDA-IP combination used for project portfolio selection where the only output is the selected and rejected projects without any information of the sensitivity of the selected projects and the portfolio itself on the selected weights of criteria.

As it was illustrated with the implementation of the method in the application of 540 R&D projects, the results are fruitful and can be effectively visualized as in Fig. 4. The decision maker can easily recognize the “stable” from the “borderline” projects and use this information in post optimization procedures if fine tuning in the decision process of project portfolio selection is needed.

For future research, we can study more cases where the thrichotomic approach can be implemented in similar decision situations, as for example, when besides the multiple criteria, there are multiple decision makers that express their opinion about the project’s evaluation or the importance of criteria. The gradual convergence to a final consensus and its effect to the selected or discarded projects can be essential information in a decision or negotiation process.