Özturk, Tsoukiàs, and Guerrand address an evaluation problem in the context of an important activity in many organizations: procurement. This chapter starts by presenting the problem definition and formulation stages (see Chap. 2), and then describes the definition and application of an aggregation procedure to a particular subproblem within a large project. This work illustrates the sorting problematic based on a variant of the ELECTRE TRI method (see Chap. 4, Sect. 4.4.4). In this case, ELECTRE TRI is used to evaluate alternatives according to a specific axis of evaluation, corresponding to one high-level criterion among other criteria. It therefore describes how the evaluation of a single (top-level) criterion can originate further aggregation problems concerning different aspects that need to be taken into account concerning that criterion, which in turn can be further decomposed in a hierarchical structure.
The French railways company SNCF, the client of this study, needs to evaluate the comfort of trains not only to select suppliers but also to define specifications in calls for tenders. The objective of the intervention was to improve a simplistic evaluation method used by the client, using comfort evaluation as a case study to demonstrate the value of a thorough MCDA study.
The decision process involved directly many actors: experts in decision aiding (the analysts), an expert in psycholinguistics, and company experts from the comfort and acquisition departments. Actors involved indirectly were the train passengers, whose voice was heard.
Several stages can be identified in this intervention: analysis of passenger survey data, definition of the evaluation criteria, definition of parameter values of the model, and application of the model. Among different dimensions of comfort, this chapter focusses on the passenger experience on the train. The hierarchy of criteria was developed based on a content analysis of passenger survey answers about what they value in terms of comfort, as well as on previous knowledge of the client organization. There were five top-level criteria which are subdivided into many more elementary attributes of the train. It is interesting to note that the same type of content analysis was used to define the criteria weights, under the assumption that aspects mentioned more often would correspond to higher importance for the passengers, instead of following a typical elicitation process. The analysts deemed these weights were acceptable and they set the remaining parameters of the evaluation model. This weight elicitation could be improved by performing new surveys based on the final list of criteria, possibly based on choosing among alternatives as often is the case in transportation research (Hensher 1994; Louviere 1988). Nevertheless, in their conclusions the authors warn that an implementation of this tool will require dealing with divergent opinions and performing sensitivity analysis.
The authors opted for a sorting problem statement because they intended to evaluate the intrinsic merit of potential offers in a call for tenders from the comfort viewpoint. This means they did not intend to select the most comfortable option, or to rank the options in terms of comfort. Actually, the chosen method, ELECTRE TRI, was used as a means to obtain a qualitative scale (this is a common use for ELECTRE TRI, see e.g., André and Roy 2007). ELECTRE TRI was the chosen aggregation method, using a simplified outranking relation which is slightly different from the one used in Chap. 15 in this book. The choice of ELECTRE TRI, besides fitting the sorting problem statement, is also justified for being a noncompensatory method, not requiring the definition of substitution rates among the criteria. ELECTRE TRI also presents the advantage of allowing the use of veto thresholds. Such thresholds can be used to prevent that an offer which is very bad in one of the subcriteria reaches a high category, which is a quite realistic requirement when we are dealing with comfort assessment. For most lower levels in the hierarchy, ELECTRE TRI was again used to aggregate subcriteria, but in some simpler cases a weighted sum was used.
As an illustration, the authors evaluate a set of alternatives consisting of three fictitious offers (confidentiality agreements do not allow presenting the true alternatives). These examples are characterized by a list of their characteristics in the comfort-related attributes, and are then sorted into their respective categories. Such categories correspond to qualitative grades that can be taken into account for a global evaluation of each offer considering other dimensions besides comfort.
The tangible results of this decision process were the definition of a criteria hierarchy and the characterization of an ELECTRE TRI sorting model—using inputs from the passengers—for building a global comfort scale. Concerning intangible results, the client understood the methodology as being useful, wishing to use it again in the future, and to extend it to other evaluation problems.
As for the relevance of this chapter, it demonstrates how the evaluation of alternatives under a single criterion (in this case comfort) can be in itself a complex MCDA problem. It also illustrates that the list of criteria does not necessarily have to be elicited from the client. Other stakeholders, in this case the train passengers, can be the source of the criteria list and contribute to the inclusion of aspects that might otherwise not be valued.