In an article in this journal, Pereira et al. (2018) use radar plots to communicate the sustainability performance of industrial activities. Their Fig. 1 presents, for three alternatives, the (normalized) score on six impact categories (energy consumption, emissions, toxicity potential, risk potential, material consumption and soil use). This format is taken from BASF’s eco-efficiency plots (Saling et al. 2002). Those authors use it for displaying graphical information, and they could alternatively have used a bar chart or another non-circular presentation. To Pereira et al. (2018), however, the circular aspect is crucial, because these authors add a metric to the diagram, in terms of the area enclosed by the polygon, for each alternative. Through this, they reduce the multi-dimensional problem to a single-dimensional metric, facilitating a ranking of options. That is a convenient result, but the validity of that result clearly depends on the scientific acceptability of the dimensionality reduction procedure. Below we will give two arguments why the approach is wrong, and its application must be rejected.

The first argument is the fact that the area of the polygon depends on the order in which the axes have been introduced. A simple example may demonstrate this. In an analysis in 6-dimensional space, a product with coordinates \(\left(\text{1,0},\text{1,0},\text{1,0}\right)\) defines a polygon with an area \(0\). But if we change the order of the coordinates, such that the product is at \(\left(\text{1,1},\text{1,0},\text{0,0}\right)\), the area is approximately \(0.87\) (Pereira et al. 2018). This dependence on the position of the axes was raised before (Feldman 2013; Albo et al. 2016), but unfortunately it is not yet common knowledge.

The second argument is based on the fact that in the radar plot all dimensions are normalized, such that the highest score is 1. Through this, it is possible that the addition or removal of irrelevant options influences the ranking of the relevant options. An example is given in Table 1, and the corresponding radar plots are in Fig. 1. Also this issue has been discussed before (Arrow 1963). In relation to the fingerprint that is used by Pereira et al. (2018), it has been discussed by Dyckhoff et al. (2015).

Table 1 Example scores for two relevant product alternatives, 1 and 2. Alternative 1 has a smaller polygon area in the radar plot, so it is better than alternative 2. Alternative 3 is irrelevant, because it is inferior on all aspects. If we add it, alternative 2 scores better than alternative 1
Full size table
Fig. 1
figure 1

Left: Radar plot of alternatives 1 and 2, without alternative 3. Right: Radar plot of alternatives 1 and 2, with alternative 3. The presence of irrelevant alternative 3 distorts the shapes of the polygons of alternatives 1 and 2, such that the preference changes

Full size image

There are even more arguments; we refer to earlier studies (Draper et al. 2009; Zhou n.d.) for a critical treatment. Altogether, we dare to doubt the conclusion by Pereira et al. (2018) that “the results have evidenced that the [Eco-efficiency Comparison Index] is a useful tool for eco-efficiency analysis”. Usefulness can not be established without the consideration of methodological consistency.