Measuring efficiency under fixed proportion technologies

Abstract

Data Envelopment Analysis (DEA) applications frequently involve nonsubstitutable inputs and nonsubstitutable outputs (that is, fixed proportion technologies). However, DEA theory requires substitutability. In this paper, we illustrate the consequences of nonsubstitutability on DEA efficiency estimates, and we develop new efficiency indicators that are similar to those of conventional DEA models except that they require nonsubstitutability. Then, using simulated and real-world datasets that encompass fixed proportion technologies, we compare DEA efficiency estimates with those of the new indicators. The examples demonstrate that DEA efficiency estimates are biased when inputs and outputs are nonsubstitutable. The degree of bias varies considerably among Decision Making Units, resulting in substantial differences in efficiency rankings between DEA and the new measures. And, over 90% of the units that DEA identifies as efficient are, in truth, not efficient. We conclude that when inputs and outputs are not substituted for either technological or other reasons, conventional DEA models should be replaced with models that account for nonsubstitutability.

Appendix: Identifying fixed vs. variable proportion relationships

A rigorous, comprehensive examination of statistical methodologies for identifying fixed vs. variable proportion relationships is far beyond the scope of this paper, but a brief discussion of some of the issues may be worthwhile.

If variables are nonsubstitutable because of the technology, then no further evidence is necessary and models accounting for their fixed proportions must be used. However, inputs (outputs) that are technologically substitutable may, in practice, not be substituted for each other, for reasons such as those identified in the opening paragraphs of this paper. Reasons for nonsubstitutability in a dataset are immaterial for deciding whether fixed or variable proportion efficiency models should be used.

If fixed proportion relationships are not mandated by the technology, then empirical evidence supporting the hypothesized relationship could be important. The Olesen and Petersen 2009 JPA paper illustrates the problem. Although reading and math score outputs could be substitutes as input resources are shifted from one to the other, they are not substituted for each other in Olsen and Petersen’s dataset.

Recall that different measures of inputs, outputs and environmental factors might affect the results, as might adjusting for endogeneity and other statistical concerns. However, it is not the true population relationships that we seek to identify, but the actual relationships in the data being used for the DEA. DEA estimates are based on the extant data’s characteristics and relationships, and it is for these relationships that the presence or absence of substitutability must be determined.

Econometricians long ago developed sophisticated methods for measuring elasticity between inputs and between outputs (Fuss 1977; Fuss and McFadden 1978), and such methods may sometimes be desirable. Herein we only illustrate some patterns that might be observed, using artificial datasets in which two inputs and one constant output illustrate either fixed or variable proportion relationships.

If any two inputs (outputs) are substitutes, their relationship must be negative when all remaining DEA inputs and outputs are held constant. That is, if output is constant for all observations, the relationship between the inputs must be negative if they are substitutes (Beattie and Taylor 1985). If a relationship between two inputs (outputs) is not negative, then the variables in question have not been substituted for each other, and valid DEA scores cannot be estimated.

Because DEA constructs the production frontier by forming convex combinations of extreme observations, the DEA frontier will always exhibit a negative slope between inputs (outputs), whether the variables are substitutable or nonsubstitutable. Further, negative relationships on the DEA frontier would be present if the DMUs forming it exhibit different degrees of efficiency for each variable. For example, suppose that in truth a fixed ratio of 10 units of Input A and 5 units of Input B are necessary to efficiently produce one unit of output. If one DMU used 11 units of the first input and 5 units of the second, and another DMU used 10 units of the first input and 6 of the second, then both would be on the DEA frontier and would exhibit a negative relationship between the inputs. In short, negative relationships between variables on the frontier provide no evidence whatsoever concerning their substitutability.

It is sometimes assumed that relations among variables on the production frontier differ from their relations within the production possibility set. So, for example, for a fixed amount of output, it sometimes is contended that even if labor and capital are, in truth, substituted for each other on the frontier, they might not exhibit a negative relationship elsewhere in the production space. If this surmise were true, we would find negative relationships between inputs (outputs) if only efficient DMUs are analyzed, but not if inefficient DMUs are analyzed.

Unfortunately, determining whether a frontier technology in truth differs from the technology used by inefficient units is a very difficult statistical problem. As has been observed by Schmidt, “I have tried (and failed) to find a reasonable specification which would allow one to say how features of the technology vary with distance from the frontier” (Schmidt 1985-86, p. 322). Even if a difference does exist in some datasets, assuming that it is present for a particular dataset would be problematic.

In estimating elasticity among inputs or outputs, econometricians, including those employing stochastic frontier analysis, use all observations in the data set, making the assumption that even inefficient units will yield similar relationships among inputs (outputs). Indeed, we asked Schmidt that, if a sample as a whole shows no substitutability between two inputs, would it be reasonable to assume that little or no substitutability between these inputs exists on the frontier. He replied that, at an intuitive level, the answer should be yes (email from Peter Schmidt, 6 June 2010).

Further, it seems to us that even firms inefficiently using substitutable inputs (for example) would adjust the input ratio if the price ratio between the inputs changed significantly. To believe otherwise would mean that inefficient DMUs completely ignore economics in making decisions, which seems unlikely to the extreme. Further, even if the elasticity of substitution was unequal on the frontier and in the interior of the production space, it again seems unlikely to the extreme that elasticity would change directions, going from negative to either zero or positive, rather than just becoming weaker or stronger in the interior as compared to the frontier.

We illustrate possible outcomes with artificial data for two inputs, assuming other inputs and all outputs have been held constant, so only the relationship between the two inputs remains. For substitutable inputs, Fig. 6 shows efficient DMUs, and the resulting piecewise isoquant, which we use in the simulation.

Fig. 6

Variable factor proportion technology, frontier DMUs

We simulate inefficiencies in using each of the two inputs for 200 DMUs, randomly generating percentage inefficiencies from a half-normal distribution with a mean and a standard deviation of 1. This means that a third of the inputs will be between double and triple the efficient amount, with about a 1% chance that an input will be more than 3 times the efficient amount. It is unlikely that such substantial proportions of DMUs would use inputs so inefficiently, but this will make the resulting graphic results more credible. Further, we used the exact same set of percentage inefficiencies for both the substitutable input scenario and the nonsubstitutable input scenario, so any differences observed would not be related to the magnitude of the inefficiencies involved.

For the substitutable inputs scenario, we randomly generated 200 efficient input combinations on the frontier of the isoquant shown in Fig. 6. Then, we multiplied each of the two inputs by one of the randomly-generated inefficiency proportions noted above, with the products indicating a DMU’s actual use of the two inputs. The results are shown in Fig. 7. Figure 7a illustrates the case when the inefficient percentages are independent, Fig. 7b shows the situation when the inefficient percentages are perfectly correlated, and Fig. 7c illustrates the case when there is an R-square of 0.42 between the inefficient percentages. Note that it is the inefficiencies only that are correlated, not the aggregated efficient and inefficient amounts.

Fig. 7

Variable factor proportion technology, inefficient DMUs

For the scenario in which the inputs are not substitutable, the sole efficient input combination is chosen to be 0.35 of Input 1 and 0.75 of Input 2, which would be the point efficiency coordinate for the observations in Fig. 6. We multiplied each of the two inputs by one of the randomly-generated inefficiency proportions noted above, with the products indicating a DMU’s actual use of the two inputs. The results are shown in Fig. 8. Figure 8a illustrates the case when the inefficiencies are independent, Fig. 8b shows the situation when the inefficiencies are perfectly correlated, and Fig. 8c illustrates the case when there is an R-squared of 0.42 between the inefficiencies.

Fig. 8

Fixed factor proportion technology, inefficient DMUs

Recall that the same proportional inefficiencies are used for both scenarios, the only difference being that efficient points along the entire isoquant were used for substitutable inputs, and the one efficient point was used for nonsubstitutable inputs. Even though the inefficiencies are much greater than would usually be expected, the patterns are clear. In all cases the substitutable inputs reflect strong down-sloping relationships between inefficiency levels. And, for nonsubstitutable inputs, there is no relationship between the inefficiencies when the inefficiencies are independent but an increasing strong positive relationship as correlation increases.

On the other hand, in our artificial dataset, even though the inefficiencies are relatively large compared to the efficient input amounts, the length of the frontier is relatively large compared to the inefficiencies. Assuming that in truth the inputs are substitutable, the shorter the relative length of the frontier when compared to the values of the inefficiencies, the less likely it will be to find a statistically significant negative relationship between the inputs.

However, as the frontier gets shorter, the de-facto technology approaches nonsubstitutability. So, DEA might be inappropriate in such cases even if there is a minute amount of substitutability.

Further, if substitutable inputs are not substituted to any great degree in the dataset at hand, then a negative relationship may not be detectable even if present in truth. Indeed, if the range of substitution is so small when compared to the aggregation of noise and random inefficiencies that a negative relationship cannot be detected, then attempts to empirically estimate a frontier with DEA may be futile.

Finally, recall that the purpose of this appendix is only to introduce some of the estimation issues, and present simple scenarios to better illustrate the concepts involved. We certainly do not purport to offer definitive answers to what often are very complex estimation questions!