Skip to main content

Multiple indicators, partially ordered sets, and linear extensions: Multi-criterion ranking and prioritization


This paper is concerned with the question of ranking a finite collection of objects when a suite of indicator values is available for each member of the collection. The objects can be represented as a cloud of points in indicator space, but the different indicators (coordinate axes) typically convey different comparative messages and there is no unique way to rank the objects while taking all indicators into account. A conventional solution is to assign a composite numerical score to each object by combining the indicator information in some fashion. Consciously or otherwise, every such composite involves judgments (often arbitrary or controversial) about tradeoffs or substitutability among indicators. Rather than trying to combine indicators, we take the view that the relative positions in indicator space determine only a partial ordering and that a given pair of objects may not be inherently comparable. Working with Hasse diagrams of the partial order, we study the collection of all rankings that are compatible with the partial order (linear extensions). In this way, an interval of possible ranks is assigned to each object. The intervals can be very wide, however. Noting that ranks near the ends of each interval are usually infrequent under linear extensions, a probability distribution is obtained over the interval of possible ranks. This distribution, called the rank-frequency distribution, turns out to be unimodal (in fact, log-concave) and represents the degree of ambiguity involved in attempting to assign a rank to the corresponding object. Stochastic ordering of probability distributions imposes a partial order on the collection of rank-frequency distributions. This collection of distributions is in one-to-one correspondence with the original collection of objects and the induced ordering on these objects is called the cumulative rank-frequency (CRF) ordering; it extends the original partial order. Although the CRF ordering need not be linear, it can be iterated to yield a fixed point of the CRF operator. We hypothesize that the fixed points of the CRF operator are exactly the linear orderings. The CRF operator treats each linear extension as an equal “voter” in determining the CRF ranking. It is possible to generalize to a weighted CRF operator by giving linear extensions differential weights either on mathematical grounds (e.g., number of jumps) or empirical grounds (e.g., indicator concordance). Explicit enumeration of all possible linear extensions is computationally impractical unless the number of objects is quite small. In such cases, the rank-frequencies can be estimated using discrete Markov chain Monte Carlo (MCMC) methods.

This is a preview of subscription content, access via your institution.


  • Aldous, D. (1987) On the Markov chain simulation method for uniform combinatorial distributions and simulated annealing. Probability in the Engineering and Informational Sciences, 1, 33-46.

    Google Scholar 

  • Brightwell, G. and Winkler, P. (1991) Counting linear extensions. Orders, 8, 225-42.

    Google Scholar 

  • Budach, L., Graw, B., Meinel, C., and Waack, S. (1988) Algebraic and Topological Properties of Finite Partially Ordered Sets, Teubner-Texte zur Mathematik, Band 109, Leipzig.

  • Charnes, A., Cooper, A.Y., Lewin, A.Y., and Seiford, L.M. (1994) Data Envelopment Analysis: Theory, Methodology, and Applications, Kluwer, Boston.

    Google Scholar 

  • Cormen, T.H., Leierson, C.E., Rivest, R.L., and Stein, C. (2001) Introduction to Algorithms (second edition), MIT Press, Cambridge, Massachusetts.

    Google Scholar 

  • Di Battista, G., Eades, P., Tamassia, R., and Tollis, I.G. (1999) Graph Drawing: Algorithms for the Visualization of Graphs, Prentice Hall, Upper Saddle River, New Jersey.

    Google Scholar 

  • Filar, J.A. and Ross, N.P. (2001) Generalized data envelopment analysis, and environmental indicators. Invited Paper. Plenary Forum on Environmental Indicators and their Integration for Quality of Life, Index 2001 Congress, Rome, Italy.

  • Fishburn, P.C. (1985) Interval Orders and Interval Graphs: A Study of Partially Ordered Sets, Wiley, New York.

    Google Scholar 

  • Haggstrom, O. (2002) Finite Markov Chains and Algorithmic Applications, Cambridge University Press, Cambridge.

    Google Scholar 

  • Knuth, D.E. (1973) The Art of Computer Programming: Volume 1, Fundamental Algorithms (second edition), Addison-Wesley, Reading, Massachusetts.

    Google Scholar 

  • Kumbhakar, S.C. and Knox Lovell, C.A. (2002) Stochastic Frontier Analysis, Cambridge University Press, Cambridge.

    Google Scholar 

  • Lehmann, E.L. (1986) Testing Statistical Hypotheses (second edition), Wiley, New York.

    Google Scholar 

  • Neggers, J. and Kim, H.S. (1998) Basic Posets, World Scientific, Singapore.

    Google Scholar 

  • Sugiyama, K. (2002) Graph Drawing and Applications, World Scientific, Singapore.

    Google Scholar 

  • Trotter, W.T. (1992) Combinatorics and Partially Ordered Sets, Johns Hopkins University Press, Baltimore.

    Google Scholar 

  • USEPA (1997) An Ecological Assessment of the United States Mid-Atlantic Region, EPA/600/R-97/130, United States Department of Environmental Protection, Office of Research and Development, Washington, DC.

    Google Scholar 

Download references

Author information

Authors and Affiliations


Rights and permissions

Reprints and Permissions

About this article

Cite this article

Patil, G., Taillie, C. Multiple indicators, partially ordered sets, and linear extensions: Multi-criterion ranking and prioritization. Environmental and Ecological Statistics 11, 199–228 (2004).

Download citation

  • Issue Date:

  • DOI:

  • CRF operator
  • Hasse diagram
  • MCMC
  • partial order
  • pre-order
  • poset
  • rank-interval
  • rank-frequency distribution
  • weighted distributions
  • zeta matrix