, Volume 26, Issue 2, pp 173–183 | Cite as

A general procedure for obtaining paired comparisons from multiple rank orders

  • Harold Gulliksen
  • Ledyard R Tucker


From a theoretical point of view, paired comparisons and the law of comparative judgment provide an excellent approach to the problem of psychological measurement. However, if a reasonably large number of stimuli are to be investigated, paired comparisons become extremely time-consuming and fatiguing to the subjects. A balanced incomplete block design, requiring multiple rank order judgments for each subject, provides an efficient experimental method for obtaining paired comparisons judgments. Features of the analysis proposed for this design are discussed in detail. A program for the analysis is available for the IBM 650 electronic computer.


Fatigue Public Policy Experimental Method General Procedure Statistical Theory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Benson, P. Increasing the predictive efficiency of preference counts from paired comparison of personality traits.Educ. psychol. Measmt, 1958,18, 283–291.Google Scholar
  2. [2]
    Bradley, R. A. and Terry, M. E. Rank analysis of incomplete block designs. The method of paired comparisons.Biometrika, 1952,39, 324–345.Google Scholar
  3. [3]
    Calvin, L. D. Doubly balanced incomplete block designs for experiments in which the treatment effects are correlated.Biometrics, 1954,10, 61–88.Google Scholar
  4. [4]
    Cochran, W. G. and Cox, G. M.Experimental designs. (2nd ed.) New York: Wiley, 1957.Google Scholar
  5. [5]
    Coombs, C. A method for the study of interstimulus similarity.Psychometrika, 1954,19, 183–194.Google Scholar
  6. [6]
    Dunkel, H. B.General education in the humanities. Washington: Amer. Counc. Educ., 1947.Google Scholar
  7. [7]
    Durbin, J. Incomplete blocks in ranking experiments.Brit. J. Psychol., Statist. Sect., 1951,4, 85–90.Google Scholar
  8. [8]
    Hall, J. and Jones, D. C. Social grading of occupations.Brit. J. Sociol., 1950,1, 31–55.Google Scholar
  9. [9]
    Kendall, M. G.Rank correlation methods. (2nd ed.) London: Griffin, 1955. Ch. 11.Google Scholar
  10. [10]
    Gulliksen, H. A least squares solution for paired comparisons with incomplete data.Psychometrika, 1956,21, 125–134.Google Scholar
  11. [11]
    Gulliksen, H. An IBM 650 program for a complete paired comparisons schedule (Parcoplet 2-21). Tech. Rep. ONR contract Nonr 1858(15). (AnAppendix to this technical report gives details on this program, including input-output formats and operating directions for this program which is recorded in the IBM 650 Program Library—file no. 6.0.045.)Google Scholar
  12. [12]
    Gulliksen, H. and Tucker, L. R. An IBM 650 program for paired comparisons from balanced incomplete blocks—a 6–31 design (Parcobib 6–31). Res. Memo. 59-5. Princeton, N. J.: Educ. Testing Serv., 1959. (This program is recorded in the IBM 650 Program Library—file no. 6.0.038.)Google Scholar
  13. [13]
    Torgerson, W. S.Theory and methods of scaling. New York: Wiley, 1958.Google Scholar

Copyright information

© Psychometric Society 1961

Authors and Affiliations

  • Harold Gulliksen
    • 1
  • Ledyard R Tucker
    • 1
  1. 1.Princeton University and Educational Testing ServiceUSA

Personalised recommendations