Autonomous Agents and Multi-Agent Systems

, Volume 28, Issue 5, pp 721–748 | Cite as

Theoretical and empirical evaluation of data reduction for exact Kemeny Rank Aggregation

  • Nadja Betzler
  • Robert Bredereck
  • Rolf Niedermeier


Kemeny Rank Aggregation is a consensus finding problem important in many areas ranging from classical voting over web search and databases to bioinformatics. The underlying decision problem Kemeny Score is NP-complete even in case of four input rankings to be aggregated into a “median ranking”. We analyze efficient polynomial-time data reduction rules with provable performance bounds that allow us to find even all optimal median rankings. We show that our reduced instances contain at most Open image in new window candidates where \(d_a\) denotes the average Kendall’s tau distance between the input votes. On the theoretical side, this improves a corresponding result for a “partial problem kernel” from quadratic to linear size. In this context we provide a theoretical analysis of a commonly used data reduction. On the practical side, we provide experimental results with data based on web search and sport competitions, e.g., computing optimal median rankings for real-world instances with more than 100 candidates within milliseconds. Moreover, we perform experiments with randomly generated data based on two random distribution models for permutations.


Kemeny score NP-hardness Parameterized algorithmics Preprocessing Average parameterization Partial problem kernel Experiments 



We are grateful to the anonymous referees of the Fifth International Symposium on Parameterized and Exact Computation (IPEC-2010) and of the Third International Workshop on Computational Social Choice (COMSOC-2010) for constructive feedback helping to improve this work. We are indebted to three anonymous referees of JAAMAS for providing numerous insightful remarks that helped to significantly improve the paper. In particular, the more efficient and effective data reduction rule exploiting the extended Condorcet property helped to improve our theoretical and practical results. We thank Christian Komusiewicz for pointing us to an improved (compared to the conference version) analysis for the bound of Theorem 1, and our student research assistant Leila Arras for her great support in doing implementations and experiments with synthetic data. Nadja Betzler and Robert Bredereck were supported by the DFG, research project PAWS, NI 369/10.

Supplementary material

10458_2013_9236_MOESM1_ESM.pdf (382 kb)
Supplementary material 1 (pdf 382 KB)


  1. 1.
    Ailon, N., Charikar, M., & Newman, A. (2008). Aggregating inconsistent information: ranking and clustering. Journal of the ACM, 55(5), 1–27.CrossRefMathSciNetGoogle Scholar
  2. 2.
    Ali, A., & Meilă, M. (2012). Experiments with Kemeny ranking: What works when? Mathematical Social Sciences, 64(1), 28–40.CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Bartholdi, J, I. I. I., Tovey, C. A., & Trick, M. A. (1989). Voting schemes for which it can be difficult to tell who won the election. Social Choice and Welfare, 6, 157–165.CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Betzler, N. (2010). A multivariate complexity analysis of voting problems. PhD thesis, Friedrich-Schiller-Universität Jena.Google Scholar
  5. 5.
    Betzler, N., Bredereck, R., Chen, J., & Niedermeier, R. (2012). Studies in computational aspects of voting—a parameterized complexity perspective. In The multivariate algorithmic revolution and beyond. LNCS (Vol. 7370, pp. 318–363). New York: Springer.Google Scholar
  6. 6.
    Betzler, N., Fellows, M. R., Guo, J., Niedermeier, R., & Rosamond, F. A. (2009). Fixed-parameter algorithms for Kemeny rankings. Theoretical Computer Science, 410(45), 4554–4570.CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Betzler, N., Guo, J., Komusiewicz, C., & Niedermeier, R. (2011). Average parameterization and partial kernelization for computing medians. Journal of Computer and System Sciences, 77(4), 774–789.CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Biedl, T., Brandenburg, F. J., & Deng, X. (2009). On the complexity of crossings in permutations. Discrete Mathematics, 309(7), 1813–1823.CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Brandenburg, F., Gleißner, A., & Hofmeier, A. (2012). Comparing and aggregating partial orders with Kendall tau distances. In Proceedings of the 6th international workshop on algorithms and computation (WALCOM ’12). LNCS (Vol. 7157, pp. 88–99). New York: Springer.Google Scholar
  10. 10.
    Brandt, F., Brill, M., & Seedig, H. G. (2011). On the fixed-parameter tractability of composition-consistent tournament solutions. In Proceedings of the 22nd international joint conference on artificial intelligence (IJCAI ’11) (pp. 85–90). Menlo Park, CA: AAAI Press.Google Scholar
  11. 11.
    Bredereck, R., (2010). Fixed-parameter algorithms for computing Kemeny scores—Theory and practice. Studienarbeit, Universität Jena, CoRR abs/1001.4003.Google Scholar
  12. 12.
    Cohen, W. W., Schapire, R. E., & Singer, Y. (1999). Learning to order things. Journal of Artificial Intelligence Research, 10(1), 243–270.zbMATHMathSciNetGoogle Scholar
  13. 13.
    Conitzer, V., Davenport, A., & Kalagnanam, J. (2006). Improved bounds for computing Kemeny rankings. In Proceedings of the 21st national conference on artificial intelligence (AAAI’06) (pp. 620–626). Menlo Park, CA: AAAI Press.Google Scholar
  14. 14.
    Davenport, A., & Kalagnanam, J. (2004). A computational study of the Kemeny rule for preference aggregation. In Proceedings of the 19th national conference on artificial intelligence (AAAI’04) (pp. 697–702). Menlo Park, CA: AAAI Press.Google Scholar
  15. 15.
    Downey, R. G., & Fellows, M. R. (1999). Parameterized complexity. New York: Springer.CrossRefGoogle Scholar
  16. 16.
    Dwork, C., Kumar, R., Naor, M., & Sivakumar, D. (2001). Rank aggregation methods for the Web. In Proceedings of the 10th international World Wide Web conference (WWW’01) (pp. 613–622).Google Scholar
  17. 17.
    Fagin, R., Kumar, R., & Sivakumar, D. (2003). Efficient similarity search and classification via rank aggregation. In Proceedings of the 22nd ACM SIGMOD international conference on management of data (SIGMOD’03) (pp. 301–312). New York: ACM.Google Scholar
  18. 18.
    Fernau, H., Fomin, F. V., Lokshtanov, D., Mnich, M., Philip, G., & Saurabh, S. (2011). Ranking and drawing in subexponential time. In Proceedings of the 21st international conference on combinatorial algorithms. LNCS (Vol. 6460, pp. 337–348). New York: Springer.Google Scholar
  19. 19.
    Fligner, M. A., & Verducci, J. S. (1986). Distance based ranking models. Journal of the Royal Statistical Society, 48(3), 359–369.zbMATHMathSciNetGoogle Scholar
  20. 20.
    Flum, J., & Grohe, M. (2006). Parameterized complexity theory. Berlin: Springer.Google Scholar
  21. 21.
    Hemaspaandra, E., Spakowski, H., & Vogel, J. (2005). The complexity of Kemeny elections. Theoretical Computer Science, 349(3), 382–391.CrossRefzbMATHMathSciNetGoogle Scholar
  22. 22.
    Jackson, B. N., Schnable, P. S., & Aluru, S. (2008). Consensus genetic maps as median orders from inconsistent sources. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 5(2), 161–171.CrossRefGoogle Scholar
  23. 23.
    Karpinski, M., & Schudy, W. (2010). Faster algorithms for feedback arc set tournament, Kemeny rank aggregation and betweenness tournament. In Proceedings of the 21st international symposium on algorithms and computation (ISAAC’10). LNCS (Vol. 6506, pp. 3–14). Berlin: Springer.Google Scholar
  24. 24.
    Kenyon-Mathieu, C., & Schudy, W. (2007). How to rank with few errors. In Proceedings of the 39th annual ACM symposium on theory of computing (STOC’07) (pp. 95–103). New York: ACM.Google Scholar
  25. 25.
    Luce, R. D. (2005). Individual choice behavior: A theoretical analysis. New York: Dover Publications.CrossRefGoogle Scholar
  26. 26.
    Mahajan, M., Raman, V., & Sikdar, S. (2009). Parameterizing above or below guaranteed values. Journal of Computer and System Sciences, 75, 137–153.CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Mallows, C. (1957). Non-null ranking models. I. Biometrika, 44(1/2), 114–130.CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    Mandhani, B., & Meilǎ, M. (2009). Tractable search for learning exponential models of rankings. Journal of Machine Learning Research—Proceedings Track, 5, 392–399.Google Scholar
  29. 29.
    Meilǎ, M., & Bao, L. (2010). An exponential model for infinite rankings. Journal of Machine Learning Research, 11, 3481–3518.Google Scholar
  30. 30.
    Meilǎ, M., Phadnis, K., Patterson, A., & Bilmes, J. (2007). Consensus ranking under the exponential model. In Proceedings of the 23rd Conference on Uncertainty in artificial intelligence (UAI’07) (pp. 285–294).Google Scholar
  31. 31.
    Niedermeier, R. (2006). Invitation to fixed-parameter algorithms. Oxford: Oxford University Press.CrossRefzbMATHGoogle Scholar
  32. 32.
    Nishimura, N., & Simjour, N. (2013). Parameterized enumeration of (locally-) optimal aggregations. In 13th International symposium on algorithms and data structures (WADS’13). LNCS (Vol. 8037, pp. 512–523). New York: Springer.Google Scholar
  33. 33.
    Placket, R. L. (1975). The analysis of permutations. Applied Statistics, 24, 193–202.CrossRefGoogle Scholar
  34. 34.
    Raman, V., & Saurabh, S. (2007). Improved fixed parameter tractable algorithms for two “edge” problems: MAXCUT and MAXDAG. Information Processing Letters, 104(2), 65–72.CrossRefzbMATHMathSciNetGoogle Scholar
  35. 35.
    Schalekamp, F., & van Zuylen, A. (2009). Rank aggregation: Together we’re strong. In Proceedings of the 11th workshop on algorithm engineering and experiments (ALENEX’09) (pp. 38–51). Philadelphia: SIAM.Google Scholar
  36. 36.
    Simjour, N. (2009). Improved parameterized algorithms for the Kemeny aggregation problem. Proceedings of the 4th international workshop on parameterized and exact computation (IWPEC’09). LNCS (Vol. 5917, pp. 312–323). New York: Springer.Google Scholar
  37. 37.
    Simjour, N. (2013). Parameterized enumeration of neighbour strings and Kemeny aggregations. PhD thesis, University of Waterloo.Google Scholar
  38. 38.
    Truchon, M. (1998). An extension of the Condorcet criterion and Kemeny orders. Technical report, cahier 98–15 du Centre de Recherche en Économie et Finance Appliquées, Université Laval, Québec, Candada.Google Scholar
  39. 39.
    Young, H. P. (1995). Optimal voting rules. Journal of Economic Perspectives, 9(1), 51–64.CrossRefGoogle Scholar
  40. 40.
    Young, H. P., & Levenglick, A. (1978). A consistent extension of Condorcet’s election principle. SIAM Journal on Applied Mathematics, 35(2), 285–300.CrossRefzbMATHMathSciNetGoogle Scholar
  41. 41.
    van Zuylen, A., & Williamson, D. P. (2009). Deterministic pivoting algorithms for constrained ranking and clustering problems. Mathematics of Operations Research, 34(3), 594–620.CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© The Author(s) 2013

Authors and Affiliations

  • Nadja Betzler
    • 1
  • Robert Bredereck
    • 1
  • Rolf Niedermeier
    • 1
  1. 1.Institut für Softwaretechnik und Theoretische InformatikTU BerlinBerlinGermany

Personalised recommendations