A Note on Machine Learning Approach to Analyze the Results of Pairwise Comparison Based Parametric Evaluation of Research Units

  • Mateusz Baran
  • Konrad Kułakowski
  • Antoni Ligęza
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8468)


This paper presents an attempt at an analysis of parametric evaluation of research units with machine learning toolkit. The main goal was to investigate if the rules of evaluation can be expressed in a readable, transparent, and easy to interpret way. A further attempt was made at investigating consistency of the applied procedure and presentation of some observed anomalies.


Machine Learn Linear Order Multiple Criterion Decision Make Principal Eigenvector Reference Entity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rozporządzenie Ministra Nauki i Szkolnictwa Wyższego z dn. 13 lipca 2012 w sprawie kryteriów i trybu przyznawania kategorii naukowej jednostkom naukowym. Dziennik Ustaw Rzeczypospolitej Polskiej, Warszawa, 1 sierpnia (2012)Google Scholar
  2. 2.
    Cios, K.J., Pedrycz, W., Swiniarski, R.W., Kurgan, L.A.: Data Mining. A Knowledge Discovery Approach. Springer Science+Business Media, LLC, New York (2007)Google Scholar
  3. 3.
    Flach, P.: Machine Learning. The Art and Science of Algorithms that Make the Sense of Data. Cambridge University Press, Cambridge (2012)CrossRefzbMATHGoogle Scholar
  4. 4.
    Frank, E., Witten, I.H.: Generating Accurate Rule Sets Without Global Optimization. In: Fifteenth International Conference on Machine Learning, pp. 144–151 (1998)Google Scholar
  5. 5.
    Huehn, J.C., Huellermeier, E.: FURIA: An Algorithm for Unordered Fuzzy Rule Induction. Data Mining and Knowledge Discovery (2009)Google Scholar
  6. 6.
    Hwang, C.-L., Masud, A.S.M.: Multiple objective decision making, methods and applications: a state-of-the-art survey. Springer (1979)Google Scholar
  7. 7.
    Gaines, B.R., Compton, P.: Induction of ripple-down rules applied to modeling large databases. Journal of Intelligent Information Systems 5(3), 211–228 (1995)CrossRefGoogle Scholar
  8. 8.
    Miettinen, K.: Nonlinear Multiobjective Optimization. Springer (1999)Google Scholar
  9. 9.
    Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Mateo (1993)Google Scholar
  10. 10.
    Landwehr, N., Hall, M., Frank, E.: Logistic Model Trees. Machine Learning 95(1-2), 161–205 (2005)CrossRefGoogle Scholar
  11. 11.
    Sumner, M., Frank, E., Hall, M.: Speeding Up Logistic Model Tree Induction. In: Jorge, A.M., Torgo, L., Brazdil, P.B., Camacho, R., Gama, J. (eds.) PKDD 2005. LNCS (LNAI), vol. 3721, pp. 675–683. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Chang, C.-C., Lin, C.-J.: LIBSVM — A Library for Support Vector Machines (2001),
  13. 13.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA Data Mining Software: An Update. SIGKDD Explorations 11(1) (2009)Google Scholar
  14. 14.
  15. 15.
    Colomer, J.M.: Ramon Llull: from ‘Ars electionis’ to social choice theory. Social Choice and Welfare 40(2), 317–328 (2011)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Bana e Costa, C.A., Vansnick, J.: A critical analysis of the eigenvalue method used to derive priorities in AHP. European Journal of Operational Research 187(3), 1422–1428 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Dyer, J.S.: Remarks on the analytic hierarchy process. Management Science 36(3), 249–258 (1990)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Fedrizzi, M., Giove, S.: Incomplete pairwise comparison and consistency optimization. European Journal of Operational Research 183(1), 303–313 (2007)CrossRefzbMATHGoogle Scholar
  19. 19.
    Greco, S., Matarazzo, B., Słowiński, R.: Dominance-based rough set approach on pairwise comparison tables to decision involving multiple decision makers. In: Yao, J., Ramanna, S., Wang, G., Suraj, Z. (eds.) RSKT 2011. LNCS, vol. 6954, pp. 126–135. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  20. 20.
    Koczkodaj, W.W.: A new definition of consistency of pairwise comparisons. Math. Comput. Model. 18(7), 79–84 (1993)CrossRefzbMATHGoogle Scholar
  21. 21.
    Koczkodaj, W.W., Kułakowski, K., Ligęza, A.: On the Quality Evaluation of Scientific Entities in Poland supported by Consistency-Driven Pairwise Comparisons Method. Scientometrics (accepted for publication, 2014)Google Scholar
  22. 22.
    Kułakowski, K.: A heuristic rating estimation algorithm for the pairwise comparisons method. Central European Journal of Operations Research, 1–17 (2013)Google Scholar
  23. 23.
    Perron, O.: Zur Theorie der Matrices. Mathematische Annalen 64(2), 248–263 (1907)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Saaty, T.L.: A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 15(3), 234–281 (1977)CrossRefzbMATHMathSciNetGoogle Scholar
  25. 25.
    Thurstone, L.L.: A law of comparative judgment, reprint of an original work published in 1927. Psychological Review 101, 266–270 (1994)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Mateusz Baran
    • 1
    • 2
  • Konrad Kułakowski
    • 1
  • Antoni Ligęza
    • 1
  1. 1.AGH University of Science and TechnologyKrakówPoland
  2. 2.Cracow University of TechnologyCracowPoland

Personalised recommendations