Skip to main content
SpringerLink
Log in

The development of a reviewer selection method: a multi-level hesitant fuzzy VIKOR and TOPSIS approaches

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

This paper proposes a new approach for the selection of reviewers to evaluate research and development (R&D) projects using a new integrated hesitant fuzzy VIKOR and TOPSIS methodology. A reviewer selection model must have a multi-level framework in which reviewer selection strategies and related objectives guide the second level of the reviewer performance ranking process. The model must measure reviewer performance related to the activities that are necessary for the R&D project evaluation to be successful. A novel model is presented in this paper. In the proposed methodology, the aim is to select a reviewer in a hierarchical decision-making structure. The selection criteria values and their weights were obtained using the hesitant fuzzy VIKOR method. For the selection of a suitable reviewer, the conventional TOPSIS model was used. We developed a simpler procedure for effectively performing the reviewer selection process. The new approach was tested with a real case study and satisfactory results were obtained. A comparative analysis is also included in the article for illustrative purposes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

adapted from Sen and Yang 1998; Ic and Simsek 2019)

Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Ataei Y, Mahmoudi A, Feylizadeh MR, Li DF (2020) Ordinal priority approach (OPA) in multiple attribute decision-making. Appl Soft Comput 86:105893

    Google Scholar 

  • Atanassov KT (1986) Intuitionistic fuzzy sets. Fuzzy Set Syst 20:87–96

    MATH  Google Scholar 

  • Benferhat S, Lang J (2001) Conference paper assignment. Int J Intell Syst 16:1183–1192

    MATH  Google Scholar 

  • Charlin L, Zemel R (2013) The Toronto Paper matching system: an automated paper-reviewer assignment system. In: Proceedings of the 30th international conference on machine learning, Atlanta, Georgia, USA, vol 28. JMLR: W&CP, pp 1–9

  • Cook W, Golany B, Kress M, Penn M, Raviv T (2005) Optimal allocation of proposals to reviewers to facilitate effective ranking. Manage Sci 51(4):655–661

    Google Scholar 

  • Das GD, Gocken T (2014) A fuzzy approach for the reviewer assignment problem. Comput Ind Eng 72:50–57

    Google Scholar 

  • Davoodi E, Kianmehr K, Afsharchi M (2013) A semantic social network-based expert recommender system. Appl Intell 39(1):1–13

    Google Scholar 

  • Dey A, Senapati T, Pal M, Chen G (2020) A novel approach to hesitant multi-fuzzy soft set based decision-making. AIMS Mathematics 5(3):1985–2008

    MathSciNet  MATH  Google Scholar 

  • Di Mauro N, Basile TMA, Ferilli S (2005) GRAPE: an expert review assignment component for scientific conference management systems. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), 3533 LNAI, pp 789–798. https://doi.org/10.1007/11504894_109

  • Dumais ST, Nielsen J (1992) Automating the assignment of submitted manuscripts to reviewers. In: SIGIR’92 15th international conference on research and development in information retrieval, pp 233–244. https://doi.org/10.1145/133160.133205

  • Dung ND, Cong NH, Anh NT (2017) Algorithm of dynamic programming for paper-reviewer assignment problem. Int Res J Eng Technol 4(11):1928–1932

    Google Scholar 

  • Fatimah F, Alcantud JCR (2021) The multi-fuzzy N-soft set and its applications to decision-making. Neural Comput Appl:1–10

  • Ferilli S, Mauro ND, Basile TMA, Esposito F, Biba M (2006) Automatic topics identification for reviewer assignment. In: 19th international conference on industrial, engineering and other applications of applied intelligent systems IEA/AIE 2006 Annecy, France, pp 721–730

  • Hartvigsen H, Wei JC, Czuchlewski R (1999) The conference paper-reviewer assignment problem. Decis Sci 30(3):865–876

    Google Scholar 

  • Hwang CL, Yoon K (1981) Multiple attribute decision making. Lecture notes in economics and mathematical systems. Springer, Berlin

    MATH  Google Scholar 

  • Ic YT, Şimşek E (2019) Operating window perspective integrated TOPSIS approach for hybrid electrical automobile selection. SN Appl Sci 1(11):1–11

    Google Scholar 

  • Janak SL, Taylor MS, Floudas CA, Burka M, Mountziaris TJ (2006) Novel and effective integer optimization approach for the NSF panel-assignment problem: a multiresource and preference-constrained generalized assignment problem. Ind Eng Chem Res 45(1):258–265

    Google Scholar 

  • Jin J, Geng Q, Zhao Q, Zhang L (2017) Integrating the trend of research interest for reviewer assignment. In: WWW 2017 companion: proceedings of the 26th international conference on world wide web companion, pp 1233–1341. https://doi.org/10.1145/3041021.3053053

  • Joshi R (2020a) A novel decision-making method using R-Norm concept and VIKOR approach under picture fuzzy environment. Expert Syst Appl 147:113228

    Google Scholar 

  • Joshi R (2020b) A new picture fuzzy information measure based on Tsallis–Havrda–Charvat concept with applications in presaging poll outcome. Comput Appl Math 39(2):1–24

    MathSciNet  MATH  Google Scholar 

  • Joshi R (2021) Multi-criteria decision-making based on bi-parametric exponential fuzzy information measures and weighted correlation coefficients. Granul Comput. https://doi.org/10.1007/s41066-020-00249-9

  • Joshi R, Kumar S (2019) A novel fuzzy decision-making method using entropy weights-based correlation coefficients under intuitionistic fuzzy environment. Int J Fuzzy Syst 21(1):232–242

    MathSciNet  Google Scholar 

  • Jung U, Seo DW (2010) An ANP approach for R&D project evaluation based on interdependencies between research objectives and evaluation criteria. Decis Support Syst 49:335–342

    Google Scholar 

  • Kale A, Kharat R, Bodkhe S, Apte P, Dhonde H (2015) Automated fair paper reviewer assignment for conference management system. In: 2015 international conference on computing communication control and automation, Pune, India, pp 408–411. https://doi.org/10.1109/ICCUBEA.2015.85

  • Karaaslan F, Özlü Ş (2020) Correlation coefficients of dual type-2 hesitant fuzzy sets and their applications in clustering analysis. Int J Intell Syst 35(7):1200–1229

    MATH  Google Scholar 

  • Karimzadehgan M, Zhai CX (2009) Constrained multi-aspect expertise matching for committee review assignment. In: Proceedings of the 18th ACM conference on information and knowledge management, CIKM 2009, pp 1697–1700

  • Karimzadehgan M, Zhai CX, Belford G (2008) Multi-aspect expertise matching for review assignment. In: CIKM'08 Proceedings of the 17th ACM Conference On Information And Knowledge Management, Napa Valley, California, USA, pp 1113–1122

  • Karimzadehgan M, Zhai CX (2012) Integer linear programming for constrained multi-aspect committee review assignment. Inf Process Manage 48(4):725–740

    Google Scholar 

  • Keshavarz-Ghorabaee M, Amiri M, Zavadskas EK, Turskis Z (2017) Multi-Criteriagroup Decision-Making Using an Extended EDAS Method with Interval Type-2fuzzy Sets. E & M Ekonomie a Manag 20(1):48–68

    Google Scholar 

  • Kolasa T, Krol D (2010) ACO-GA approach to paper-reviewer assignment problem in CMS. In: 4th KES International Symposium KES-AMSTA, Gdynia, Poland, pp 360–369

  • Kolasa T, Krol D (2011) A survey of algorithms for paper-reviewer assignment problem. IETE Tech Rev 28(2):123–134

    Google Scholar 

  • Kou NM, LH U, Mamoulis N, Gong Z (2015a) Weighted coverage based reviewer assignment. In: The 2015 ACM SIGMOD International Conference, 2031–2046. Melbourne, Victoria

  • Kou NM, LH U, Mamoulis N, Yuhong L, Ye L, Gong Z (2015b) A topic-based reviewer assignment system. Proc VLDB Endow 8(12):1852–1855

  • Li B, Hou YT (2016) The new automated IEEE INFOCOM review assignment system. IEEE Netw 30(5):18–24

    Google Scholar 

  • Li X, Watanabe T (2013) Automatic paper-to-reviewer assignment, based on the matching degree of the reviewers. Procedia Comput Sci 22:633–642

    Google Scholar 

  • Li D, Zeng W, Li J (2015) New distance and similarity measures on hesitant fuzzy sets and their applications in multiple criteria decision making. Eng Appl Artif Intell 40:11–16

    Google Scholar 

  • Lian JW, Mattei N, Noble R, Walsh T (2018) The conference paper assignment problem: using order weighted averages to assign indivisible goods. In: Thirty-second AAAI conference on artificial intelligence, pp 1138–1145

  • Liao H, Xu Z (2017) Hesitant fuzzy decision making methodologies and applications. Uncertainty and operations research. Springer, Singapore

    Google Scholar 

  • Liu X, Suel T, Memon N (2014) A robust model for paper-reviewer assignment. In: RecSys 2014—proceedings of the 8th ACM conference on recommender systems, CA, USA, pp 25–32. https://doi.org/10.1145/2645710.2645749

  • Liu O, Wang J, Mac J, Sun Y (2016) An intelligent decision support approach for reviewer assignment in R&D project selection. Comput Ind 76:1–10

    Google Scholar 

  • Long, C, Wong, RC, Peng, Y, Ye, L (2013) On Good and Fair Paper-Reviewer Assignment. In: 2013 IEEE 13th international conference on data mining, pp 1145–1150. https://doi.org/10.1109/ICDM.2013.13

  • Mahmoudi A, Deng X, Javed SA, Zhang N (2020) Sustainable supplier selection in megaprojects: grey ordinal priority approach. Bus Strateg Environ 30(1):318–339

    Google Scholar 

  • Merelo-Guervós JJ, Castillo-Valdivieso P (2004) Conference paper assignment using a combined greedy/evolutionary algorithm. In: Yao X et al (eds) Parallel problem solving from nature - PPSN VIII. PPSN 2004. Lecture notes in computer science, vol 3242. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30217-9_61

  • Mishra AR, Rani P, Krishankumar R, Zavadskas EK, Cavallaro F, Ravichandran KS (2021) A hesitant fuzzy combined compromise solution framework-based on discrimination measure for ranking sustainable third-party reverse logistic providers. Sustainability 13(4):2064

    Google Scholar 

  • Nguyen J, Sánchez-Hernández G, Agell N, Rovira X, Angulo C (2018) A decision support tool using order weighted averaging for conference review assignment. Pattern Recogn Lett 105:114–120

    Google Scholar 

  • Opricovic S, Tzeng GH (2004) Compromise Solution By MCDM Methods: A Comparative Analysis of VIKOR and TOPSIS. Eur J Oper Res 156(2):445–455

    MATH  Google Scholar 

  • Opricovic S, Tzeng GH (2007) Extended VIKOR method in comparison with outranking methods. Eur J Oper Res 178:514–529

    MATH  Google Scholar 

  • Özlü Ş, Karaaslan F (2019) Some distance measures for type 2 hesitant fuzzy sets and their applications to multi-criteria group decision-making problems. Soft Comput 24:1–16

    MATH  Google Scholar 

  • Peng DH, Gao CY, Gao ZF (2013) Generalized hesitant fuzzy synergetic weighted distance measures and their application to multiple criteria decision-making. Appl Math Model 37(8):5837–5850

    MathSciNet  MATH  Google Scholar 

  • Peng DH, Wang TD, Gao CY, Wang H (2017) Enhancing relative ratio method for MCDM via attitudinal distance measures of interval-valued hesitant fuzzy sets. Int J Mach Learn Cybern 8(4):1347–1368

    Google Scholar 

  • Qian G, Wanga H, Feng X (2013) Generalized hesitant fuzzy sets and their application in decision support system. Knowl Based Syst 37:357–365

    Google Scholar 

  • Ren Z, Zeshui Xu, Wang H (2018) Multi-Criteria Group Decision-Making Based On Quasi-Order for Dual Hesitant Fuzzy Sets and Professional Degrees of Decision Makers. Appl Soft Comput 71:20–35

    Google Scholar 

  • Rodriguez MA, Bollen J, Sompel HV (2006) Mapping the bid behavior of conference referees. J Informet 1:68–82

    Google Scholar 

  • Rodriguez MA, Bollen J, Van de Sompel H (2007) Mapping the bid behavior of conference referees. J Informet 1(1):68–82

    Google Scholar 

  • Sen P, Yang J-B (1998) Multiple attributes decision support in engineering design. Springer, London

    Google Scholar 

  • Senapati T, Yager RR, Chen G (2021) Cubic intuitionistic WASPAS technique and its application in multi-criteria decision-making. J Ambient Intell Human Comput 12:8823–8833. https://doi.org/10.1007/s12652-020-02667-8

    Article  Google Scholar 

  • Siegel S, Castellan NJ (1956) Nonparametric statistics for the behavioral sciences, vol 7. McGraw-hill, New York, pp 202–204

    Google Scholar 

  • Sun YH, Ma J, Fan ZP, Wang J (2008a) A group decision support approach to evaluate experts for R&D project selection. IEEE Trans Eng Manage 55(1):158–170. https://doi.org/10.1109/TEM.2007.912934

    Article  Google Scholar 

  • Sun YH, Ma J, Fan ZP, Wang J (2008b) A hybrid knowledge and model approach for reviewer assignment. Expert Syst Appl 34(2):817–824

    Google Scholar 

  • Tang W, Tang J, Tan C (2010) Expertise matching via constraint-based Optimization. In: IEEE/WIC/ACM International Conference on Web Intelligence and intelligent agent technology. Toronto, pp 33–41

  • Tayal DK, Saxena PC, Sharma S, Khanna G, Gupta S (2014) New method for solving reviewer assignment problem using Type-2 Fuzzy sets and fuzzy functions. Appl Intell 40:54–73

    Google Scholar 

  • Torra V (2010) Hesitant Fuzzy sets. Int J Intell Syst 25:529–539

    MATH  Google Scholar 

  • Torra V, Narukawa Y (2009) On hesitant fuzzy sets and decision. In: 2009 IEEE International Conference on Fuzzy Systems, pp 1378–1382

  • Wang F, Zhou S, Shi N (2013) Group-to-group reviewer assignment problem. Comput Oper Res 40(5):1351–1362

    MathSciNet  MATH  Google Scholar 

  • Wang Y, Wang X, Jiang Y, Liang Y, Liu Y (2016) A code reviewer assignment model incorporating the competence differences and participant preferences. Found Comput Decis Sci 41(1):77–91

    MathSciNet  Google Scholar 

  • Wang F, Chen B, Miao Z (2008) A survey on reviewer assignment problem. In: 21st International Conference on industrial, engineering and other applications of applied intelligent systems IEA/AIE, Wrocław, Poland, pp 718–727

  • Xu Z, Xia M (2011a) Distance and Similarity Measures for Hesitant Fuzzy Sets. Inf Sci 181:2128–2138

    MathSciNet  MATH  Google Scholar 

  • Xu Z, Xia M (2011b) On distance and correlation measures of hesitant Fuzzy information. Int J Intell Syst 26:410–425

    MATH  Google Scholar 

  • Xu Y, Ma J, Sun Y, Hao G, Xu W, Zhao D (2010) A decision support approach for assigning reviewers to proposals. Expert Syst Appl 37:6948–6956

    Google Scholar 

  • Xu YH, Zuo X (2016) A LDA model based text-mining method to recommend reviewer for proposal of research project selection. In: 13th International Conference on Service Systems and Service Management (ICSSSM), pp 1–5

  • Yager RR (2014) Pythagorean membership grades in multicriteria decision making. IEEE Trans Fuzzy Syst 22(4):958–965

    Google Scholar 

  • Yang KH, Kuo TL, Lee HM (2009) Ho JM (2009) A Reviewer Recommendation System Based on Collaborative Intelligence. IEEE/WIC/ACM Int Conf Web Intell Intell Agent Technol 1:564–567

    Google Scholar 

  • Yimam-Seid D, Kobsa A (2003) Expert-finding systems for organizations: problem and domain analysis and the DEMOIR approach. J Organ Comput Electron Commer 13(1):1–24

    Google Scholar 

  • Yue M, Tian K, Ma T (2017) An accurate and impartial expert assignment method for scientific project review. J Data Inf Sci 2:65–80

    Google Scholar 

  • Yurdakul M, İç YT, Atalay KD (2020) Development of an intuitionistic fuzzy ranking model for nontraditional machining processes. Soft Comput 24:10095–10110

    Google Scholar 

  • Zadeh LA (1965) Fuzzy sets. Inf Control 8:338–353

    MATH  Google Scholar 

  • Zhang N, Wei G (2013) Extension of VIKOR method for decision making problem based on hesitant Fuzzy set. Appl Math Model 37(7):4938–4947

    MathSciNet  MATH  Google Scholar 

  • Zhao S, Zhang D, Duan Z, Chen J, Zhang YP, Tang J (2018) A novel classification method for paper-reviewer recommendation. Scientometrics 115:1293–1313

    Google Scholar 

  • Zhu B, Xu Z (2014) Some results for dual hesitant fuzzy sets. J Intell Fuzzy Syst 26(4):1657–1668

    MathSciNet  MATH  Google Scholar 

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. The Scientific and Technological Research Council of Turkey, 06100, Kavaklıdere, Ankara, Turkey

    Serdar Koçak

  2. Department of Industrial Engineering, Baskent University, 06790, Etimesgut, Ankara, Turkey

    Yusuf Tansel İç, Kumru Didem Atalay & Berna Dengiz

  3. Department of Computer Engineering, Baskent University, 06790, Etimesgut, Ankara, Turkey

    Mustafa Sert

Authors
  1. Serdar Koçak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yusuf Tansel İç
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kumru Didem Atalay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mustafa Sert
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Berna Dengiz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yusuf Tansel İç.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Ethical approval

This paper does not contain any studies with human participants or animals performed by any of the authors.

Consent to Participate

Not applicable.

Consent to publish

All authors have read and agreed to the published version of the manuscript.

Availability of data and materials

All data generated or analyzed during this study are included in this published article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koçak, S., İç, Y., Atalay, K.D. et al. The development of a reviewer selection method: a multi-level hesitant fuzzy VIKOR and TOPSIS approaches. J Ambient Intell Human Comput 14, 3275–3302 (2023). https://doi.org/10.1007/s12652-021-03466-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-021-03466-5

Keywords

Search

Navigation