Skip to main content

Advertisement

Log in

Sustainable supplier selection model with a trade-off between supplier development and supplier switching

  • Original Research
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Due to growing concerns regarding sustainability, purchasing decisions are challenging and difficult tasks for decision-makers. This difficulty has compelled purchasing companies to know and understand the whole purchasing process and to give importance to purchasing associated decisions. A long-term relationship and investment are required for purchasing decisions because they impact significantly on a company’s performance and supply chain. Hence, supplier selection and sourcing strategy selection decisions are among a firm’s most important problems. In this study, sourcing strategy decisions include supplier development, which helps suppliers to improve their performance, and supplier switching, which searches for more proficient alternatives for supply. To solve these problems, this study provides an integrated multi-criteria decision-making (MCDM) model. The model contains four stages. First, the right set of sustainable key performance indicators (SKPIs) for evaluating the performance of suppliers is identified through a literature survey and discussions with the decision-making team. Second, the best worst method-measurement of alternatives and ranking according to the compromise solution method (BWM-MARCOS) approach is applied to determine the priority weights of SKPIs and the priority weights of incumbent and new suppliers based on identified SKPIs. Third, a bi-objective mathematical model is developed to determine which optimum sourcing strategy and potential supplier should be chosen based on the priority of incumbent and new suppliers while optimizing cost and sustainable performance. Fourth, the mathematical model is solved using Epsilon constraint method and min–max fuzzy approach. The applicability and efficiency of the proposed integrated MCDM model is demonstrated with a real case study from a home appliance manufacturing company. The key findings reveal that the proposed model can be utilized for strategic and effective sourcing planning. One of the important contributions of this work is to provide suggestions for deciding the appropriate sourcing strategy for suppliers using the outputs of the mathematical model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Abdullah, L., Chan, W., & Afshari, A. (2019). Application of PROMETHEE method for green supplier selection: A comparative result based on preference functions. Journal of Industrial Engineering International, 15(2), 271–285.

    Google Scholar 

  • Ahmadi, H. B., Kusi-Sarpong, S., & Rezaei, J. (2017). Assessing the social sustainability of supply chains using Best Worst Method. Resources, Conservation and Recycling, 126, 99–106.

    Google Scholar 

  • Alavi, B., Tavana, M., & Mina, H. (2021). A dynamic decision support system for sustainable supplier selection in circular economy. Sustainable Production and Consumption, 27, 905–920.

    Google Scholar 

  • Amid, A., Ghodsypour, S. H., & O’Brien, C. (2011). A weighted max–min model for fuzzy multi-objective supplier selection in a supply chain. International Journal of Production Economics, 131(1), 139–145.

    Google Scholar 

  • Amiri, M., Ayazi, S. A., Olfat, L., & Moradi, J. S. (2011). Group decision making process for supplier selection with VIKOR under fuzzy circumstance case study: An Iranian car parts supplier. International Bulletin of Business Administration, 10(6), 66–75.

    Google Scholar 

  • Badri Ahmadi, H., Hashemi Petrudi, S., & Wang, X. (2017). Integrating sustainability into supplier selection with analytical hierarchy process and improved grey relational analysis: A case of telecom industry. International Journal of Advanced Manufacturing Technology90.

  • Bahadori, M., Hosseini, S. M., Teymourzadeh, E., Ravangard, R., Raadabadi, M., & Alimohammadzadeh, K. (2020). A supplier selection model for hospitals using a combination of artificial neural network and fuzzy VIKOR. International Journal of Healthcare Management, 13(4), 286–294.

    Google Scholar 

  • Bai, C., Kusi-Sarpong, S., Badri Ahmadi, H., & Sarkis, J. (2019). Social sustainable supplier evaluation and selection: A group decision-support approach. International Journal of Production Research, 57(22), 7046–7067.

    Google Scholar 

  • Bevilacqua, M., & Petroni, A. (2002). From traditional purchasing to supplier management: A fuzzy logic-based approach to supplier selection. International Journal of Logistics, 5(3), 235–255.

    Google Scholar 

  • Bartos, K. E., Schwarzkopf, J., Mueller, M., & Hofmann-Stoelting, C. (2022). Explanatory factors for variation in supplier sustainability performance in the automotive sector–A quantitative analysis. Cleaner Logistics and Supply Chain, 5, 100068.

  • Blome, C., Hollos, D., & Paulraj, A. (2014). Green procurement and green supplier development: Antecedents and effects on supplier performance. International Journal of Production Research, 52(1), 32–49.

    Google Scholar 

  • Boran, F. E., Genç, S., Kurt, M., & Akay, D. (2009). A multi-criteria intuitionistic fuzzy group decision making for supplier selection with TOPSIS method. Expert Systems with Applications, 36(8), 11363–11368.

    Google Scholar 

  • Bottani, E., Centobelli, P., Murino, T., & Shekarian, E. (2018). A QFD-ANP method for supplier selection with benefits, opportunities, costs and risks considerations. International Journal of Information Technology & Decision Making, 17(03), 911–939.

    Google Scholar 

  • Burke, G. J., Carrillo, J. E., & Vakharia, A. J. (2009). Sourcing decisions with stochastic supplier reliability and stochastic demand. Production and Operations Management, 18(4), 475–484.

    Google Scholar 

  • Calvi, R., Le Dain, M. A., Fendt, T. C., & Herrmann, C. J. (2010). Supplier selection for strategic supplier development.

  • Cao, Y., Wang, Q., Du, J., Nojavan, S., Jermsittiparsert, K., & Ghadimi, N. (2019). Optimal operation of CCHP and renewable generation-based energy hub considering environmental perspective: An epsilon constraint and fuzzy methods. Sustainable Energy, Grids and Networks, 20, 100274.

    Google Scholar 

  • Chen, Y. J. (2011). Structured methodology for supplier selection and evaluation in a supply chain. Information Sciences, 181(9), 1651–1670.

    Google Scholar 

  • Ching, L. H., & Kwangsun, Y. (1981). Multiple attribute decision making: methods and applications: a state-of-the-art survey (lecture notes in economics and mathematical systems). Springer.

    Google Scholar 

  • Chowdhury, P., & Paul, S. K. (2020). Applications of MCDM methods in research on corporate sustainability: a systematic literature review. Management of Environmental Quality: An International Journal.

  • Demski, J. S., Sappington, D. E., & Spiller, P. T. (1987). Managing supplier switching. The RAND Journal of Economics, 77–97.

  • Dey, P. K., Bhattacharya, A., Ho, W., & Clegg, B. (2015). Strategic supplier performance evaluation: A case-based action research of a UK manufacturing organisation. International Journal of Production Economics, 166, 192–214.

    Google Scholar 

  • Dickson, G. W. (1966). An analysis of vendor selection systems and decisions. Journal of Purchasing, 2(1), 5–17.

    Google Scholar 

  • Diouf, M., & Kwak, C. (2018). Fuzzy AHP, DEA, and Managerial analysis for supplier selection and development; from the perspective of open innovation. Sustainability, 10(10), 3779.

    Google Scholar 

  • Dobos, I., & Vörösmarty, G. (2014). Green supplier selection and evaluation using DEA-type composite indicators. International Journal of Production Economics, 157, 273–278.

    Google Scholar 

  • Engau, A., & Wiecek, M. M. (2007). Generating ε-efficient solutions in multi objective programming. European Journal of Operational Research, 177(3), 1566–1579.

    Google Scholar 

  • Fei, L., Xia, J., Feng, Y., & Liu, L. (2019). An ELECTRE-based multiple criteria decision making method for supplier selection using Dempster-Shafer theory. IEEE Access, 7, 84701–84716.

    Google Scholar 

  • Freeman, J., & Chen, T. (2015). Green supplier selection using an AHP-Entropy-TOPSIS framework. Supply Chain Management: An International Journal.

  • Friedl, G., & Wagner, S. M. (2012). Supplier development or supplier switching? International Journal of Production Research, 50(11), 3066–3079.

    Google Scholar 

  • Govindan, K. (2022a). How Artificial Intelligence Drives Sustainable Frugal Innovation: A Multitheoretical Perspective. IEEE Transactions on Engineering Management.

  • Govindan, K. (2022b). Tunneling the barriers of blockchain technology in remanufacturing for achieving sustainable development goals: A circular manufacturing perspective. Business Strategy and the Environment.

  • Govindan, K., Kannan, D., Jørgensen., T.B., Nielsen, T.S (2022a) Supply Chain 4.0 performance measurement: A Systematic Literature Review and framework development, Transportation Research Part E

  • Govindan, K., Nasr, A. K., Karimi, F., & Mina, H. (2022b). Circular economy adoption barriers: An extended fuzzy best–worst method using fuzzy DEMATEL and Supermatrix structure. Business Strategy and the Environment.

  • Govindan, K., Nasr, A. K., Saeed Heidary, M., Nosrati-Abargooee, S., & Mina, H. (2022c). Prioritizing adoption barriers of platforms based on blockchain technology from balanced scorecard perspectives in healthcare industry: a structural approach. International Journal of Production Research, 1–15.

  • Ghadimi, P., Dargi, A., & Heavey, C. (2017). Making sustainable sourcing decisions: Practical evidence from the automotive industry. International Journal of Logistics Research and Applications, 20(4), 297–321.

    Google Scholar 

  • Glock, C. H., Grosse, E. H., & Ries, J. M. (2017). Reprint of “Decision support models for supplier development: Systematic literature review and research agenda.” International Journal of Production Economics, 194, 246–260.

    Google Scholar 

  • Haimes, Y. (1971). On a bicriterion formulation of the problems of integrated system identification and system optimization. IEEE Transactions on Systems, Man, and Cybernetics, 1(3), 296–297.

    Google Scholar 

  • Hamdan, S., & Cheaitou, A. (2017). Supplier selection and order allocation with green criteria: An MCDM and multi-objective optimization approach. Computers & Operations Research, 81, 282–304.

    Google Scholar 

  • Hashim, M., Nazam, M., Yao, L., Baig, S. A., Abrar, M., & Zia-ur-Rehman, M. (2017). Application of multi-objective optimization based on genetic algorithm for sustainable strategic supplier selection under fuzzy environment. Journal of Industrial Engineering and Management, 10(2), 188–212.

    Google Scholar 

  • Holma, A. M., Bask, A., Laakso, A., & Andersson, D. (2021). Conceptualizing the supplier switching process: an example from public procurement. Journal of Business & Industrial Marketing.

  • Hsiao, Y. T., Chiang, H. D., Liu, C. C., & Chen, Y. L. (1994). A computer package for optimal multi-objective VAR planning in large scale power systems. IEEE Transactions on Power Systems, 9(2), 668–676.

    Google Scholar 

  • Hsu, C. W., Kuo, T. C., Chen, S. H., & Hu, A. H. (2013). Using DEMATEL to develop a carbon management model of supplier selection in green supply chain management. Journal of Cleaner Production, 56, 164–172.

    Google Scholar 

  • Humphreys, P. K., Li, W. L., & Chan, L. Y. (2004). The impact of supplier development on buyer–supplier performance. Omega, 32(2), 131–143.

    Google Scholar 

  • Hwang, C. L., Lai, Y. J., & Liu, T. Y. (1993). A new approach for multiple objective decision making. Computers & Operations Research, 20(8), 889–899.

    Google Scholar 

  • Jafarian, M., Lotfi, M. M., & Pishvaee, M. S. (2021). Supplier switching versus supplier development under risk: A mathematical modelling approach. Computers & Industrial Engineering, 162, 107737.

    Google Scholar 

  • Jain, N., & Singh, A. R. (2020). Sustainable supplier selection under must-be criteria through Fuzzy inference system. Journal of Cleaner Production, 248, 119275.

    Google Scholar 

  • Karami, S., Ghasemy Yaghin, R., & Mousazadegan, F. (2021). Supplier selection and evaluation in the garment supply chain: An integrated DEA–PCA–VIKOR approach. The Journal of the Textile Institute, 112(4), 578–595.

    Google Scholar 

  • Kannan, D. (2021). Sustainable procurement drivers for extended multi-tier context: A multi-theoretical perspective in the Danish supply chain. Transportation Research Part E: Logistics and Transportation Review, 146, 102092.

    Google Scholar 

  • Kaur, H., Singh, S. P., & Glardon, R. (2016). An integer linear program for integrated supplier selection: A sustainable flexible framework. Global Journal of Flexible Systems Management, 17(2), 113–134.

    Google Scholar 

  • Kannan, D., Shankar, K. M. & Gholipour, P. (2022). Paving the way for a green transition through mitigation of green manufacturing challenges: A systematic literature review. Journal of Cleaner Production, 132578.

  • Kazemi, N., Ehsani, E., Glock, C. H., & Schwindl, K. (2015). A mathematical programming model for a multi-objective supplier selection and order allocation problem with fuzzy objectives. International Journal of Services and Operations Management, 21(4), 435–465.

    Google Scholar 

  • Khan, S. A., Kusi-Sarpong, S., Arhin, F. K., & Kusi-Sarpong, H. (2018). Supplier sustainability performance evaluation and selection: A framework and methodology. Journal of Cleaner Production, 205, 964–979.

    Google Scholar 

  • Kheybari, S., Kazemi, M., & Rezaei, J. (2019). Bioethanol facility location selection using best-worst method. Applied Energy, 242, 612–623.

    Google Scholar 

  • Kusi-Sarpong, S., Gupta, H., & Sarkis, J. (2019). A supply chain sustainability innovation framework and evaluation methodology. International Journal of Production Research, 57(7), 1990–2008.

    Google Scholar 

  • Lahane, S., & Kant, R. (2021). Evaluating the circular supply chain implementation barriers using Pythagorean fuzzy AHP-DEMATEL approach. Cleaner Logistics and Supply Chain, 2, 100014.

    Google Scholar 

  • Lee, J., Cho, H., & Kim, Y. S. (2015). Assessing business impacts of agility criterion and order allocation strategy in multi-criteria supplier selection. Expert Systems with Applications, 42(3), 1136–1148.

    Google Scholar 

  • Luthra, S., Govindan, K., Kannan, D., Mangla, S. K., & Garg, C. P. (2017). An integrated framework for sustainable supplier selection and evaluation in supply chains. Journal of Cleaner Production, 140, 1686–1698.

    Google Scholar 

  • Malek, J., & Desai, T. N. (2019). Prioritization of sustainable manufacturing barriers using Best Worst Method. Journal of Cleaner Production, 226, 589–600.

    Google Scholar 

  • Malek, J., & Desai, T. N. (2021). A framework for prioritizing the solutions to overcome sustainable manufacturing barriers. Cleaner Logistics and Supply Chain, 1, 100004.

    Google Scholar 

  • Mani, V., Agrawal, R., & Sharma, V. (2014). Supplier selection using social sustainability: AHP based approach in India. International Strategic Management Review, 2(2), 98–112.

    Google Scholar 

  • Memari, A., Dargi, A., Jokar, M. R. A., Ahmad, R., & Rahim, A. R. A. (2019). Sustainable supplier selection: A multi-criteria intuitionistic fuzzy TOPSIS method. Journal of Manufacturing Systems, 50, 9–24.

    Google Scholar 

  • Matthess, M., Kunkel, S., Xue, B., & Beier, G. (2022). Supplier sustainability assessment in the age of Industry 4.0–Insights from the electronics industry. Cleaner Logistics and Supply Chain, 4, 100038.

  • Merzifonluoglu, Y. (2015). Impact of risk aversion and backup supplier on sourcing decisions of a firm. International Journal of Production Research, 53(22), 6937–6961.

    Google Scholar 

  • Mina, H., Kannan, D., Gholami-Zanjani, S. M., & Biuki, M. (2021). Transition towards circular supplier selection in petrochemical industry: A hybrid approach to achieve sustainable development goals. Journal of Cleaner Production, 286, 125273.

    Google Scholar 

  • Mir, S., Aloysius, J. A., & Eckerd, S. (2017). Understanding supplier switching behavior: The role of psychological contracts in a competitive setting. Journal of Supply Chain Management, 53(3), 3–18.

    Google Scholar 

  • Modi, S. B., & Mabert, V. A. (2007). Supplier development: Improving supplier performance through knowledge transfer. Journal of Operations Management, 25(1), 42–64.

    Google Scholar 

  • Mohammed, A., Harris, I., & Govindan, K. (2019). A hybrid MCDM-FMOO approach for sustainable supplier selection and order allocation. International Journal of Production Economics, 217, 171–184.

    Google Scholar 

  • Mousavi-Nasab, S. H., & Sotoudeh-Anvari, A. (2017). A comprehensive MCDM-based approach using TOPSIS, COPRAS and DEA as an auxiliary tool for material selection problems. Materials & Design, 121, 237–253.

    Google Scholar 

  • Mukherjee, K. (2017). Modeling and optimization of traditional supplier selection. In Supplier Selection (pp. 31–58). Springer, New Delhi.

  • Nielsen, I. E., Banaeian, N., Golińska, P., Mobli, H., & Omid, M. (2014). Green supplier selection criteria: from a literature review to a flexible framework for determination of suitable criteria. Logistics operations, supply chain management and sustainability, 79–99.

  • Nouri, A., Khodaei, H., Darvishan, A., Sharifian, S., & Ghadimi, N. (2018). RETRACTED: Optimal performance of fuel cell-CHP-battery based micro-grid under real-time energy management: An epsilon constraint method and fuzzy satisfying approach.

  • Önder, E., & Kabadayi, N. (2015). Supplier selection in hospitality industry using ANP. International Journal of Academic Research in Business and Social Sciences5(1).

  • Orji, I. J., & Wei, S. (2015). An innovative integration of fuzzy-logic and systems dynamics in sustainable supplier selection: A case on manufacturing industry. Computers & Industrial Engineering, 88, 1–12.

    Google Scholar 

  • Qin, J., & Liu, X. (2019). Interval type-2 fuzzy group decision making by integrating improved best worst method with COPRAS for emergency material supplier selection. In Type-2 fuzzy decision-making theories, methodologies and applications (pp. 249–271). Springer, Singapore.

  • Rahimi, M., Kumar, P., Moomivand, B., & Yari, G. (2021). An intuitionistic fuzzy entropy approach for supplier selection. Complex & Intelligent Systems, 1–8.

  • Rashidi, K., Noorizadeh, A., Kannan, D., & Cullinane, K. (2020). Applying the triple bottom line in sustainable supplier selection: A meta-review of the state-of-the-art. Journal of Cleaner Production, 269, 122001.

    Google Scholar 

  • Rezaei, A., Rahiminezhad Galankashi, M., Mansoorzadeh, S., & Mokhatab Rafiei, F. (2020). Supplier selection and order allocation with lean manufacturing criteria: An integrated MCDM and Bi-objective modelling approach. Engineering Management Journal, 32(4), 253–271.

    Google Scholar 

  • Rezaei, J., Nispeling, T., Sarkis, J., & Tavasszy, L. (2016). A supplier selection life cycle approach integrating traditional and environmental criteria using the best worst method. Journal of Cleaner Production, 135, 577–588.

    Google Scholar 

  • Roy, S. A., Ali, S. M., Kabir, G., Enayet, R., Suhi, S. A., Haque, T., & Hasan, R. (2020). A framework for sustainable supplier selection with transportation criteria. International Journal of Sustainable Engineering, 13(2), 77–92.

    Google Scholar 

  • Salimi, N., & Rezaei, J. (2018). Evaluating firms’ R&D performance using best worst method. Evaluation and Program Planning, 66, 147–155.

    Google Scholar 

  • Schmitz, J., & Platts, K. W. (2004). Supplier logistics performance measurement: Indications from a study in the automotive industry. International Journal of Production Economics, 89(2), 231–243.

    Google Scholar 

  • Schramm, V. B., Cabral, L. P. B., & Schramm, F. (2020). Approaches for supporting sustainable supplier selection-A literature review. Journal of Cleaner Production, 273, 123089.

    Google Scholar 

  • Shang, Z., Yang, X., Barnes, D., & Wu, C. (2022). Supplier selection in sustainable supply chains: Using the integrated BWM, fuzzy Shannon entropy, and fuzzy MULTIMOORA methods. Expert Systems with Applications, 195, 116567.

    Google Scholar 

  • Sillanpää, I., Shahzad, K., & Sillanpää, E. (2015). Supplier development and buyer-supplier relationship strategies–a literature review. International Journal of Procurement Management, 8(1–2), 227–250.

    Google Scholar 

  • Solanki, R., Kannan, D., Kaul, A., & Jha, P. C. (2022). Barrier analysis for carbon regulatory environmental policies implementation in manufacturing supply chains to achieve zero carbon. Journal of Cleaner Production, 131910.

  • Sharma, R., Kannan, D., Darbari, J. D., & Jha, P. C. (2022). Analysis of Collaborative Sustainable Practices in multi-tier food supply chain using integrated TISM-Fuzzy MICMAC model: A supply chain practice view. Journal of Cleaner Production, 354, 131271.

    Google Scholar 

  • Sonar, H., Gunasekaran, A., Agrawal, S., & Roy, M. (2022). Role of lean, agile, resilient, green, and sustainable paradigm in supplier selection. Cleaner Logistics and Supply Chain, 100059.

  • Singh, M., Rathi, R., & Garza-Reyes, J. A. (2021). Analysis and prioritization of Lean Six Sigma enablers with environmental facets using best worst method: A case of Indian MSMEs. Journal of Cleaner Production, 279, 123592.

    Google Scholar 

  • Steuer, R. E. (1986). Multiple criteria optimization: theory, computation, and applications. New York: Wiley.

    Google Scholar 

  • Stević, Ž, Pamučar, D., Puška, A., & Chatterjee, P. (2020). Sustainable supplier selection in healthcare industries using a new MCDM method: Measurement of alternatives and ranking according to COmpromise solution (MARCOS). Computers & Industrial Engineering, 140, 106231.

    Google Scholar 

  • Tavana, M., Yazdani, M., & Di Caprio, D. (2017). An application of an integrated ANP–QFD framework 1237 for sustainable supplier selection. International Journal of Logistics Research and Applications, 20(3), 254–275.

  • Tavana, M., Shaabani, A., Di Caprio, D., & Bonyani, A. (2021). An integrated group fuzzy best-worst method and combined compromise solution with Bonferroni functions for supplier selection in reverse supply chains. Cleaner Logistics and Supply Chain, 2, 100009.

    Google Scholar 

  • Thongchattu, C., & Siripokapirom, S. (2010, August). Notice of Retraction: Green supplier selection consensus by neural network. In 2010 2nd International Conference on Mechanical and Electronics Engineering (Vol. 2, pp. V2–313). IEEE.

  • Tidwell, A., & Sutterfield, J. S. (2012). Supplier selection using QFD: a consumer products case study. International Journal of Quality & Reliability Management.

  • Tong, L. Z., Wang, J., & Pu, Z. (2022). Sustainable supplier selection for SMEs based on an extended PROMETHEE II approach. Journal of Cleaner Production, 330, 129830.

    Google Scholar 

  • Törn, A. A. (1980). A sampling-search-clustering approach for exploring the feasible/efficient solutions of MCDM problems. Computers & Operations Research, 7(1–2), 67–79.

    Google Scholar 

  • Uluskan, M., Godfrey, A. B., & Joines, J. A. (2017). Impact of competitive strategy and cost-focus on global supplier switching (reshore and relocation) decisions. The Journal of the Textile Institute, 108(8), 1308–1318.

    Google Scholar 

  • Vahidi, F., Torabi, S. A., & Ramezankhani, M. J. (2018). Sustainable supplier selection and order allocation under operational and disruption risks. Journal of Cleaner Production, 174, 1351–1365.

    Google Scholar 

  • van der Westhuizen, J., & Ntshingila, L. (2020). The Effect of supplier selection, supplier development and information sharing on sme’s business performance in sedibeng. International Journal of Economics and Finance Studies, 12(2), 153–167.

    Google Scholar 

  • Vasiljević, M., Fazlollahtabar, H., Stević, Ž, & Vesković, S. (2018). A rough multicriteria approach for evaluation of the supplier criteria in automotive industry. Decision Making: Applications in Management and Engineering, 1(1), 82–96.

    Google Scholar 

  • Wagner, S. M., & Friedl, G. (2007). Supplier switching decisions. European Journal of Operational Research, 183(2), 700–717.

    Google Scholar 

  • Wu, L. C. (2009). Supplier selection under uncertainty: a switching options perspective. Industrial Management & Data Systems.

  • Wu, Y., Chen, K., Zeng, B., Xu, H., & Yang, Y. (2016). Supplier selection in nuclear power industry with extended VIKOR method under linguistic information. Applied Soft Computing, 48, 444–457.

    Google Scholar 

  • Xu, C., & Xiang-Yang, L. (2007, August). Multiphase supplier selection model based on supplier development orientation. In 2007 international conference on management science and engineering (pp. 826–831). IEEE.

  • Yazdani, M., Torkayesh, A. E., Stević, Ž, Chatterjee, P., Ahari, S. A., & Hernandez, V. D. (2021). An interval valued neutrosophic decision-making structure for sustainable supplier selection. Expert Systems with Applications, 183, 115354.

    Google Scholar 

  • Yazdi, M., Nedjati, A., Zarei, E., & Abbassi, R. (2020). A reliable risk analysis approach using an extension of best-worst method based on democratic-autocratic decision-making style. Journal of Cleaner Production, 256, 120418.

    Google Scholar 

  • Zavadskas, E. K., & Turskis, Z. (2011). Multiple criteria decision making (MCDM) methods in economics: An overview. Technological and Economic Development of Economy, 17(2), 397–427.

    Google Scholar 

  • Zarbakhshnia, N., Govindan, K., Kannan, D., & Goh, M. (2022). Outsourcing logistics operations in circular economy towards to sustainable development goals. Business Strategy and the Environment (in press).

  • Zavadskas, E. K., Turskis, Z., & Kildienė, S. (2014). State of art surveys of overviews on MCDM/MADM methods. Technological and Economic Development of Economy, 20(1), 165–179.

    Google Scholar 

  • Zhan, Y., Chung, L., Lim, M. K., Ye, F., Kumar, A., & Tan, K. H. (2021). The impact of sustainability on supplier selection: A behavioural study. International Journal of Production Economics, 236, 108118.

    Google Scholar 

  • Zhang, J., Tang, W., & Hu, M. (2015). Optimal supplier switching with volume-dependent switching costs. International Journal of Production Economics, 161, 96–104.

    Google Scholar 

  • Zimmer, K., Fröhling, M., & Schultmann, F. (2016). Sustainable supplier management–a review of models supporting sustainable supplier selection, monitoring and development. International Journal of Production Research, 54(5), 1412–1442.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Devika Kannan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A

1.1 A.1 Multi-criteria decision making (MCDM)

MCDM is the study of methods that are applied to make decisions in the presence of multiple, usually conflicting, criteria (Törn, 1980). MCDM has become a powerful tool to solve design problems that involve multiple conflicting quantitative and qualitative criteria. It includes different perspectives and varied knowledge domains of the decision makers (DMs) so that all the criteria are properly catered for optimizing the problem and attaining appropriate decisions as per the objective (Mousavi-Nasab, and Sotoudeh-Anvari, 2017). Hence, MCDM methods are best utilized in situations where conjoint decision analysis is required and the conflicts among conjoint DMs are needed to transform into priority weights (Chowdhury & Paul, 2020). Ching and Kwangsun (1981) have classified the problems of MCDM broadly into two categories: 1) Multiple Attribute Decision Making (MADM), 2) Multiple Objective Decision Making (MODM). The aim of the study is to utilize the combined efficiencies and effectiveness of both MODM and MADM. The combined approach is extremely beneficial in decision making problem situations that include the multi-dimensions of sustainability objectives. It facilitates an efficient framework to model the complexities while formulating the supply chain problems and enables DMs for participating actively in decision making. The key concepts of MODM and MADM techniques used in the study are briefly described below.

1.1.1 A.1.1 Multiple attribute decision making (MADM)

MADM methods are applied to evaluate and solve discrete problems with limited number of predetermined alternatives (Zavadskas & Turskis, 2011). Evaluating a limited number of alternatives inherent in a range of attributes is difficult; prioritizing them adds further complexity. To solve such arrays of problems, MADM methods are best suited. In MADM, the final selection of the alternative is decided by various alternatives’ comparisons with respect to each inter- and intra-attribute. The comparisons may include explicit or implicit tradeoffs. The MADM methods BWM and MARCOS applied in this study to evaluate the supplier performance are explained briefly below while we provide the advantages of both over other MADM methods.

1.1.1.1 A.1.1.1 Best–worst method

The best worst method (BWM) is a MADM method that has been proposed by Rezaei (2015). BWM is based on pairwise comparisons; the advantages of this method over other MCDM methods are: 1) it requires less pairwise comparison data entries as compared to other full pairwise comparison based MADM methods, and 2) the consistency of results generated from BWM are better than the other full pairwise comparison-based MADM methods. Given the above advantages of BWM, it has become a popular MADM technique and is extremely useful in various applications such as supplier selection (Rezaei et al., 2016), social sustainability assessment of supply chains (Ahmadi et al., 2017), sustainable Manufacturing (Malek & Desai, 2021), R&D performance evaluation of firms (Salimi & Rezaei, 2018), facility location selection (Kheybari et al., 2019), sustainable manufacturing barriers prioritization (Malek & Desai, 2019), risk analysis (Yazdi et al., 2020), and Lean six sigma enablers prioritization (Singh et al., 2021).

As discussed previously, a set of SKPIs \(\{{SKPI}_{1}, {SKPI}_{2},\dots ,{SKPI}_{U}\}\) are divided into three categories. The sets of economic, environmental, and social sub-SKPIs \(\{{SKPI}_{1}^{Eco},{SKPI}_{2}^{Eco},\dots ,{SKPI}_{L}^{Eco}\}\), \(\{{SKPI}_{1}^{Env}, {SKPI}_{2}^{Env}, \dots , {SKPI}_{M}^{Env}\}\), and \(\{{SKPI}_{1}^{Soc}, {SKPI}_{2}^{Soc}, \dots , {SKPI}_{N}^{Soc}\}\) respectively are chosen for sustainable evaluation of suppliers. The steps of BWM for finding the importance of ranking of SKPIs are structured as follows:

Step 1: Determine the best and worst SKPI:

The best economic \({SKPI}^{Eco}\) \(\left({SKPI}_{B}^{Eco}, B\in \left\{\mathrm{1,2},\dots ,L\right\}\right)\) i.e., the most important economic SKPI and the worst economic \({SKPI}^{Eco}\) \(({SKPI}_{W}^{Eco}, W\in \left\{\mathrm{1,2},\dots ,L\right\}))\) i.e., the least important economic SKPI are identified by each of the k DMs.

Step 2: Compute the preference of \({SKPI}_{B}^{Eco},\) over all the other SKPIs:

Using a scale of 1–9, the preference of \({SKPI}_{B}^{Eco}\) over each \({SKPI}_{h}^{Eco}\) is calculated for kth DM, denoted as \({a}_{Bh}^{k} \mathrm{with }{a}_{BB}^{k}=1 .\) This results in the best-to-others (BO) vector \({A}_{B}^{k} = ({a}_{B1}^{k}, {a}_{B2}^{k}, \dots , {a}_{BL}^{k})\)

Step 3: Compute the preference of \({SKPI}_{W}^{Eco}\) over all the other SKPIs:

Using the same scale, the preference of each \({SKPI}_{h}^{Eco}\) over worst KPI \({SKPI}_{W}^{Eco}\) is calculated for kth DM, denoted as \({a}_{hW}^{k} \mathrm{with }{a}_{WW}^{k}=1.\) This results in the others-to-worst (OW) vector \({A}_{W}^{k} = ({a}_{1W}^{k}, {a}_{2W}^{k}, \dots , {a}_{LW}^{k})\)

Step 4: Calculate the optimal weights of each \({SKPI}^{Eco}\) for each DM.

To determine the unique optimal weighting vector \(\left({w}_{1}^{k*},{ w}_{2}^{k*},\dots ,{w}_{L}^{k*}\right)\) of the set of SKPIs for kth DM, the following maximum absolute difference is to be minimized:

$$ max\left\{ {\left| {w_{B}^{k*} - a_{Bl}^{k} w_{l}^{k*} } \right|,\left| {w_{l}^{k*} - a_{lW}^{k} w_{W}^{k*} } \right|;l = 1, 2, \ldots ,L} \right\} $$

This is achieved by the following optimization model:

$$ \begin{aligned} & min\mathop {\max }\limits_{l} \{ |\frac{{w_{B}^{k*} }}{{w_{l}^{k*} }} - a_{Bl}^{k} \left| , \right|\frac{{w_{l}^{k*} }}{{w_{W}^{k*} }} - a_{lW}^{k} |\} \\ & {\text{Subject to}} \\ & \mathop \sum \limits_{l} w_{l}^{k*} = {1} \\ & w_{l}^{k*} \ge 0,\quad \forall l \in \left\{ {1 ,2, \ldots ,L} \right\} \\ \end{aligned} $$
(P1)

The above fractional programming problem formulation (P1) is transformed into a linear programming problem formulation as given below (Rezaei, 2016):

$$ \begin{aligned} & \min {\upxi }^{k} \\ & {\text{Subject to}} \\ & \left| {w_{B}^{k} - a_{Bl}^{k} w_{l}^{k} } \right|\} \le {\upxi }^{k} \quad \forall l \in \left\{ {1, 2, \ldots ,L} \right\} \\ & \left| {w_{l}^{k} - a_{lW}^{k} w_{W}^{k} } \right|\} \le {\upxi }^{k} \quad \forall l \in \left\{ {1, 2, \ldots ,L} \right\} \\ & \mathop \sum \limits_{l} w_{l}^{k} = {1} \\ & w_{l}^{k} \ge 0, \quad \forall { }l \in \left\{ {1, 2, \ldots ,L} \right\} \\ \end{aligned} $$
(P2)

Problem (P2) provides a unique optimal weighting vector \(\left( {w_{1}^{k*} , w_{2}^{k*} , \ldots ,w_{L}^{k*} } \right)\) and optimal value \(\xi^{k*}\) for kth DM. The desirable value of \(\xi^{k*}\) is closer to zero as it indicates a high consistency and high reliability.

Problem (P2) is solved to compute the optimal weighting vector for each DM. Further, for computing the final weights of each \(SKPI^{Eco}\), the average of all the attained optimal weights for DMs is calculated by given formula.

$$ w_{l}^{Eco} = \frac{{\mathop \sum \nolimits_{k = 1}^{K} w_{l}^{k*} }}{K}\quad \forall l = 1,2, \ldots ,L. $$
(11)

The final optimal weighting vector (\(w_{1}^{Eco} , w_{2}^{Eco} , \ldots ,w_{L}^{Eco} )\) is determined and provides the priority weights of each \(SKPI^{Eco}\). Similarly, this procedure is performed for \(SKPI^{Env}\) and \(SKPI^{Soc}\) to determine the optimal weighting vectors (\(w_{1}^{Env} , w_{2}^{Env} , \ldots ,w_{M}^{Env} )\) and (\(w_{1}^{Soc} , w_{2}^{Soc} , \ldots ,w_{N}^{Soc} )\) respectively.

Further, the procedure is repeated for three categories (economic, environmental, and social) and determines the optimal weighting vector \(\left( {w^{Eco} , w^{Env} , w^{Soc} } \right)\).

To compute the global weights of each SKPI \(w_{1}^{SKPI} , w_{2}^{SKPI} , \ldots ,w_{U}^{SKPI}\), the weight obtained for each SKPI belong to each category is multiplied by the weight of the category.

1.1.1.2 A.1.1.2 MARCOS

Measurement of Alternatives and Ranking according to Compromise Solution (MARCOS) is a MADM technique developed by Stevic, Pamučar, Puška, and Chatterjee (2020). The main advantage of this method over other MADM methods is its consideration of ideal and anti-ideal alternatives at the initial stage of the evaluation process. The closer determination of utility degree with respect to ideal and anti-ideal solutions provides greater stability while considering a large set of criteria and alternatives. The steps of the MARCOS method for computing the priority weights of incumbent and new suppliers with respect to SKPIs are described as follows:

Step 1: Develop an initial decision-making matrix

In this step, a decision matrix is developed for incumbent suppliers with respect to each DM. The opinions of all DMs are included to form a decision matrix for supplier evaluation with respect to SKPIs. DMs are asked to evaluate the U SKPIs for each incumbent supplier on the scale of five degrees: 1,3,5,7 and 9, where 1 = very poor, 3 = poor, 5 = average, 7 = good, 9 = very good. The evaluation decision matrix filled by kth DM is

$$ T^{k} = \left[ {\begin{array}{*{20}l} {x_{11}^{k} } \hfill & {x_{12}^{k} } \hfill & \cdots \hfill & {x_{1U}^{k} } \hfill \\ {x_{21}^{k} } \hfill & {x_{22}^{k} } \hfill & \cdots \hfill & {x_{2U}^{k} } \hfill \\ \cdots \hfill & \cdots \hfill & \cdots \hfill & \cdots \hfill \\ {x_{I1}^{k} } \hfill & {x_{I2}^{k} } \hfill & \cdots \hfill & {x_{IU}^{k} } \hfill \\ \end{array} } \right] $$
(12)

In group decision-making, evaluation decision matrices formed by members of expert panel are aggregated into initial decision-making matrix by taking the average of evaluation decision matrices. The initial decision-making matrix is

$$ T = \left[ {\begin{array}{*{20}l} {x_{11} } \hfill & {x_{12} } \hfill & \cdots \hfill & {x_{1U} } \hfill \\ {x_{21} } \hfill & {x_{22} } \hfill & \cdots \hfill & {x_{2U} } \hfill \\ \cdots \hfill & \cdots \hfill & \cdots \hfill & \cdots \hfill \\ {x_{I1} } \hfill & {x_{I2} } \hfill & \cdots \hfill & {x_{IU} } \hfill \\ \end{array} } \right] $$
(13)

Step 2: Form an extended initial matrix

To extend the initial decision-making matrix, the ideal (AI) and anti-ideal (AAI) solutions are computed. The AI solution is the best incumbent supplier with respect to SKPIs, whereas the AAI solution is the worst incumbent supplier with respect to SKPIs. Based on the nature of the SKPIs, AI and AAI are defined as follows:

$$ AI = \mathop {\max }\limits_{i} x_{iu } \, if\,u\epsilon Ben\,and\, \mathop {\min }\limits_{i} \,x_{iu} \,if\,u\epsilon Cos $$
(14)
$$ AAI = \mathop {\min }\limits_{i} x_{iu} \, if\,u\epsilon Ben\,and\,\mathop {\max }\limits_{i} \,x_{iu } \,if\, u\epsilon Cos $$
(15)

where Ben: a benefit group of SKPIs, and Cos: a group of cost SKPIs.

The extended initial matrix X is performed as follows:

(16)

Step 3: Normalize the extended initial matrix

The extended initial matrix X is normalized to obtain normalized extended initial matrix \(Y = \left[ {y_{iu} } \right]_{I \times U}\) by using the following equations:

$$ y_{iu} = \frac{{x_{ai} }}{{x_{iu} }} \,if\, u\epsilon {\text{Cos}} $$
(17)
$$ y_{iu} = \frac{{x_{iu} }}{{x_{ai} }}\,if\, u\epsilon Ben $$
(18)

where \(x_{ai}\) and \(x_{iu}\) are the elements from the initial decision matrix.

Step 4: Determine the weighted matrix

The weighted matrix \(Z = \left[ {z_{iu} } \right]_{I \times U}\) is computed by multiplying the normalized matrix Y with the weight coefficient of the SKPI \(w_{u}^{SKPI}\), using the following expression:

$$ z_{iu} = y_{iu} \times w_{u}^{SKPI} $$
(19)

Step 5: Calculate the utility degree of suppliers

The utility degree of each incumbent supplier with regard to anti-ideal and ideal solution are computed using the following expressions:

$$ P_{i}^{ - } = \frac{{Q_{i} }}{{Q_{aai} }} $$
(20)
$$ P_{i}^{ + } = \frac{{Q_{i} }}{{Q_{ai} }} $$
(21)

where \(Q_{i} \) represents the sum of the elements of the weighted matrix Z using following expression:

$$ Q_{i} = \mathop \sum \limits_{i = 1}^{I} z_{iu} $$
(22)

Step 6: Determine the utility function of alternatives

The utility function of each incumbent supplier with regard to the ideal and anti-ideal solution is determined using the following expression:

$$ f\left( {P_{i} } \right) = \frac{{P_{i}^{ + } + P_{i}^{ - } }}{{1 + \frac{{1 - f\left( {P_{i}^{ + } } \right)}}{{f\left( {P_{i}^{ + } } \right)}} + \frac{{1 - f\left( {P_{i}^{ - } } \right)}}{{f\left( {P_{i}^{ - } } \right)}}}}; $$
(23)

where \(f\left( {P_{i}^{ - } } \right)\): the utility function with regard to the anti-ideal solution and \(f\left( {P_{i}^{ + } } \right):\) the utility function with regard to the ideal solution. These are computed by using the following expression:

$$ f\left( {P_{i}^{ - } } \right) = \frac{{P_{i}^{ + } }}{{\left( {P_{i}^{ + } + P_{i}^{ - } } \right)}} $$
(24)
$$ f\left( {P_{i}^{ + } } \right) = \frac{{P_{i}^{ - } }}{{\left( {P_{i}^{ + } + P_{i}^{ - } } \right)}} $$
(25)

Step 7: Compute the weightage of alternatives

The priority weights of incumbent suppliers are computed by normalizing the final utility function values. For each incumbent supplier, it is desirable to achieve highest utility function value.

This procedure is repeated for the set of new suppliers to determine their priority weights.

1.1.2 A.1.2 Multiple objective decision making (MODM)

The Multiple objective decision making (MODM) technique is best suited for continuous decision spaces and associated with the problem where the alternatives are non-predetermined (Zavadskas et al., 2014). The aim of considered problem is to design the 'best' alternative under consideration of a set of quantifiable objectives and a set of well-defined designed constraints. MODM facilitates a process of obtaining some tradeoff information, implicit or explicit, between the stated quantifiable objectives and also between stated or unstated nonquantifiable objectives.

Real world decision-making problems generally have multi-objectives that cannot be optimized simultaneously due to inherent incommensurability and conflict between these objectives. Thus, to get the best compromise solution, achievement of trade-off between these objective plays a main role. Several methodologies are proposed for solving MODM problems in the existing literature (Hwang et al., 1993).

Mathematically, MODM problems can be expressed as follows:

$$ \min F\left( x \right) = [f_{1} \left( x \right),f_{2} \left( x \right), \ldots ,f_{O} \left( x \right)]^{T} $$

s.t.

$$ x\epsilon S = {\text{\{ }}x{|}g_{h} \left( x \right){)}\left\{ { \ge , = , \le } \right\}0, h = 1,2, \ldots ,H\} $$

where \(S \subseteq R^{n}\) is the feasible space (Steuer, 1986).

The problem consists of H constraints and O objectives. \(f_{o} \left( x \right)\) and \(g_{h} \left( x \right)\), can be linear or non-linear.

As discussed earlier, optimization of all the objectives simultaneously is not possible. In this case, the concept of non-inferiority i.e., known as efficiency or Pareto optimality is utilized to obtain the solution to the MODM problems. This concept is based on the following definitions (Hsiao et al., 1994):

Definition 1

The feasible region S is the set of state vectors x that satisfy the constraints: \(S = {\text{\{ }}x{|}g_{h} \left( x \right){)}\left\{ { \ge , = , \le } \right\}0, h = 1,2, \ldots ,H\}\).

Definition 2

A point \(\check{x}\epsilon S\) is a local non-inferior point if there exists an \(\rho > 0\) such that in the neighborhood \(N\left( {\check{x},\rho } \right)\) of \(\check{x}\), there exists no other point x such that (i) \(f_{o} \left( x \right) \le f_{o} \left( {\check{x}} \right),o = 1,2, \ldots ,O\) and (ii) \(f_{{o^{{\prime }} }} \left( x \right) < f_{{o^{{\prime }} }} \left( {\check{x}} \right),\) for some \(o^{{\prime }} \epsilon \left\{ {1,2, \ldots ,O^{{\prime }} } \right\}\).

Definition 3

A point \(\check{x}\epsilon S\) is a global non-inferior point if and only if there exists no other point x such that (i) \(f_{o} \left( x \right) \le f_{o} \left( {\check{x}} \right),i = 1,2, \ldots ,O\) and (ii) \(f_{{o^{{\prime }} }} \left( x \right) < f_{{o^{{\prime }} }} \left( {\check{x}} \right),\) for some \(o^{{\prime }} \epsilon \left\{ {1,2, \ldots ,O^{{\prime }} } \right\}\).

A non-inferior solution of a MODM problem is one in which any improvement of one objective function can be achieved only at the expense of at least one of the other objectives. In general, there are an infinite number of (global) non-inferior points for a given MODM problem; this makes the task of finding the collection of such points called the non-inferior set extremely difficult.

For generating an entire non-inferior set, there are few methods such as the weight sum method, kth-objective method, and the Epsilon constraint method. In the current study, the Epsilon constraint method, applied to solve the proposed optimization model, is discussed in detail as follows.

1.1.2.1 A.1.2.1 Epsilon constraint method

The Epsilon constraint is a MODM approach, proposed by Haimes (1971), to generate Pareto optimal solutions. These solutions provide a clear understanding of the Pareto optimal set. In this method, initially the primary objective function and secondary objective functions are decided (Engau & Wiecek, 2007). Further, the MODM problem is transformed into single objective problem by shifting the other objectives in constraints with some allowable amount \(\varepsilon\) (Nouri et al., 2018). The advantages of \(\varepsilon\)-constraint method over other MODM methods are such as (i) Weighting approach results in merely Pareto optimal extreme solutions for linear models, whereas non-extreme solutions can be produced through applying ε-constraint method. (ii) Unlike the weighting approach, in multi-objective problems via integer programming as well as mixed-integer linear programming ones, unsupported Pareto optimal solutions can be produced by ε-constraint method. (iii) Weighting approach highly depends on the scaling of objective functions while this issue is not important in the ε-constraint method. (iv) The controlled number of Pareto optimal solutions can be produced in ε-constraint method by effectively tuning the number of grid points in the range of each objective function.

Based on \(\varepsilon\)-constraint method, a single-objective model of problem (P3) is represented as follows:

$$ \begin{aligned} & {\text{Objective}}\,{\text{Function}} = Min f_{1} \left( x \right) \\ & s.t. \\ & f_{o} \left( x \right) \ge \epsilon_{o} ,o = 2,3, \ldots ,O \\ & x\varepsilon S = {\text{\{ }}x{|}g_{h} \left( x \right){)}\left\{ { \ge , = , \le } \right\}0, h = 1,2, \ldots ,H\} \\ \end{aligned} $$

For generating the Pareto optimal solutions, the \(\varepsilon_{o}\) are altered from minimum to maximum amount of \(f_{o} \left( x \right)\) and the goal is obtained.

1.2 A.2 Fuzzy decision

For deciding the best compromised solution, max–min fuzzy method is utilized in this study. This method a fuzzy membership function in the interval [0,1] to each solution in the Pareto front (Cao et al., 2019). The fuzzy membership functions for oth objective function are as follows:

$$ \mu_{o}^{r} = \left\{ {\begin{array}{*{20}l} 1 \hfill & {\theta_{o}^{r} \le \theta_{o}^{min} } \hfill \\ {\frac{{\theta_{o}^{max} - \theta_{o}^{r} }}{{\theta_{o}^{max} - \theta_{o}^{min} }}} \hfill & {\theta_{o}^{min} \le \theta_{o}^{r} \le \theta_{o}^{max} } \hfill \\ 0 \hfill & {\theta_{o}^{r} \ge \theta_{o}^{max} } \hfill \\ \end{array} } \right. $$
(26)

where \(\mu_{o}^{r}\): the optimality degree of the rth solution of oth objective function.

In the oth objective function \(f_{o}^{min}\) and \(f_{o}^{max}\) are the minimum and maximum values of Pareto solutions. Then, per unit quantities are compared and the minimum value is selected based on Eq. (17). Finally, the best point as trade-off solution for multi-objective solution is the maximum amount of all the minimums Eq. (18).

$$ \mu^{r} = \min \left( {\mu_{1}^{r} , \ldots ,\mu_{o}^{r} } \right);\forall r = 1,2, \ldots .Rp $$
(27)
$$ \mu^{max} = \max \left( {\mu^{1} ,\mu^{2} , \ldots ,\mu^{Rp} } \right). $$
(28)

Appendix B

See Table 5.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aditi, Kannan, D., Darbari, J.D. et al. Sustainable supplier selection model with a trade-off between supplier development and supplier switching. Ann Oper Res 331, 351–392 (2023). https://doi.org/10.1007/s10479-022-04812-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-022-04812-2

Keywords

Navigation