A hybrid method for evaluating the effectiveness of giant systems with indicator correlations: an application for naval formation decision making in multiple scenarios

  • Xiaowei Xu
  • Xinlian XieEmail author
  • Bofei Zhang
  • Wei Pan
Methodologies and Application


This article discusses the giant system effectiveness evaluation (GSEE) problem with inevitable correlation in indicator systems due to their high specificity and complexity and proposes a hybrid method that is then applied to the naval formation decision-making process in multiple scenarios. The indicator correlation in a large-scale system will generate bias in its evaluation of effectiveness; the proposal that the lower the correlation is, the better the performance of the evidential reasoning approach (ERA) has been proven mathematically. In light of this proposition, a corollary was put forward: Fewer indicators would improve the precision of the result of the ERA application when considering the correlation. Considering that the giant system can be split into respective subsystems, which can then be analyzed by experts in their own fields, a hybrid method was developed for the GSEE problem based on the ERA and prospect theory. The core of the method is the construction of a nonlinear optimization model (NOM) aimed at minimizing the correlation and maximizing the evaluation ability of the prospect value of the indicator system. By constraint, the NOM also includes the optimized weight value of each indicator. For demonstration purposes, a naval formation operation effectiveness evaluation (NFOEE) was performed to assess the feasibility of the proposed method and the NOM. The results show that the proposed method can solve the NFOEE effectively and allow the decision maker to obtain useful information for naval formation-type decisions in multiple scenarios. Furthermore, the evaluation method is a general tool that can be applied to other GSEE problems.


Giant system Effectiveness evaluation Indicator correlation Naval formation Evidential reasoning approach Prospect theory 



The authors are very grateful to the anonymous reviewers and editor for their very valuable comments and suggestions, which were greatly helpful in revising the manuscript. This work is supported by National Key R&D Program of China (Grant No. 2017YFC0805309) and the Fundamental Research Funds for the Central Universities (Logistics Research Institute, Dalian Maritime University Grant No. 3132019303).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.


  1. Alomair Y, Ahmad I, Alghamdi A (2015) A review of evaluation methods and techniques for simulation packages. Procedia Comput Sci 62:249–256CrossRefGoogle Scholar
  2. Bao T, Xie X, Long P, Wei Z (2017) MADM method based on prospect theory and evidential reasoning approach with unknown attribute weights under intuitionistic fuzzy environment. Expert Syst Appl 88:305–317CrossRefGoogle Scholar
  3. Cao J, Chen G, Khoveyni M, Eslami R, Yang G (2016) Specification of a performance indicator using the evidential-reasoning approach. Knowl Based Syst 92(C):138–150CrossRefGoogle Scholar
  4. Chen X, Jing Y, Li C, Jiang N (2010) Effectiveness evaluation of warfare command systems with dissymmetrical warfare information. In: IEEE American control conference, pp 5556–5560Google Scholar
  5. Chin KS, Fu C (2014) Integrated evidential reasoning approach in the presence of cardinal and ordinal preferences and its applications in software selection. Pergamon Press Inc, OxfordCrossRefGoogle Scholar
  6. Chin KS, Fu C, Wang Y (2015) A method of determining attribute weights in evidential reasoning approach based on incompatibility among attributes. Pergamon Press Inc, OxfordCrossRefGoogle Scholar
  7. Cho V (2007) A study of the impact of organizational learning on information system effectiveness. Int J Bus Inf 2(1):127–158Google Scholar
  8. Dubois D, Pap E, Prade H (2000) Hybrid probabilistic-possibilistic mixtures and utility functions. In: Fodor J, de Baets B, Perny P (eds) Preferences and Decisions under Incomplete Knowledge. Studies in Fuzziness and Soft Computing, vol 51. Springer, Heidelberg, pp 51–73zbMATHCrossRefGoogle Scholar
  9. Dubrova E (2013) Information redundancy. In: Dubrova E (ed) Fault-tolerant design. Springer, New YorkzbMATHCrossRefGoogle Scholar
  10. Fahmi A, Abdullah S, Amin F, Ali A (2017) Precursor selection for sol–gel synthesis of titanium carbide nano powders by a new cubic fuzzy multi-attribute group decision-making model. J Intell Fuzzy Syst. CrossRefGoogle Scholar
  11. Fahmi A, Fazli A, Saleem A, Asad A (2018a) Cubic fuzzy Einstein aggregation operators and its application to decision-making. Int J Syst Sci 49:1–13MathSciNetCrossRefGoogle Scholar
  12. Fahmi A, Abdullah S, Amin F, Sajjad Ali Khan M (2018b) Trapezoidal cubic fuzzy number einstein hybrid weighted averaging operators and its application to decision making. Soft Comput 23:5753zbMATHCrossRefGoogle Scholar
  13. Fazli A, Fahmi A, Abdullah S, Ahmed A, Rehan A, Ghani F (2018) Triangular cubic linguistic hesitant fuzzy aggregation operators and their application in group decision making. J Intell Fuzzy Syst 34:2401–2416CrossRefGoogle Scholar
  14. Grabisch M, Marichal JL, Mesiar R, Pap E (2009) Aggregation functions. Cambridge University Press, CambridgezbMATHCrossRefGoogle Scholar
  15. Grabisch M, Marichal JL, Mesiar R, Pap E (2011a) Aggregation functions: means. Inf Sci 181(1):1–22MathSciNetzbMATHCrossRefGoogle Scholar
  16. Grabisch M, Marichal JL, Mesiar R, Pap E (2011b) Aggregation functions: construction methods, conjunctive, disjunctive and mixed classes. Inf Sci 181(1):23–43MathSciNetzbMATHCrossRefGoogle Scholar
  17. Gul M, Celik E, Akyuz E (2017) A hybrid risk-based approach for maritime applications: the case of ballast tank maintenance. Hum Ecol Risk Assess 23(6):1389–1403CrossRefGoogle Scholar
  18. Huang YY, Wang JY, Wang J, Dai YW (2009) An evaluation method of operational effectiveness for the emergency plans. In: IEEE informs international conference on service operations, logistics and informatics, pp 169–172Google Scholar
  19. Huo JH, Zhang GH, Cai HL, Yuan L (2011) Research on an quantitative method for C^(4)ISR system effectiveness evaluation. Int Conf Intell Hum Mach Syst Cybern IEEE Comput Soc 2:23–26Google Scholar
  20. Hwang CL, Yoon KP (1981) Multiple attribute decision making: methods and applications. A state-of- the-art survey. Eur J Oper Res 4(4):287–288zbMATHGoogle Scholar
  21. Jeong E, Oh C (2017) Evaluating the effectiveness of active vehicle safety systems. Accid Anal Prev 100:85–96CrossRefGoogle Scholar
  22. Kahneman D, Tversky A (1979) Prospect theory: an analysis of decision under risk. Econometrica 47(2):263–291MathSciNetzbMATHCrossRefGoogle Scholar
  23. Kanada N, Morioka K, Sumiya Y, Yonemoto N, Shioji M, Kohmura A et al (2014) Evaluation by scale model experiments for aeronautical MIMO systems effectiveness of channel capacity evaluation in 5.1GHz band. In: IEEE antenna technology, pp 246–249Google Scholar
  24. Kang J, Zhang J, Bai Y (2016) Modeling and evaluation of the oil-spill emergency response capability based on linguistic variables. Mar Pollut Bull 113(1–2):293–301CrossRefGoogle Scholar
  25. Kong G, Xu DL, Yang JB, Ma X (2015) Combined medical quality assessment using the evidential reasoning approach. Expert Syst Appl 42(13):5522–5530CrossRefGoogle Scholar
  26. Li J, Fu C, Chen Y, Yang K, Zhang X (2014) An operational efficiency evaluation method for weapon system-of-systems combat networks based on operation loop. In: IEEE international conference on system of systems engineering, pp 219–223Google Scholar
  27. Liang H, Xiong W, Dong Y (2018) A prospect theory-based method for fusing the individual preference-approval structures in group decision making. Comput Ind Eng 117:237–248CrossRefGoogle Scholar
  28. Liberatore MJ (1982) Book review of the analytic hierarchy process: planning, priority setting, resource allocation by Thomas L. Saaty. Am J Math Manag Sci 2(2):165–172Google Scholar
  29. Liu SL, Li H (2017) Modified ADC method and its application for weapon system effectiveness evaluation. J Natl Univ Def Technol 39(3):130–135Google Scholar
  30. Liu P, Jin F, Zhang X, Su Y, Wang M (2011) Research on the multi-attribute decision-making under risk with interval probability based on prospect theory and the uncertain linguistic variables. Knowl Based Syst 24(4):554–561CrossRefGoogle Scholar
  31. Liu Y, Meng M, Zhang H, Yao J (2014) Effectiveness evaluation method of naval ship oriented to multi-mission base on MAS and FS. J Harbin Inst Technol 46(2):121–128Google Scholar
  32. Meng F, Wang N, Li B (2015) Selection method of evaluation indicators with three-parameter interval grey number. Open J Appl Sci 5(12):833–840CrossRefGoogle Scholar
  33. Muñoz-Villamizar A, Santos J, Montoya-Torres JR, Jaca C (2018) Using oee to evaluate the effectiveness of urban freight transportation systems: a case study. Int J Prod Econ 197(C):232–242CrossRefGoogle Scholar
  34. Ng CY (2016) An evidential reasoning-based AHP approach for the selection of environmentally-friendly designs. Environ Impact Assess Rev 61:1–7CrossRefGoogle Scholar
  35. Pap E (1997) Pseudo-analysis as a mathematical base for soft computing. Soft Comput 1(2):61–68MathSciNetCrossRefGoogle Scholar
  36. Pap E (2002) Aggregation operators in the engineering design. In: Calvo T, Mayor G, Mesiar R (eds) Aggregation Operators. Studies in Fuzziness and Soft Computing, vol 97. Springer, Heidelberg, pp 195–223zbMATHCrossRefGoogle Scholar
  37. Qi ZF, Wang GS (2014) Grey synthetic relational analysis method-based effectiveness evaluation of ewcc system with incomplete information. Appl Mech Mater 638–640:2409–2412CrossRefGoogle Scholar
  38. Rudas IJ, Pap E, Fodor J (2013) Information aggregation in intelligent systems: an application oriented approach. Knowl Based Syst 38:3–13CrossRefGoogle Scholar
  39. Saaty TL (1980) The analytic hierarchy process: planning, priority setting, resource allocation. McGraw-Hill, New YorkzbMATHGoogle Scholar
  40. Stolzer AJ, Friend MA, Truong D, Tuccio WA, Aguiar M (2018) Measuring and evaluating safety management system effectiveness using data envelopment analysis. Saf Sci 104:55–69CrossRefGoogle Scholar
  41. Wu QB, Ye FC, Wan XJ (2010) Study on measures of effectiveness of weapon system. Electron Prod Reliab Environ Test 28(4):27–31Google Scholar
  42. Wu W, Yan S, Feng R, Song D, Chen X (2017) Development of an environmental performance indicator framework to evaluate management effectiveness for jiaozhou bay coastal wetland special marine protected area, qingdao, China. Ocean Coast Manag 142:71–89CrossRefGoogle Scholar
  43. Wu Y, Xu C, Zhang T (2018) Evaluation of renewable power sources using a fuzzy mcdm based on cumulative prospect theory: a case in china. Energy 147:1227–1239CrossRefGoogle Scholar
  44. Xu Y, Wen X, Zhang W (2018) A two-stage consensus method for large-scale multi-attribute group decision making with an application to earthquake shelter selection. Comput Ind Eng 116:113–129CrossRefGoogle Scholar
  45. Yanyan H (2009) A methodology of simulation and evaluation on the operational effectiveness of weapon equipment. IEEE, Guilin, China, pp 131–136Google Scholar
  46. Yang JB, Singh MG (1994) An evidential reasoning approach for multiple-attribute decision making with uncertainty. IEEE Trans Syst Man Cybern 24(1):1–18CrossRefGoogle Scholar
  47. Yu J, Dai R (1993) A new discipline of science: the study of open complex giant system and its methodology. J Syst Eng Electron 4(2):2–12Google Scholar
  48. Zadeh LA (1965) Fuzzy sets. Inf Control 8(3):338–353zbMATHCrossRefGoogle Scholar
  49. Zhao X, Chen G, Yue Q (2011) Equipment support quality management effectiveness evaluation based on fce method. Procedia Eng 15(1):4377–4381CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Integrated Transport InstituteDalian Maritime UniversityDalianChina

Personalised recommendations