WMU Journal of Maritime Affairs

, Volume 17, Issue 1, pp 71–90 | Cite as

Consistency in the development of performance assessment methods in the maritime domain

  • Jørgen ErnstsenEmail author
  • Salman NazirEmail author


The maritime industry is considered to be a backbone of the global economy. It is therefore imperative to ensure that maritime operations run safely and efficiently. Assessment of maritime performance is necessary for designers and engineers to be able to pinpoint the weakest links in the system and make impactful system improvements. The current article presents a systematic quantitative literature review of research on performance assessment in the maritime industry with the goal of establishing an understanding of accuracy and consistency in the development of methods used to assess performance. The review focuses on four major segments within the industry—port logistics, ship handling, safety and environmental research—and investigates their uses in developing accurate and consistent performance assessment methods. After the completion of an exclusion process, 62 articles published in a wide range of academic journals were used in the analysis. Two important conclusions were drawn from the analysis. First, performance assessment is generally consistent throughout the maritime industry; most papers used accurate and consistent approaches to develop the methods (n = 43). A subsequent bivariate analysis revealed a call for increased attention to the development of assessment methods within the maritime segment of ship handling. The current study suggests and discusses certain directions with regard to assessment research in the maritime industry.


Maritime performance Assessment Performance indicators Literature review Complex operations 


  1. Abril M, Barber F, Ingolotti L, Salido MA, Tormos P, Lova A (2008) An assessment of railway capacity. Transp Res E-Log 44(5):774–806. CrossRefGoogle Scholar
  2. Accident Investigation Board, Norway (2010a) Crete Cement - IMO NO. 9037161, grounding at Aspond Island in the Oslo Fjord, Norway, on 19 November 2008. Report Sjø 1Google Scholar
  3. Accident Investigation Board, Norway (2010b) Report on marine accident Federal Kivalina-IMO NO. 9205885 grounding at Årsundøya, Norway 6 October 2008. Report Sjø 1Google Scholar
  4. Accident Investigation Board, Norway 2012 Report on investigation into marine accident M/V Godafoss V2PM7 grounding in Løperen, Hvaler on 17 February 2011. Report Sjø 1Google Scholar
  5. Aditya P, Uday K, Diego G, Christer S (2015) Performance measurement and management for maintenance: a literature review. J Qual Maint Eng 21(1):2–33. CrossRefGoogle Scholar
  6. Akyuz E, Celik M (2014a) A hybrid decision-making approach to measure effectiveness of safety management system implementations on-board ships. Saf Sci 68:169–179. CrossRefGoogle Scholar
  7. Akyuz E, Celik M (2014b) Utilisation of cognitive map in modelling human error in marine accident analysis and prevention. Saf Sci 70:19–28. CrossRefGoogle Scholar
  8. Altuntaş C, Tuna O (2013) Greening logistics centers: the evolution of industrial buying criteria towards green. Asian Shipp Logist 29(1):59–80. CrossRefGoogle Scholar
  9. Antão P, Calderón M, Puig M, Michail A, Wooldridge C, Darbra RM (2016) Identification of occupational health, safety, security (OHSS) and environmental performance indicators in port areas. Saf Sci 85:266–275CrossRefGoogle Scholar
  10. Bergantino AS, Musso E, Porcelli F (2013) Port management performance and contextual variables: which relationship? Methodological and empirical issues. Res Transp Bus Manag 8:39–49. CrossRefGoogle Scholar
  11. Bichou K, Gray R (2004) A logistics and supply chain management approach to port performance measurement. Marit Policy Manag 31(1):47–67. CrossRefGoogle Scholar
  12. BIMCO (2015) The global supply and demand for seafarers in 2015. Manpower report, InGoogle Scholar
  13. Bouejla A, Chaze X, Guarnieri F, Napoli A (2014) A Bayesian network to manage risks of maritime piracy against offshore oil fields. Saf Sci 68(Supplement C):222–230. CrossRefGoogle Scholar
  14. Chen L, Zhang D, Ma X, Wang L, Li S, Wu Z, Pan G (2016) Container port performance measurement and comparison leveraging ship GPS traces and maritime open data. IEEE Trans Intell Transp Syst 17(5):1227–1242. CrossRefGoogle Scholar
  15. Chhetri P, Jayatilleke GB, Gekara VO, Manzoni A, Corbitt B (2016) Container terminal operations simulator (CTOS)—simulating the impact of extreme weather events on port operation. Eur J Transp Infrastruct Res 16(1):195–213Google Scholar
  16. Chiu R-H, Lirn T-C, Li C-Y, Lu B-Y, Shang K-C (2011) An evaluation of free trade port zone in Taiwan. Asian Shipp Logist 27(3):423–445. CrossRefGoogle Scholar
  17. Chu S, Cui Y, Liu N (2017) The path towards sustainable energy. Nat Mater 16(1):16–22CrossRefGoogle Scholar
  18. Coast Guard, United States (2017) Steam ship El Faro sinking and loss of the vessel with 33 persons missing and presumed deceased northeast of acklins and crooked island, Bahamas on October 1, 2015. In Marine Board's Report. U.S. Department of Homeland SecurityGoogle Scholar
  19. Cruz MR, Ferreira JJM, Azevedo SM (2012) Operational performance and physical capacity of Iberian seaport: a multi-criteria analysis. In Studies in Fuzziness and Soft Computing, 449–63Google Scholar
  20. Da Cruz RP, Ferreira JJM, Azevedo SG (2013) Logistics resources in seaport performance: multi-criteria analysis. Marit Policy Manag 40(6):588–613. CrossRefGoogle Scholar
  21. De Langen PW, Sharypova K (2013) Intermodal connectivity as a port performance indicator. Res Transp Bus Manag 8:97–102. CrossRefGoogle Scholar
  22. Delandshere G, Petrosky AR (1998) Assessment of complex performances: limitations of key measurement assumptions. Educ Res 27(2):14–24. CrossRefGoogle Scholar
  23. Dong Y, Frangopol DM, Sabatino S (2016) A decision support system for mission-based ship routing considering multiple performance criteria. Reliab Eng Syst Saf 150:190–201. CrossRefGoogle Scholar
  24. Douma A, Schutten M, Schuur P (2009) Waiting profiles: an efficient protocol for enabling distributed planning of container barge rotations along terminals in the port of Rotterdam. Transp Res Part C: Emerg Technol 17(2):133–148. CrossRefGoogle Scholar
  25. Downing SM (2003) Validity: on the meaningful interpretation of assessment data. Med Educ 37(9):830–837CrossRefGoogle Scholar
  26. Ernstsen J, Nazir S, Røed BK (2017) Human reliability analysis of a pilotage operation. Saf Sea Transp:293–300.
  27. Ernstsen J, Nazir S, Røed BK, Manca D (2016) Systemising performance indicators in the assessment of complex sociotechnical systems. Chem Eng Trans 53:187–192. Google Scholar
  28. Fabiano B, Currò F, Reverberi AP, Pastorino R (2010) Port safety and the container revolution: a statistical study on human factor and occupational accidents over the long period. Saf Sci 48(8):980–990. CrossRefGoogle Scholar
  29. Grabowski M, You Z, Song H, Wang H, Merrick JRW (2010) Sailing on friday: developing the link between safety culture and performance in safety-critical systems. IEEE Trans Syst Man Cybern Part A Syst Humans 40(2):263–284. CrossRefGoogle Scholar
  30. Heij C, Knapp S (2012) Evaluation of safety and environmental risk at individual ship and company level. Transp Res Part D: Transp Environ 17(3):228–236. CrossRefGoogle Scholar
  31. Himes AH (2007) Performance indicators in MPA management: using questionnaires to analyze stakeholder preferences. Ocean Coast Manag 50(5–6):329–351. CrossRefGoogle Scholar
  32. Hinkin TR, Bruce Tracey J, Enz CA (1997) Scale construction: developing reliable and valid measurement instruments. J Hosp Tour Res 21(1):100–120CrossRefGoogle Scholar
  33. Hu Z-H (2015) Multi-objective genetic algorithm for berth allocation problem considering daytime preference. Comput Ind Eng 89:2–14. CrossRefGoogle Scholar
  34. Iannone R, Miranda S, Prisco L, Riemma S, Sarno D (2016) Proposal for a flexible discrete event simulation model for assessing the daily operation decisions in a Ro–Ro terminal. Simul Model Pract Theory 61:28–46. CrossRefGoogle Scholar
  35. Kandakoglu A, Celik M, Akgun I (2009) A multi-methodological approach for shipping registry selection in maritime transportation industry. Math Comput Model 49(3–4):586–597. CrossRefGoogle Scholar
  36. Kim TS (2015) The revealed competitiveness of major ports in the East Asian region: an additive market share analysis. Asian Shipp Logist 31(4):429–435. CrossRefGoogle Scholar
  37. Kim T-e, Nazir S, Øvergård KI (2016) A STAMP-based causal analysis of the Korean Sewol ferry accident. Saf Sci 83:93–101CrossRefGoogle Scholar
  38. Kluge A (2014) The acquisition of knowledge and skills for taskwork and teamwork to control complex technical systems: a cognitive and macroergonomics perspective: SpringerGoogle Scholar
  39. Lam JSL (2015) Designing a sustainable maritime supply chain: a hybrid QFD–ANP approach. Transp Res E-Log 78:70–81CrossRefGoogle Scholar
  40. Leriche D, Oudani M, Cabani A, Hoblos G, Mouzna J, Boukachour J, El Hilali Alaoui A (2015) Simulating new logistics system of Le Havre Port. IFAC-PapersOnLine 48(3):418–423. CrossRefGoogle Scholar
  41. Leveson N (2011) Engineering a safer world: systems thinking applied to safety. MIT Press, CambridgeGoogle Scholar
  42. Li J, Jiang B (2014) Cooperation performance evaluation between seaport and dry port; case of Qingdao port and Xi’an port*. Int J e-Navig Marit Econ 1:99–109. Google Scholar
  43. Lin K-L, Jhan H-T, Ting K-H, Lin C-L, Liu W-H (2014) Using indicators to evaluate the Taiwanese distant-water fishery-policy performance. Ocean Coast Manag 96:29–41. CrossRefGoogle Scholar
  44. Lu C-S, Tseng P-H (2012) Identifying crucial safety assessment criteria for passenger ferry services. Saf Sci 50(7):1462–1471. CrossRefGoogle Scholar
  45. Lun YHV (2011) Green management practices and firm performance: a case of container terminal operations. Resour Conserv Recycl 55(6):559–566. CrossRefGoogle Scholar
  46. Manca D, Nazir S, Colombo S, Kluge A (2014) Procedure for automated assessment of industrial operators. Chem Eng Trans 36:391–396Google Scholar
  47. Marlow PB, Paixão Casaca AC (2003) Measuring lean ports performance. Int J Transp Manag 1(4):189–202. Google Scholar
  48. Mavin TJ, Roth W-M (2014) A holistic view of cockpit performance: an analysis of the assessment discourse of flight examiners. Int J Aviat Psychol 24(3):210–227. CrossRefGoogle Scholar
  49. Monteiro JGR (2010) Measuring productivity and efficiency of major ports of India. Econ Polit Wkly 45(26/27):325–331Google Scholar
  50. Moon DSH, Woo JK (2014) The impact of port operations on efficient ship operation from both economic and environmental perspectives. Marit Policy Manag 41(5):444–461. CrossRefGoogle Scholar
  51. Naderpour M, Nazir S, Jie L (2015) The role of situation awareness in accidents of large-scale technological systems. Process Saf Environ Prot 97:13–24. CrossRefGoogle Scholar
  52. Nazir S, Kluge A, Manca D (2014) Automation in process industry: cure or curse? How can training improve operator’s performance. Comput Aided Chem Eng.
  53. Nazir S, Øvergård KI, Yang Z (2015) Towards effective training for process and maritime industries. Procedia Manuf 3:1519–1526. CrossRefGoogle Scholar
  54. Okstad E, Jersin E, Tinmannsvik RK (2012) Accident investigation in the Norwegian petroleum industry—common features and future challenges. Saf Sci 50(6):1408–1414. CrossRefGoogle Scholar
  55. Pak JY, Yeo GT, Oh SW, Yang Z (2015) Port safety evaluation from a captain’s perspective: the Korean experience. Saf Sci 72:172–181. CrossRefGoogle Scholar
  56. Park N-k, Yoon D-g, Park S-k (2014) Port capacity evaluation formula for general cargo. Asian Shipp Logist 30(2):175–192. CrossRefGoogle Scholar
  57. Peris-Mora E, Orejas JMD, Subirats A, Ibáñez S, Alvarez P (2005) Development of a system of indicators for sustainable port management. Mar Pollut Bull 50(12):1649–1660. CrossRefGoogle Scholar
  58. Prajogo D, Tang AKY, Lai K-h (2012) Do firms get what they want from ISO 14001 adoption?: an Australian perspective. J Clean Prod 33:117–126. CrossRefGoogle Scholar
  59. Rødseth, ØJ, Perera LP, Mo B (2016) Big data in shipping - challenges and opportunities. In 15th International conference on computer and IT Applications in the Maritime Industries - COMPIT '16. Schriftenreihe Schiffbau, Hamburg, pp 361–373Google Scholar
  60. Roh H-S, Lalwani CS, Naim MM (2007) Modelling a port logistics process using the structured analysis and design technique. Int J Log Res Appl 10(3):283–302CrossRefGoogle Scholar
  61. Roussanaly S, Hognes ES, Jakobsen JP (2013a) Multi-criteria analysis of two CO2 transport technologies. Energy Procedia 37:2981–2988. CrossRefGoogle Scholar
  62. Roussanaly S, Jakobsen JP, Hognes EH, Brunsvold AL (2013b) Benchmarking of CO2 transport technologies: part I—onshore pipeline and shipping between two onshore areas. Int J Greenhouse Gas Control 19:584–594. CrossRefGoogle Scholar
  63. Roussanaly S, Brunsvold AL, Hognes ES (2014) Benchmarking of CO2 transport technologies: part II—offshore pipeline and shipping to an offshore site. Int J Greenhouse Gas Control 28:283–299. CrossRefGoogle Scholar
  64. Sadovaya E, Thai VV (2015) Impacts of implementation of the effective maritime security management model (EMSMM) on organizational performance of shipping Companies1. Asian Shipp Logist 31(2):195–215. CrossRefGoogle Scholar
  65. Satumanatpan S, Senawongse P, Thansuporn W, Kirkman H (2014) Enhancing management effectiveness of environmental protected areas, Thailand. Ocean Coast Manag 89:1–10. CrossRefGoogle Scholar
  66. Saus E-R, Johnsen BH, Eid J, Thayer JF (2012) Who benefits from simulator training: personality and heart rate variability in relation to situation awareness during navigation training. Comput Hum Behav 28(4):1262–1268. CrossRefGoogle Scholar
  67. Schinas O, Stefanakos CN (2014) Selecting technologies towards compliance with MARPOL Annex VI: The perspective of operators. Transp Res Part D: Transp Environ 28:28–40. CrossRefGoogle Scholar
  68. Seguí X, Puig M, Quintieri E, Wooldridge C, Darbra RM (2016) New environmental performance baseline for inland ports: a benchmark for the European inland port sector. Environ Sci Pol 58:29–40. CrossRefGoogle Scholar
  69. Serebrisky T, Sarriera JM, Suárez-Alemán A, Araya G, Briceño-Garmendía C, Schwartz J (2016) Exploring the drivers of port efficiency in Latin America and the Caribbean. Transp Policy 45:31–45. CrossRefGoogle Scholar
  70. Sleire H, Dale E (2009) The shipping KPI standard. Retrieved from​ Accessed January 2017
  71. Somanathan S, Flynn P, Szymanski J (2009) The northwest passage: a simulation. Transp Res A Policy Pract 43(2):127–135. CrossRefGoogle Scholar
  72. Song D-P, Dong L, Drake P (2015) Multi-objective optimization for planning liner shipping service with uncertain port times. Transp Res E-Log 84:1–22. CrossRefGoogle Scholar
  73. Standal D, Utne IB (2011) The hard choices of sustainability. Mar Policy 35(4):519–527. CrossRefGoogle Scholar
  74. Stefanidaki E, Lekakou M (2014) Cruise carrying capacity: a conceptual approach. Res Transp Bus Manag 13:43–52. CrossRefGoogle Scholar
  75. Suárez-Alemán A, Sarriera JM, Serebrisky T, Trujillo L (2016) When it comes to container port efficiency, are all developing regions equal? Transp Res A Policy Pract 86:56–77. CrossRefGoogle Scholar
  76. Sutomo H, Soemardjito J (2012) Assessment model of the port effectiveness and efficiency (case study: western Indonesia region). Procedia - Soc Behavl Sci 43:24–32. CrossRefGoogle Scholar
  77. Talley WK, Ng MW, Marsillac E (2014) Port service chains and port performance evaluation. Transp Res E-Log 69:236–247. CrossRefGoogle Scholar
  78. Tichavska M, Tovar B (2015a) Environmental cost and eco-efficiency from vessel emissions in Las Palmas Port. Transp Res E-Log 83:126–140. CrossRefGoogle Scholar
  79. Tichavska M, Tovar B (2015b) Port-city exhaust emission model: an application to cruise and ferry operations in Las Palmas Port. Transp Res A Policy Pract 78:347–360. CrossRefGoogle Scholar
  80. Tongzon JL (2009) Port choice and freight forwarders. Transp Res E-Log 45(1):186–195. CrossRefGoogle Scholar
  81. Utne IB (2007a) Acceptable sustainability in the fishing fleet. Mar Policy 32(3):475–482. CrossRefGoogle Scholar
  82. Utne IB (2007b) System evaluation of sustainability in the Norwegian cod-fisheries. Mar Policy 31(4):390–401. CrossRefGoogle Scholar
  83. Vaghi C, Lucietti L (2016) Costs and benefits of speeding up reporting formalities in maritime transport. Transp Res Procedia 14:213–222. CrossRefGoogle Scholar
  84. Wang Y, Yeo GT, Ng AKY (2014) Choosing optimal bunkering ports for liner shipping companies: a hybrid fuzzy-Delphi-TOPSIS approach. Transp Policy 35:358–365. CrossRefGoogle Scholar
  85. Wiegmans B, Witte P, Spit T (2015) Characteristics of European inland ports: a statistical analysis of inland waterway port development in Dutch municipalities. Transp Res A Policy Pract 78:566–577. CrossRefGoogle Scholar
  86. Wiggins G (1993) Assessment to improve performance, not just monitor it: assessment reform in the social sciences. Soc Sci Rec 30(2):5–12Google Scholar
  87. Woo SH, Pettit S, Beresford AKC (2011) Port evolution and performance in changing logistics environments. Marit Econ Logist 13(3):250–277. CrossRefGoogle Scholar
  88. Yang Z, Ng AKY, Wang J (2014) A new risk quantification approach in port facility security assessment. Transp Res A Policy Pract 59:72–90. CrossRefGoogle Scholar

Copyright information

© World Maritime University 2018

Authors and Affiliations

  1. 1.Training and Assessment Research Group, Department of Maritime OperationsUniversity College of Southeast NorwayBorreNorway

Personalised recommendations