Consistency in the development of performance assessment methods in the maritime domain

Abstract

The maritime industry is considered to be a backbone of the global economy. It is therefore imperative to ensure that maritime operations run safely and efficiently. Assessment of maritime performance is necessary for designers and engineers to be able to pinpoint the weakest links in the system and make impactful system improvements. The current article presents a systematic quantitative literature review of research on performance assessment in the maritime industry with the goal of establishing an understanding of accuracy and consistency in the development of methods used to assess performance. The review focuses on four major segments within the industry—port logistics, ship handling, safety and environmental research—and investigates their uses in developing accurate and consistent performance assessment methods. After the completion of an exclusion process, 62 articles published in a wide range of academic journals were used in the analysis. Two important conclusions were drawn from the analysis. First, performance assessment is generally consistent throughout the maritime industry; most papers used accurate and consistent approaches to develop the methods (n = 43). A subsequent bivariate analysis revealed a call for increased attention to the development of assessment methods within the maritime segment of ship handling. The current study suggests and discusses certain directions with regard to assessment research in the maritime industry.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. Abril M, Barber F, Ingolotti L, Salido MA, Tormos P, Lova A (2008) An assessment of railway capacity. Transp Res E-Log 44(5):774–806. https://doi.org/10.1016/j.tre.2007.04.001

    Article  Google Scholar 

  2. Accident Investigation Board, Norway (2010a) Crete Cement - IMO NO. 9037161, grounding at Aspond Island in the Oslo Fjord, Norway, on 19 November 2008. Report Sjø 1

  3. Accident Investigation Board, Norway (2010b) Report on marine accident Federal Kivalina-IMO NO. 9205885 grounding at Årsundøya, Norway 6 October 2008. Report Sjø 1

  4. Accident Investigation Board, Norway 2012 Report on investigation into marine accident M/V Godafoss V2PM7 grounding in Løperen, Hvaler on 17 February 2011. Report Sjø 1

  5. Aditya P, Uday K, Diego G, Christer S (2015) Performance measurement and management for maintenance: a literature review. J Qual Maint Eng 21(1):2–33. https://doi.org/10.1108/JQME-10-2013-0067

    Article  Google Scholar 

  6. Akyuz E, Celik M (2014a) A hybrid decision-making approach to measure effectiveness of safety management system implementations on-board ships. Saf Sci 68:169–179. https://doi.org/10.1016/j.ssci.2014.04.003

    Article  Google Scholar 

  7. Akyuz E, Celik M (2014b) Utilisation of cognitive map in modelling human error in marine accident analysis and prevention. Saf Sci 70:19–28. https://doi.org/10.1016/j.ssci.2014.05.004

    Article  Google Scholar 

  8. Altuntaş C, Tuna O (2013) Greening logistics centers: the evolution of industrial buying criteria towards green. Asian Shipp Logist 29(1):59–80. https://doi.org/10.1016/j.ajsl.2013.05.004

    Article  Google Scholar 

  9. Antão P, Calderón M, Puig M, Michail A, Wooldridge C, Darbra RM (2016) Identification of occupational health, safety, security (OHSS) and environmental performance indicators in port areas. Saf Sci 85:266–275

    Article  Google Scholar 

  10. Bergantino AS, Musso E, Porcelli F (2013) Port management performance and contextual variables: which relationship? Methodological and empirical issues. Res Transp Bus Manag 8:39–49. https://doi.org/10.1016/j.rtbm.2013.07.002

    Article  Google Scholar 

  11. Bichou K, Gray R (2004) A logistics and supply chain management approach to port performance measurement. Marit Policy Manag 31(1):47–67. https://doi.org/10.1080/0308883032000174454

    Article  Google Scholar 

  12. BIMCO (2015) The global supply and demand for seafarers in 2015. Manpower report, In

    Google Scholar 

  13. Bouejla A, Chaze X, Guarnieri F, Napoli A (2014) A Bayesian network to manage risks of maritime piracy against offshore oil fields. Saf Sci 68(Supplement C):222–230. https://doi.org/10.1016/j.ssci.2014.04.010

    Article  Google Scholar 

  14. Chen L, Zhang D, Ma X, Wang L, Li S, Wu Z, Pan G (2016) Container port performance measurement and comparison leveraging ship GPS traces and maritime open data. IEEE Trans Intell Transp Syst 17(5):1227–1242. https://doi.org/10.1109/TITS.2015.2498409

    Article  Google Scholar 

  15. Chhetri P, Jayatilleke GB, Gekara VO, Manzoni A, Corbitt B (2016) Container terminal operations simulator (CTOS)—simulating the impact of extreme weather events on port operation. Eur J Transp Infrastruct Res 16(1):195–213

    Google Scholar 

  16. Chiu R-H, Lirn T-C, Li C-Y, Lu B-Y, Shang K-C (2011) An evaluation of free trade port zone in Taiwan. Asian Shipp Logist 27(3):423–445. https://doi.org/10.1016/S2092-5212(11)80020-9

    Article  Google Scholar 

  17. Chu S, Cui Y, Liu N (2017) The path towards sustainable energy. Nat Mater 16(1):16–22

    Article  Google Scholar 

  18. Coast Guard, United States (2017) Steam ship El Faro sinking and loss of the vessel with 33 persons missing and presumed deceased northeast of acklins and crooked island, Bahamas on October 1, 2015. In Marine Board's Report. U.S. Department of Homeland Security

  19. Cruz MR, Ferreira JJM, Azevedo SM (2012) Operational performance and physical capacity of Iberian seaport: a multi-criteria analysis. In Studies in Fuzziness and Soft Computing, 449–63

  20. Da Cruz RP, Ferreira JJM, Azevedo SG (2013) Logistics resources in seaport performance: multi-criteria analysis. Marit Policy Manag 40(6):588–613. https://doi.org/10.1080/03088839.2013.777979

    Article  Google Scholar 

  21. De Langen PW, Sharypova K (2013) Intermodal connectivity as a port performance indicator. Res Transp Bus Manag 8:97–102. https://doi.org/10.1016/j.rtbm.2013.06.003

    Article  Google Scholar 

  22. Delandshere G, Petrosky AR (1998) Assessment of complex performances: limitations of key measurement assumptions. Educ Res 27(2):14–24. https://doi.org/10.2307/1176194

    Article  Google Scholar 

  23. Dong Y, Frangopol DM, Sabatino S (2016) A decision support system for mission-based ship routing considering multiple performance criteria. Reliab Eng Syst Saf 150:190–201. https://doi.org/10.1016/j.ress.2016.02.002

    Article  Google Scholar 

  24. Douma A, Schutten M, Schuur P (2009) Waiting profiles: an efficient protocol for enabling distributed planning of container barge rotations along terminals in the port of Rotterdam. Transp Res Part C: Emerg Technol 17(2):133–148. https://doi.org/10.1016/j.trc.2008.06.003

    Article  Google Scholar 

  25. Downing SM (2003) Validity: on the meaningful interpretation of assessment data. Med Educ 37(9):830–837

    Article  Google Scholar 

  26. Ernstsen J, Nazir S, Røed BK (2017) Human reliability analysis of a pilotage operation. Saf Sea Transp:293–300. https://doi.org/10.1201/9781315099088-51

  27. Ernstsen J, Nazir S, Røed BK, Manca D (2016) Systemising performance indicators in the assessment of complex sociotechnical systems. Chem Eng Trans 53:187–192. https://doi.org/10.3303/CET1653032

    Google Scholar 

  28. Fabiano B, Currò F, Reverberi AP, Pastorino R (2010) Port safety and the container revolution: a statistical study on human factor and occupational accidents over the long period. Saf Sci 48(8):980–990. https://doi.org/10.1016/j.ssci.2009.08.007

    Article  Google Scholar 

  29. Grabowski M, You Z, Song H, Wang H, Merrick JRW (2010) Sailing on friday: developing the link between safety culture and performance in safety-critical systems. IEEE Trans Syst Man Cybern Part A Syst Humans 40(2):263–284. https://doi.org/10.1109/TSMCA.2009.2035300

    Article  Google Scholar 

  30. Heij C, Knapp S (2012) Evaluation of safety and environmental risk at individual ship and company level. Transp Res Part D: Transp Environ 17(3):228–236. https://doi.org/10.1016/j.trd.2011.12.003

    Article  Google Scholar 

  31. Himes AH (2007) Performance indicators in MPA management: using questionnaires to analyze stakeholder preferences. Ocean Coast Manag 50(5–6):329–351. https://doi.org/10.1016/j.ocecoaman.2006.09.005

    Article  Google Scholar 

  32. Hinkin TR, Bruce Tracey J, Enz CA (1997) Scale construction: developing reliable and valid measurement instruments. J Hosp Tour Res 21(1):100–120

    Article  Google Scholar 

  33. Hu Z-H (2015) Multi-objective genetic algorithm for berth allocation problem considering daytime preference. Comput Ind Eng 89:2–14. https://doi.org/10.1016/j.cie.2015.04.035

    Article  Google Scholar 

  34. Iannone R, Miranda S, Prisco L, Riemma S, Sarno D (2016) Proposal for a flexible discrete event simulation model for assessing the daily operation decisions in a Ro–Ro terminal. Simul Model Pract Theory 61:28–46. https://doi.org/10.1016/j.simpat.2015.11.005

    Article  Google Scholar 

  35. Kandakoglu A, Celik M, Akgun I (2009) A multi-methodological approach for shipping registry selection in maritime transportation industry. Math Comput Model 49(3–4):586–597. https://doi.org/10.1016/j.mcm.2008.09.001

    Article  Google Scholar 

  36. Kim TS (2015) The revealed competitiveness of major ports in the East Asian region: an additive market share analysis. Asian Shipp Logist 31(4):429–435. https://doi.org/10.1016/j.ajsl.2016.01.001

    Article  Google Scholar 

  37. Kim T-e, Nazir S, Øvergård KI (2016) A STAMP-based causal analysis of the Korean Sewol ferry accident. Saf Sci 83:93–101

    Article  Google Scholar 

  38. Kluge A (2014) The acquisition of knowledge and skills for taskwork and teamwork to control complex technical systems: a cognitive and macroergonomics perspective: Springer

  39. Lam JSL (2015) Designing a sustainable maritime supply chain: a hybrid QFD–ANP approach. Transp Res E-Log 78:70–81

    Article  Google Scholar 

  40. Leriche D, Oudani M, Cabani A, Hoblos G, Mouzna J, Boukachour J, El Hilali Alaoui A (2015) Simulating new logistics system of Le Havre Port. IFAC-PapersOnLine 48(3):418–423. https://doi.org/10.1016/j.ifacol.2015.06.117

    Article  Google Scholar 

  41. Leveson N (2011) Engineering a safer world: systems thinking applied to safety. MIT Press, Cambridge

    Google Scholar 

  42. Li J, Jiang B (2014) Cooperation performance evaluation between seaport and dry port; case of Qingdao port and Xi’an port*. Int J e-Navig Marit Econ 1:99–109. https://doi.org/10.1016/j.enavi.2014.12.009

    Google Scholar 

  43. Lin K-L, Jhan H-T, Ting K-H, Lin C-L, Liu W-H (2014) Using indicators to evaluate the Taiwanese distant-water fishery-policy performance. Ocean Coast Manag 96:29–41. https://doi.org/10.1016/j.ocecoaman.2014.04.028

    Article  Google Scholar 

  44. Lu C-S, Tseng P-H (2012) Identifying crucial safety assessment criteria for passenger ferry services. Saf Sci 50(7):1462–1471. https://doi.org/10.1016/j.ssci.2012.01.019

    Article  Google Scholar 

  45. Lun YHV (2011) Green management practices and firm performance: a case of container terminal operations. Resour Conserv Recycl 55(6):559–566. https://doi.org/10.1016/j.resconrec.2010.12.001

    Article  Google Scholar 

  46. Manca D, Nazir S, Colombo S, Kluge A (2014) Procedure for automated assessment of industrial operators. Chem Eng Trans 36:391–396

    Google Scholar 

  47. Marlow PB, Paixão Casaca AC (2003) Measuring lean ports performance. Int J Transp Manag 1(4):189–202. https://doi.org/10.1016/j.ijtm.2003.12.002

    Google Scholar 

  48. Mavin TJ, Roth W-M (2014) A holistic view of cockpit performance: an analysis of the assessment discourse of flight examiners. Int J Aviat Psychol 24(3):210–227. https://doi.org/10.1080/10508414.2014.918434

    Article  Google Scholar 

  49. Monteiro JGR (2010) Measuring productivity and efficiency of major ports of India. Econ Polit Wkly 45(26/27):325–331

    Google Scholar 

  50. Moon DSH, Woo JK (2014) The impact of port operations on efficient ship operation from both economic and environmental perspectives. Marit Policy Manag 41(5):444–461. https://doi.org/10.1080/03088839.2014.931607

    Article  Google Scholar 

  51. Naderpour M, Nazir S, Jie L (2015) The role of situation awareness in accidents of large-scale technological systems. Process Saf Environ Prot 97:13–24. https://doi.org/10.1016/j.psep.2015.06.002

    Article  Google Scholar 

  52. Nazir S, Kluge A, Manca D (2014) Automation in process industry: cure or curse? How can training improve operator’s performance. Comput Aided Chem Eng. https://doi.org/10.1016/b978-0-444-63456-6.50149-6

  53. Nazir S, Øvergård KI, Yang Z (2015) Towards effective training for process and maritime industries. Procedia Manuf 3:1519–1526. https://doi.org/10.1016/j.promfg.2015.07.409

    Article  Google Scholar 

  54. Okstad E, Jersin E, Tinmannsvik RK (2012) Accident investigation in the Norwegian petroleum industry—common features and future challenges. Saf Sci 50(6):1408–1414. https://doi.org/10.1016/j.ssci.2010.12.012

    Article  Google Scholar 

  55. Pak JY, Yeo GT, Oh SW, Yang Z (2015) Port safety evaluation from a captain’s perspective: the Korean experience. Saf Sci 72:172–181. https://doi.org/10.1016/j.ssci.2014.09.007

    Article  Google Scholar 

  56. Park N-k, Yoon D-g, Park S-k (2014) Port capacity evaluation formula for general cargo. Asian Shipp Logist 30(2):175–192. https://doi.org/10.1016/j.ajsl.2014.09.003

    Article  Google Scholar 

  57. Peris-Mora E, Orejas JMD, Subirats A, Ibáñez S, Alvarez P (2005) Development of a system of indicators for sustainable port management. Mar Pollut Bull 50(12):1649–1660. https://doi.org/10.1016/j.marpolbul.2005.06.048

    Article  Google Scholar 

  58. Prajogo D, Tang AKY, Lai K-h (2012) Do firms get what they want from ISO 14001 adoption?: an Australian perspective. J Clean Prod 33:117–126. https://doi.org/10.1016/j.jclepro.2012.04.019

    Article  Google Scholar 

  59. Rødseth, ØJ, Perera LP, Mo B (2016) Big data in shipping - challenges and opportunities. In 15th International conference on computer and IT Applications in the Maritime Industries - COMPIT '16. Schriftenreihe Schiffbau, Hamburg, pp 361–373

    Google Scholar 

  60. Roh H-S, Lalwani CS, Naim MM (2007) Modelling a port logistics process using the structured analysis and design technique. Int J Log Res Appl 10(3):283–302

    Article  Google Scholar 

  61. Roussanaly S, Hognes ES, Jakobsen JP (2013a) Multi-criteria analysis of two CO2 transport technologies. Energy Procedia 37:2981–2988. https://doi.org/10.1016/j.egypro.2013.06.184

    Article  Google Scholar 

  62. Roussanaly S, Jakobsen JP, Hognes EH, Brunsvold AL (2013b) Benchmarking of CO2 transport technologies: part I—onshore pipeline and shipping between two onshore areas. Int J Greenhouse Gas Control 19:584–594. https://doi.org/10.1016/j.ijggc.2013.05.031

    Article  Google Scholar 

  63. Roussanaly S, Brunsvold AL, Hognes ES (2014) Benchmarking of CO2 transport technologies: part II—offshore pipeline and shipping to an offshore site. Int J Greenhouse Gas Control 28:283–299. https://doi.org/10.1016/j.ijggc.2014.06.019

    Article  Google Scholar 

  64. Sadovaya E, Thai VV (2015) Impacts of implementation of the effective maritime security management model (EMSMM) on organizational performance of shipping Companies1. Asian Shipp Logist 31(2):195–215. https://doi.org/10.1016/j.ajsl.2015.06.002

    Article  Google Scholar 

  65. Satumanatpan S, Senawongse P, Thansuporn W, Kirkman H (2014) Enhancing management effectiveness of environmental protected areas, Thailand. Ocean Coast Manag 89:1–10. https://doi.org/10.1016/j.ocecoaman.2013.12.001

    Article  Google Scholar 

  66. Saus E-R, Johnsen BH, Eid J, Thayer JF (2012) Who benefits from simulator training: personality and heart rate variability in relation to situation awareness during navigation training. Comput Hum Behav 28(4):1262–1268. https://doi.org/10.1016/j.chb.2012.02.009

    Article  Google Scholar 

  67. Schinas O, Stefanakos CN (2014) Selecting technologies towards compliance with MARPOL Annex VI: The perspective of operators. Transp Res Part D: Transp Environ 28:28–40. https://doi.org/10.1016/j.trd.2013.12.006

    Article  Google Scholar 

  68. Seguí X, Puig M, Quintieri E, Wooldridge C, Darbra RM (2016) New environmental performance baseline for inland ports: a benchmark for the European inland port sector. Environ Sci Pol 58:29–40. https://doi.org/10.1016/j.envsci.2015.12.014

    Article  Google Scholar 

  69. Serebrisky T, Sarriera JM, Suárez-Alemán A, Araya G, Briceño-Garmendía C, Schwartz J (2016) Exploring the drivers of port efficiency in Latin America and the Caribbean. Transp Policy 45:31–45. https://doi.org/10.1016/j.tranpol.2015.09.004

    Article  Google Scholar 

  70. Sleire H, Dale E (2009) The shipping KPI standard. Retrieved from​ https://www.shipping-kpi.org. Accessed January 2017

  71. Somanathan S, Flynn P, Szymanski J (2009) The northwest passage: a simulation. Transp Res A Policy Pract 43(2):127–135. https://doi.org/10.1016/j.tra.2008.08.001

    Article  Google Scholar 

  72. Song D-P, Dong L, Drake P (2015) Multi-objective optimization for planning liner shipping service with uncertain port times. Transp Res E-Log 84:1–22. https://doi.org/10.1016/j.tre.2015.10.001

    Article  Google Scholar 

  73. Standal D, Utne IB (2011) The hard choices of sustainability. Mar Policy 35(4):519–527. https://doi.org/10.1016/j.marpol.2011.01.001

    Article  Google Scholar 

  74. Stefanidaki E, Lekakou M (2014) Cruise carrying capacity: a conceptual approach. Res Transp Bus Manag 13:43–52. https://doi.org/10.1016/j.rtbm.2014.11.005

    Article  Google Scholar 

  75. Suárez-Alemán A, Sarriera JM, Serebrisky T, Trujillo L (2016) When it comes to container port efficiency, are all developing regions equal? Transp Res A Policy Pract 86:56–77. https://doi.org/10.1016/j.tra.2016.01.018

    Article  Google Scholar 

  76. Sutomo H, Soemardjito J (2012) Assessment model of the port effectiveness and efficiency (case study: western Indonesia region). Procedia - Soc Behavl Sci 43:24–32. https://doi.org/10.1016/j.sbspro.2012.04.074

    Article  Google Scholar 

  77. Talley WK, Ng MW, Marsillac E (2014) Port service chains and port performance evaluation. Transp Res E-Log 69:236–247. https://doi.org/10.1016/j.tre.2014.05.008

    Article  Google Scholar 

  78. Tichavska M, Tovar B (2015a) Environmental cost and eco-efficiency from vessel emissions in Las Palmas Port. Transp Res E-Log 83:126–140. https://doi.org/10.1016/j.tre.2015.09.002

    Article  Google Scholar 

  79. Tichavska M, Tovar B (2015b) Port-city exhaust emission model: an application to cruise and ferry operations in Las Palmas Port. Transp Res A Policy Pract 78:347–360. https://doi.org/10.1016/j.tra.2015.05.021

    Article  Google Scholar 

  80. Tongzon JL (2009) Port choice and freight forwarders. Transp Res E-Log 45(1):186–195. https://doi.org/10.1016/j.tre.2008.02.004

    Article  Google Scholar 

  81. Utne IB (2007a) Acceptable sustainability in the fishing fleet. Mar Policy 32(3):475–482. https://doi.org/10.1016/j.marpol.2007.09.017

    Article  Google Scholar 

  82. Utne IB (2007b) System evaluation of sustainability in the Norwegian cod-fisheries. Mar Policy 31(4):390–401. https://doi.org/10.1016/j.marpol.2006.10.006

    Article  Google Scholar 

  83. Vaghi C, Lucietti L (2016) Costs and benefits of speeding up reporting formalities in maritime transport. Transp Res Procedia 14:213–222. https://doi.org/10.1016/j.trpro.2016.05.057

    Article  Google Scholar 

  84. Wang Y, Yeo GT, Ng AKY (2014) Choosing optimal bunkering ports for liner shipping companies: a hybrid fuzzy-Delphi-TOPSIS approach. Transp Policy 35:358–365. https://doi.org/10.1016/j.tranpol.2014.04.009

    Article  Google Scholar 

  85. Wiegmans B, Witte P, Spit T (2015) Characteristics of European inland ports: a statistical analysis of inland waterway port development in Dutch municipalities. Transp Res A Policy Pract 78:566–577. https://doi.org/10.1016/j.tra.2015.07.004

    Article  Google Scholar 

  86. Wiggins G (1993) Assessment to improve performance, not just monitor it: assessment reform in the social sciences. Soc Sci Rec 30(2):5–12

    Google Scholar 

  87. Woo SH, Pettit S, Beresford AKC (2011) Port evolution and performance in changing logistics environments. Marit Econ Logist 13(3):250–277. https://doi.org/10.1057/mel.2011.12

    Article  Google Scholar 

  88. Yang Z, Ng AKY, Wang J (2014) A new risk quantification approach in port facility security assessment. Transp Res A Policy Pract 59:72–90. https://doi.org/10.1016/j.tra.2013.10.025

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding authors

Correspondence to Jørgen Ernstsen or Salman Nazir.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ernstsen, J., Nazir, S. Consistency in the development of performance assessment methods in the maritime domain. WMU J Marit Affairs 17, 71–90 (2018). https://doi.org/10.1007/s13437-018-0136-5

Download citation

Keywords

  • Maritime performance
  • Assessment
  • Performance indicators
  • Literature review
  • Complex operations